Posterior dorsomedial striatum is critical for both selective instrumental and Pavlovian reward learning

Outcome devaluation as a method for identifying goal-directed behaviors in rats

Goal-directed behaviors allow animals to act to satisfy needs and desires. The outcome devaluation task is an effective method for identifying goal-directed behaviors and distinguishing these from other types of behavior. Rats can be trained to lever-press for one or multiple distinct food rewards. During testing, the previously earned food-or a control food for comparison-is devalued by allowing the animal to freely feed on it until they are sated before testing lever-press performance under extinction conditions (no rewards are delivered). Behavior that adapts to reflect the new value of the outcome is considered goal-directed, whereas behavior that continues as in previous training despite the change in outcome value, is not. As more research groups have used this task, variability in the procedures used has increased. Here, we provide a reliable procedure for conducting the outcome devaluation task with appropriate controls. We describe the most common variants of the task and control conditions and discuss troubleshooting measures such as outcome pre-exposure, habituation to pre-feeding chambers and attention to animals' hunger levels. The method outlined can be executed in ~2 weeks including training (~8 d) and testing (1-4 d) by researchers who are familiar with performing behavioral tasks in laboratory rodents, although longer training may be considered for those who are interested in observing habitual control of behavior. This protocol should facilitate the comparison of results from different studies and laboratories, while allowing flexibility in the application of the outcome devaluation task to different research questions.

Ceftriaxone reverses diet-induced deficits in goal-directed control

RATIONALE

Obesity is associated with numerous health risks and ever-increasing rates are a significant global concern. However, despite weight loss attempts many people have difficulty maintaining weight loss. Previous studies in animals have shown that chronic access to an obesogenic diet can disrupt goal-directed behavior, impairing the ability of animals to flexibly adjust food-seeking behavior following changes in the value of earned outcomes. Changes in behavioral control have been linked to disruption of glutamate transmission in the dorsal medial striatum (DMS), a region critical for the acquisition and expression of goal-directed behavior.

OBJECTIVES

The goal of this study was to test whether ceftriaxone, a beta-lactam antibiotic shown elsewhere to upregulate the expression of the glutamate transporter GLT-1, would improve goal-directed control following long-term exposure to an obesogenic diet.

METHODS

Male and female rats were given access to either standard chow or chow plus sweetened condensed milk (SCM) for 6 weeks. Access to SCM was ended and rats received daily injections of either ceftriaxone or saline for 6 days. Rats were then trained to press a lever to earn a novel food reward and, finally, were assessed for sensitivity to outcome devaluation. Histological analyses examined changes to GLT-1 protein levels and morphological changes to astrocytes, within the DMS.

RESULTS

We found that ceftriaxone robustly restored goal-directed behavior in animals following long-term exposure to SCM. While we did not observe changes in protein levels of GLT-1 in the DMS, we observed that SCM induced changes in the morphology of astrocytes in the DMS, and that ceftriaxone mitigated these changes.

CONCLUSIONS

These results demonstrate that long-term access to a SCM diet impairs goal-directed behavior while also altering the morphology of astrocytes in the DMS. Furthermore, these results suggest that ceftriaxone administration can reverse the impairment of goal-directed behavior potentially through its actions on astrocytes in decision-making circuitry.

A dual-pathway architecture enables chronic stress to disrupt agency and promote habit formation

Chronic stress can change how we learn and, thus, how we make decisions. Here we investigated the neuronal circuit mechanisms that enable this. Using a multifaceted systems neuroscience approach in male and female mice, we reveal a dual pathway, amygdala-striatal neuronal circuit architecture by which a recent history of chronic stress disrupts the action-outcome learning underlying adaptive agency and promotes the formation of inflexible habits. We found that the basolateral amygdala projection to the dorsomedial striatum is activated by rewarding events to support the action-outcome learning needed for flexible, goal-directed decision making. Chronic stress attenuates this to disrupt action-outcome learning and, therefore, agency. Conversely, the central amygdala projection to the dorsomedial striatum mediates habit formation. Following stress this pathway is progressively recruited to learning to promote the premature formation of inflexible habits. Thus, stress exerts opposing effects on two amygdala-striatal pathways to disrupt agency and promote habit. These data provide neuronal circuit insights into how chronic stress shapes learning and decision making, and help understand how stress can lead to the disrupted decision making and pathological habits that characterize substance use disorders and mental health conditions.

Dorsomedial Striatum CB1R signaling is required for Pavlovian outcome devaluation in male Long Evans rats and reduces inhibitory synaptic transmission in both sexes

Cannabinoid-1 receptor (CB1R) signaling in the dorsal striatum regulates the shift from flexible to habitual behavior in instrumental outcome devaluation. Based on prior work establishing individual, sex, and experience-dependent differences in Pavlovian behaviors, we predicted a role for dorsomedial striatum CB1R signaling in driving rigid responding in Pavlovian autoshaping and outcome devaluation. We trained male and female Long Evans rats in Pavlovian Lever Autoshaping (PLA). We gave intra-dorsomedial striatum (DMS) infusions of the CB1R inverse agonist, rimonabant, before satiety-induced outcome devaluation test sessions, where we sated rats on training pellets or home cage chow and tested them in brief nonreinforced Pavlovian Lever Autoshaping sessions. Overall, inhibition of DMS CB1R signaling prevented Pavlovian outcome devaluation but did not affect behavior in reinforced PLA sessions. Males were sensitive to devaluation while females were not and DMS CB1R inhibition impaired devaluation sensitivity in males. We then investigated how DMS CB1R signaling impacts local inhibitory synaptic transmission in male and female Long Evans rats. We recorded spontaneous inhibitory postsynaptic currents (sIPSC) from DMS neurons at baseline and before and after application of a CB1R agonist, WIN 55,212-2. We found that male rats showed decreased sIPSC frequency compared to females, and that CB1R activation reduced DMS inhibitory transmission independent of sex. Altogether our results demonstrate that DMS CB1Rs regulate Pavlovian devaluation sensitivity and inhibitory synaptic transmission and suggest that basal sex differences in inhibitory synaptic transmission may underly sex differences in DMS function and behavioral flexibility.

Pharmacological manipulations of the dorsomedial and dorsolateral striatum during fear extinction reveal opposing roles in fear renewal

Systemic manipulations that enhance dopamine (DA) transmission around the time of fear extinction can strengthen fear extinction and reduce conditioned fear relapse. Prior studies investigating the brain regions where DA augments fear extinction focus on targets of mesolimbic and mesocortical DA systems originating in the ventral tegmental area, given the role of these DA neurons in prediction error. The dorsal striatum (DS), a primary target of the nigrostriatal DA system originating in the substantia nigra (SN), is implicated in behaviors beyond its canonical role in movement, such as reward and punishment, goal-directed action, and stimulus-response associations, but whether DS DA contributes to fear extinction is unknown. We have observed that chemogenetic stimulation of SN DA neurons during fear extinction prevents the return of fear in contexts different from the extinction context, a form of relapse called renewal. This effect of SN DA stimulation is mimicked by a DA D1 receptor (D1R) agonist injected into the DS, thus implicating DS DA in fear extinction. Different DS subregions subserve unique functions of the DS, but it is unclear where in the DS D1R agonist acts during fear extinction to reduce renewal. Furthermore, although fear extinction increases neural activity in DS subregions, whether neural activity in DS subregions is causally involved in fear extinction is unknown. To explore the role of DS subregions in fear extinction, adult, male Long-Evans rats received microinjections of either the D1R agonist SKF38393 or a cocktail consisting of GABAA/GABAB receptor agonists muscimol/baclofen selectively into either dorsomedial (DMS) or dorsolateral (DLS) DS subregions immediately prior to fear extinction, and extinction retention and renewal were subsequently assessed drug-free. While increasing D1R signaling in the DMS during fear extinction did not impact fear extinction retention or renewal, DMS inactivation reduced later renewal. In contrast, DLS inactivation had no effect on fear extinction retention or renewal but increasing D1R signaling in the DLS during extinction reduced fear renewal. These data suggest that DMS and DLS activity during fear extinction can have opposing effects on later fear renewal, with the DMS promoting renewal and the DLS opposing renewal. Mechanisms through which the DS could influence the contextual gating of fear extinction are discussed.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration. We then exposed them to four days of punishment testing in which footshock was delivered randomly on one-third of trials. Before and after punishment testing (four days pre-punishment and ≥ four days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance for cocaine was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment in males, although it was associated with habitual responding both pre- and post-punishment in females, indicating that punishment resistance was predicted by habitual responding in food-seeking females. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Opening new vistas on obsessive-compulsive disorder with the observing response task

Obsessive-compulsive disorder (OCD), a highly prevalent and debilitating disorder, is incompletely understood in terms of underpinning behavioural, psychological, and neural mechanisms. This is attributable to high symptomatic heterogeneity; cardinal features comprise obsessions and compulsions, including clinical subcategories. While obsessive and intrusive thoughts are arguably unique to humans, dysfunctional behaviours analogous to those seen in clinical OCD have been examined in nonhuman animals. Genetic, ethological, pharmacological, and neurobehavioural approaches all contribute to understanding the emergence and persistence of compulsive behaviour. One behaviour of particular interest is maladaptive checking, whereby human patients excessively perform checking rituals despite these serving no purpose. Dysfunctional and excessive checking is the most common symptom associated with OCD and can be readily operationalised in rodents. This review considers animal models of OCD, the neural circuitries associated with impairments in habit-based and goal-directed behaviour, and how these may link to the compulsions observed in OCD. We further review the Observing Response Task (ORT), an appetitive instrumental learning procedure that distinguishes between functional and dysfunctional checking, with translational application in humans and rodents. By shedding light on the psychological and neural bases of compulsive-like checking, the ORT has potential to offer translational insights into the underlying mechanisms of OCD, in addition to being a platform for testing psychological and neurochemical treatment approaches.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Striatum supports fast learning but not memory recall

Animals learn to carry out motor actions in specific sensory contexts to achieve goals. The striatum has been implicated in producing sensory-motor associations, yet its contribution to memory formation or recall is not clear. To investigate the contribution of striatum to these processes, mice were taught to associate a cue, consisting of optogenetic activation of striatum-projecting neurons in visual cortex, with forelimb reaches to access food pellets. As necessary to direct learning, striatal neural activity encoded both the sensory context and outcome of reaching. With training, the rate of cued reaching increased, but brief optogenetic inhibition of striatal activity arrested learning and prevented trial-to-trial improvements in performance. However, the same manipulation did not affect performance improvements already consolidated into short- (within an hour) or long-term (across days) memories. Hence, striatal activity is necessary for trial-to-trial improvements in task performance, leading to plasticity in other brain areas that mediate memory recall.

The Motivational Determinants of Human Action, Their Neural Bases and Functional Impact in Adolescents With Obsessive-Compulsive Disorder

Background

Establishing the motivational influences on human action is essential for understanding choice and decision making in health and disease. Here we used tests of value-based decision making, manipulating both predicted and experienced reward values to assess the motivational control of goal-directed action in healthy adolescents and those with obsessive-compulsive disorder (OCD).

Methods

After instrumental training on a two action-two outcome probabilistic task, adolescents (n = 21) underwent Pavlovian conditioning using distinct stimuli predicting either the instrumental outcomes, a third outcome, or nothing. We then assessed functional magnetic resonance imaging during choice tests in which we varied the predicted value, using specific and general Pavlovian-instrumental transfer, and the experienced value, using outcome devaluation. To establish functional significance, we tested a matched cohort of adolescents with OCD (n = 20).

Results

In healthy adolescents, both predicted and experienced values influenced the performance of goal-directed actions, mediated by distinct orbitofrontal-striatal circuits involving the lateral orbitofrontal cortex (OFC) and medial OFC, respectively. However, in adolescents with OCD, choice was insensitive to changes in either predicted or experienced values. These impairments were related to hypoactivity in the lateral OFC and hyperactivity in the medial OFC during specific Pavlovian-instrumental transfer and hypoactivity in the anterior prefrontal cortex, caudate nucleus, and their connectivity in the devaluation test.

Conclusions

We found that predicted and experienced values exerted a potent influence on the performance of goal-directed actions in adolescents via distinct orbitofrontal- and prefrontal-striatal circuits. Furthermore, the influence of these motivational processes was severely blunted in OCD, as was the functional segregation of circuits involving medial and lateral OFC, producing dysregulated action control.

Inhibition of the dorsomedial striatal direct pathway is essential for the execution of action sequences

Contrary to the previous notion that the dorsomedial striatum (DMS) is crucial for acquiring new learning, accumulated evidence has suggested that the DMS also plays a role in the execution of already learned action sequences. Here, we examined how the direct and indirect pathways in the DMS regulate action sequences using a task that requires animals to press a lever consecutively. Cell-type-specific bulk Ca2+ recording revealed that the direct pathway was inhibited at the time of sequence execution. The sequence-related response was blunted in trials where the sequential behaviors were disrupted. Optogenetic activation at the sequence start caused distraction of action sequences without affecting motor function or memory of the task structure. By contrast with the direct pathway, the indirect pathway was slightly activated at the start of the sequence, but the optogenetic suppression of such sequence-related signaling did not impact the behaviors. These results suggest that the inhibition of the DMS direct pathway promotes sequence execution potentially by suppressing the formation of a new association.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration (2 h/day). We then exposed them to 4 days of punishment testing, in which footshock (0.4 mA, 0.3 s) was delivered randomly on one-third of trials, immediately following completion of seeking and prior to extension of the taking lever. Before and after punishment testing (4 days pre-punishment and ≥4 days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Activating the dorsomedial and ventral midbrain projections to the striatum differentially impairs goal-directed action in male mice

The cognitive symptoms of schizophrenia are wide ranging and include impaired goal-directed action. This could be driven by an increase in dopamine transmission in the dorsomedial striatum, a pathophysiological hallmark of schizophrenia. Although commonly associated with psychotic symptoms, dopamine signalling in this region also modulates associative learning that aids in the execution of actions. To gain a better understanding of the role of subcortical dopamine in learning and decision-making, we assessed goal-directed action in male mice using the cross-species outcome-specific devaluation task (ODT). First, we administered systemic amphetamine during training to determine the impact of altered dopaminergic signaling on associative learning. Second, we used pathway-specific chemogenetic approaches to activate the dorsomedial and ventral striatal pathways (that originate in the midbrain) to separately assess learning and performance. Amphetamine treatment during learning led to a dose-dependent impairment in goal-directed action. Activation of both striatal pathways during learning also impaired performance. However, when these pathways were activated during choice, only activation of the ventral pathway impaired goal-directed action. This suggests that elevated transmission in the dorsomedial striatal pathway impairs associative learning processes that guide the goal-directed execution of actions. By contrast, elevated transmission of the ventral striatal pathway disrupts the encoding of outcome values that are important for both associative learning and choice performance. These findings highlight the differential roles of the dorsomedial and ventral inputs into the striatum in goal-directed action and provides insight into how striatal dopamine signaling may contribute to the cognitive problems in those with schizophrenia.

Distributed processing for value-based choice by prelimbic circuits targeting anterior-posterior dorsal striatal subregions in male mice

Fronto-striatal circuits have been implicated in cognitive control of behavioral output for social and appetitive rewards. The functional diversity of prefrontal cortical populations is strongly dependent on their synaptic targets, with control of motor output mediated by connectivity to dorsal striatum. Despite evidence for functional diversity along the anterior-posterior striatal axis, it is unclear how distinct fronto-striatal sub-circuits support value-based choice. Here we found segregated prefrontal populations defined by anterior/posterior dorsomedial striatal target. During a feedback-based 2-alternative choice task, single-photon imaging revealed circuit-specific representations of task-relevant information with prelimbic neurons targeting anterior DMS (PL::A-DMS) robustly modulated during choices and negative outcomes, while prelimbic neurons targeting posterior DMS (PL::P-DMS) encoded internal representations of value and positive outcomes contingent on prior choice. Consistent with this distributed coding, optogenetic inhibition of PL::A-DMS circuits strongly impacted choice monitoring and responses to negative outcomes while inhibition of PL::P-DMS impaired task engagement and strategies following positive outcomes. Together our data uncover PL populations engaged in distributed processing for value-based choice.

The role of dorsomedial striatum adenosine 2A receptors in the loss of goal-directed behaviour

RATIONALE

Adenosine A_2A receptors (A_2AR) in the dorsal striatum have been implicated in goal-directed behaviour. While activation of these receptors with several methods has resulted in an insensitivity to outcome devaluation, particular explanations for how they disrupt behaviour have not been explored. We both confirm a role for A_2A receptors in goal-directed responding and evaluate additional behavioural aspects of goal-directed control to more fully understand the role of A_2A receptors in instrumental behaviour.

OBJECTIVES

To examine the effects of the adenosine A_2A agonist CGS-21680 in the DMS on response-outcome encoding, updating representations of outcome value and on the ability to inhibit behaviour when reward is not available.

METHODS

Male rats were trained to lever press for food reward. The A_2AR agonist CGS-21680 was infused into the dorsomedial striatum either before an outcome devaluation test, prior to training with two distinct response-outcome associations or prior to a test of discriminative stimulus control over instrumental performance.

RESULTS

Intra-DMS administration of CGS-21680 impaired sensitivity to outcome devaluation. CGS-21680 treatment did not impair acquisition of specific response-outcome associations, selective control of responding based on the presence of stimuli that signaled when reward was or was not available, discrimination between stimuli or lever choices nor did it influence the effect of devaluation on the amounts of food eaten in a consumption test.

CONCLUSIONS

CGS-21680 impairs the ability to modulate responding based on recent changes to outcome value, an effect that is not accounted for by impairments in behavioural inhibition, discrimination, encoding the specific outcome of a response or the effectiveness of specific satiety.

Lack of action monitoring as a prerequisite for habitual and chunked behavior: Behavioral and neural correlates

We previously reported the rapid development of habitual behavior in a discrete-trials instrumental task in which lever insertion and retraction act as reward-predictive cues delineating sequence execution. Here we asked whether lever cues or performance variables reflective of skill and automaticity might account for habitual behavior in male rats. Behavior in the discrete-trials habit-promoting task was compared with two task variants lacking the sequence-delineating cues of lever extension and retraction. We find that behavior is under goal-directed control in absence of sequence-delineating cues but not in their presence, and that skilled performance does not predict goal-directed vs. habitual behavior. Neural activity recordings revealed an engagement of dorsolateral striatum and a disengagement of dorsomedial striatum during the sequence execution of the habit-promoting task, specifically. Together, these results indicate that sequence delineation cues promote habit and differential engagement of striatal subregions during instrumental responding, a pattern that may reflect cue-elicited behavioral chunking.

Discriminating goal-directed and habitual cocaine seeking in rats using a novel outcome devaluation procedure

Habits are theorized to play a key role in compulsive cocaine seeking, yet there is limited methodology for assessing habitual responding for intravenous (IV) cocaine. We developed a novel outcome devaluation procedure to discriminate goal-directed from habitual responding in cocaine-seeking rats. This procedure elicits devaluation temporarily and requires no additional training, allowing repeated testing at different time points. After training male rats to self-administer IV cocaine, we devalued the drug outcome via experimenter-administered IV cocaine (a "satiety" procedure) prior to a 10-min extinction test. Many rats were sensitive to outcome devaluation, a hallmark of goal-directed responding. These animals reduced responding when given a dose of experimenter-administered cocaine that matched or exceeded satiety levels during self-administration. However, other rats were insensitive to experimenter-administered cocaine, suggesting their responding was habitual. Importantly, reinforcement schedules and neural manipulations that produce goal-directed responding (i.e., ratio schedules or dorsolateral striatum lesions) caused sensitivity to outcome devaluation, whereas reinforcement schedules and neural manipulations that produce habitual responding (i.e., interval schedules or dorsomedial striatum lesions) caused insensitivity. Satiety-based outcome devaluation is an innovative new tool to dissect the neural and behavioral mechanisms underlying IV cocaine-seeking behavior.

Striatum expresses region-specific plasticity consistent with distinct memory abilities

The striatum mediates two learning modalities: goal-directed behavior in dorsomedial (DMS) and habits in dorsolateral (DLS) striata. The synaptic bases of these learnings are still elusive. Indeed, while ample research has described DLS plasticity, little remains known about DMS plasticity and its involvement in procedural learning. Here, we find symmetric and asymmetric anti-Hebbian spike-timing-dependent plasticity (STDP) in DMS and DLS, respectively, with opposite plasticity dominance upon increasing corticostriatal activity. During motor-skill learning, plasticity is engaged in DMS and striatonigral DLS neurons only during early learning stages, whereas striatopallidal DLS neurons are mobilized only during late phases. With a mathematical modeling approach, we find that symmetric anti-Hebbian STDP favors memory flexibility, while asymmetric anti-Hebbian STDP favors memory maintenance, consistent with memory processes at play in procedural learning.

A Role for Serotonin in Modulating Opposing Drive and Brake Circuits of Impulsivity

Impulsivity generally refers to a deficit in inhibition, with a focus on understanding the neural circuits which constitute the "brake" on actions and gratification. It is likely that increased impulsivity can arise not only from reduced inhibition, but also from a heightened or exaggerated excitatory "drive." For example, an action which has more vigor, or is fueled by either increased incentive salience or a stronger action-outcome association, may be harder to inhibit. From this perspective, this review focuses on impulse control as a competition over behavioral output between an initially learned response-reward outcome association, and a subsequently acquired opposing inhibitory association. Our goal is to present a synthesis of research from humans and animal models that supports this dual-systems approach to understanding the behavioral and neural substrates that contribute to impulsivity, with a focus on the neuromodulatory role of serotonin. We review evidence for the role of serotonin signaling in mediating the balance of the "drive" and "brake" circuits. Additionally, we consider parallels of these competing instrumental systems in impulsivity within classical conditioning processes (e.g., extinction) in order to point us to potential behavioral and neural mechanisms that may modulate the competing instrumental associations. Finally, we consider how the balance of these competing associations might contribute to, or be extracted from, our experimental assessments of impulsivity. A careful understanding of the underlying behavioral and circuit level contributions to impulsivity is important for understanding the pathogenesis of increased impulsivity present in a number of psychiatric disorders. Pathological levels of impulsivity in such disorders are likely subserved by deficits in the balance of motivational and inhibitory processes.

Diet-induced deficits in goal-directed control are rescued by agonism of group II metabotropic glutamate receptors in the dorsomedial striatum

Many overweight or obese people struggle to sustain the behavioural changes necessary to achieve and maintain weight loss. In rodents, obesogenic diet can disrupt goal-directed control of responding for food reinforcers, which may indicate that diet can disrupt brain regions associated with behavioural control. We investigated a potential glutamatergic mechanism to return goal-directed control to rats who had been given an obesogenic diet prior to operant training. We found that an obesogenic diet reduced goal-directed control and that systemic injection of LY379268, a Group II metabotropic glutamate receptor (mGluR2/3) agonist, returned goal-directed responding in these rats. Further, we found that direct infusion of LY379268 into the dorsomedial striatum, a region associated with goal-directed control, also restored goal-directed responding in the obesogenic-diet group. This indicates that one mechanism through which obesogenic diet disrupts goal-directed control is glutamatergic, and infusion of a mGluR2/3 agonist into the DMS is sufficient to ameliorate deficits in goal-directed control.

Adolescent Dopamine Neurons Represent Reward Differently during Action and State Guided Learning

Neuronal underpinning of learning cause-and-effect associations in the adolescent brain remains poorly understood. Two fundamental forms of associative learning are Pavlovian (classical) conditioning, where a stimulus is followed by an outcome, and operant (instrumental) conditioning, where outcome is contingent on action execution. Both forms of learning, when associated with a rewarding outcome, rely on midbrain dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SN). We find that, in adolescent male rats, reward-guided associative learning is encoded differently by midbrain dopamine neurons in each conditioning paradigm. Whereas simultaneously recorded VTA and SN adult neurons have a similar phasic response to reward delivery during both forms of conditioning, adolescent neurons display a muted reward response during operant but a profoundly larger reward response during Pavlovian conditioning. These results suggest that adolescent neurons assign a different value to reward when it is not gated by action. The learning rate of adolescents and adults during both forms of conditioning was similar, supporting the notion that differences in reward response in each paradigm may be because of differences in motivation and independent of state versus action value learning. Static characteristics of dopamine neurons, such as dopamine cell number and size, were similar in the VTA and SN of both ages, but there were age-related differences in stimulated dopamine release and correlated spike activity, suggesting that differences in reward responsiveness by adolescent dopamine neurons are not because of differences in intrinsic properties of these neurons but engagement of different dopaminergic networks.SIGNIFICANCE STATEMENT Reckless behavior and impulsive decision-making by adolescents suggest that motivated behavioral states are encoded differently by the adolescent brain. Motivated behavior, which is dependent on the function of the dopamine system, follows learning of cause-and-effect associations in the environment. We find that dopamine neurons in adolescents encode reward differently depending on the cause-and-effect relationship of the means to receive that reward. Compared with adults, reward contingent on action led to a muted response, whereas reward that followed a cue but was not gated by action produced an augmented phasic response. These data demonstrate an age-related difference in dopamine neuron response to reward that is not uniform and is guided by processes that differentiate between state and action values.

Dorsomedial Striatal Activity Tracks Completion of Behavioral Sequences in Rats

For proper execution of goal-directed behaviors, individuals require both a general representation of the goal and an ability to monitor their own progress toward that goal. Here, we examine how dorsomedial striatum (DMS), a region pivotal for forming associations among stimuli, actions, and outcomes, encodes the execution of goal-directed action sequences that require self-monitoring of behavior. We trained rats to complete a sequence of at least five consecutive lever presses (without visiting the reward port) to obtain a reward and recorded the activity of individual cells in DMS while rats performed the task. We found that the pattern of DMS activity gradually changed during the execution of the sequence, permitting accurate decoding of sequence progress from neural activity at a population level. Moreover, this sequence-related activity was blunted on trials where rats did not complete a sufficient number of presses. Overall, these data suggest a link between DMS activity and the execution of behavioral sequences that require monitoring of ongoing behavior.

Dopamine sensitization by methamphetamine treatment prior to instrumental training delays the transition into habit in female rats

Early in instrumental learning, behavior is goal-directed and sensitive to changes in the value of the instrumental outcome. With sufficient repetition, responding becomes insensitive to changes in outcome value, or habitual. We have previously found that females transition into habit over a distinct range of training from 120 to 160 reinforced responses. This low level of instrumental training is markedly less than what has been shown to support habitual responding in male rats. To begin to investigate the early development of habit in females, we conducted a series of experiments in which we pretreated female rats with methamphetamine (METH) with the aim of sensitizing central dopamine, a major modulator of striatal function, prior to instrumental nose-poke training at the beginning and at the endpoint of the transition range in females. Following training, we tested for sensitivity to reinforcer devaluation (RD), which was conducted by repeatedly pairing reinforcers previously earned during training with lithium chloride (LiCl)-induced illness. As a counterpoint, a series of similar experiments was conducted separately in male rats. Additionally, in order to ascertain the validity of using nose-poke as an instrumental response, we compared sensitivity to devaluation between the Pavlovian approach towards the food magazine and the nose-poke response. In females, Vehicle groups responded in a habitual manner at both training levels (120 and 160 reinforced responses), whereas METH groups remained sensitive to devaluation. This suggests that increasing central dopamine delays habit formation in female rats. In male rats, Vehicle groups demonstrated goal-directed responding following training with 120 and 320 reinforced responses, and marginally goal-directed responding,with 160. METH-pretreated males were sensitive to devaluation at the 120 and 160 training levels, however, following more extended training to 320 reinforced responses, METH-pretreated males responded in a habitual manner, indicating that increasing central dopamine can advance habit formation in male rats. Overall, these results suggest that METH pretreatment maintains goal-directed responding in female rats when they are typically transitioning to habitual control of instrumental behavior and can advance habit formation in male rats given sufficient instrumental training. In addition, we found differential RD sensitivity of the nose-poke response used during instrumental training compared to Pavlovian approach towards the food magazine, confirming that there is a distinction between these two behaviors and that nose-poking is a valid instrumental response.

Pharmacologically induced N-methyl-D-aspartate receptor hypofunction impairs goal-directed food seeking in rats

AIM

Acute N-methyl-D-aspartate (NMDA) receptor antagonism is an important pharmacological animal model of schizophrenia. In previous studies, schizophrenia patients show impaired goal-directed behavior in an outcome-specific devaluation procedure. In this study, we investigated whether the rat model of the NMDA receptor blockade also showed altered goal-directed behavior in a satiety-induced outcome devaluation paradigm.

METHODS

In experiments 1 and 2, we aimed to establish the satiety-induced outcome devaluation test using sucrose and lipid rewards in operant conditioning and free consumption paradigms. In experiment 3, we tested the effect of MK-801 (0.1 mg/kg, i.p.) on outcome-specific devaluation.

RESULTS

Experiments 1 and 2 demonstrated that 1-h ad libitum food consumption is sufficient to induce outcome-specific devaluation in both lever-press and free consumption tests in rats. Experiment 3 showed that the administration of MK-801 impaired satiety-induced devaluation in the lever-press test but not in the subsequent free consumption test.

CONCLUSIONS

Our results suggest that acute pharmacological NMDA receptor antagonism in rats is a useful animal model for impaired goal-directed behavior in schizophrenia.

Inactivation of posterior but not anterior dorsomedial caudate-putamen impedes learning with self-administered nicotine stimulus in male rats

The rodent caudate-putamen is a large heterogeneous neural structure with distinct anatomical connections that differ in their control of learning processes. Previous research suggests that the anterior and posterior dorsomedial caudate-putamen (a- and p-dmCPu) differentially regulate associative learning with a non-contingent nicotine stimulus. The current study used bilateral NMDA-induced excitotoxic lesions to the a-dmCPu and p-dmCPu to determine the functional involvement of a-dmCPu and p-dmCPu in appetitive learning with contingent nicotine stimulus. Rats with a-dmCPu, p-dmCPu, or sham lesions were trained to lever-press for intravenous nicotine (0.03 mg/kg/inf) followed by access to sucrose 30 s later. After 1, 3, 9, and 20 nicotine-sucrose training sessions, appetitive learning in the form of a goal-tracking response was assessed using a non-contingent nicotine-alone test. All rats acquired nicotine self-administration and learned to retrieve sucrose from a receptacle at equal rates. However, rats with lesions to p-dmCPu demonstrated blunted learning of the nicotine-sucrose association. Our primary findings show that rats with lesions to p-dmCPu had a blunted goal-tracking response to a non-contingent nicotine administration after 20 consecutive days of nicotine-sucrose pairing. Our findings extend previous reports to a contingent model of nicotine self-administration and show that p-dmCPu is involved in associative learning with nicotine stimulus using a paradigm where rats voluntarily self-administer nicotine infusions that are paired with access to sucrose-a paradigm that closely resembles learning processes observed in humans.

Controlling one's world: Identification of sub-regions of primate PFC underlying goal-directed behavior

Impaired detection of causal relationships between actions and their outcomes can lead to maladaptive behavior. However, causal roles of specific prefrontal cortex (PFC) sub-regions and the caudate nucleus in mediating such relationships in primates are unclear. We inactivated and overactivated five PFC sub-regions, reversibly and pharmacologically: areas 24 (perigenual anterior cingulate cortex), 32 (medial PFC), 11 (anterior orbitofrontal cortex, OFC), 14 (rostral ventromedial PFC/medial OFC), and 14-25 (caudal ventromedial PFC) and the anteromedial caudate to examine their role in expressing learned action-outcome contingencies using a contingency degradation paradigm in marmoset monkeys. Area 24 or caudate inactivation impaired the response to contingency change, while area 11 inactivation enhanced it, and inactivation of areas 14, 32, or 14-25 had no effect. Overactivation of areas 11 and 24 impaired this response. These findings demonstrate the distinct roles of PFC sub-regions in goal-directed behavior and illuminate the candidate neurobehavioral substrates of psychiatric disorders, including obsessive-compulsive disorder.

•

Monkey pgACC-24 is necessary for detecting causal control of actions over outcomes

•

Its projection target in the caudate nucleus is also implicated

•

Three other subregions of the ventromedial prefrontal cortex are not necessary

•

Anterior OFC-11 may mediate Pavlovian influences on goal-directed behavior

Aberrant prefrontal cortical-striatal functional connectivity in children with primary complex motor stereotypies

Motor stereotypies are rhythmic, repetitive, prolonged, predictable, and purposeless movements that stop with distraction. Although once believed to occur only in children with neurodevelopmental disorders such as autism, the presence and persistence of complex motor stereotypies (CMS) in otherwise typically developing children (primary CMS) has been well-established. Little, however, is known about the underlying pathophysiology of these unwanted actions. The aim of the present study was to use resting-state functional magnetic resonance imaging to evaluate functional connectivity within frontal-striatal circuits that are essential for goal-directed and habitual activity in children with primary complex motor stereotypies. Functional connectivity between prefrontal cortical and striatal regions, considered essential for developing goal-directed behaviors, was reduced in children with primary CMS compared to their typically developing peers. In contrast, functional connectivity between motor/premotor and striatal regions, critical for developing and regulating habitual behaviors, did not differ between groups. This documented alteration of prefrontal to striatal connectivity could provide the underlying mechanism for the presence and persistence of complex motor stereotypies in otherwise developmentally normal children.

Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment

Significant evidence supports the view that dopamine shapes learning by encoding reward prediction errors. However, it is unknown whether striatal targets receive tailored dopamine dynamics based on regional functional specialization. Here, we report wave-like spatiotemporal activity patterns in dopamine axons and release across the dorsal striatum. These waves switch between activational motifs and organize dopamine transients into localized clusters within functionally related striatal subregions. Notably, wave trajectories were tailored to task demands, propagating from dorsomedial to dorsolateral striatum when rewards are contingent on animal behavior and in the opponent direction when rewards are independent of behavioral responses. We propose a computational architecture in which striatal dopamine waves are sculpted by inference about agency and provide a mechanism to direct credit assignment to specialized striatal subregions. Supporting model predictions, dorsomedial dopamine activity during reward-pursuit signaled the extent of instrumental control and interacted with reward waves to predict future behavioral adjustments.

Whole-Brain Mapping of Direct Inputs to Dopamine D1 and D2 Receptor-Expressing Medium Spiny Neurons in the Posterior Dorsomedial Striatum

The posterior dorsomedial striatum (pDMS) is mainly composed of medium spiny neurons (MSNs) expressing either dopamine D1 receptors (D1Rs) or D2Rs. Activation of these two MSN types produces opposing effects on addictive behaviors. However, it remains unclear whether pDMS D1-MSNs or D2-MSNs receive afferent inputs from different brain regions or whether the extrastriatal afferents express distinct dopamine receptors. To assess whether these afferents also contained D1Rs or D2Rs, we generated double transgenic mice, in which D1R-expressing and D2R-expressing neurons were fluorescently labeled. We used rabies virus-mediated retrograde tracing in these mice to perform whole-brain mapping of direct inputs to D1-MSNs or D2-MSNs in the pDMS. We found that D1-MSNs preferentially received inputs from the secondary motor, secondary visual, and cingulate cortices, whereas D2-MSNs received inputs from the primary motor and primary sensory cortices, and the thalamus. We also discovered that the bed nucleus of the stria terminalis (BNST) and the central nucleus of the amygdala (CeA) contained abundant D2R-expressing, but few D1R-expressing, neurons in a triple transgenic mouse model. Remarkably, although limited D1R or D2R expression was observed in extrastriatal neurons that projected to D1-MSNs or D2-MSNs, we found that cortical structures preferentially contained D1R-expressing neurons that projected to D1-MSNs or D2-MSNs, while the thalamus, substantia nigra pars compacta (SNc), and BNST had more D2R-expressing cells that projected to D2-MSNs. Taken together, these findings provide a foundation for future understanding of the pDMS circuit and its role in action selection and reward-based behaviors.

Prior Cocaine Use Alters the Normal Evolution of Information Coding in Striatal Ensembles during Value-Guided Decision-Making

Substance use disorders (SUDs) are characterized by maladaptive behavior. The ability to properly adjust behavior according to changes in environmental contingencies necessitates the interlacing of existing memories with updated information. This can be achieved by assigning learning in different contexts to compartmentalized "states." Though not often framed this way, the maladaptive behavior observed in individuals with SUDs may result from a failure to properly encode states because of drug-induced neural alterations. Previous studies found that the dorsomedial striatum (DMS) is important for behavioral flexibility and state encoding, suggesting the DMS may be an important substrate for these effects. Here, we recorded DMS neural activity in cocaine-experienced male rats during a decision-making task where blocks of trials represented distinct states to probe whether the encoding of state and state-related information is affected by prior drug exposure. We found that DMS medium spiny neurons (MSNs) and fast-spiking interneurons (FSIs) encoded such information and that prior cocaine experience disrupted the evolution of representations both within trials and across recording sessions. Specifically, DMS MSNs and FSIs from cocaine-experienced rats demonstrated higher classification accuracy of trial-specific rules, defined by response direction and value, compared with those drawn from sucrose-experienced rats, and these overly strengthened trial-type representations were related to slower switching behavior and reaction times. These data show that prior cocaine experience paradoxically increases the encoding of state-specific information and rules in the DMS and suggest a model in which abnormally specific and persistent representation of rules throughout trials in DMS slows value-based decision-making in well trained subjects.SIGNIFICANCE STATEMENT Substance use disorders (SUDs) may result from a failure to properly encode rules guiding situationally appropriate behavior. The dorsomedial striatum (DMS) is thought to be important for such behavioral flexibility and encoding that defines the situation or "state." This suggests that the DMS may be an important substrate for the maladaptive behavior observed in SUDs. In the current study, we show that prior cocaine experience results in over-encoding of state-specific information and rules in the DMS, which may impair normal adaptive decision-making in the task, akin to what is observed in SUDs.

Dose-dependent effects of alcohol injections on omission-contingency learning have an inverted-U pattern

Previous examinations of the long-term effects of alcohol exposure on omission-contingency learning have produced mixed results across different age or sex groups, with evidence for faster learning or no effect. However, none of these experiments made comparisons using the same exposure-dose across the age/sex groups. Here, we exposed rats to 6 weeks of alcohol injections (3 days/week, 1.75 or 3.5 g/kg/24-h, i.p. broken up into 2 injections/day) in adolescent/early adult males or females (PND27-66) or adult males (PND62-101). We then tested the rats in autoshaping and omission-contingency tasks. In contrast to our hypotheses, the low 1.75-g/kg/24-h dose led to slower omission learning and the higher 3.5-g/kg/24-h dose had no effect. There were no age- or sex-differences in omission learning. Additionally, during autoshaping training, rats exposed in adolescence/early adulthood had a faster shift to sign-tracking in their sign-tracking/goal-tracking ratios than rats exposed in adulthood, with no consistent effect of alcohol exposure or sex-differences. Our results suggest complex effects of alcohol on the neural substrates of omission-contingency learning at different doses, which will require future investigation.

Neural substrates of habit

Active reward pursuit is supported by the balance between the cognitive and habitual control of behavior. The cognitive, goal-directed strategy relies on the prospective evaluation of anticipated consequences, which allows behavior to readily adapt when circumstances change. Repetition of successful actions promotes less cognitively taxing habits, in which behavior is automatically executed without prospective consideration. Disruption in either of these behavioral regulatory systems contributes to the symptoms that underlie many psychiatric disorders. Here, I review recently identified neural substrates, at multiple neural levels, that contribute to habits and outline gaps in knowledge that must be addressed to fully understand the neural mechanisms of behavioral control.

Dorsal and ventral striatal dopamine D1 and D2 receptors differentially modulate distinct phases of serial visual reversal learning

Impaired cognitive flexibility in visual reversal-learning tasks has been observed in a wide range of neurological and neuropsychiatric disorders. Although both human and animal studies have implicated striatal D2-like and D1-like receptors (D2R; D1R) in this form of flexibility, less is known about the contribution they make within distinct sub-regions of the striatum and the different phases of visual reversal learning. The present study investigated the involvement of D2R and D1R during the early (perseverative) phase of reversal learning as well as in the intermediate and late stages (new learning) after microinfusions of D2R and D1R antagonists into the nucleus accumbens core and shell (NAcC; NAcS), the anterior and posterior dorsomedial striatum (DMS) and the dorsolateral striatum (DLS) on a touchscreen visual serial reversal-learning task. Reversal learning was improved after dopamine receptor blockade in the nucleus accumbens; the D1R antagonist, SCH23390, in the NAcS and the D2R antagonist, raclopride, in the NAcC selectively reduced early, perseverative errors. In contrast, reversal learning was impaired by D2R antagonism, but not D1R antagonism, in the dorsal striatum: raclopride increased errors in the intermediate phase after DMS infusions, and increased errors across phases after DLS infusions. These findings indicate that D1R and D2R modulate different stages of reversal learning through effects localised to different sub-regions of the striatum. Thus, deficits in behavioral flexibility observed in disorders linked to dopamine perturbations may be attributable to specific D1R and D2R dysfunction in distinct striatal sub-regions.

Characterizing impulsivity and resting-state functional connectivity in normal-weight binge eaters

OBJECTIVE

Binge eating is characterized by episodes of uncontrolled eating, within discrete periods of time. Although it is usually described in obese individuals or as a symptom of Binge Eating Disorder (BED), this behavior can also occur in the normal-weight (NW) population. An interesting premise suggests that impulsivity might contribute to the onset of binge eating and the progression toward weight gain. Drawing upon this evidence, here we explored impulsivity in NW individuals reporting binge-eating episodes through a functional connectivity approach. We hypothesized that, even in the absence of an eating disorder, NW binge eaters would be characterized by connectivity pattern changes in corticostriatal regions implicated in impulsivity, similarly to the results described in BED individuals.

METHODS

A resting-state functional magnetic resonance imaging study tested 39 NW men and women, with and without binge eating (binge eaters, BE and non-BE). Brain functional connectivity was explored by means of graph theoretic centrality measures and traditional seed-based analysis; trait impulsivity was assessed with self-report questionnaires.

RESULTS

The BE group was characterized by a higher degree of trait impulsivity. Brain functional connectivity measures revealed lower degree centrality within the right middle frontal gyrus, left insula/putamen and left temporoparietal regions and a lower functional connectivity between the right middle frontal gyrus and right insula in the BE group.

DISCUSSION

The results support previous evidence on BED of altered functional connectivity and higher impulsivity at the roots of overeating behavior, but further extend this concept excluding any potential confounding effect exerted by the weight status.

Cortical endogenous opioids and their role in facilitating repetitive behaviors in deer mice

Deer mice provide a non-pharmacologically induced model for the study of repetitive behaviors. In captivity, these animals develop frequent jumping and rearing that resemble clinical symptoms of obsessive-compulsive behavior (OCB), autism spectrum disorder (ASD), complex motor stereotypies (CMS), and Tourette's syndrome (TS). In this study, we pursue the mechanism of repetitive behaviors by performing stereological analyses and liquid chromatography/ mass spectrometry (LC-MS/MS) measurements of glutamate (Glut), GABA, 3,4-dihydroxyphenylacetic acid (DOPAC), dopamine (DA), leu-enkephalin (leu-enk), and dynorphin-A (dyn-A) in frontal cortex (FC), prefrontal cortex (PFC), and basal ganglia. The only significant stereological alteration was a negative correlation between repetitive behaviors and the cell count in the ventromedial striatum (VMS). Neurochemical analyses demonstrated a significant negative correlation between repetitive behaviors and endogenous opioids (leu-enk and dyn-A) in the FC - the site of origin of habitual behaviors and cortical projections to striatal MSNs participating in direct and indirect pathways. The precise neurochemical process by which endogenous opioids influence synaptic neurotransmission is unknown. One postulated cortical mechanism, supported by our findings, is an opioid effect on cortical interneuron GABA release and a consequent effect on glutamatergic cortical pyramidal cells. Anatomical changes in the VMS could have a role in repetitive behaviors, recognizing that this region influences goal-directed and habitual behaviors.

Fractionating the all-or-nothing definition of goal-directed and habitual decision-making

Goal-directed and habitual decision-making are fundamental processes that support the ongoing adaptive behavior. There is a growing interest in examining their disruption in psychiatric disease, often with a focus on a disease shifting control from one process to the other, usually a shift from goal-directed to habitual control. However, several different experimental procedures can be used to probe whether decision-making is under goal-directed or habitual control, including outcome devaluation and contingency degradation. These different experimental procedures may recruit diverse behavioral and neural processes. Thus, there are potentially many opportunities for these disease phenotypes to manifest as alterations to both goal-directed and habitual controls. In this review, we highlight the examples of behavioral and neural circuit divergence and similarity, and suggest that interpretation based on behavioral processes recruited during testing may leave more room for goal-directed and habitual decision-making to coexist. Furthermore, this may improve our understanding of precisely what the involved neural mechanisms underlying aspects of goal-directed and habitual behavior are, as well as how disease affects behavior and these circuits.

Chemogenetic Silencing of Prelimbic Cortex to Anterior Dorsomedial Striatum Projection Attenuates Operant Responding

Operant (instrumental) conditioning is a laboratory analog for voluntary behavior and involves learning to make a response for a reinforcing outcome. The prelimbic cortex (PL), a region of the rodent medial prefrontal cortex, and the dorsomedial striatum (DMS), have been separately established as important in the acquisition of minimally-trained operant behavior. Despite dense anatomical connections between the two regions, experimenters have only recently linked projections from the PL to the posterior DMS (pDMS) in the acquisition of an operant response. Yet, it is still unknown if these projections mediate behavioral expression, and if more anterior regions of the DMS (aDMS), which receive dense projections from the PL, are also involved. Therefore, we utilized designer receptors exclusively activated by designer drugs (DREADDs) to test whether or not projections from the PL to the aDMS influence the expression of operant behavior. Rats underwent bilateral PL-targeted infusions of either a DREADD virus (AAV8-hSyn-hM4D(Gi)-mCherry) or a control virus (AAV8-hSyn-GFP). In addition, guide cannulae were implanted bilaterally in the aDMS. Rats were tested with both clozapine-N-oxide (CNO) (DREADD ligand) and vehicle infusions into the aDMS. Animals that had received the DREADD virus, but not the control virus, showed attenuated responding when they received CNO microinfusions into the aDMS, compared to vehicle infusions. Patch clamp electrophysiology verified the inhibitory effect of CNO on DREADDs-expressing PL neurons in acute brain slices. GFP-expressing control PL neurons were unaffected by CNO. The results add to the recent literature suggesting that connections between the PL and aDMS are important for the expression of minimally-trained operant responding.

Distinct recruitment of dorsomedial and dorsolateral striatum erodes with extended training

Hypotheses of striatal orchestration of behavior ascribe distinct functions to striatal subregions, with the dorsolateral striatum (DLS) especially implicated in habitual and skilled performance. Thus neural activity patterns recorded from the DLS, but not the dorsomedial striatum (DMS), should be correlated with habitual and automatized performance. Here, we recorded DMS and DLS neural activity in rats during training in a task promoting habitual lever pressing. Despite improving performance across sessions, clear changes in corresponding neural activity patterns were not evident in DMS or DLS during early training. Although DMS and DLS activity patterns were distinct during early training, their activity was similar following extended training. Finally, performance after extended training was not associated with DMS disengagement, as would be predicted from prior work. These results suggest that behavioral sequences may continue to engage both striatal regions long after initial acquisition, when skilled performance is consolidated.

Acute alcohol and cognition: Remembering what it causes us to forget

Addiction has been conceptualized as a specific form of memory that appropriates typically adaptive neural mechanisms of learning to produce the progressive spiral of drug-seeking and drug-taking behavior, perpetuating the path to addiction through aberrant processes of drug-related learning and memory. From that perspective, to understand the development of alcohol use disorders, it is critical to identify how a single exposure to alcohol enters into or alters the processes of learning and memory, so that involvement of and changes in neuroplasticity processes responsible for learning and memory can be identified early. This review characterizes the effects produced by acute alcohol intoxication as a function of brain region and memory neurocircuitry. In general, exposure to ethanol doses that produce intoxicating effects causes consistent impairments in learning and memory processes mediated by specific brain circuitry, whereas lower doses either have no effect or produce a facilitation of memory under certain task conditions. Therefore, acute ethanol does not produce a global impairment of learning and memory, and can actually facilitate particular types of memory, perhaps particular types of memory that facilitate the development of excessive alcohol use. In addition, the effects on cognition are dependent on brain region, task demands, dose received, pharmacokinetics, and tolerance. Additionally, we explore the underlying alterations in neurophysiology produced by acute alcohol exposure that help to explain these changes in cognition and highlight future directions for research. Through understanding the impact that acute alcohol intoxication has on cognition, the preliminary changes potentially causing a problematic addiction memory can better be identified.

The role of serotonin 1B in the representation of outcomes

Disrupted serotonin neurotransmission has been implicated in the etiology of psychopathic traits. Empirical research has found that people with high levels of psychopathic traits have a deficit in reinforcement learning that is thought to be linked with amygdala dysfunction. Altered serotonin neurotransmission provides a plausible explanation for amygdala dysfunction in psychopathic traits and recent research suggests that this may be associated with serotonin 1B (5-HT_1B) receptor function. This research used an animal model to test the hypothesis that 5-HT_1B receptors are involved in the encoding of the specific features of reinforcing outcomes. An outcome devaluation task was used to test the effect of the systemic administration of a selective 5-HT_1B receptor agonist administered before encoding of “action-outcome” associations. Results showed that while administration of a 5-HT_1B receptor agonist allowed rats to acquire instrumental responding for food, when the content of that learning was further probed using an outcome devaluation task, performance differed from controls. 5-HT_1B agonism impaired learning about the specific sensory qualities of food rewards associated with distinct instrumental responses, required to direct choice performance when the value of one outcome changed. These findings suggest a role for 5-HT_1B receptor function in the encoding of the specific features of reinforcing outcomes.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

Control of habitual instrumental actions by anticipation of postingestive sensations

In animal behavioral experiments, extended training causes instrumental actions that deliver ingestible substances to lose sensitivity to outcome devaluation and contingency degradation, and to gain direct sensitivity to the current motivational state. These features of habitual control have been attributed to a process that relies on stimulus-response (S-R) associations linking the context to instrumental actions and Pavlovian associations linking the same context to orally-sensed properties of substances attained there. This Pavlovian process was conceived based on the results of irrelevant incentive experiments, but it is not supported by the results of all such experiments. An alternative process is therefore proposed here. In this process, recall of the instrumental action is evoked by an S-R association, but the rate at which this action is then performed is controlled by anticipation of postingestive sensations that have frequently followed it. This anticipation relies on recall of an association linking the action directly to the postingestive sensations. This association is learned during the formation of a chunked action series that begins with the instrumental action and ends with a consummatory response. It enables a prediction of subjective value that is directly influenced by the current motivational state, but is not influenced by devaluation or non-contingent delivery of the substance that has produced the postingestive sensations.

Habits Are Negatively Regulated by Histone Deacetylase 3 in the Dorsal Striatum

BACKGROUND

Optimal behavior and decision making result from a balance of control between two strategies, one cognitive/goal-directed and one habitual. These systems are known to rely on the anatomically distinct dorsomedial and dorsolateral striatum, respectively. However, the transcriptional regulatory mechanisms required to learn and transition between these strategies are unknown. Here we examined the role of one chromatin-based transcriptional regulator, histone modification via histone deacetylases (HDACs), in this process.

METHODS

We combined procedures that diagnose behavioral strategy in rats with pharmacological and viral-mediated HDAC manipulations, chromatin immunoprecipitation, and messenger RNA quantification.

RESULTS

The results indicate that dorsal striatal HDAC3 activity constrains habit formation. Systemic HDAC inhibition following instrumental (lever press → reward) conditioning increased histone acetylation throughout the dorsal striatum and accelerated habitual control of behavior. HDAC3 was removed from the promoters of key learning-related genes in the dorsal striatum as habits formed with overtraining and with posttraining HDAC inhibition. Decreasing HDAC3 function, either by selective pharmacological inhibition or by expression of dominant-negative mutated HDAC3, in either the dorsolateral striatum or the dorsomedial striatum accelerated habit formation, while HDAC3 overexpression in either region prevented habit.

CONCLUSIONS

These results challenge the strict dissociation between dorsomedial striatum and dorsolateral striatum function in goal-directed versus habitual behavioral control and identify dorsostriatal HDAC3 as a critical molecular directive of the transition to habit. Because this transition is disrupted in many neurodegenerative and psychiatric diseases, these data suggest a potential molecular mechanism for the negative behavioral symptoms of these conditions and a target for therapeutic intervention.

Insular and Ventrolateral Orbitofrontal Cortices Differentially Contribute to Goal-Directed Behavior in Rodents

The medial prefrontal cortex (mPFC) has long been considered a critical site in action control. However, recent evidence indicates that the contribution of cortical areas to goal-directed behavior likely extends beyond mPFC. Here, we examine the function of both insular (IC) and ventrolateral orbitofrontal (vlOFC) cortices in action-dependent learning. We used chemogenetics to study the consequences of IC or vlOFC inhibition on acquisition and performance of instrumental actions using the outcome devaluation task. Rats first learned to associate actions with desirable outcomes. Then, one of these outcomes was devalued and we assessed the rats' choice between the 2 actions. Typically, rats will bias their selection towards the action that delivers the still valued outcome. We show that chemogenetic-induced inhibition of IC during choice abolishes goal-directed control whereas inhibition during instrumental acquisition is without effect. IC is therefore necessary for action selection based on current outcome value. By contrast, vlOFC inhibition during acquisition or the choice test impaired goal-directed behavior but only following a shift in the instrumental contingencies. Our results provide clear evidence that vlOFC plays a critical role in action-dependent learning, which challenges the popular idea that this region of OFC is exclusively involved in stimulus-dependent behaviors.

The dorsomedial striatum mediates Pavlovian appetitive conditioning and food consumption

The dorsomedial striatum (DMS) is an important sensorimotor region mediating the acquisition of goal-directed instrumental reward learning and behavioral flexibility. However, whether the DMS also regulates Pavlovian cue-food learning is less clear. The current study used excitotoxic lesions to determine whether the DMS is critical in Pavlovian appetitive learning and behavior, using discriminative conditioning and reversal paradigms. The results showed that DMS lesions transiently retarded cue-food learning and subsequent reversal of this learning. Rats with DMS lesions selectively attenuated responding to a food cue but not a control cue, early in training, suggesting the DMS is involved when initial associations are formed. Similarly, initial reversal learning was attenuated in rats with DMS lesions, which suggests impaired flexibility to adjust behavior when the cue meaning is reversed. We also examined the effect of DMS lesions on food intake during tests with access to a highly palatable food along with standard chow diet. Rats with DMS lesions showed an altered pattern of intake, with an initial reduction in high-fat diet followed by an increase in chow consumption. These results demonstrate that the DMS has a role in mediating cue-food learning and its subsequent reversal, as well as changes in food intake when a choice is provided. Together, these results demonstrate the DMS is involved in reward associative learning and reward consumption, when behavioral flexibility is needed to adjust responding or consumption to match the current value. (PsycINFO Database Record

Methamphetamine promotes habitual action and alters the density of striatal glutamate receptor and vesicular proteins in dorsal striatum

Goal-directed actions are controlled by the value of the consequences they produce and so increase when what they produce is valuable and decrease when it is not. With continued invariant practice, however, goal-directed actions can become habits, controlled not by their consequences but by antecedent, reward-related states and stimuli. Here, we show that pre-exposure to methamphetamine (METH) caused abnormally rapid development of habitual control. Furthermore, these drug-induced habits differed strikingly from conventional habits; we found that they were insensitive both to changes in reward value and to the effects of negative feedback. In addition to these behavioral changes, METH exposure produced bidirectional changes to synaptic proteins in the dorsal striatum. In the dorsomedial striatum, a structure critical for goal-directed action, METH exposure was associated with a reduction in glutamate receptor and glutamate vesicular proteins, whereas in the dorsolateral striatum, a region that has previously been implicated in habit learning, there was an increase in these proteins. Together, these results indicate that METH exposure promotes habitual control of action that appears to be the result of bidirectional changes in glutamatergic transmission in the circuits underlying goal-directed and habit-based learning.

Volitional Modulation of Primary Visual Cortex Activity Requires the Basal Ganglia

Animals acquire behaviors through instrumental conditioning. Brain-machine interfaces have used instrumental conditioning to reinforce patterns of neural activity directly, especially in frontal and motor cortices, which are a rich source of signals for voluntary action. However, evidence suggests that activity in primary sensory cortices may also reflect internally driven processes, instead of purely encoding antecedent stimuli. Here, we show that rats and mice can learn to produce arbitrary patterns of neural activity in their primary visual cortex to control an auditory cursor and obtain reward. Furthermore, learning was prevented when neurons in the dorsomedial striatum (DMS), which receives input from visual cortex, were optogenetically inhibited, but not during inhibition of nearby neurons in the dorsolateral striatum. After learning, DMS inhibition did not affect production of the rewarded patterns. These data demonstrate that cortico-basal ganglia circuits play a general role in learning to produce cortical activity that leads to desirable outcomes.

Dopamine, psychosis and schizophrenia: the widening gap between basic and clinical neuroscience

The stagnation in drug development for schizophrenia highlights the need for better translation between basic and clinical research. Understanding the neurobiology of schizophrenia presents substantial challenges but a key feature continues to be the involvement of subcortical dopaminergic dysfunction in those with psychotic symptoms. Our contemporary knowledge regarding dopamine dysfunction has clarified where and when dopaminergic alterations may present in schizophrenia. For example, clinical studies have shown patients with schizophrenia show increased presynaptic dopamine function in the associative striatum, rather than the limbic striatum as previously presumed. Furthermore, subjects deemed at high risk of developing schizophrenia show similar presynaptic dopamine abnormalities in the associative striatum. Thus, our view of subcortical dopamine function in schizophrenia continues to evolve as we accommodate this newly acquired information. However, basic research in animal models has been slow to incorporate these clinical findings. For example, psychostimulant-induced locomotion, the commonly utilised phenotype for positive symptoms in rodents, is heavily associated with dopaminergic activation in the limbic striatum. This anatomical misalignment has brought into question how we assess positive symptoms in animal models and represents an opportunity for improved translation between basic and clinical research. The current review focuses on the role of subcortical dopamine dysfunction in psychosis and schizophrenia. We present and discuss alternative phenotypes that may provide a more translational approach to assess the neurobiology of positive symptoms in schizophrenia. Incorporation of recent clinical findings is essential if we are to develop meaningful translational animal models.

Regulation of habit formation in the dorsal striatum

Habits are an essential and pervasive component of our daily lives that allow us to efficiently perform routine tasks. But their disruption contributes to the symptoms that underlie many psychiatric diseases. Emerging data are revealing the cellular and molecular mechanisms of habit formation in the dorsal striatum. New data suggest that in both the dorsolateral and dorsomedial striatum histone deacetylase (HDAC) activity acts as a critical negative regulator of the transcriptional processes underlying habit formation. In this review, we discuss this recent work and draw conclusions relevant to the treatment of diseases marked by maladaptive habits.