Accelerated habit formation following amphetamine exposure is reversed by D1, but enhanced by D2, receptor antagonists

Neither Amphetamine nor Sub-Anesthetic Ketamine Treatment during Adolescence Impairs Devaluation in Rats Tested during Adulthood

BACKGROUND

Much of the existing animal literature on the devaluation task suggests that prior repeated exposure to drugs of abuse during adulthood can impair goal-directed action, but the literature on human drug users is mixed. Also, the initiation of drug use often occurs during adolescence, but examinations of the effects of drug exposure during adolescence on behavior in the devaluation task are lacking.

METHODS

We examined whether repeated exposure during adolescence to amphetamine (3 mg/kg injections every-other day from post-natal day 27-45) or ketamine (twice daily 30 mg/kg injections from post-natal day 35-44) would impair behavior in a devaluation test when tested drug-free in adulthood. Rats were trained to press a left lever with a steady cue-light above it for one reinforcer and a right lever with a flashing cue-light above it for a different reinforcer. We tested whether any impairments in goal-directed action could be overcome by compensation between strategies by giving rats information based on lever-location and cue-lights during the test that was either congruent (allowing compensation) or incongruent (preventing compensation between strategies) with the configurations during training.

RESULTS

Our results provided no evidence for impairment of goal-directed action during adulthood after adolescent amphetamine or ketamine exposure.

CONCLUSIONS

We discuss possible reasons for this discrepancy with the prior literature, including (1) the age of exposure and (2) the pattern in the previous literature that most previous demonstrations of drug exposure impairing devaluation in laboratory animals may be attributed to either drug-associated cues present in the testing environment and/or accelerated habit learning in tasks that predispose laboratory animals towards habit formation with extended training (with training procedures that should resist the formation of habits in the current experiment). However, additional research is needed to examine the effects of these factors, as well a potential role for the particular doses and washout periods to determine the cause of our finding of no devaluation impairment after drug exposure.

Acute stress facilitates habitual behavior in female rats

Instrumental behavior can reflect the influence of goal-directed and habitual systems. Contemporary research suggests that stress may facilitate control by the habitual system under conditions where the behavior would otherwise reflect control by the goal-directed system. However, it is unclear how stress modulates the influence of these systems on instrumental responding to achieve this effect, particularly in females. Here, we examine whether a mild psychogenic stressor experienced before acquisition training (Experiment 1), or prior to the test of expression (Experiment 2) would influence goal-directed and habitual control of instrumental responding in female rats. In both experiments, rats acquired an instrumental nose-poke response for a sucrose reward. This was followed by a reinforcer devaluation phase in which half the rats in Stressed and Non-Stressed conditions received pairings of the sucrose pellet with illness induced by lithium chloride until they rejected the pellet when offered. The remaining rats received a control treatment consisting of pellets and illness on separate days (Unpaired). Control by goal-directed and habitual systems was evaluated in a subsequent nonreinforced test of nose poking. The results of Experiment 1 indicated that the Non-Stressed Paired group reduced nose-poking compared to the Unpaired controls, identifying the response as goal directed, whereas the Stressed Paired and Unpaired groups made a similar number of nose pokes identifying the response as habitual despite a similar amount of training. Results from Experiment 2 indicated habitual control of nose-poke responding was present when stress was experienced just prior to the test. Collectively, these data suggest that stress may facilitate habitual control by altering the relative influence of goal-directed and habitual processes underpinning instrumental behavior. These results may be clinically relevant for understanding the contributions of stress to dysregulated instrumental behavior in compulsive pathologies.

Antagonism of D2 receptors via raclopride ameliorates amphetamine-induced associative learning deficits in male mice

Dopamine levels in the dorsomedial striatum (DMS) are highly dynamic and are thought to underly the encoding of action-outcome associations. Although it is known that amphetamine disrupts the learning that is required for goal-directed action, the role of D1 and D2 receptors in this process has not been established. In this study, we examined the role of D1 and D2 receptor antagonists on learning in response to amphetamine. We used the outcome-specific devaluation task to examine goal-directed action in male C57BL6/J mice treated systemically with either a D1 antagonist (SCH-23990; 0.01 mg/kg) or a D2 antagonist (raclopride; 0.5 mg/kg) and then administered amphetamine (1 mg/kg). The mice were injected repeatedly throughout the instrumental training phase of the task to assess the impact on the learning of action-outcomes, and the subsequent choice test assessing performance of goal-directed action was conducted drug free. Effects of chronic drug administration on locomotor behaviour was assessed before and after the choice test. Treatment during learning with either amphetamine, or the D1 or D2 antagonists, impaired the subsequent performance of goal-directed action. The amphetamine-induced impairment in goal-directed action was reversed in mice treated with raclopride, but not when treated with SCH-23990. By contrast, amphetamine-induced hyperactivity was reversed in mice treated with SCH-23990, but not in mice treated with raclopride. Taken together, these data support the role of a balance of dopamine receptor signalling after amphetamine treatment. While overall D1 receptor availability is necessary to promote learning, in a state of elevated dopamine, modifying D2 receptor function can ameliorate learning deficits.

Drug reinforcement impairs cognitive flexibility by inhibiting striatal cholinergic neurons

Addictive substance use impairs cognitive flexibility, with unclear underlying mechanisms. The reinforcement of substance use is mediated by the striatal direct-pathway medium spiny neurons (dMSNs) that project to the substantia nigra pars reticulata (SNr). Cognitive flexibility is mediated by striatal cholinergic interneurons (CINs), which receive extensive striatal inhibition. Here, we hypothesized that increased dMSN activity induced by substance use inhibits CINs, reducing cognitive flexibility. We found that cocaine administration in rodents caused long-lasting potentiation of local inhibitory dMSN-to-CIN transmission and decreased CIN firing in the dorsomedial striatum (DMS), a brain region critical for cognitive flexibility. Moreover, chemogenetic and time-locked optogenetic inhibition of DMS CINs suppressed flexibility of goal-directed behavior in instrumental reversal learning tasks. Notably, rabies-mediated tracing and physiological studies showed that SNr-projecting dMSNs, which mediate reinforcement, sent axonal collaterals to inhibit DMS CINs, which mediate flexibility. Our findings demonstrate that the local inhibitory dMSN-to-CIN circuit mediates the reinforcement-induced deficits in cognitive flexibility.

Long-term effects of THC exposure on reward learning and motivated behavior in adolescent and adult male rats

RATIONALE

The endocannabinoid system makes critical contributions to reward processing, motivation, and behavioral control. Repeated exposure to THC or other cannabinoid drugs can cause persistent adaptions in the endocannabinoid system and associated neural circuitry. It remains unclear how such treatments affect the way rewards are processed and pursued.

OBJECTIVE AND METHODS

We examined if repeated THC exposure (5 mg/kg/day for 14 days) during adolescence or adulthood led to long-term changes in rats' capacity to flexibly encode and use action-outcome associations for goal-directed decision making. Effects on hedonic feeding and progressive ratio responding were also assessed.

RESULTS

THC exposure had no effect on rats' ability to flexibly select actions following reward devaluation. However, instrumental contingency degradation learning, which involves avoiding an action that is unnecessary for reward delivery, was augmented in rats with a history of adult but not adolescent THC exposure. THC-exposed rats also displayed more vigorous instrumental behavior in this study, suggesting a motivational enhancement. A separate experiment found that while THC exposure had no effect on hedonic feeding behavior, it increased rats' willingness to work for food on a progressive ratio schedule, an effect that was more pronounced when THC was administered to adults. Adolescent and adult THC exposure had opposing effects on the CB1 receptor dependence of progressive ratio performance, decreasing and increasing sensitivity to rimonabant-induced behavioral suppression, respectively.

CONCLUSIONS

Our findings reveal that exposure to a translationally relevant THC exposure regimen induces long-lasting, age-dependent alterations in cognitive and motivational processes that regulate the pursuit of rewards.

Activating the dorsomedial and ventral midbrain projections to the striatum differentially impairs goal-directed action in male mice

The cognitive symptoms of schizophrenia are wide ranging and include impaired goal-directed action. This could be driven by an increase in dopamine transmission in the dorsomedial striatum, a pathophysiological hallmark of schizophrenia. Although commonly associated with psychotic symptoms, dopamine signalling in this region also modulates associative learning that aids in the execution of actions. To gain a better understanding of the role of subcortical dopamine in learning and decision-making, we assessed goal-directed action in male mice using the cross-species outcome-specific devaluation task (ODT). First, we administered systemic amphetamine during training to determine the impact of altered dopaminergic signaling on associative learning. Second, we used pathway-specific chemogenetic approaches to activate the dorsomedial and ventral striatal pathways (that originate in the midbrain) to separately assess learning and performance. Amphetamine treatment during learning led to a dose-dependent impairment in goal-directed action. Activation of both striatal pathways during learning also impaired performance. However, when these pathways were activated during choice, only activation of the ventral pathway impaired goal-directed action. This suggests that elevated transmission in the dorsomedial striatal pathway impairs associative learning processes that guide the goal-directed execution of actions. By contrast, elevated transmission of the ventral striatal pathway disrupts the encoding of outcome values that are important for both associative learning and choice performance. These findings highlight the differential roles of the dorsomedial and ventral inputs into the striatum in goal-directed action and provides insight into how striatal dopamine signaling may contribute to the cognitive problems in those with schizophrenia.

Long-term memory, synaptic plasticity and dopamine in rodent medial prefrontal cortex: Role in executive functions

Mnemonic functions, supporting rodent behavior in complex tasks, include both long-term and (short-term) working memory components. While working memory is thought to rely on persistent activity states in an active neural network, long-term memory and synaptic plasticity contribute to the formation of the underlying synaptic structure, determining the range of possible states. Whereas, the implication of working memory in executive functions, mediated by the prefrontal cortex (PFC) in primates and rodents, has been extensively studied, the contribution of long-term memory component to these tasks received little attention. This review summarizes available experimental data and theoretical work concerning cellular mechanisms of synaptic plasticity in the medial region of rodent PFC and the link between plasticity, memory and behavior in PFC-dependent tasks. A special attention is devoted to unique properties of dopaminergic modulation of prefrontal synaptic plasticity and its contribution to executive functions.

Sustained inhibitory transmission but dysfunctional dopamine D2 receptor signaling in dorsal striatal subregions following protracted abstinence from amphetamine

Behavioral sensitization to amphetamine is a complex phenomenon that engages several neurotransmitter systems and brain regions. While dysregulated signaling in the mesolimbic dopamine system repeatedly has been linked to behavioral sensitization, later research has implicated dorsal striatal circuits and GABAergic neurotransmission in contributing to behavioral transformation elicited by amphetamine. The aim of this study was thus to determine if repeated amphetamine exposure followed by abstinence would alter inhibitory neurotransmission in dorsal striatal subregions. To this end, male Wistar rats received amphetamine (2.0 mg/kg) in an intermittent manner for a total of five days. Behavioral sensitization to amphetamine was measured in locomotor-activity boxes, while neuroadaptations were recorded in the dorsolateral (DLS) and dorsomedial striatum (DMS) using ex vivo electrophysiology at different timepoints of amphetamine abstinence (2 weeks, 4-5 weeks, 10-11 weeks). Data show that repeated drug-exposure produces behavioral sensitization to the locomotor-stimulatory properties of amphetamine, which sustains for at least ten weeks. Electrophysiological recordings demonstrated a long-lasting suppression of evoked population spikes in both striatal subregions. Furthermore, following ten weeks of abstinence, the responsiveness to a dopamine D2 receptor agonist was significantly impaired in brain slices from rats previously receiving amphetamine. However, neither the frequency nor the amplitude of spontaneous inhibitory currents was affected by treatment at any of the time points analyzed. In conclusion, passive administration of amphetamine initiates long-lasting neuroadaptations in brain regions associated with goal-directed behavior and habitual performance, but these transformations do not appear to be driven by changes in GABAergic neurotransmission.

Cyclic estrogen and progesterone during instrumental acquisition contributes to habit formation in female rats

Habit formation is thought to involve two parallel processes that are mediated by distinct neural substates: one that suppresses goal-directed behavior, and one that facilitates stimulus-response (S-R) learning, which underscores habitual behavior. In previous studies we showed that habitual responding emerges early during instrumental training in gonadally-intact female, compared to male, rats. The present study aimed to determine the role of ovarian hormones during instrumental acquisition in the transition from goal-directed to habitual behavior in female rats. Ovariectomized (OVX) female rats were given subcutaneous silastic capsules that released low levels of 17-β estradiol (E2) to maintain estrogen receptor availability. Rats were assigned to one of three hormone treatment conditions: no additional hormone replacement (Control group), replacement with high E2 (High E2 group), or replacement with high E2 followed by progesterone (High E2 + P4 group). Hormone replacement occurred twice during acquisition to mimic natural hormone fluctuations. At test, the Control and High E2 groups demonstrated responding that was sensitive to devaluation by lithium chloride-induced illness, indicating goal-directed behavior. In contrast, the High E2 + P4 group exhibited a pattern of devaluation-insensitive, habitual responding, that suggested the suppression of goal-directed processes. In a follow-up experiment, similar procedures were conducted, however during acquisition, OVX rats were given cyclic high E2 plus medroxy-progesterone (MPA), a form of progesterone that does not metabolize to neuroactive metabolites. In this group, goal-directed behavior was observed. These data indicate that habit formation is not facilitated in low estrogen states, nor in the presence of cyclic high E2. However, cyclic high E2, together with progesterone during acquisition, appears to facilitate the early emergence of habitual responding. Furthermore, these data suggest that a neuroactive progesterone metabolite, like allopregnanolone, in combination with high cyclic E2, supports this phenomenon.

Regulation of Voluntary Physical Activity Behavior: A Review of Evidence Involving Dopaminergic Pathways in the Brain

Physical activity leads to well-established health benefits. Current efforts to enhance physical activity have targeted mainly socioeconomic factors. However, despite these efforts, only a small number of adults engage in regular physical activity to the point of meeting current recommendations. Evidence collected in rodent models and humans establish a strong central nervous system component that regulates physical activity behavior. In particular, dopaminergic pathways in the central nervous system are among the best-characterized biological mechanisms to date with respect to regulating reward, motivation, and habit formation, which are critical for establishing regular physical activity. Herein, we discuss evidence for a role of brain dopamine in the regulation of voluntary physical activity behavior based on selective breeding and pharmacological studies in rodents, as well as genetic studies in both rodents and humans. While these studies establish a role of dopamine and associated mechanisms in the brain in the regulation of voluntary physical activity behavior, there is clearly need for more research on the underlying biology involved in motivation for physical activity and the formation of a physical activity habit. Such knowledge at the basic science level may ultimately be translated into better strategies to enhance physical activity levels within the society.

Dopaminergic and noradrenergic modulation of stress-induced alterations in brain activation associated with goal-directed behaviour

BACKGROUND

Acute stress is thought to reduce goal-directed behaviour, an effect purportedly associated with stress-induced release of catecholamines. In contrast, experimentally increased systemic catecholamine levels have been shown to increase goal-directed behaviour. Whether experimentally increased catecholamine function can modulate stress-induced reductions in goal-directed behaviour and its neural substrates, is currently unknown.

AIM

To assess whether and how experimentally induced increases in dopamine and noradrenaline contribute to the acute stress effects on goal-directed behaviour and associated brain activation.

METHODS

One hundred participants underwent a stress induction protocol (Maastricht acute stress test; MAST) or a control procedure and received methylphenidate (MPH) (40 mg, oral) or placebo according to a 2 × 2 between-subjects design. In a well-established instrumental learning paradigm, participants learnt stimulus-response-outcome associations, after which rewards were selectively devalued. Participants' brain activation and associated goal-directed behaviour were assessed in a magnetic resonance imaging scanner at peak cortisol/MPH concentrations.

RESULTS

The MAST and MPH increased physiological measures of stress (salivary cortisol and blood pressure), but only MAST increased subjective measures of stress. MPH modulated stress effects on activation of brain areas associated with goal-directed behaviour, including insula, putamen, amygdala, medial prefrontal cortex, frontal pole and orbitofrontal cortex. However, MPH did not modulate the tendency of stress to induce a reduction in goal-directed behaviour.

CONCLUSION

Our neuroimaging data suggest that MPH-induced increases in dopamine and noradrenaline reverse stress-induced changes in key brain regions associated with goal-directed behaviour, while behavioural effects were absent. These effects may be relevant for preventing stress-induced maladaptive behaviour like in addiction or binge eating disorder.

The effects of the dopamine D2/3 agonist quinpirole on incentive value and palatability-based choice in a rodent model of attention-deficit/hyperactivity disorder

RATIONALE

Palatability and incentive value influence animal food choice. Dopamine D2/3 receptor signaling may mediate the effects of palatability and incentive value on choice. Dopamine signaling is disrupted in attention-deficit hyperactivity disorder (ADHD). Investigating behavioral choice processes under D2/3 receptor agonists will help elucidate behavioral and pharmacological correlates of ADHD.

OBJECTIVES

To determine (1) how changes in incentive value affects choice of actions for outcomes that differ in palatability; (2) the effects of the D2/3 agonist quinpirole on choice based on palatability and incentive value; (3) how choice differs in spontaneously hypertensive rats (SHR; ADHD model) compared with control strains.

METHODS

Rats responded instrumentally for two food outcomes (chocolate and grain pellets) that differed in palatability. Following specific satiety of one outcome, rats underwent a choice test. Prior to the choice test, rats were given intra-peritoneal quinpirole (0.01-0.1 mg/kg) body weight. These manipulations were conducted in three strains of rats: SHR rats; the normotensive Wistar-Kyoto (WKY) controls; and Wistar outbred (WIS) controls.

RESULTS

All rat strains responded more vigorously for chocolate pellets compared with grain pellets. Quinpirole reduced the effects of palatability and dose-dependently increased the effects of incentive value on choice. SHR rats were the least influenced by incentive value, whereas WKY rats were the least influenced by palatability.

CONCLUSIONS

These results show that D2/3 signaling modulates choice based on palatability and incentive value. Disruption of this process in SHR rats may mirror motivational impairments observed in ADHD.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Dopamine D1 and D2 Receptors Are Important for Learning About Neutral-Valence Relationships in Sensory Preconditioning

Dopamine neurotransmission has been ascribed multiple functions with respect to both motivational and associative processes in reward-based learning, though these have proven difficult to tease apart. In order to better describe the role of dopamine in associative learning, this series of experiments examined the potential of dopamine D1- and D2-receptor antagonism (or combined antagonism) to influence the ability of rats to learn neutral valence stimulus-stimulus associations. Using a sensory preconditioning task, rats were first exposed to pairings of two neutral stimuli (S2-S1). Subsequently, S1 was paired with a mild foot-shock and resulting fear to both S1 (directly conditioned) and S2 (preconditioned) was examined. Initial experiments demonstrated the validity of the procedure in that measures of sensory preconditioning were shown to be contingent on pairings of the two sensory stimuli. Subsequent experiments indicated that systemic administration of dopamine D1- or D2-receptor antagonists attenuated learning when administered prior to S2-S1 pairings. However, the administration of a more generic D1R/D2R antagonist was without effect. These effects remained constant regardless of the affective valence of the conditioning environment and did not differ between male and female rats. The results are discussed in the context of recent suggestions that dopaminergic systems encode more than a simple reward prediction error, and provide potential avenues for future investigation.

Problematic eating as an issue of habitual control

Obesity has reached alarming rates worldwide. Although many people attempt to control weight by modifying their food-related behaviours, this typically only has short-term effects and most dieters regain the weight that was lost. Why do so many people struggle to regulate their food-related behaviours? One possible explanation is that these behaviours have become habits that are not immediately sensitive to their consequences. Here we review experimental evidence for a shift to habitual control over food-related behaviours and the neural systems that control them and how this relates to difficulty changing ones' eating behavior.

Prolonged Amphetamine Exposures Increase the Endogenous Human Dopamine Receptors 2 at the Cellular Membrane in Cells Lacking the Dopamine Transporter

The dopamine 2 receptors (D2R) are G-protein coupled receptors expressed both in pre- and post-synaptic terminals that play an important role in mediating the physiological and behavioral effects of amphetamine (Amph). Previous studies have indicated that the effects of Amph at the D2R mainly rely on the ability of Amph to robustly increase extracellular dopamine through the dopamine transporter (DAT). This implies that the effects of Amph on D2R require the neurotransmitter dopamine. However, because of its lipophilic nature, Amph can cross the cellular membrane and thus potentially affect D2R expression independently of dopamine and DAT, e.g., in post-synaptic terminals. Here we used an in vitro system to study whether Amph affects total expression, cellular distribution, and function of the human D2R (hD2R), endogenously expressed in HEK293 cells. By performing Western blot experiments, we found that prolonged treatments with 1 or 50 μM Amph cause a significant decrease of the endogenous hD2R in cells transfected with human DAT (hDAT). On the other hand, in cells lacking expression of DAT, quantification of the hD2R-mediated changes in cAMP, biotinylation assays, Western blots and imaging experiments demonstrated an increase of hD2R at the cellular membrane after 15-h treatments with Amph. Moreover, imaging data suggested that barbadin, a specific inhibitor of the βarrestin-βadaptin interaction, blocked the Amph-induced increase of hD2R. Taken together our data suggest that prolonged exposures to Amph decrease or increase the endogenous hD2R at the cellular membrane in HEK293 cells expressing or lacking hDAT, respectively. Considering that this drug is often consumed for prolonged periods, during which tolerance develops, our data suggest that even in absence of DAT or dopamine, Amph can still alter D2R distribution and function.

K369I Tau Mice Demonstrate a Shift Towards Striatal Neuron Burst Firing and Goal-directed Behaviour

Pathological forms of the microtubule-associated protein tau are involved in a large group of neurodegenerative diseases named tauopathies, including frontotemporal lobar degeneration (FTLD-tau). K369I mutant tau transgenic mice (K3 mice) recapitulate neural and behavioural symptoms of FTLD, including tau aggregates in the cortex, alterations to nigrostriatum, memory deficits and parkinsonism. The aim of this study was to further characterise the K3 mouse model by examining functional alterations to the striatum. Whole-cell patch-clamp electrophysiology was used to investigate the properties of striatal neurons in K3 mice and wildtype controls. Additionally, striatal-based instrumental learning tasks were conducted to assess goal-directed versus habitual behaviours (i.e., by examining sensitivity to outcome devaluation and progressive ratios). The K3 model demonstrated significant alterations in the discharge properties of striatal neurons relative to wildtype mice, which manifested as a shift in neuronal output towards a burst firing state. K3 mice acquired goal-directed responding faster than control mice and were goal-directed at test unlike wildtype mice, which is likely to indicate reduced capacity to develop habitual behaviour. The observed pattern of behaviour in K3 mice is suggestive of deficits in dorsal lateral striatal function and this was supported by our electrophysiological findings. Thus, both the electrophysiological and behavioural alterations indicate that K3 mice have early deficits in striatal function. This finding adds to the growing literature which indicate that the striatum is impacted in tau-related neuropathies such as FTLD, and further suggests that the K3 model is a unique mouse model for investigating FTLD especially with striatal involvement.

Accelerated habitual learning resulting from L-dopa exposure in rats is prevented by N-acetylcysteine

Instrumental actions are initially goal-directed and driven by their associated outcome. However, with repeated experience habitual actions develop which are automated and efficient, as they are instead driven by antecedent stimuli. Dopamine is thought to facilitate the transition from goal-directed to habitual actions. This idea has been largely derived from evidence that psychostimulants accelerate the development of habitual actions. In the current study, we examined the impact of L-dopa (levodopa or L-dihydroxyphenylalanine), which also potentiates dopamine activity, on habitual learning. L-dopa was systemically administered prior to training rats to press a lever for a food outcome. When tested, L-dopa exposed animals were insensitive to changes in the value of the food outcome, and hence demonstrated accelerated habitual behavioral control compared to control animals that remained goal directed. We also showed that when N-acetylcysteine (NAC), an antioxidant and regulator of glutamate activity, was co-administered with L-dopa, it prevented the transition to habitual behavior; an effect demonstrated previously for cocaine. Therefore, this study establishes similarities between L-dopa and psychostimulants in both the development and prevention of habitual actions, and supports the notion that excess dopamine potentiates habitual learning. This finding extends the limited existing knowledge of the impact of L-dopa on learning and behavior, and has implications for neurological disorders where L-dopa is the primary treatment.

Neural substrates of habit

Active reward pursuit is supported by the balance between the cognitive and habitual control of behavior. The cognitive, goal-directed strategy relies on the prospective evaluation of anticipated consequences, which allows behavior to readily adapt when circumstances change. Repetition of successful actions promotes less cognitively taxing habits, in which behavior is automatically executed without prospective consideration. Disruption in either of these behavioral regulatory systems contributes to the symptoms that underlie many psychiatric disorders. Here, I review recently identified neural substrates, at multiple neural levels, that contribute to habits and outline gaps in knowledge that must be addressed to fully understand the neural mechanisms of behavioral control.

Addiction is driven by excessive goal-directed drug choice under negative affect: translational critique of habit and compulsion theory

Drug addiction may be a goal-directed choice driven by excessive drug value in negative affective states, a habit driven by strong stimulus-response associations, or a compulsion driven by insensitivity to costs imposed on drug seeking. Laboratory animal and human evidence for these three theories is evaluated. Excessive goal theory is supported by dependence severity being associated with greater drug choice/economic demand. Drug choice is demonstrably goal-directed (driven by the expected value of the drug) and can be augmented by stress/negative mood induction and withdrawal-effects amplified in those with psychiatric symptoms and drug use coping motives. Furthermore, psychiatric symptoms confer risk of dependence, and coping motives mediate this risk. Habit theory of addiction has weaker support. Habitual behaviour seen in drug-exposed animals often does not occur in complex decision scenarios, or where responding is rewarded, so habit is unlikely to explain most human addictive behaviour where these conditions apply. Furthermore, most human studies have not found greater propensity to habitual behaviour in drug users or as a function of dependence severity, and the minority that have can be explained by task disengagement producing impaired explicit contingency knowledge. Compulsion theory of addiction also has weak support. The persistence of punished drug seeking in animals is better explained by greater drug value (evinced by the association with economic demand) than by insensitivity to costs. Furthermore, human studies have provided weak evidence that propensity to discount cost imposed on drug seeking is associated with dependence severity. These data suggest that human addiction is primarily driven by excessive goal-directed drug choice under negative affect, and less by habit or compulsion. Addiction is pathological because negative states powerfully increase expected drug value acutely outweighing abstinence goals.

The Pathophysiology of Tics; An Evolving Story

Background:

Tics, defined as quick, rapid, sudden, recurrent, non-rhythmic motor movements or vocalizations are required components of Tourette Syndrome (TS) - a complex disorder characterized by the presence of fluctuating, chronic motor and vocal tics, and the presence of co-existing neuropsychological problems. Despite many advances, the underlying pathophysiology of tics/TS remains unknown.

Objective:

To address a variety of controversies surrounding the pathophysiology of TS. More specifically: 1) the configuration of circuits likely involved; 2) the role of inhibitory influences on motor control; 3) the classification of tics as either goal-directed or habitual behaviors; 4) the potential anatomical site of origin, e.g. cortex, striatum, thalamus, cerebellum, or other(s); and 5) the role of specific neurotransmitters (dopamine, glutamate, GABA, and others) as possible mechanisms (Abstract figure).

Methods:

Existing evidence from current clinical, basic science, and animal model studies are reviewed to provide: 1) an expanded understanding of individual components and the complex integration of the Cortico-Basal Ganglia-Thalamo-Cortical (CBGTC) circuit - the pathway involved with motor control; and 2) scientific data directly addressing each of the aforementioned controversies regarding pathways, inhibition, classification, anatomy, and neurotransmitters.

Conclusion:

Until a definitive pathophysiological mechanism is identified, one functional approach is to consider that a disruption anywhere within CBGTC circuitry, or a brain region inputting to the motor circuit, can lead to an aberrant message arriving at the primary motor cortex and enabling a tic. Pharmacologic modulation may be therapeutically beneficial, even though it might not be directed toward the primary abnormality.

The neurochemical substrates of habitual and goal-directed control

Our daily decisions are governed by the arbitration between goal-directed and habitual strategies. However, the neurochemical basis of this arbitration is unclear. We assessed the contribution of dopaminergic, serotonergic, and opioidergic systems to this balance across reward and loss domains. Thirty-nine participants (17 healthy controls, 15 patients with pathological gambling, and 7 with binge eating disorder) underwent positron emission tomography (PET) imaging with [¹⁸F]FDOPA, [¹¹C]MADAM and [¹¹C]carfentanil to assess presynaptic dopamine, and serotonin transporter and mu-opioid receptor binding potential. Separately, participants completed a modified two-step task, which quantifies the degree to which decision-making is influenced by goal-directed or habitual strategies. All participants completed a version with reward outcomes; healthy controls additionally completed a version with loss outcomes. In the context of rewarding outcomes, we found that greater serotonin transporter binding potential in prefrontal regions was associated with habitual control, while greater serotonin transporter binding potential in the putamen was marginally associated with goal-directed control; however, the findings were no longer significant when controlling for the opposing valence (loss). In blocks with loss outcomes, we found that the opioidergic system, specifically greater [¹¹C]carfentanil binding potential, was positively associated with goal-directed control and negatively associated with habit-directed control. Our findings illuminate the complex neurochemical basis of goal-directed and habitual behavior, implicating differential roles for prefrontal and subcortical serotonin in decision-making across healthy and pathological populations.

Complementary Control over Habits and Behavioral Vigor by Phasic Activity in the Dorsolateral Striatum

Despite clear evidence linking the basal ganglia to the control of outcome insensitivity (i.e., habit) and behavioral vigor (i.e., its behavioral speed/fluidity), it remains unclear whether or how these functions relate to one another. Here, using male Long-Evans rats in response-based and cue-based maze-running tasks, we demonstrate that phasic dorsolateral striatum (DLS) activity occurring at the onset of a learned behavior regulates how vigorous and habitual it is. In a response-based task, brief optogenetic excitation at the onset of runs decreased run duration and the occurrence of deliberative behaviors, whereas midrun stimulation carried little effect. Outcome devaluation showed these runs to be habitual. DLS inhibition at run start did not produce robust effects on behavior until after outcome devaluation. At that time, when the DLS was plausibly most critically required for performance (i.e., habitual), inhibition reduced performance vigor measures and caused a dramatic loss of habitual responding (i.e., animals quit the task). In a second cue-based "beacon" task requiring behavior initiation at the start of the run and again in the middle of the run, DLS excitation at both time points could improve the vigor of runs. Postdevaluation testing showed behavior on the beacon task to be habitual as well. This pattern of results suggests that one role for phasic DLS activity at behavior initiation is to promote the execution of the behavior in a vigorous and habitual fashion by a diverse set of measures.SIGNIFICANCE STATEMENT Our research expands the literature twofold. First, we find that features of a habitual behavior that are typically studied separately (i.e., maze response performance, deliberation movements, running vigor, and outcome insensitivity) are quite closely linked together. Second, efforts have been made to understand "what" the dorsolateral striatum (DLS) does for habitual behavior, and our research provides a key set of results showing "when" it is important (i.e., at behavior initiation). By showing such dramatic control over habits by DLS activity in a phasic time window, plausible real-world applications could involve more informed DLS perturbations to curb intractably problematic habits.

Controversies Surrounding the Pathophysiology of Tics

Tics are sudden, rapid, recurrent, nonrhythmic motor movements or vocalizations (phonic productions) that are commonly present in children and are required symptoms for the diagnosis of Tourette syndrome. Despite their frequency, the underlying pathophysiology of tics/Tourette syndrome remains unknown. In this review, we discuss a variety of controversies surrounding the pathophysiology of tics, including the following: Are tics voluntary or involuntary? What is the role of the premonitory urge? Are tics due to excess excitatory or deficient inhibition? Is it time to adopt the contemporary version of the cortico-basal ganglia-thalamocortical (CBGTC) circuit? and Do we know the primary abnormal neurotransmitter in Tourette syndrome? Data from convergent clinical and animal model studies support complex interactions among the various CBGTC sites and neurotransmitters. Advances are being made; however, numerous pathophysiologic questions persist.

Mesolimbic dopamine projections mediate cue-motivated reward seeking but not reward retrieval in rats

Efficient foraging requires an ability to coordinate discrete reward-seeking and reward-retrieval behaviors. We used pathway-specific chemogenetic inhibition to investigate how rats’ mesolimbic and mesocortical dopamine circuits contribute to the expression and modulation of reward seeking and retrieval. Inhibiting ventral tegmental area dopamine neurons disrupted the tendency for reward-paired cues to motivate reward seeking, but spared their ability to increase attempts to retrieve reward. Similar effects were produced by inhibiting dopamine inputs to nucleus accumbens, but not medial prefrontal cortex. Inhibiting dopamine neurons spared the suppressive effect of reward devaluation on reward seeking, an assay of goal-directed behavior. Attempts to retrieve reward persisted after devaluation, indicating they were habitually performed as part of a fixed action sequence. Our findings show that complete bouts of reward seeking and retrieval are behaviorally and neurally dissociable from bouts of reward seeking without retrieval. This dichotomy may prove useful for uncovering mechanisms of maladaptive behavior.

Merging the Pathophysiology and Pharmacotherapy of Tics

Background

Anatomically, cortical-basal ganglia-thalamo-cortical (CBGTC) circuits have an essential role in the expression of tics. At the biochemical level, the proper conveyance of messages through these circuits requires several functionally integrated neurotransmitter systems. In this manuscript, evidence supporting proposed pathophysiological abnormalities, both anatomical and chemical is reviewed. In addition, the results of standard and emerging tic-suppressing therapies affecting nine separate neurotransmitter systems are discussed. The goal of this review is to integrate our current understanding of the pathophysiology of Tourette syndrome (TS) with present and proposed pharmacotherapies for tic suppression.

Methods

For this manuscript, literature searches were conducted for both current basic science and clinical information in PubMed, Google-Scholar, and other scholarly journals to September 2018.

Results

The precise primary site of abnormality for tics remains undetermined. Although many pathophysiologic hypotheses favor a specific abnormality of the cortex, striatum, or globus pallidus, others recognize essential influences from regions such as the thalamus, cerebellum, brainstem, and ventral striatum. Some prefer an alteration within direct and indirect pathways, whereas others believe this fails to recognize the multiple interactions within and between CBGTC circuits. Although research and clinical evidence supports involvement of the dopaminergic system, additional data emphasizes the potential roles for several other neurotransmitter systems.

Discussion

A greater understanding of the primary neurochemical defect in TS would be extremely valuable for the development of new tic-suppressing therapies. Nevertheless, recognizing the varied and complex interactions that exist in a multi-neurotransmitter system, successful therapy may not require direct targeting of the primary abnormality.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

An extended history of drug self-administration results in multiple sources of control over drug seeking behavior

It is widely recognized that across the development of drug addiction, cues associated with drug use come to exert increasing control over drug seeking and taking behaviors. However, there remain gaps in our knowledge regarding how the different types of drug related cues affect drug seeking and taking behaviors, and how the emergence of cue control over these behaviors relates to the onset of drug seeking compulsions. This paper reviews the literature on drug self-administration in animals to address these gaps. It first identifies the different types of cues that acquire control over reward seeking behavior generally, and examines whether the same types of cues acquire control over drug seeking behavior specifically. It then examines how the role of drug related cues in motivating and reinforcing drug seeking behavior changes across an extended drug-taking history, with a particular focus on the case of nicotine. The evidence reviewed shows that, after an extended history of drug taking, drug seeking behaviors are controlled by contextual cues associated with the development of drug seeking habits, response contingent cues that accompany delivery of the drug, as well as internal states that correlate with levels of drug intake. These multiple sources of control over drug seeking are discussed in relation to the generation of an addicted phenotype in animal models and the hypothesized progression from internal control over drug use to compulsive drug seeking.

Dopaminergic Modulation of Goal-Directed Behavior in a Rodent Model of Attention-Deficit/Hyperactivity Disorder

Aside from its clinical symptoms of inattention, impulsivity and hyperactivity, patients with Attention/Deficit-Hyperactivity Disorder (ADHD) display reward and motivational impairments. These impairments may reflect a deficit in action control, that is, an inability to flexibly adapt behavior to changing consequences. We previously showed that spontaneously hypertensive rats (SHR), an inbred rodent model of ADHD, show impairments in goal-directed action control, and instead are predominated by habits. In this study, we examined the effects of specific dopamine receptor sub-type (D1 and D2) agonists and antagonists on goal-directed behavior in SHR and the normotensive inbred control strain Wistar-Kyoto (WKY) rats. Rats acquired an instrumental response for different-flavored food rewards. A selective-satiety outcome devaluation procedure followed by a choice test in extinction revealed outcome-insensitive habitual behavior in SHR rats. Outcome-sensitive goal-directed behavior was restored in SHR rats following injection prior to the choice test of the dopamine D2 receptor agonist Quinpirole or dopamine D1 receptor antagonist SCH23390, whereas WKY rats showed habitual responding following exposure to these drugs. This novel finding indicates that the core behavioral deficit in ADHD might not be a consequence of dopamine hypofunction, but rather is due to a misbalance between activation of dopamine D1 and D2 receptor pathways that govern action control.

G protein-coupled receptor heteromers are key players in substance use disorder

G protein-coupled receptors (GPCR) represent the largest family of membrane proteins in the human genome. Physical association between two different GPCRs is linked to functional interactions which generates a novel entity, called heteromer, with specific ligand binding and signaling properties. Heteromerization is increasingly recognized to take place in the mesocorticolimbic pathway and to contribute to various aspects related to substance use disorder. This review focuses on heteromers identified in brain areas relevant to drug addiction. We report changes at the molecular and cellular levels that establish specific functional impact and highlight behavioral outcome in preclinical models. Finally, we briefly discuss selective targeting of native heteromers as an innovative therapeutic option.

Thiamine tetrahydrofurfuryl disulfide promotes voluntary activity through dopaminergic activation in the medial prefrontal cortex

A physically active lifestyle is associated with better health in body and mind, and it is urgent that supporting agents for such lifestyles be developed. In rodents, voluntary locomotor activity as an active physical behavior may be mediated by dopaminergic neurons (DNs). Thiamine phosphate esters can stimulate DNs, and we thus hypothesized that thiamine tetrahydrofurfuryl disulfide (TTFD), a thiamine derivative, promotes locomotor activity via DNs in rats. Acute i.p. administration of TTFD enhanced rat locomotor activity in a normal cage. In vivo microdialysis revealed that TTFD-enhanced locomotor activity was synchronized with dopamine release in the medial prefrontal cortex (mPFC). Antagonism of the dopamine D1 receptor, but not D2 receptor, in the mPFC fully suppressed TTFD-enhanced locomotor activity. Finally, we found a TTFD dose-dependent increase in voluntary wheel running. Our findings demonstrate that DNs in the mPFC mediates TTFD-enhanced locomotor activity, suggesting the potential of TTFD to induce active physical behavior.

Acute drug effects on habitual and non-habitual responding in crossed high alcohol preferring mice

RATIONALE

Drug reward plays a central role in acquiring drug-seeking behavior. However, subjects may continue using drugs despite negative consequences because self-administration becomes habitual, and divorced from outcome values. Although a history of drug and alcohol use expedite habit acquisition, and in spite of the fact that self-administration leads to intoxication, the acute effects of drugs on habitual responding are not well understood.

OBJECTIVES

We sought to observe how acute ethanol and amphetamine affect the balance between habitual and goal-directed behavior, as measured by a fluid-reinforced operant conditioning task.

METHODS

Selectively bred crossed high-alcohol-preferring (cHAP) mice were trained on an operant conditioning task reinforced on a variable interval schedule with 1% banana solution, which was subsequently devalued via LiCl pairing in half the animals. Ethanol (1.0 g/kg), amphetamine (2.0 mg/kg), or saline was administered prior to a post-devaluation test.

RESULTS

Overall, mice showed habitual behavior, but when divided into high- or low-responding groups based on training response rates, saline-treated, low-responding animals devalued, while saline-treated high-responding animals did not. Furthermore, amphetamine elicited devaluation even in high-responding animals, while ethanol prevented devaluation even in low-responding animals.

CONCLUSIONS

These data show that ethanol shifts animals toward behaving habitually. This may illuminate why alcohol-intoxicated individuals display impaired judgment about the relative merits of drinking, and potentially serve as a mechanism by which intoxicated subjects resume previously devalued behaviors, such as comorbid drug use. These findings also show that high variable interval response rates facilitate a shift from goal-directed to habitual behavior.

GABA and glutamate in children with Tourette syndrome: A 1H MR spectroscopy study at 7T

Tourette syndrome (TS) is characterized by presence of chronic, fluctuating motor and phonic tics. The underlying neurobiological basis for these movements is hypothesized to involve cortical-striatal-thalamo-cortical (CSTC) pathways. Two major neurotransmitters within these circuits are γ-aminobutyric acid (GABA) and glutamate. Seventy-five participants (32 with TS, 43 controls) ages 5-12 years completed ¹H MRS at 7T. GABA and glutamate were measured in dorsolateral prefrontal cortex (DLPFC), ventromedial prefrontal cortex (VMPFC), premotor cortex (PMC), and striatum, and metabolites quantified using LCModel. Participants also completed neuropsychological assessment emphasizing inhibitory control. Scans were well tolerated by participants. Across ROIs combined, glutamate was significantly higher in the TS group, compared to controls, with no significant group differences in GABA observed. ROI analyses revealed significantly increased PMC glutamate in the TS group. Among children with TS, increased PMC glutamate was associated with improved selective motor inhibition; however, no significant associations were identified between levels of glutamate or GABA and tic severity. The dopaminergic system has long been considered to have a dominant role in TS. Accumulating evidence, however, suggests involvement of other neurotransmitter systems. Data obtained using ¹H MRS at 7T supports alteration of glutamate within habitual behavior-related CSTC pathways of children with TS.

Model-Based Control in Dimensional Psychiatry

We use parallel interacting goal-directed and habitual strategies to make our daily decisions. The arbitration between these strategies is relevant to inflexible repetitive behaviors in psychiatric disorders. Goal-directed control, also known as model-based control, is based on an affective outcome relying on a learned internal model to prospectively make decisions. In contrast, habit control, also known as model-free control, is based on an integration of previous reinforced learning autonomous of the current outcome value and is implicit and more efficient but at the cost of greater inflexibility. The concept of model-based control can be further extended into pavlovian processes. Here we describe and compare tasks that tap into these constructs and emphasize the clinical relevance and translation of these tasks in psychiatric disorders. Together, these findings highlight a role for model-based control as a transdiagnostic impairment underlying compulsive behaviors and representing a promising therapeutic target.

Prior Cocaine Self-Administration Increases Response-Outcome Encoding That Is Divorced from Actions Selected in Dorsal Lateral Striatum

Dorsal lateral striatum (DLS) is a highly associative structure that encodes relationships among environmental stimuli, behavioral responses, and predicted outcomes. DLS is known to be disrupted after chronic drug abuse; however, it remains unclear what neural signals in DLS are altered. Current theory suggests that drug use enhances stimulus-response processing at the expense of response-outcome encoding, but this has mostly been tested in simple behavioral tasks. Here, we investigated what neural correlates in DLS are affected by previous cocaine exposure as rats performed a complex reward-guided decision-making task in which predicted reward value was independently manipulated by changing the delay to or size of reward associated with a response direction across a series of trial blocks. After cocaine self-administration, rats exhibited stronger biases toward higher-value reward and firing in DLS more strongly represented action-outcome contingencies independent from actions subsequently taken rather than outcomes predicted by selected actions (chosen-outcome contingencies) and associations between stimuli and actions (stimulus-response contingencies). These results suggest that cocaine self-administration strengthens action-outcome encoding in rats (as opposed to chosen-outcome or stimulus-response encoding), which abnormally biases behavior toward valued reward when there is a choice between two options during reward-guided decision-making.SIGNIFICANCE STATEMENT Current theories suggest that the impaired decision-making observed in individuals who chronically abuse drugs reflects a decrease in goal-directed behaviors and an increase in habitual behaviors governed by neural representations of response-outcome (R-O) and stimulus-response associations, respectively. We examined the impact that prior cocaine self-administration had on firing in dorsal lateral striatum (DLS), a brain area known to be involved in habit formation and affected by drugs of abuse, during performance of a complex reward-guided decision-making task. Surprisingly, we found that previous cocaine exposure enhanced R-O associations in DLS. This suggests that there may be more complex consequences of drug abuse than current theories have explored, especially when examining brain and behavior in the context of a complex two-choice decision-making task.

Age and Sex Interact to Mediate the Effects of Intermittent, High-Dose Ethanol Exposure on Behavioral Flexibility

Human alcoholics have been shown to have impaired cognitive control over actions and increased reliance on habitual response strategies. While it is unclear in humans whether these differences predate ethanol exposure or result from chronic drinking, data from animal studies suggest that ethanol acts to promote the development of inflexible behaviors. Here, we investigated how intermittent exposure to high doses of ethanol impacts the ability to flexibly regulate behavior in a habit model. As adolescence, may represent a period of increased drug taking and developmental vulnerability that may impact adult behavior, we compared the effects of high-dose ethanol exposure during adolescence to exposure during adulthood in male and female rats. Our findings indicated that the effects of intermittent, high-dose ethanol exposure on habitual behavior is mediated by age and sex such that ethanol exposure during adolescence promoted the use of habitual response strategies in adult females, but not males, and that the opposite pattern emerged following intermittent, high-dose ethanol exposure in adult rats.

What makes you tic? Translational approaches to study the role of stress and contextual triggers in Tourette syndrome

Tourette syndrome (TS) is a neurodevelopmental condition characterized by multiple, recurring motor and phonic tics. Rich empirical evidence shows that the severity of tics and associated manifestations is increased by several stressors and contextual triggers; however, the neurobiological mechanisms responsible for symptom exacerbation in TS remain poorly understood. This conceptual gap partially reflects the high phenotypic variability in tics, as well as the existing difficulties in operationalizing and standardizing stress and its effects in a clinical setting. Animal models of TS may be highly informative tools to overcome some of these limitations; these experimental preparations have already provided critical insights on key aspects of TS pathophysiology, and may prove useful to identify the neurochemical alterations induced by different stressful contingencies. In particular, emerging knowledge on the role of contextual triggers in animal models of TS may inform the development of novel pharmacological interventions to reduce tic fluctuations in this disorder.

Prefrontal cortical BDNF: A regulatory key in cocaine- and food-reinforced behaviors

Brain-derived neurotrophic factor (BDNF) affects synaptic plasticity and neural structure and plays key roles in learning and memory processes. Recent evidence also points to important, yet complex, roles for BDNF in rodent models of cocaine abuse and addiction. Here we examine the role of prefrontal cortical (PFC) BDNF in reward-related decision making and behavioral sensitivity to, and responding for, cocaine. We focus on BDNF within the medial and orbital PFC, its regulation by cocaine during early postnatal development and in adulthood, and how BDNF in turn influences responding for drug reinforcement, including in reinstatement models. When relevant, we draw comparisons and contrasts with experiments using natural (food) reinforcers. We also summarize findings supporting, or refuting, the possibility that BDNF in the medial and orbital PFC regulate the development and maintenance of stimulus-response habits. Further investigation could assist in the development of novel treatment approaches for cocaine use disorders.

Going and stopping: Dichotomies in behavioral control by the prefrontal cortex

The rodent dorsal medial prefrontal cortex (PFC), specifically the prelimbic cortex (PL), regulates the expression of conditioned fear and behaviors interpreted as reward-seeking. Meanwhile, the ventral medial PFC, namely the infralimbic cortex (IL), is essential to extinction conditioning in both appetitive and aversive domains. Here we review evidence that supports, or refutes, this "PL-go/IL-stop" dichotomy. We focus on the extinction of conditioned fear and the extinction and reinstatement of cocaine- or heroin-reinforced responding. We then synthesize evidence that the PL is essential for developing goal-directed response strategies, while the IL supports habit behavior. Finally, we propose that some functions of the orbital PFC parallel those of the medial PFC in the regulation of response selection. Integration of these discoveries may provide points of intervention for inhibiting untethered drug seeking in drug use disorders, failures in extinction in Post-traumatic Stress Disorder, or co-morbidities between the two.

Daily Exposure to Sucrose Impairs Subsequent Learning About Food Cues: A Role for Alterations in Ghrelin Signaling and Dopamine D2 Receptors

The prevalence of hedonic foods and associated advertising slogans has contributed to the rise of the obesity epidemic in the modern world. Research has shown that intake of these foods disrupt dopaminergic systems. It may be that a disruption of these circuits produces aberrant learning about food-cue relationships. We found that rodents given 28 days of intermittent access to sucrose exhibited a deficit in the ability to block learning about a stimulus when it is paired in compound with food and another stimulus that has already been established as predictive of the food outcome. This deficit was characterized by an approach to a cue signaling food delivery that is usually blocked by prior learning, an effect dependent on dopaminergic prediction-error signaling in the midbrain. Administering the D2 agonist quinpirole during learning restored blocking in animals with a prior history of sucrose exposure. Further, repeated central infusions of ghrelin produced a deficit in blocking in the same manner as sucrose exposure. We argue that changes in dopaminergic systems resulting from sucrose exposure are mediated by a disruption of ghrelin signaling as rodents come to anticipate delivery of the highly palatable sucrose outside of normal feeding schedules. This suggestion is supported by our finding that both sucrose and ghrelin treatments resulted in increases in amphetamine-induced locomotor responding. Thus, for the first time, we have provided evidence of a potential link between alterations in D2 receptors caused by the intake of hedonic foods and aberrant learning about cue-food relationships capable of promoting inappropriate feeding habits. In addition, we have found preliminary evidence to suggest that this is mediated by changes in ghrelin signaling, a finding that should stimulate further research into modulation of ghrelin activity to treat obesity.

A Closer Look at the Effects of Repeated Cocaine Exposure on Adaptive Decision-Making under Conditions That Promote Goal-Directed Control

It has been proposed that compulsive drug seeking reflects an underlying dysregulation in adaptive behavior that favors habitual (automatic and inflexible) over goal-directed (deliberative and highly flexible) action selection. Rodent studies have established that repeated exposure to cocaine or amphetamine facilitates the development of habits, producing behavior that becomes unusually insensitive to a reduction in the value of its outcome. The current study more directly investigated the effects of cocaine pre-exposure on goal-directed learning and action selection using an approach that discourages habitual performance. After undergoing a 15-day series of cocaine (15 or 30 mg/kg, i.p.) or saline injections and a drug withdrawal period, rats were trained to perform two different lever-press actions for distinct reward options. During a subsequent outcome devaluation test, both cocaine- and saline-treated rats showed a robust bias in their choice between the two actions, preferring whichever action had been trained with the reward that retained its value. Thus, it appears that the tendency for repeated cocaine exposure to promote habit formation does not extend to a more complex behavioral scenario that encourages goal-directed control. To further explore this issue, we assessed how prior cocaine treatment would affect the rats' ability to learn about a selective reduction in the predictive relationship between one of the two actions and its outcome, which is another fundamental feature of goal-directed behavior. Interestingly, we found that cocaine-treated rats showed enhanced, rather than diminished, sensitivity to this action-outcome contingency degradation manipulation. Given their mutual dependence on striatal dopamine signaling, we suggest that cocaine's effects on habit formation and contingency learning may stem from a common adaptation in this neurochemical system.

Memory Systems and the Addicted Brain

The view that anatomically distinct memory systems differentially contribute to the development of drug addiction and relapse has received extensive support. The present brief review revisits this hypothesis as it was originally proposed 20 years ago (1) and highlights several recent developments. Extensive research employing a variety of animal learning paradigms indicates that dissociable neural systems mediate distinct types of learning and memory. Each memory system potentially contributes unique components to the learned behavior supporting drug addiction and relapse. In particular, the shift from recreational drug use to compulsive drug abuse may reflect a neuroanatomical shift from cognitive control of behavior mediated by the hippocampus/dorsomedial striatum toward habitual control of behavior mediated by the dorsolateral striatum (DLS). In addition, stress/anxiety may constitute a cofactor that facilitates DLS-dependent memory, and this may serve as a neurobehavioral mechanism underlying the increased drug use and relapse in humans following stressful life events. Evidence supporting the multiple systems view of drug addiction comes predominantly from studies of learning and memory that have employed as reinforcers addictive substances often considered within the context of drug addiction research, including cocaine, alcohol, and amphetamines. In addition, recent evidence suggests that the memory systems approach may also be helpful for understanding topical sources of addiction that reflect emerging health concerns, including marijuana use, high-fat diet, and video game playing.