Pulling habits out of rats: adenosine 2A receptor antagonism in dorsomedial striatum rescues meth-amphetamine-induced deficits in goal-directed action

Outcome devaluation as a method for identifying goal-directed behaviors in rats

Goal-directed behaviors allow animals to act to satisfy needs and desires. The outcome devaluation task is an effective method for identifying goal-directed behaviors and distinguishing these from other types of behavior. Rats can be trained to lever-press for one or multiple distinct food rewards. During testing, the previously earned food-or a control food for comparison-is devalued by allowing the animal to freely feed on it until they are sated before testing lever-press performance under extinction conditions (no rewards are delivered). Behavior that adapts to reflect the new value of the outcome is considered goal-directed, whereas behavior that continues as in previous training despite the change in outcome value, is not. As more research groups have used this task, variability in the procedures used has increased. Here, we provide a reliable procedure for conducting the outcome devaluation task with appropriate controls. We describe the most common variants of the task and control conditions and discuss troubleshooting measures such as outcome pre-exposure, habituation to pre-feeding chambers and attention to animals' hunger levels. The method outlined can be executed in ~2 weeks including training (~8 d) and testing (1-4 d) by researchers who are familiar with performing behavioral tasks in laboratory rodents, although longer training may be considered for those who are interested in observing habitual control of behavior. This protocol should facilitate the comparison of results from different studies and laboratories, while allowing flexibility in the application of the outcome devaluation task to different research questions.

Reduced inhibitory synaptic transmission onto striatopallidal neurons may underlie aging-related motor skill deficits

Human beings are living longer than ever before and aging is accompanied by an increased incidence of motor deficits, including those associated with the neurodegenerative conditions, Parkinson's disease (PD) and Huntington's disease (HD). However, the biological correlates underlying this epidemiological finding, especially the functional basis at the synapse level, have been elusive. This study reveals that motor skill performance examined via rotarod, beam walking and pole tests is impaired in aged mice. This study, via electrophysiology recordings, further identifies an aging-related reduction in the efficacy of inhibitory synaptic transmission onto dorsolateral striatum (DLS) indirect-pathway medium spiny neurons (iMSNs), i.e., a disinhibition effect on DLS iMSNs. In addition, pharmacologically enhancing the activity of DLS iMSNs by infusing an adenosine A2A receptor (A2AR) agonist, which presumably mimics the disinhibition effect, impairs motor skill performance in young mice, simulating the behavior in aged naïve mice. Conversely, pharmacologically suppressing the activity of DLS iMSNs by infusing an A2AR antagonist, in order to offset the disinhibition effect, restores motor skill performance in aged mice, mimicking the behavior in young naïve mice. In conclusion, this study identifies a functional inhibitory synaptic plasticity in DLS iMSNs that likely contributes to the aging-related motor skill deficits, which would potentially serve as a striatal synaptic basis underlying age being a prominent risk factor for neurodegenerative motor deficits.

Differential effects of acute and prolonged morphine withdrawal on motivational and goal-directed control over reward-seeking behaviour

Opioid addiction is a relapsing disorder marked by uncontrolled drug use and reduced interest in normally rewarding activities. The current study investigated the impact of spontaneous withdrawal from chronic morphine exposure on emotional, motivational and cognitive processes involved in regulating the pursuit and consumption of food rewards in male rats. In Experiment 1, rats experiencing acute morphine withdrawal lost weight and displayed somatic signs of drug dependence. However, hedonically driven sucrose consumption was significantly elevated, suggesting intact and potentially heightened reward processing. In Experiment 2, rats undergoing acute morphine withdrawal displayed reduced motivation when performing an effortful response for palatable food reward. Subsequent reward devaluation testing revealed that acute withdrawal disrupted their ability to exert flexible goal-directed control over reward seeking. Specifically, morphine-withdrawn rats were impaired in using current reward value to select actions both when relying on prior action-outcome learning and when given direct feedback about the consequences of their actions. In Experiment 3, rats tested after prolonged morphine withdrawal displayed heightened rather than diminished motivation for food rewards and retained their ability to engage in flexible goal-directed action selection. However, brief re-exposure to morphine was sufficient to impair motivation and disrupt goal-directed action selection, though in this case, rats were only impaired in using reward value to select actions in the presence of morphine-paired context cues and in the absence of response-contingent feedback. We suggest that these opioid-withdrawal induced deficits in motivation and goal-directed control may contribute to addiction by interfering with the pursuit of adaptive alternatives to drug use.

Neither Amphetamine nor Sub-Anesthetic Ketamine Treatment during Adolescence Impairs Devaluation in Rats Tested during Adulthood

BACKGROUND

Much of the existing animal literature on the devaluation task suggests that prior repeated exposure to drugs of abuse during adulthood can impair goal-directed action, but the literature on human drug users is mixed. Also, the initiation of drug use often occurs during adolescence, but examinations of the effects of drug exposure during adolescence on behavior in the devaluation task are lacking.

METHODS

We examined whether repeated exposure during adolescence to amphetamine (3 mg/kg injections every-other day from post-natal day 27-45) or ketamine (twice daily 30 mg/kg injections from post-natal day 35-44) would impair behavior in a devaluation test when tested drug-free in adulthood. Rats were trained to press a left lever with a steady cue-light above it for one reinforcer and a right lever with a flashing cue-light above it for a different reinforcer. We tested whether any impairments in goal-directed action could be overcome by compensation between strategies by giving rats information based on lever-location and cue-lights during the test that was either congruent (allowing compensation) or incongruent (preventing compensation between strategies) with the configurations during training.

RESULTS

Our results provided no evidence for impairment of goal-directed action during adulthood after adolescent amphetamine or ketamine exposure.

CONCLUSIONS

We discuss possible reasons for this discrepancy with the prior literature, including (1) the age of exposure and (2) the pattern in the previous literature that most previous demonstrations of drug exposure impairing devaluation in laboratory animals may be attributed to either drug-associated cues present in the testing environment and/or accelerated habit learning in tasks that predispose laboratory animals towards habit formation with extended training (with training procedures that should resist the formation of habits in the current experiment). However, additional research is needed to examine the effects of these factors, as well a potential role for the particular doses and washout periods to determine the cause of our finding of no devaluation impairment after drug exposure.

Impairments in expression of devaluation in a Pavlovian goal-tracking task, but not a free operant devaluation task, after fentanyl exposure in female rats

In laboratory animals, there are numerous demonstrations that past exposure to drugs of abuse can lead to devaluation impairments weeks after the final drug exposure, with the majority of these demonstrations examining effects of exposure to psychostimulants. There has been minimal investigation into whether prior exposure to opiates can lead to devaluation impairments. Here, we first trained female rats that two separate cuelights predicted two different foods and measured Pavlovian goal-tracking responses (Experiment 1) or trained female rats to press two levers to earn two different foods and measured this operant response (Experiment 2). In both experiments, we subsequently gave the rats injections of fentanyl twice daily for 6 days, and then tested rats for conditioned responses after satiation on one of the foods 48-h after the final injection. We found that rats were impaired in the expression of devaluation in the Pavlovian task after fentanyl exposure, but were unimpaired in the expression of devaluation in the operant task. The pattern of results is most consistent with an impairment in lateral orbitofrontal cortex function, but additional research is needed to determine the neurobiological cause of this pattern of results.

Effects of Maternal Separation on Effort-based Responding for Sucrose Are Associated with c-Fos Expression in the Nucleus Accumbens Core

In both people and animals, exposure to adverse experiences early in life can alter neurodevelopment and lead to long-term behavioral effects, including effects on reward processing. In the current study, we use a well-validated rodent model of maternal neglect, maternal separation (MS), to investigate the impact of early life adversity on reward learning and motivation and identify associated modifications in cellular activation in reward-relevant areas. Litters of Long-Evans rats were separated from the dam for either 15 min (brief) or 180 min (prolonged)/day from postnatal day (PND)2 to PND14. As adults, offspring were trained to lever press for a sucrose pellet using fixed ratio (FR) schedules and motivation was tested using a progressive ratio (PR) schedule over 10 daily sessions to assess sustained effects on effort-based responding. Immunohistochemical staining for c-Fos was conducted in a subset of animals that underwent an additional PR session. While there were no effects on reward learning, both MS180 males and females demonstrated increased effort-based responding on the first day of PR testing, while only MS180 males demonstrated a sustained increase in effort across all 10 days. MS180-induced changes in c-Fos expression in the dorsal and ventral striatum were observed, with subregion-specific effects along the rostrocaudal axis. Moreover, regression analyses suggest that motivated responding for a sucrose food reward in MS180-exposed, but not MS15-exposed animals, was associated with increased c-Fos expression in the rostral nucleus accumbens core. These findings implicate specific striatal regions in sex-specific modulation of sustained effort-based reward behavior following early life adversity.

Bipolar mania and epilepsy pathophysiology and treatment may converge in purine metabolism: A new perspective on available evidence

Decreased ATPergic signaling is an increasingly recognized pathophysiology in bipolar mania disease models. In parallel, adenosine deficit is increasingly recognized in epilepsy pathophysiology. Under-recognized ATP and/or adenosine-increasing mechanisms of several antimanic and antiseizure therapies including lithium, valproate, carbamazepine, and ECT suggest a fundamental pathogenic role of adenosine deficit in bipolar mania to match the established role of adenosine deficit in epilepsy. The depletion of adenosine-derivatives within the purine cycle is expected to result in a compensatory increase in oxopurines (uric acid precursors) and secondarily increased uric acid, observed in both bipolar mania and epilepsy. Cortisol-based inhibition of purine conversion to adenosine-derivatives may be reflected in observed uric acid increases and the well-established contribution of cortisol to both bipolar mania and epilepsy pathology. Cortisol-inhibited conversion from IMP to AMP as precursor of both ATP and adenosine may represent a mechanism for treatment resistance common in both bipolar mania and epilepsy. Anti-cortisol therapies may therefore augment other treatments both in bipolar mania and epilepsy. Evidence linking (i) adenosine deficit with a decreased need for sleep, (ii) IMP/cGMP excess with compulsive hypersexuality, and (iii) guanosine excess with grandiose delusions may converge to suggest a novel theory of bipolar mania as a condition characterized by disrupted purine metabolism. The potential for disease-modification and prevention related to adenosine-mediated epigenetic changes in epilepsy may be mirrored in mania. Evaluating the purinergic effects of existing agents and validating purine dysregulation may improve diagnosis and treatment in bipolar mania and epilepsy and provide specific targets for drug development.

Differential effects of acute and prolonged morphine withdrawal on motivational and goal-directed control over reward-seeking behavior

Opioid addiction is a relapsing disorder marked by uncontrolled drug use and reduced interest in normally rewarding activities. The current study investigated the impact of spontaneous withdrawal from chronic morphine exposure on emotional, motivational, and cognitive processes involved in regulating the pursuit and consumption of natural food rewards in male rats. In Experiment 1, rats experiencing acute morphine withdrawal lost weight and displayed somatic signs of drug dependence. However, hedonically-driven sucrose consumption was significantly elevated, suggesting intact and potentially heightened emotional reward processing. In Experiment 2, rats undergoing acute morphine withdrawal displayed reduced motivation when performing an effortful response for palatable food reward. Subsequent reward devaluation testing revealed that acute withdrawal also disrupted their ability to exert flexible goal-directed control over their reward-seeking behavior. Specifically, morphine-withdrawn rats displayed insensitivity to reward devaluation both when relying on prior action-outcome learning and when given direct feedback about the consequences of their actions. In Experiment 3, rats tested after prolonged morphine withdrawal displayed heightened rather than diminished motivation for food rewards and retained their ability to engage in flexible goal-directed action selection. However, brief re-exposure to morphine was sufficient to impair motivation and disrupt goal-directed action selection, though in this case insensitivity to reward devaluation was only observed in the presence of morphine-paired context cues and in the absence of response-contingent feedback. We suggest that these opioid-withdrawal induced deficits in motivation and goal-directed control may contribute to addiction by interfering with the pursuit of adaptive alternatives to drug use.

Drug reinforcement impairs cognitive flexibility by inhibiting striatal cholinergic neurons

Addictive substance use impairs cognitive flexibility, with unclear underlying mechanisms. The reinforcement of substance use is mediated by the striatal direct-pathway medium spiny neurons (dMSNs) that project to the substantia nigra pars reticulata (SNr). Cognitive flexibility is mediated by striatal cholinergic interneurons (CINs), which receive extensive striatal inhibition. Here, we hypothesized that increased dMSN activity induced by substance use inhibits CINs, reducing cognitive flexibility. We found that cocaine administration in rodents caused long-lasting potentiation of local inhibitory dMSN-to-CIN transmission and decreased CIN firing in the dorsomedial striatum (DMS), a brain region critical for cognitive flexibility. Moreover, chemogenetic and time-locked optogenetic inhibition of DMS CINs suppressed flexibility of goal-directed behavior in instrumental reversal learning tasks. Notably, rabies-mediated tracing and physiological studies showed that SNr-projecting dMSNs, which mediate reinforcement, sent axonal collaterals to inhibit DMS CINs, which mediate flexibility. Our findings demonstrate that the local inhibitory dMSN-to-CIN circuit mediates the reinforcement-induced deficits in cognitive flexibility.

Striatopallidal adenosine A2A receptor modulation of goal-directed behavior: Homeostatic control with cognitive flexibility

Dysfunction of goal-directed behaviors under stressful or pathological conditions results in impaired decision-making and loss of flexibility of thoughts and behaviors, which underlie behavioral deficits ranging from depression, obsessive-compulsive disorders and drug addiction. Tackling the neuromodulators fine-tuning this core behavioral element may facilitate the development of effective strategies to control these deficits present in multiple psychiatric disorders. The current investigation of goal-directed behaviors has concentrated on dopamine and glutamate signaling in the corticostriatal pathway. In accordance with the beneficial effects of caffeine intake on mood and cognitive dysfunction, we now propose that caffeine's main site of action - adenosine A2A receptors (A2AR) - represent a novel target to homeostatically control goal-directed behavior and cognitive flexibility. A2AR are abundantly expressed in striatopallidal neurons and colocalize and interact with dopamine D2, NMDA and metabotropic glutamate 5 receptors to integrate dopamine and glutamate signaling. Specifically, striatopallidal A2AR (i) exert an overall "break" control of a variety of cognitive processes, making A2AR antagonists a novel strategy for improving goal-directed behavior; (ii) confer homeostatic control of goal-directed behavior by acting at multiple sites with often opposite effects, to enhance cognitive flexibility; (iii) integrate dopamine and adenosine signaling through multimeric A2AR-D2R heterocomplexes allowing a temporally precise fine-tuning in response to local signaling changes. As the U.S. Food and Drug Administration recently approved the A2AR antagonist Nourianz® (istradefylline) to treat Parkinson's disease, striatal A2AR-mediated control of goal-directed behavior may offer a new and real opportunity for improving deficits of goal-directed behavior and enhance cognitive flexibility under various neuropsychiatric conditions. This article is part of the Special Issue on "Purinergic Signaling: 50 years".

The role of dorsomedial striatum adenosine 2A receptors in the loss of goal-directed behaviour

RATIONALE

Adenosine A_2A receptors (A_2AR) in the dorsal striatum have been implicated in goal-directed behaviour. While activation of these receptors with several methods has resulted in an insensitivity to outcome devaluation, particular explanations for how they disrupt behaviour have not been explored. We both confirm a role for A_2A receptors in goal-directed responding and evaluate additional behavioural aspects of goal-directed control to more fully understand the role of A_2A receptors in instrumental behaviour.

OBJECTIVES

To examine the effects of the adenosine A_2A agonist CGS-21680 in the DMS on response-outcome encoding, updating representations of outcome value and on the ability to inhibit behaviour when reward is not available.

METHODS

Male rats were trained to lever press for food reward. The A_2AR agonist CGS-21680 was infused into the dorsomedial striatum either before an outcome devaluation test, prior to training with two distinct response-outcome associations or prior to a test of discriminative stimulus control over instrumental performance.

RESULTS

Intra-DMS administration of CGS-21680 impaired sensitivity to outcome devaluation. CGS-21680 treatment did not impair acquisition of specific response-outcome associations, selective control of responding based on the presence of stimuli that signaled when reward was or was not available, discrimination between stimuli or lever choices nor did it influence the effect of devaluation on the amounts of food eaten in a consumption test.

CONCLUSIONS

CGS-21680 impairs the ability to modulate responding based on recent changes to outcome value, an effect that is not accounted for by impairments in behavioural inhibition, discrimination, encoding the specific outcome of a response or the effectiveness of specific satiety.

Habitual behaviour associated with exposure to high-calorie diet is prevented by an orexin-receptor-1 antagonist

Habitual actions, which are associated with addictive behaviours, contribute to the loss of control of food seeking seen following exposure to calorie-dense foods in rats. Antagonism of orexin-receptor-1 (ORX-R1) has been shown to reduce a range of stimulus-driven feeding behaviours, but have yet to be implicated in the regulation of habitual actions. In the current study, male Long-Evans rats were given 'binge-like' access to high-calorie diet (HCD) or standard chow diet, and were subsequently trained to press a lever for food outcome. When lever responses were tested following outcome devaluation, chow-fed rats displayed goal-directed actions, whereas HCD-exposed rats displayed habitual actions. In study 1, it was shown that systemic administration of the ORX-R1 antagonist, SB-334867, prior to test restored goal-directed behaviour in HCD-exposed rats. In study 2, intra-nigral administration of SB-334867 similarly restored goal-directed behaviour, thereby implicating the substantia nigra as an important site for this effect. This study demonstrates that targeting ORX-R1 reduces habitual food seeking in male rats which may be important for understanding and treating compulsive feeding, obesity and binge eating disorder. This study also implicates the lateral hypothalamus, where ORX is produced, in mediating the expression of habits for the first time, and thus extends on the neurocircuits known to regulate habitual actions. Further investigation is required to determine whether the same effects are also seen in female rats, given that there are recognised sexual dimorphisms in feeding behaviour and a higher incidence of disordered eating in female than male populations.

The role of the bed nucleus of the stria terminalis in the motivational control of instrumental action

We review recent studies assessing the role of the bed nucleus of the stria terminalis (BNST) in the motivational control of instrumental conditioning. This evidence suggests that the BNST and central nucleus of the amygdala (CeA) form a circuit that modulates the ventral tegmental area (VTA) input to the nucleus accumbens core (NAc core) to control the influence of Pavlovian cues on instrumental performance. In support of these claims, we found that activity in the oval region of BNST was increased by instrumental conditioning, as indexed by phosphorylated ERK activity (Experiment 1), but that this increase was not due to exposure to the instrumental contingency or to the instrumental outcome per se (Experiment 2). Instead, BNST activity was most significantly incremented in a test conducted when the instrumental outcome was anticipated but not delivered, suggesting a role for BNST in the motivational effects of anticipated outcomes on instrumental performance. To test this claim, we examined the effect of NMDA-induced cell body lesions of the BNST on general Pavlovian-to-instrumental transfer (Experiment 3). These lesions had no effect on instrumental performance or on conditioned responding during Pavlovian conditioning to either an excitory conditioned stimulus (CS) or a neutral CS (CS0) but significantly attenuated the excitatory effect of the Pavlovian CS on instrumental performance. These data are consistent with the claim that the BNST mediates the general excitatory influence of Pavlovian cues on instrumental performance and suggest BNST activity may be central to CeA-BNST modulation of a VTA-NAc core circuit in incentive motivation.

Negative Urgency Exacerbates Relapse to Cocaine Seeking After Abstinence

BACKGROUND

The mechanisms through which drug-cue-induced negative affective states are involved in relapse have not been defined. We tested the hypothesis that in individuals having developed a dorsolateral striatum (DLS)-dependent cue-controlled cocaine-seeking habit, the loss of the opportunity to enact the drug-seeking response during abstinence results in an urge to act that exacerbates relapse severity mediated by negative urgency.

METHODS

Eighty-seven male Sprague Dawley rats were trained to seek cocaine under the influence of the conditioned reinforcing properties of drug-paired cues or not. We investigated whether the tendency to relapse depended on the aversive state of withdrawal or instead on the loss of opportunity to perform the ingrained drug-seeking response after periods of abstinence. The striatal locus of control over cocaine seeking at baseline and relapse was investigated using in situ hybridization of the cellular activity marker C-fos and assessment of the sensitivity of instrumental drug seeking to dopamine receptor blockade in the dorsomedial striatum-dependent goal-directed and DLS-dependent habit systems.

RESULTS

The development of a DLS-dependent cue-controlled cocaine-seeking habit prior to abstinence resulted in a marked increase in drug seeking at relapse, which was not motivated by a cocaine withdrawal state and was no longer dependent on the DLS habit system. Instead, it reflected the emergence of negative urgency caused by the prevention of the performance of the habit during abstinence and underpinned by transient engagement of the goal-directed system.

CONCLUSIONS

These results show that ingrained cue-controlled drug-seeking habits increase the pressure to relapse.

Animal models of action control and cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) has historically been considered a motor disorder induced by a loss of dopaminergic neurons in the substantia nigra pars compacta. More recently, it has been recognized to have significant non-motor symptoms, most prominently cognitive symptoms associated with a dysexecutive syndrome. It is common in the literature to see motor and cognitive symptoms treated separately and, indeed, there has been a general call for specialized treatment of the latter, particularly in the more severe cases of PD with mild cognitive impairment and dementia. Animal studies have similarly been developed to model the motor or non-motor symptoms. Nevertheless, considerable research has established that segregating consideration of cognition from the precursors to motor movement, particularly movement associated with goal-directed action, is difficult if not impossible. Indeed, on some contemporary views cognition is embodied in action control, something that is particularly prevalent in theory and evidence relating to the integration of goal-directed and habitual control processes. The current paper addresses these issues within the literature detailing animal models of cognitive dysfunction in PD and their neural and neurochemical bases. Generally, studies using animal models of PD provide some of the clearest evidence for the integration of these action control processes at multiple levels of analysis and imply that consideration of this integrative process may have significant benefits for developing new approaches to the treatment of PD.

Diet-induced deficits in goal-directed control are rescued by agonism of group II metabotropic glutamate receptors in the dorsomedial striatum

Many overweight or obese people struggle to sustain the behavioural changes necessary to achieve and maintain weight loss. In rodents, obesogenic diet can disrupt goal-directed control of responding for food reinforcers, which may indicate that diet can disrupt brain regions associated with behavioural control. We investigated a potential glutamatergic mechanism to return goal-directed control to rats who had been given an obesogenic diet prior to operant training. We found that an obesogenic diet reduced goal-directed control and that systemic injection of LY379268, a Group II metabotropic glutamate receptor (mGluR2/3) agonist, returned goal-directed responding in these rats. Further, we found that direct infusion of LY379268 into the dorsomedial striatum, a region associated with goal-directed control, also restored goal-directed responding in the obesogenic-diet group. This indicates that one mechanism through which obesogenic diet disrupts goal-directed control is glutamatergic, and infusion of a mGluR2/3 agonist into the DMS is sufficient to ameliorate deficits in goal-directed control.

New directions in modelling dysregulated reward seeking for food and drugs

Behavioral models are central to behavioral neuroscience. To study the neural mechanisms of maladaptive behaviors (including binge eating and drug addiction), it is essential to develop and utilize appropriate animal models that specifically focus on dysregulated reward seeking. Both food and cocaine are typically consumed in a regulated manner by rodents, motivated by reward and homeostatic mechanisms. However, both food and cocaine seeking can become dysregulated, resulting in binge-like consumption and compulsive patterns of intake. The speakers in this symposium for the 2021 International Behavioral Neuroscience Meeting utilize behavioral models of dysregulated reward-seeking to investigate the neural mechanisms of binge-like consumption, enhanced cue-driven reward seeking, excessive motivation, and continued use despite negative consequences. In this review, we outline examples of maladaptive patterns of intake and explore recent animal models that drive behavior to become dysregulated, including stress exposure and intermittent access to rewards. Lastly, we explore select behavioral and neural mechanisms underlying dysregulated reward-seeking for both food and drugs.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Dopamine D1 and D2 Receptors Are Important for Learning About Neutral-Valence Relationships in Sensory Preconditioning

Dopamine neurotransmission has been ascribed multiple functions with respect to both motivational and associative processes in reward-based learning, though these have proven difficult to tease apart. In order to better describe the role of dopamine in associative learning, this series of experiments examined the potential of dopamine D1- and D2-receptor antagonism (or combined antagonism) to influence the ability of rats to learn neutral valence stimulus-stimulus associations. Using a sensory preconditioning task, rats were first exposed to pairings of two neutral stimuli (S2-S1). Subsequently, S1 was paired with a mild foot-shock and resulting fear to both S1 (directly conditioned) and S2 (preconditioned) was examined. Initial experiments demonstrated the validity of the procedure in that measures of sensory preconditioning were shown to be contingent on pairings of the two sensory stimuli. Subsequent experiments indicated that systemic administration of dopamine D1- or D2-receptor antagonists attenuated learning when administered prior to S2-S1 pairings. However, the administration of a more generic D1R/D2R antagonist was without effect. These effects remained constant regardless of the affective valence of the conditioning environment and did not differ between male and female rats. The results are discussed in the context of recent suggestions that dopaminergic systems encode more than a simple reward prediction error, and provide potential avenues for future investigation.

Depression in the Direct Pathway of the Dorsomedial Striatum Permits the Formation of Habitual Action

In order to achieve optimal outcomes in an ever-changing environment, humans and animals generally manage their action control via either goal-directed action or habitual action. These two action strategies are thought to be encoded in distinct parallel circuits in the dorsal striatum, specifically, the posterior dorsomedial striatum (DMS) and the dorsolateral striatum (DLS), respectively. The striatum is primarily composed of two subtypes of medium spiny neurons (MSNs): the direct-pathway striatonigral and the indirect-pathway striatopallidal MSNs. MSN-subtype-specific synaptic plasticity in the DMS and the DLS has been revealed to underlie goal-directed action and habitual action, respectively. However, whether any MSN-subtype-specific synaptic plasticity in the DMS is associated with habitual action, and if so, whether the synaptic plasticity affects the formation of habitual action, are not known. This study demonstrates that postsynaptic depression in the excitatory synapses of the direct-pathway striatonigral MSNs in the DMS is formed after habit learning. Moreover, chemogenetically rescuing this depression compromises the acquisition, but not the expression, of habitual action. These findings reveal that an MSN-subtype-specific synaptic plasticity in the DMS affects habitual action and suggest that plasticity in the DMS as well as in the DLS contributes to the formation of habitual action.

Renewal of goal direction with a context change after habit learning

An instrumental action can be goal-directed after a moderate amount of practice and then convert to habit after more extensive practice. Recent evidence suggests, however, that habits can return to action status after different environmental manipulations. The present experiments therefore asked whether habit learning interferes with goal direction in a context-dependent manner like other types of retroactive interference (e.g., extinction, punishment, counterconditioning). In Experiment 1, rats were given a moderate amount of instrumental training to form an action in one context (Context A) and then more extended training of the same response to form a habit in another context (Context B). We then performed reinforcer devaluation with taste aversion conditioning in both contexts, and tested the response in both contexts. The response remained habitual in Context B, but was goal-directed in Context A, indicating renewal of goal direction after habit learning. Experiment 2 expanded on Experiment 1 by testing the response in a third context (Context C). It found that the habitual response also renewed as action in this context. Together, the results establish a parallel between habit and extinction learning: Conversion to habit does not destroy action knowledge, but interferes with it in a context-specific way. They are also consistent with other results suggesting that habit is specific to the context in which it is learned, whereas goal-direction can transfer between contexts. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

'Liking' and 'wanting' in eating and food reward: Brain mechanisms and clinical implications

It is becoming clearer how neurobiological mechanisms generate 'liking' and 'wanting' components of food reward. Mesocorticolimbic mechanisms that enhance 'liking' include brain hedonic hotspots, which are specialized subregions that are uniquely able to causally amplify the hedonic impact of palatable tastes. Hedonic hotspots are found in nucleus accumbens medial shell, ventral pallidum, orbitofrontal cortex, insula cortex, and brainstem. In turn, a much larger mesocorticolimbic circuitry generates 'wanting' or incentive motivation to obtain and consume food rewards. Hedonic and motivational circuitry interact together and with hypothalamic homeostatic circuitry, allowing relevant physiological hunger and satiety states to modulate 'liking' and 'wanting' for food rewards. In some conditions such as drug addiction, 'wanting' is known to dramatically detach from 'liking' for the same reward, and this may also occur in over-eating disorders. Via incentive sensitization, 'wanting' selectively becomes higher, especially when triggered by reward cues when encountered in vulnerable states of stress, etc. Emerging evidence suggests that some cases of obesity and binge eating disorders may reflect an incentive-sensitization brain signature of cue hyper-reactivity, causing excessive 'wanting' to eat. Future findings on the neurobiological bases of 'liking' and 'wanting' can continue to improve understanding of both normal food reward and causes of clinical eating disorders.

Adenosine A2A receptor blockade improves neuroprosthetic learning by volitional control of population calcium signal in M1 cortical neurons

Volitional control is at the core of brain-machine interfaces (BMI) adaptation and neuroprosthetic-driven learning to restore motor function for disabled patients, but neuroplasticity changes and neuromodulation underlying volitional control of neuroprosthetic learning are largely unexplored. To better study volitional control at annotated neural population, we have developed an operant neuroprosthetic task with closed-loop feedback system by volitional conditioning of population calcium signal in the M1 cortex using fiber photometry recording. Importantly, volitional conditioning of the population calcium signal in M1 neurons did not improve within-session adaptation, but specifically enhanced across-session neuroprosthetic skill learning with reduced time-to-target and the time to complete 50 successful trials. With brain-behavior causality of the neuroprosthetic paradigm, we revealed that proficiency of neuroprosthetic learning by volitional conditioning of calcium signal was associated with the stable representational (plasticity) mapping in M1 neurons with the reduced calcium peak. Furthermore, pharmacological blockade of adenosine A_2A receptors facilitated volitional conditioning of neuroprosthetic learning and converted an ineffective volitional conditioning protocol to be the effective for neuroprosthetic learning. These findings may help to harness neuroplasticity for better volitional control of neuroprosthetic training and suggest a novel pharmacological strategy to improve neuroprosthetic learning in BMI adaptation by targeting striatal A_2A receptors.

Neural substrates of habit

Active reward pursuit is supported by the balance between the cognitive and habitual control of behavior. The cognitive, goal-directed strategy relies on the prospective evaluation of anticipated consequences, which allows behavior to readily adapt when circumstances change. Repetition of successful actions promotes less cognitively taxing habits, in which behavior is automatically executed without prospective consideration. Disruption in either of these behavioral regulatory systems contributes to the symptoms that underlie many psychiatric disorders. Here, I review recently identified neural substrates, at multiple neural levels, that contribute to habits and outline gaps in knowledge that must be addressed to fully understand the neural mechanisms of behavioral control.

Addiction is driven by excessive goal-directed drug choice under negative affect: translational critique of habit and compulsion theory

Drug addiction may be a goal-directed choice driven by excessive drug value in negative affective states, a habit driven by strong stimulus-response associations, or a compulsion driven by insensitivity to costs imposed on drug seeking. Laboratory animal and human evidence for these three theories is evaluated. Excessive goal theory is supported by dependence severity being associated with greater drug choice/economic demand. Drug choice is demonstrably goal-directed (driven by the expected value of the drug) and can be augmented by stress/negative mood induction and withdrawal-effects amplified in those with psychiatric symptoms and drug use coping motives. Furthermore, psychiatric symptoms confer risk of dependence, and coping motives mediate this risk. Habit theory of addiction has weaker support. Habitual behaviour seen in drug-exposed animals often does not occur in complex decision scenarios, or where responding is rewarded, so habit is unlikely to explain most human addictive behaviour where these conditions apply. Furthermore, most human studies have not found greater propensity to habitual behaviour in drug users or as a function of dependence severity, and the minority that have can be explained by task disengagement producing impaired explicit contingency knowledge. Compulsion theory of addiction also has weak support. The persistence of punished drug seeking in animals is better explained by greater drug value (evinced by the association with economic demand) than by insensitivity to costs. Furthermore, human studies have provided weak evidence that propensity to discount cost imposed on drug seeking is associated with dependence severity. These data suggest that human addiction is primarily driven by excessive goal-directed drug choice under negative affect, and less by habit or compulsion. Addiction is pathological because negative states powerfully increase expected drug value acutely outweighing abstinence goals.

Unexpected food outcomes can return a habit to goal-directed action

Three experiments examined the return of a habitual instrumental response to the status of goal-directed action. In all experiments, rats received extensive training in which lever pressing was reinforced with food pellets on a random-interval schedule of reinforcement. In Experiment 1, the extensively-trained response was not affected by conditioning a taste aversion to the reinforcer, and was therefore considered a habit. However, if the response had earned a new and unexpected food pellet during the final training session, the response was affected by taste aversion conditioning to the (first) reinforcer, and had thus been converted to a goal-directed action. In Experiment 3, 30 min of prefeeding with an irrelevant food pellet immediately before the test also converted a habit back to action, as judged by the taste-aversion devaluation method. That result was consistent with difficulty in finding evidence of habit with the sensory-specific satiety method after extensive instrumental training (Experiment 2). The results suggest that an instrumental behavior's status as a habit is not permanent, and that a habit can be returned to action status by associating it with a surprising reinforcer (Experiment 1) or by giving the animal an unexpected prefeeding immediately prior to the action/habit test (Experiment 3).

Prediction and control of operant behavior: What you see is not all there is

Prediction and control of operant behavior are major goals of behavior analysis. We suggest that achieving these goals can benefit from doing more than identifying the three-term contingency between the behavior, its setting stimulus, and its consequences. Basic research now underscores the idea that prediction and control require consideration of the behavior's history. As one example, if an operant is a goal-directed action, it is controlled by the current value of the reinforcer, as illustrated by the so-called reinforcer devaluation effect. In contrast, if the behavior is a habit, it occurs automatically, without regard to the reinforcer's value, as illustrated by its insensitivity to the reinforcer devaluation effect. History variables that distinguish actions and habits include the extent of their prior practice and their schedule of reinforcement. Other operants can appear to have very low or zero strength. However, if the behavior has reached that level through extinction or punishment, it may precipitously increase in strength by changing the context, allowing time to pass, presenting the reinforcer contingently or noncontingently, or extinguishing an alternative behavior. Behaviors that are not suppressed by extinction or punishment are not affected the same way. When predicting the strength of an operant behavior, what you see is not all there is. The behavior's history counts.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

An extended history of drug self-administration results in multiple sources of control over drug seeking behavior

It is widely recognized that across the development of drug addiction, cues associated with drug use come to exert increasing control over drug seeking and taking behaviors. However, there remain gaps in our knowledge regarding how the different types of drug related cues affect drug seeking and taking behaviors, and how the emergence of cue control over these behaviors relates to the onset of drug seeking compulsions. This paper reviews the literature on drug self-administration in animals to address these gaps. It first identifies the different types of cues that acquire control over reward seeking behavior generally, and examines whether the same types of cues acquire control over drug seeking behavior specifically. It then examines how the role of drug related cues in motivating and reinforcing drug seeking behavior changes across an extended drug-taking history, with a particular focus on the case of nicotine. The evidence reviewed shows that, after an extended history of drug taking, drug seeking behaviors are controlled by contextual cues associated with the development of drug seeking habits, response contingent cues that accompany delivery of the drug, as well as internal states that correlate with levels of drug intake. These multiple sources of control over drug seeking are discussed in relation to the generation of an addicted phenotype in animal models and the hypothesized progression from internal control over drug use to compulsive drug seeking.

G protein-coupled receptor heteromers are key players in substance use disorder

G protein-coupled receptors (GPCR) represent the largest family of membrane proteins in the human genome. Physical association between two different GPCRs is linked to functional interactions which generates a novel entity, called heteromer, with specific ligand binding and signaling properties. Heteromerization is increasingly recognized to take place in the mesocorticolimbic pathway and to contribute to various aspects related to substance use disorder. This review focuses on heteromers identified in brain areas relevant to drug addiction. We report changes at the molecular and cellular levels that establish specific functional impact and highlight behavioral outcome in preclinical models. Finally, we briefly discuss selective targeting of native heteromers as an innovative therapeutic option.

Dorsolateral Striatum Engagement Interferes with Early Discrimination Learning

In current models, learning the relationship between environmental stimuli and the outcomes of actions involves both stimulus-driven and goal-directed systems, mediated in part by the DLS and DMS, respectively. However, though these models emphasize the importance of the DLS in governing actions after extensive experience has accumulated, there is growing evidence of DLS engagement from the onset of training. Here, we used in vivo photosilencing to reveal that DLS recruitment interferes with early touchscreen discrimination learning. We also show that the direct output pathway of the DLS is preferentially recruited and causally involved in early learning and find that silencing the normal contribution of the DLS produces plasticity-related alterations in a PL-DMS circuit. These data provide further evidence suggesting that the DLS is recruited in the construction of stimulus-elicited actions that ultimately automate behavior and liberate cognitive resources for other demands, but with a cost to performance at the outset of learning.

Methamphetamine promotes habitual action and alters the density of striatal glutamate receptor and vesicular proteins in dorsal striatum

Goal-directed actions are controlled by the value of the consequences they produce and so increase when what they produce is valuable and decrease when it is not. With continued invariant practice, however, goal-directed actions can become habits, controlled not by their consequences but by antecedent, reward-related states and stimuli. Here, we show that pre-exposure to methamphetamine (METH) caused abnormally rapid development of habitual control. Furthermore, these drug-induced habits differed strikingly from conventional habits; we found that they were insensitive both to changes in reward value and to the effects of negative feedback. In addition to these behavioral changes, METH exposure produced bidirectional changes to synaptic proteins in the dorsal striatum. In the dorsomedial striatum, a structure critical for goal-directed action, METH exposure was associated with a reduction in glutamate receptor and glutamate vesicular proteins, whereas in the dorsolateral striatum, a region that has previously been implicated in habit learning, there was an increase in these proteins. Together, these results indicate that METH exposure promotes habitual control of action that appears to be the result of bidirectional changes in glutamatergic transmission in the circuits underlying goal-directed and habit-based learning.

Pharmacological Blockade of Adenosine A2A but Not A1 Receptors Enhances Goal-Directed Valuation in Satiety-Based Instrumental Behavior

The balance and smooth shift between flexible, goal-directed behaviors and repetitive, habitual actions are critical to optimal performance of behavioral tasks. The striatum plays an essential role in control of goal-directed versus habitual behaviors through a rich interplay of the numerous neurotransmitters and neuromodulators to modify the input, processing and output functions of the striatum. The adenosine receptors (namely A_2AR and A₁R), with their high expression pattern in the striatum and abilities to interact and integrate dopamine, glutamate and cannabinoid signals in the striatum, may represent novel therapeutic targets for modulating instrumental behavior. In this study, we examined the effects of pharmacological blockade of the A_2ARs and A₁Rs on goal-directed versus habitual behaviors in different information processing phases of instrumental learning using a satiety-based instrumental behavior procedure. We found that A_2AR antagonist acts at the coding, consolidation and expression phases of instrumental learning to modulate animals’ sensitivity to goal-directed valuation without modifying action-outcome contingency. However, pharmacological blockade and genetic knockout of A₁Rs did not affect acquisition or sensitivity to goal-valuation of instrumental behavior. These findings provide pharmacological evidence for a potential therapeutic strategy to control abnormal instrumental behaviors associated with drug addiction and obsessive-compulsive disorder by targeting the A_2AR.

Recent Insights into Corticostriatal Circuit Mechanisms underlying Habits: Invited review for Current Opinions in Behavioral Sciences

Habits have been studied for decades, but it was not until recent years that experiments began to elucidate the underlying cellular and circuit mechanisms. The latest experiments have been enabled by advances in cell-type specific monitoring and manipulation of activity in large neuronal populations. Here we will review recent efforts to understand the neural substrates underlying habit formation, focusing on rodent studies on corticostriatal circuits.

Junk Food Exposure Disrupts Selection of Food-Seeking Actions in Rats

There is growing evidence that repeated consumption of highly palatable, nutritionally poor "junk food" diets can produce deficits in cognition and behavioral control. We explored whether long-term junk-food diet exposure disrupts rats' ability to make adaptive choices about which foods to pursue based on (1) expected reward value (outcome devaluation test) and (2) cue-evoked reward expectations (Pavlovian-to-instrumental test). Rats were initially food restricted and trained on two distinct response-outcome contingencies (e.g., left press chocolate pellets, and right press sweetened condensed milk) and stimulus-outcome contingencies (e.g., white noise chocolate pellets, and clicker sweetened condensed milk). They were then given 6 weeks of unrestricted access to regular chow alone (controls) or chow and either 1 or 24 h access to junk food per day. Subsequent tests of decision making revealed that rats in both junk-food diet groups were impaired in selecting actions based on either expected food value or the presence of food-paired cues. These data demonstrate that chronic junk food consumption can disrupt the processes underlying adaptive control over food-seeking behavior. We suggest that the resulting dysregulation of food seeking may contribute to overeating and obesity.

Purinergic Signalling: Therapeutic Developments

Purinergic signalling, i.e., the role of nucleotides as extracellular signalling molecules, was proposed in 1972. However, this concept was not well accepted until the early 1990's when receptor subtypes for purines and pyrimidines were cloned and characterised, which includes four subtypes of the P1 (adenosine) receptor, seven subtypes of P2X ion channel receptors and 8 subtypes of the P2Y G protein-coupled receptor. Early studies were largely concerned with the physiology, pharmacology and biochemistry of purinergic signalling. More recently, the focus has been on the pathophysiology and therapeutic potential. There was early recognition of the use of P1 receptor agonists for the treatment of supraventricular tachycardia and A2A receptor antagonists are promising for the treatment of Parkinson's disease. Clopidogrel, a P2Y12 antagonist, is widely used for the treatment of thrombosis and stroke, blocking P2Y12 receptor-mediated platelet aggregation. Diquafosol, a long acting P2Y2 receptor agonist, is being used for the treatment of dry eye. P2X3 receptor antagonists have been developed that are orally bioavailable and stable in vivo and are currently in clinical trials for the treatment of chronic cough, bladder incontinence, visceral pain and hypertension. Antagonists to P2X7 receptors are being investigated for the treatment of inflammatory disorders, including neurodegenerative diseases. Other investigations are in progress for the use of purinergic agents for the treatment of osteoporosis, myocardial infarction, irritable bowel syndrome, epilepsy, atherosclerosis, depression, autism, diabetes, and cancer.

Long-term plasticity of corticostriatal synapses is modulated by pathway-specific co-release of opioids through κ-opioid receptors

KEY POINTS

Both endogenous opioids and opiate drugs of abuse modulate learning of habitual and goal-directed actions, and can also modify long-term plasticity of corticostriatal synapses. Striatal projection neurons of the direct pathway co-release the opioid neuropeptide dynorphin which can inhibit dopamine release via κ-opioid receptors. Theta-burst stimulation of corticostriatal fibres produces long-term potentiation (LTP) in striatal projection neurons when measured using whole-cell patch recording. Optogenetic activation of direct pathway striatal projection neurons inhibits LTP while reducing dopamine release. Because the endogenous release of opioids is activity dependent, this modulation of synaptic plasticity represents a negative feedback mechanism that may limit runaway enhancement of striatal neuron activity in response to drugs of abuse.

ABSTRACT

Synaptic plasticity in the striatum adjusts behaviour adaptively during skill learning, or maladaptively in the case of addiction. Just as dopamine plays a critical role in synaptic plasticity underlying normal skill learning and addiction, endogenous and exogenous opiates also modulate learning and addiction-related striatal plasticity. Though the role of opioid receptors in long-term depression in striatum has been characterized, their effect on long-term potentiation (LTP) remains unknown. In particular, direct pathway (dopamine D1 receptor-containing; D1R-) spiny projection neurons (SPNs) co-release the opioid neuropeptide dynorphin, which acts at presynaptic κ-opioid receptors (KORs) on dopaminergic afferents and can negatively regulate dopamine release. Therefore, we evaluated the interaction of co-released dynorphin and KOR on striatal LTP. We optogenetically facilitate the release of endogenous dynorphin from D1R-SPNs in brain slice while using whole-cell patch recording to measure changes in the synaptic response of SPNs following theta-burst stimulation (TBS) of cortical afferents. Our results demonstrate that TBS evokes corticostriatal LTP, and that optogenetic activation of D1R-SPNs during induction impairs LTP. Additional experiments demonstrate that optogenetic activation of D1R-SPNs reduces stimulation-evoked dopamine release and that bath application of a KOR antagonist provides full rescue of both LTP induction and dopamine release during optogenetic activation of D1R-SPNs. These results suggest that an increase in the opioid neuropeptide dynorphin is responsible for reduced TBS LTP and illustrate a physiological phenomenon whereby heightened D1R-SPN activity can regulate corticostriatal plasticity. Our findings have important implications for learning in addictive states marked by elevated direct pathway activation.

DRD2: Bridging the Genome and Ingestive Behavior

Recent work highlights the importance of genetic variants that influence brain structure and function in conferring risk for polygenic obesity. The neurotransmitter dopamine (DA) has a pivotal role in energy balance by integrating metabolic signals with circuits supporting cognitive, perceptual, and appetitive functions that guide feeding. It has also been established that diet and obesity alter DA signaling, leading to compulsive-like feeding and neurocognitive impairments. This raises the possibility that genetic variants that influence DA signaling and adaptation confer risk for overeating and cognitive decline. Here, we consider the role of two common gene variants, FTO and TaqIA rs1800497 in driving gene × environment interactions promoting obesity, metabolic dysfunction, and cognitive change via their influence on DA receptor subtype 2 (DRD2) signaling.

Subchronic anesthetic ketamine injections in rats impair choice reversal learning, but have no effect on reinforcer devaluation

Previous exposure to a variety of drugs of abuse has been shown to cause long-term impairments in reversal learning and reinforcer devaluation tasks. However, there is mixed evidence in the literature for a long-term effect of ketamine exposure on reversal learning and the long-term effect of ketamine exposure on devaluation is not known. We determined whether repeated injections of an anesthetic dose of ketamine would lead to impairments in choice reversal learning after discrimination learning or impairments in reinforcer devaluation. In two experiments, rats received three injections once-daily of ketamine (100 mg/kg, intraperitoneally) or saline and then began behavioral training 19 days later so that the key reversal learning and devaluation tests would occur about 1 month after the final ketamine injection. This ketamine exposure regimen did not impair learning in our discrimination task, but led to an increase in perseverative errors in reversal learning. However, the same ketamine exposure regimen (or injections of a lower 50 mg/kg dose) had no effect on behavior in the devaluation task. The behavioral patterns observed suggest possible neural mechanisms for the effects of ketamine, but future neurobiological investigations will be needed to isolate these mechanisms.

Habit Formation after Random Interval Training Is Associated with Increased Adenosine A2A Receptor and Dopamine D2 Receptor Heterodimers in the Striatum

Striatal adenosine A_2A receptors (A_2ARs) modulate striatal synaptic plasticity and instrumental learning, possibly by functional interaction with the dopamine D₂ receptors (D₂Rs) and metabotropic glutamate receptors 5 (mGluR5) through receptor-receptor heterodimers, but in vivo evidence for these interactions is lacking. Using in situ proximity ligation assay (PLA), we studied the subregional distribution of the A_2AR-D₂R and A_2AR-mGluR5 heterodimer complexes in the striatum and their adaptive changes over the random interval and random ratio training of instrumental learning. After confirming the specificity of the PLA detection of the A_2AR-D₂R heterodimers with the A_2AR knockout and D₂R knockout mice, we detected a heterogeneous distribution of the A_2AR-D₂R heterodimer complexes in the striatum, being more abundant in the dorsolateral than the dorsomedial striatum. Importantly, habit formation after the random interval training was associated with the increased formation of the A_2AR-D₂R heterodimer complexes, with prominant increase in the dorsomedial striatum. Conversely, goal-directed behavior after the random ratio schedule was not associated with the adaptive change in the A_2AR-D₂R heterodimer complexes. In contrast to the A_2AR-D₂R heterodimers, the A_2AR-mGluR5 heterodimers showed neither subregional variation in the striatum nor adaptive changes over either the random ratio (RR) or random interval (RI) training of instrumental learning. These findings suggest that development of habit formation is associated with increased formation of the A_2AR-D₂R heterodimer protein complexes which may lead to reduced dependence on D₂R signaling in the striatum.

Contexts Paired with Junk Food Impair Goal-Directed Behavior in Rats: Implications for Decision Making in Obesogenic Environments

The high prevalence of obesity and related metabolic diseases calls for greater understanding of the factors that drive excess energy intake. Calorie-dense palatable foods are readily available and often are paired with highly salient environmental cues. These cues can trigger food-seeking and consumption in the absence of hunger. Here we examined the effects of palatable food-paired environmental cues on control of instrumental food-seeking behavior. In Experiment 1, adult male rats received exposures to one context containing three “junk” foods (JFs context) and another containing chow (Chow context). Next, rats were food-deprived and trained to perform instrumental responses (lever-press) for two novel food rewards in a third, distinct context. Contextual influences on flexible control of food-seeking behavior were then assessed by outcome devaluation tests held in the JF, chow and training contexts. Devaluation was achieved using specific satiety and test order was counterbalanced. Rats exhibited goal-directed control over behavior when tested in the training and chow-paired contexts. Notably, performance was habitual (insensitive to devaluation) when tested in the JF context. In Experiment 2 we tested whether the impairment found in the JF context could be ameliorated by the presentation of a discrete auditory cue paired with the chow context, relative to a second cue paired with the JF context. Consistent with the results of Experiment 1, the devaluation effect was not significant when rats were tested in the JF context with the JF cue. However, presenting the chow cue increased the impact of the devaluation treatment leading to a robust devaluation effect. Further tests confirmed that performance in the chow context was goal-directed and that sensory-specific satiety in the JF context was intact. These results show that environments paired with palatable foods can impair goal-directed control over food-seeking behavior, but that this deficit was improved by a cue paired with chow. This has promising implications for assisting individuals in controlling their eating behavior in environments designed to dysregulate it.

A Closer Look at the Effects of Repeated Cocaine Exposure on Adaptive Decision-Making under Conditions That Promote Goal-Directed Control

It has been proposed that compulsive drug seeking reflects an underlying dysregulation in adaptive behavior that favors habitual (automatic and inflexible) over goal-directed (deliberative and highly flexible) action selection. Rodent studies have established that repeated exposure to cocaine or amphetamine facilitates the development of habits, producing behavior that becomes unusually insensitive to a reduction in the value of its outcome. The current study more directly investigated the effects of cocaine pre-exposure on goal-directed learning and action selection using an approach that discourages habitual performance. After undergoing a 15-day series of cocaine (15 or 30 mg/kg, i.p.) or saline injections and a drug withdrawal period, rats were trained to perform two different lever-press actions for distinct reward options. During a subsequent outcome devaluation test, both cocaine- and saline-treated rats showed a robust bias in their choice between the two actions, preferring whichever action had been trained with the reward that retained its value. Thus, it appears that the tendency for repeated cocaine exposure to promote habit formation does not extend to a more complex behavioral scenario that encourages goal-directed control. To further explore this issue, we assessed how prior cocaine treatment would affect the rats' ability to learn about a selective reduction in the predictive relationship between one of the two actions and its outcome, which is another fundamental feature of goal-directed behavior. Interestingly, we found that cocaine-treated rats showed enhanced, rather than diminished, sensitivity to this action-outcome contingency degradation manipulation. Given their mutual dependence on striatal dopamine signaling, we suggest that cocaine's effects on habit formation and contingency learning may stem from a common adaptation in this neurochemical system.