Sex specific effects of dorsomedial striatal cannabinoid receptor-1 signaling on Pavlovian outcome devaluation

Cannabinoid-1 receptor (CB1R) signaling in the dorsal striatum regulates the shift from flexible to habitual behavior in instrumental outcome devaluation. Based on prior work establishing individual, sex, and experience-dependent differences in Pavlovian behaviors, we predicted a role for dorsomedial striatum CB1R signaling in driving sign-tracking and rigid responding in Pavlovian outcome devaluation. We trained male and female rats in Pavlovian Lever Autoshaping to determine sign-, or goal- or intermediate tracking groups. After extended Pavlovian training, we gave intra-DMS infusions of the CB1R inverse agonist, rimonabant, before satiety-induced outcome devaluation test sessions, in which we sated rats on training pellets (devalued) or home cage chow (valued) and examined responding to cues in brief nonreinforced Pavlovian Lever Autoshaping sessions. Overall, DMS CB1R signaling inhibition blocked Pavlovian outcome devaluation. After extended training, male rats were sensitive to devaluation while female rats were not. Inhibition of DMS CB1R signaling impaired Pavlovian outcome devaluation in male sign-tracking rats making their behavior more rigid but had no effects in female sign-tracking rats. Intra-DMS rimonabant infusions before reinforced sessions had no effects on Pavlovian sign- or goal-tracking in either sex. The sex-specific and CB1R-dependent effects were specific to outcome devaluation and were consistent between sign- and goal-tracking groups. Our results demonstrate that DMS endocannabinoid receptor signaling regulates behavioral flexibility in a sex-specific manner, suggesting differences in CB1R signaling in DMS between male and female rats.

Decreased Ventral Tegmental Area CB1R Signaling Reduces Sign Tracking and Shifts Cue-Outcome Dynamics in Rat Nucleus Accumbens

Sign-tracking (ST) rats show enhanced cue sensitivity before drug experience that predicts greater discrete cue-induced drug seeking compared with goal-tracking or intermediate rats. Cue-evoked dopamine in the nucleus accumbens (NAc) is a neurobiological signature of sign-tracking behaviors. Here, we examine a critical regulator of the dopamine system, endocannabinoids, which bind the cannabinoid receptor-1 (CB1R) in the ventral tegmental area (VTA) to control cue-evoked striatal dopamine levels. We use cell type-specific optogenetics, intra-VTA pharmacology, and fiber photometry to test the hypothesis that VTA CB1R receptor signaling regulates NAc dopamine levels to control sign tracking. We trained male and female rats in a Pavlovian lever autoshaping (PLA) task to determine their tracking groups before testing the effect of VTA → NAc dopamine inhibition. We found that this circuit is critical for mediating the vigor of the ST response. Upstream of this circuit, intra-VTA infusions of rimonabant, a CB1R inverse agonist, during PLA decrease lever and increase food cup approach in sign-trackers. Using fiber photometry to measure fluorescent signals from a dopamine sensor, GRABDA (AAV9-hSyn-DA2m), we tested the effects of intra-VTA rimonabant on NAc dopamine dynamics during autoshaping in female rats. We found that intra-VTA rimonabant decreased sign-tracking behaviors, which was associated with increases in NAc shell, but not core, dopamine levels during reward delivery [unconditioned stimulus (US)]. Our results suggest that CB1R signaling in the VTA influences the balance between the conditioned stimulus-evoked and US-evoked dopamine responses in the NAc shell and biases behavioral responding to cues in sign-tracking rats.SIGNIFICANCE STATEMENT Substance use disorder (SUD) is a chronically relapsing psychological disorder that affects a subset of individuals who engage in drug use. Recent research suggests that there are individual behavioral and neurobiological differences before drug experience that predict SUD and relapse vulnerabilities. Here, we investigate how midbrain endocannabinoids regulate a brain pathway that is exclusively involved in driving cue-motivated behaviors of sign-tracking rats. This work contributes to our mechanistic understanding of individual vulnerabilities to cue-triggered natural reward seeking that have relevance for drug-motivated behaviors.

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

Investigating discriminative stimulus modulation of opioid seeking after conflict-induced abstinence in sign- and goal-tracking rats

RATIONALE

Discriminative stimuli (DS) are cues that predict reward availability. DS are resistant to extinction and motivate drug seeking even after long periods of abstinence. Previous studies have demonstrated that sign-tracking (ST) and goal-tracking (GT) differences in Pavlovian approach predict distinct cue-modulated vulnerabilities to cocaine reinstatement. GT rats show heightened reinstatement to contextual and DS, while ST rats show heightened reinstatement to discrete stimuli. Here we examine whether DS modulate reinstatement after electric barrier-induced abstinence and whether tracking-related relapse vulnerabilities generalize to opioid relapse.

OBJECTIVES

We examine whether DS-modulated reinstatement to fentanyl seeking persists in the presence of reduced adverse consequences after electric barrier-induced abstinence. We also examine whether tracking differences predict the magnitude of DS-modulated reinstatement of fentanyl seeking after electric barrier-induced abstinence.

METHODS

We used Pavlovian lever autoshaping (PLA) training to determine sign-, goal-, and intermediate tracking groups in male and female Sprague Dawley rats. We then trained rats in a DS model of intermittent fentanyl self-administration, and extinguished drug seeking by imposing an electric barrier of increasing intensity. We then measured the level of DS-modulated reinstatement in the presence of a reduced electric barrier intensity.

RESULTS

We report that DS strongly modulate fentanyl seeking after electric barrier-induced abstinence. DS-modulation of fentanyl acquisition, electric barrier-induced abstinence, and reinstatement was similar for sign- and goal-tracking groups.

CONCLUSIONS

Discriminative stimuli powerfully motivate opioid seeking, despite continued aversive consequences. Pavlovian approach differences do not predict the level of DS-modulated reinstatement to fentanyl seeking after conflict-induced abstinence.

Choose your path: Divergent basolateral amygdala efferents differentially mediate incentive motivation, flexibility and decision-making

To survive in a complex environment, individuals form associations between environmental stimuli and rewards to organize and optimize reward seeking behaviors. The basolateral amygdala (BLA) uses these learned associations to inform decision-making processes. In this review, we describe functional projections between BLA and its cortical and striatal targets that promote learning and motivational processes central to decision-making. Specifically, we compare and contrast divergent projections from the BLA to the orbitofrontal (OFC) and to the nucleus accumbens (NAc) and examine the roles of these pathways in associative learning, value-guided decision-making, choice behaviors, as well as cue and context-driven drug seeking. Finally, we consider how these projections are involved in disorders of motivation, with a focus on Substance Use Disorder.

Corrigendum: Role of BNST CRFR1 Receptors in Incubation of Fentanyl Seeking

[This corrects the article DOI: 10.3389/fnbeh.2020.00153.].

Effects of 5-HT2A receptor stimulation on economic demand for fentanyl after intermittent and continuous access self-administration in male rats

The relative value of and motivation for abused drugs often increases with drug experience and differs based on drug availability. Here, we determined how different intake patterns of fentanyl, a μ-opioid agonist, alter economic demand for fentanyl and how 5-HT_2A receptor stimulation affects economic demand for fentanyl. We used a within-session demand threshold procedure to characterize changes in economic demand for fentanyl before and after intermittent or continuous access schedules. We subsequently tested the acute effects of 5-HT_2A receptor stimulation with psychedelic 2,5-dimethoxy-4-iodoamphetamine (DOI) on economic demand for fentanyl. Extended fentanyl experience with both intermittent and continuous schedules increased fentanyl consumption at low cost (Q₀ ), increased total fentanyl consumption, and decreased demand elasticity (α), indicating both schedules elevated motivation to self-administer fentanyl. Overall, the two schedules produced similar alterations in economic demand for fentanyl, although low-cost consumption (Q₀ ) increased more in the continuous access group. Systemic injections of DOI (0.0-0.4 mg/kg, i.p.) dose-dependently produced economic demand changes in the opposite direction produced by fentanyl experience. DOI decreased motivation (increased "α"), decreased Q₀ , and decreased total fentanyl consumption. The selective 5-HT_2A antagonist, M100907 (0.3 mg/kg, i.p.), blocked the effects of DOI, indicating that DOI is acting through 5-HT_2A receptors to alter economic demand for fentanyl. In an economic food demand experiment, DOI (0.4 mg/kg) also increased demand elasticity and reduced food consumption. These results demonstrate that both intermittent and continuous fentanyl experience raise the economic demand for fentanyl, and acute 5-HT_2A receptor activation reduces economic demand for fentanyl and food.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Role of BNST CRFR1 Receptors in Incubation of Fentanyl Seeking

The time-dependent increase in cue-triggered opioid seeking, termed “incubation of opioid craving,” is modeled in rodents by examining responding for opioid-associated cues after a period of forced abstinence. With opioid drugs, withdrawal symptoms may heighten cue reactivity by recruiting brain systems involved in both reward seeking and stress responses. Corticotropin releasing factor (CRF) in the bed nucleus of the stria terminalis (BNST) is a critical driver of stress-induced relapse to drug seeking. Here, we sought to determine whether BNST CRF receptor 1 (CRFR1) signaling drives incubation of opioid craving in opioid dependent and non-dependent rats. First, we tested whether BNST CRFR1 signaling drives incubation of opioid craving in rats with short-access fentanyl self-administration experience (2.5 μg/kg/infusion, 3 h/day for 10 days). On Day 1 of forced abstinence, we gave bilateral intra-BNST vehicle injections to all rats and measured lever responding for opioid cues in the absence of fentanyl infusions. On Day 30 of forced abstinence, we gave an identical test after bilateral intra-BNST injections of vehicle or CRFR1 receptor antagonist, R121919 (1 μg/0.3 μL/hemisphere). Vehicle treated rats showed greater responding for opioid associated cues on Day 30 relative to Day 1, and this incubation effect was prevented by intra-BNST R121919 on Day 30. Next, we incorporated an opioid-dependence procedure to investigate whether BNST CRFR1 signaling drives opioid cue-reactivity to a greater extent in opioid-dependent relative to non-dependent rats. We trained rats to self-administer fentanyl for 5 days before initiating the dependence phase and resuming daily fentanyl self-administration sessions for 10 days. We gave intra-BNST R121919 or vehicle injections before testing during acute (Day 5) or protracted (Day 30) withdrawal. During acute withdrawal, antagonizing BNST CRFR1 decreased the number of press bouts without affecting bout size or duration. These patterns of responding with R121919 treatment resulted in less fentanyl-associated conditioned reinforcement during test. Together, these findings suggest a role for BNST CRFR1 signaling in driving cue-reinforced opioid seeking after periods of forced abstinence.

Effects of Limited and Extended Pavlovian Training on Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Individual differences in Pavlovian approach predict differences in devaluation sensitivity. Recent studies indicate goal-tracking (GT) rats are sensitive to outcome devaluation while sign-tracking (ST) rats are not. With extended training in Pavlovian lever autoshaping (PLA), GT rats display more lever-directed behavior, typical of ST rats, suggesting they may become insensitive to devaluation with more Pavlovian training experience. Here, we use a within-subject satiety-induced outcome devaluation procedure to test devaluation sensitivity after limited and extended PLA training in GT and ST rats. We trained rats in PLA to determine GT and ST groups. Then, we sated rats on either the training pellets (devalued condition) or homecage chow (valued condition) prior to brief non-reinforced test sessions after limited (sessions 5/6) and extended (sessions 17/18) PLA training. GT rats decreased conditioned responding under devalued relative to valued conditions after both limited and extended training, demonstrating they are sensitive to satiety devaluation regardless of the amount of PLA training. While ST rats were insensitive to satiety devaluation after limited training, their lever directed behavior became devaluation sensitive after extended training. To determine whether sign-tracking rats also displayed sensitivity to illness-induced outcome devaluation after extended training, we trained a separate cohort of rats in extended PLA and devalued the outcome with lithium chloride injections after pellet consumption in the homecage. ST rats failed to decrease conditioned responding after illness-induced outcome devaluation, while Non-ST rats (GT and intermediates) were sensitive to illness-induced outcome devaluation after extended training. Together, our results confirm devaluation sensitivity is stable in GT rats across training and devaluation approaches. Extended training unmasks devaluation sensitivity in ST rats after satiety, but not illness-induced devaluation, suggesting ST rats respond appropriately by decreasing responding to cues during state-dependent but not inference-based devaluation. The differences in behavioral flexibility across tracking groups and devaluation paradigms have translational relevance for the understanding state- vs. inference-based reward devaluation as it pertains to drug addiction vulnerability.

Stability of individual differences in sucralose taste preference

Outbred rats display variable preferences for bittersweet solutions, expressed as preference or avoidance of high concentrations of artificial sweeteners over water. This may reflect individual differences in appetitive/aversive conflict processing that may have predictive validity for disorders of motivation. Here we use a homecage two-bottle choice procedure to examine the test/retest stability and between-tastant consistency in sucralose preference to determine the reliability of bittersweet taste preference. Sucralose is a non-caloric artificial sweetener that is preferred by some rats and avoided by others. We sought to determine whether sucralose preference is consistent with preference of sucrose/quinine solutions that have known sweet and bitter taste qualities, respectively. We give fluid restricted rats 45-minutes homecage access to water and ascending concentrations of sucralose (SUCRA; 0.0025-10mM) or a compound solution of sucrose (116mM) + quinine (0.002-2mM) (SQ). We use a within-subject counterbalanced design (SUCRA or SQ testing) to determine preference of each bittersweet solution relative to water. We observed individual variability in preference for SUCRA and SQ, such that some rats preferred bittersweet solutions over water (preferring) while other rats preferred water over bittersweet solutions (avoiding). Within tastant, this preference remained stable across repeated testing. Between solutions, SUCRA preference scores correlated with SQ scores, suggesting consistent taste conflict processing for both bittersweet solutions. Population level analyses confirmed that preference generalizes across bittersweet solutions, and that rats' preferences for bittersweet solutions relative to water are stable over time. The test/retest and between-tastant reliability of this taste conflict screening procedure support the potential utility of this model for exploring individual variability in appetitive/aversive conflict processes mediating motivated behavior.

Manipulating the revision of reward value during the intertrial interval increases sign tracking and dopamine release

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change the relative ST-GT proportion as well as DA phasic responses. Here, we tested these predictions and found that lengthening the ITI increased ST, i.e., behavioral engagement with conditioned stimuli (CS) and cue-induced phasic DA release. Importantly, DA release was also present at the time of reward delivery, even after learning, and DA release was correlated with time spent in the food cup during the ITI. During conditioning with shorter ITIs, GT was prominent (i.e., engagement with food cup), and DA release responded to the CS while being absent at the time of reward delivery after learning. Hence, shorter ITIs restored the classical DA reward prediction error (RPE) pattern. These results validate the computational hypotheses, opening new perspectives on the understanding of individual differences in Pavlovian conditioning and DA signaling.

Disconnection of basolateral amygdala and insular cortex disrupts conditioned approach in Pavlovian lever autoshaping

Previously established individual differences in appetitive approach and devaluation sensitivity observed in goal- and sign-trackers may be attributed to differences in the acquisition, modification, or use of associative information in basolateral amygdala (BLA) pathways. Here, we sought to determine the extent to which communication of associative information between BLA and anterior portions of insular cortex (IC) supports ongoing Pavlovian conditioned approach behaviors in sign- and goal-tracking rats, in the absence of manipulations to outcome value. We hypothesized that the BLA mediates goal-, but not sign- tracking approach through interactions with the IC, a brain region involved in supporting flexible behavior. We first trained rats in Pavlovian lever autoshaping to determine their sign- or goal-tracking tendency. During alternating test sessions, we gave unilateral intracranial injections of vehicle or a cocktail of gamma-aminobutyric acid (GABA) receptor agonists, baclofen and muscimol, unilaterally into the BLA and contralaterally or ipsilaterally into the IC prior to reinforced lever autoshaping sessions. Consistent with our hypothesis we found that contralateral inactivation of BLA and IC increased the latency to approach the food cup and decreased the number of food cup contacts in goal-trackers. While contralateral inactivation of BLA and IC did not affect the total number of lever contacts in sign-trackers, this manipulation increased the latency to approach the lever. Ipsilateral inactivation of BLA and IC did not impact approach behaviors in Pavlovian lever autoshaping. These findings, contrary to our hypothesis, suggest that communication between BLA and IC maintains a representation of initially learned appetitive associations that commonly support the initiation of Pavlovian conditioned approach behavior regardless of whether it is directed at the cue or the location of reward delivery.

The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning

Phasic activity of midbrain dopamine neurons is currently thought to encapsulate the prediction-error signal described in Sutton and Barto’s (1981) model-free reinforcement learning algorithm. This phasic signal is thought to contain information about the quantitative value of reward, which transfers to the reward-predictive cue after learning. This is argued to endow the reward-predictive cue with the value inherent in the reward, motivating behavior toward cues signaling the presence of reward. Yet theoretical and empirical research has implicated prediction-error signaling in learning that extends far beyond a transfer of quantitative value to a reward-predictive cue. Here, we review the research which demonstrates the complexity of how dopaminergic prediction errors facilitate learning. After briefly discussing the literature demonstrating that phasic dopaminergic signals can act in the manner described by Sutton and Barto (1981), we consider how these signals may also influence attentional processing across multiple attentional systems in distinct brain circuits. Then, we discuss how prediction errors encode and promote the development of context-specific associations between cues and rewards. Finally, we consider recent evidence that shows dopaminergic activity contains information about causal relationships between cues and rewards that reflect information garnered from rich associative models of the world that can be adapted in the absence of direct experience. In discussing this research we hope to support the expansion of how dopaminergic prediction errors are thought to contribute to the learning process beyond the traditional concept of transferring quantitative value.

Individual variability in behavioral flexibility predicts sign-tracking tendency

Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS-) and second-order (SOCS+, SOCS-) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.

Effect of yohimbine on reinstatement of operant responding in rats is dependent on cue contingency but not food reward history

Yohimbine is an alpha-2 adrenoceptor antagonist that has been used in numerous studies as a pharmacological stressor in rodents, monkeys and humans. Recently, yohimbine has become the most common stress manipulation in studies on reinstatement of drug and food seeking. However, the wide range of conditions under which yohimbine promotes reward seeking is significantly greater than that of stressors like intermittent footshock. Here, we addressed two fundamental questions regarding yohimbine's effect on reinstatement of reward seeking: (1) whether the drug's effect on operant responding is dependent on previous reward history or cue contingency, and (2) whether yohimbine is aversive or rewarding under conditions typically used in reinstatement studies. We also used in vivo microdialysis to determine yohimbine's effect on dopamine levels in nucleus accumbens (NAc) and medial prefrontal cortex (mPFC). We found that the magnitude of yohimbine-induced (0.5, 1.0, 2.0 mg/kg) operant responding during the reinstatement tests was critically dependent on the contingency between lever pressing and discrete tone-light cue delivery but not the previous history with food reward during training. We also found that yohimbine (2 mg/kg) did not cause conditioned place aversion. Finally, we found that yohimbine modestly increased dopamine levels in mPFC but not NAc. Results suggest that yohimbine's effects on operant responding in reinstatement studies are likely independent of the history of contingent self-administration of food or drug rewards and may not be related to the commonly assumed stress-like effects of yohimbine.

Effect of cafeteria diet history on cue-, pellet-priming-, and stress-induced reinstatement of food seeking in female rats

BACKGROUND

Relapse to unhealthy eating habits is a major problem in human dietary treatment. The individuals most commonly seeking dietary treatment are overweight or obese women, yet the commonly used rat reinstatement model to study relapse to palatable food seeking during dieting primarily uses normal-weight male rats. To increase the clinical relevance of the relapse to palatable food seeking model, here we pre-expose female rats to a calorically-dense cafeteria diet in the home-cage to make them overweight prior to examining the effect of this diet history on cue-, pellet-priming- and footshock-induced reinstatement of food seeking.

METHODS

Post-natal day 32 female Long-Evans rats had seven weeks of home-cage access to either chow only or daily or intermittent cafeteria diet alongside chow. Next, they were trained to self-administer normally preferred 45 mg food pellets accompanied by a tone-light cue. After extinction, all rats were tested for reinstatement induced by discrete cue, pellet-priming, and intermittent footshock under extinction conditions.

RESULTS

Access to daily cafeteria diet and to a lesser degree access to intermittent cafeteria diet decreased food pellet self-administration compared to chow-only. Prior history of these cafeteria diets also reduced extinction responding, cue- and pellet-priming-induced reinstatement. In contrast, modest stress-induced reinstatement was only observed in rats with a history of daily cafeteria diet.

CONCLUSION

A history of cafeteria diet does not increase the propensity for cue- and pellet-priming-induced relapse in the rat reinstatement model but does appear to make rats more susceptible to footshock stress-induced reinstatement.

The reinstatement model of drug relapse: recent neurobiological findings, emerging research topics, and translational research

BACKGROUND AND RATIONALE

Results from many clinical studies suggest that drug relapse and craving are often provoked by acute exposure to the self-administered drug or related drugs, drug-associated cues or contexts, or certain stressors. During the last two decades, this clinical scenario has been studied in laboratory animals by using the reinstatement model. In this model, reinstatement of drug seeking by drug priming, drug cues or contexts, or certain stressors is assessed following drug self-administration training and subsequent extinction of the drug-reinforced responding.

OBJECTIVE

In this review, we first summarize recent (2009-present) neurobiological findings from studies using the reinstatement model. We then discuss emerging research topics, including the impact of interfering with putative reconsolidation processes on cue- and context-induced reinstatement of drug seeking, and similarities and differences in mechanisms of reinstatement across drug classes. We conclude by discussing results from recent human studies that were inspired by results from rat studies using the reinstatement model.

CONCLUSIONS

Main conclusions from the studies reviewed highlight: (1) the ventral subiculum and lateral hypothalamus as emerging brain areas important for reinstatement of drug seeking, (2) the existence of differences in brain mechanisms controlling reinstatement of drug seeking across drug classes, (3) the utility of the reinstatement model for assessing the effect of reconsolidation-related manipulations on cue-induced drug seeking, and (4) the encouraging pharmacological concordance between results from rat studies using the reinstatement model and human laboratory studies on cue- and stress-induced drug craving.

Effect of fenfluramine on reinstatement of food seeking in female and male rats: implications for the predictive validity of the reinstatement model

RATIONALE AND OBJECTIVES

Relapse to old unhealthy eating habits while dieting is often provoked by stress or acute exposure to palatable foods. We adapted a rat reinstatement model, which is used to study drug relapse, to study mechanisms of relapse to palatable food seeking induced by food-pellet priming (non-contingent exposure to a small amount of food pellets) or injections of yohimbine (an alpha-2 adrenoceptor antagonist that causes stress-like responses in humans and non-humans). Here, we assessed the predictive validity of the food reinstatement model by studying the effects of fenfluramine, a serotonin releaser with known anorectic effects, on reinstatement of food seeking.

METHODS

We trained food-restricted female and male rats to lever-press for 45-mg food pellets (3-h sessions) and first assessed the effect of fenfluramine (0.75, 1.5, and 3.0 mg/kg, i.p.) on food-reinforced responding. Subsequently, we extinguished the food-reinforced responding and tested the effect of fenfluramine (1.5 and 3.0 mg/kg) on reinstatement of food seeking induced by yohimbine injections (2 mg/kg, i.p.) or pellet priming (four non-contingent pellets).

RESULTS

Fenfluramine decreased yohimbine- and pellet-priming-induced reinstatement. As expected, fenfluramine also decreased food-reinforced responding, but a control condition in which we assessed fenfluramine's effect on high-rate operant responding indicated that the drug's effect on reinstatement was not due to performance deficits.

CONCLUSIONS

The present data support the predictive validity of the food reinstatement model and suggest that this model could be used to identify medications for prevention of relapse induced by stress or acute exposure to palatable food during dietary treatments.

Neural correlates of variations in event processing during learning in central nucleus of amygdala

Attention or variations in event processing help drive learning. Lesion studies have implicated the central nucleus of the amygdala (CeA) in this process, particularly when expected rewards are omitted. However, lesion studies cannot specify how information processing in CeA supports such learning. To address these questions, we recorded CeA neurons in rats performing a task in which rewards were delivered or omitted unexpectedly. We found that activity in CeA neurons increased selectively at the time of omission and declined again with learning. Increased firing correlated with CeA-inactivation sensitive measures of attention. Notably CeA neurons did not fire to the cues or in response to unexpected rewards. These results indicate that CeA contributes to learning in response to reward omission due to a specific role in signaling actual omission rather than a more general involvement in signaling expectancies, errors, or reward value.

All that glitters ... dissociating attention and outcome expectancy from prediction errors signals

Initially reported in dopamine neurons, neural correlates of prediction errors have now been shown in a variety of areas, including orbitofrontal cortex, ventral striatum, and amygdala. Yet changes in neural activity to an outcome or cues that precede it can reflect other processes. We review the recent literature and show that although activity in dopamine neurons appears to signal prediction errors, similar activity in orbitofrontal cortex, basolateral amygdala, and ventral striatum does not. Instead, increased firing in basolateral amygdala to unexpected outcomes likely reflects attention, whereas activity in orbitofrontal cortex and ventral striatum is unaffected by prior expectations and may provide information on outcome expectancy. These results have important implications for how these areas interact to facilitate learning and guide behavior.

The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes

Humans and other animals change their behavior in response to unexpected outcomes. The orbitofrontal cortex (OFC) is implicated in such adaptive responding, based on evidence from reversal tasks. Yet these tasks confound using information about expected outcomes with learning when those expectations are violated. OFC is critical for the former function; here we show it is also critical for the latter. In a Pavlovian overexpectation task, inactivation of OFC prevented learning driven by unexpected outcomes, even when performance was assessed later. We propose this reflects a critical contribution of outcome signaling by OFC to encoding of reward prediction errors elsewhere. In accord with this proposal, we report that signaling of reward predictions by OFC neurons was related to signaling of prediction errors by dopamine neurons in ventral tegmental area (VTA). Furthermore, bilateral inactivation of VTA or contralateral inactivation of VTA and OFC disrupted learning driven by unexpected outcomes.

Cocaine-paired cues activate aversive representations in accumbens neurons

Aversive states are proposed to drive addiction. Here, Wheeler and colleagues show that drug-associated cues come to activate neural representations of aversive information in nucleus accumbens and that this activation predicts subsequent drug use. These remarkable data identify a potential neural substrate through which aversive affective representations may motivate drug use.

Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards

The dopamine system is thought to be involved in making decisions about reward. Here we recorded from the ventral tegmental area in rats learning to choose between differently delayed and sized rewards. As expected, the activity of many putative dopamine neurons reflected reward prediction errors, changing when the value of the reward increased or decreased unexpectedly. During learning, neural responses to reward in these neurons waned and responses to cues that predicted reward emerged. Notably, this cue-evoked activity varied with size and delay. Moreover, when rats were given a choice between two differently valued outcomes, the activity of the neurons initially reflected the more valuable option, even when it was not subsequently selected.

Neural correlates of inflexible behavior in the orbitofrontal-amygdalar circuit after cocaine exposure

Addiction is characterized by compulsive or inflexible behavior, observed both in the context of drug-seeking and in contexts unrelated to drugs. One possible contributor to these inflexible behaviors may be drug-induced dysfunction within circuits that support behavioral flexibility, including the basolateral amygdala (ABL) and the orbitofrontal cortex (OFC). Here we describe data demonstrating that chronic cocaine exposure causes long-lasting changes in encoding properties in the ABL and the OFC during learning and reversal in an odor-guided task. In particular, these data suggest that inflexible encoding in ABL neurons may be the proximal cause of cocaine-induced behavioral inflexibility, and that a loss of outcome-expectant encoding in OFC neurons could be a more distal contributor to this impairment. A similar mechanism of drug-induced orbitofrontal-amygdalar dysfunction may cause inflexible behavior when animals and addicts are exposed to drug-associated cues and contexts.

Cocaine-induced decision-making deficits are mediated by miscoding in basolateral amygdala

Addicts and drug-experienced animals have decision-making deficits in reversal-learning tasks and more complex 'gambling' variants. Here we show evidence that these deficits are mediated by persistent encoding of outdated associative information in the basolateral amygdala. Cue-selective neurons in the basolateral amygdala, recorded in cocaine-treated rats, failed to change cue preference during reversal learning. Further, the presence of these neurons was critical to the expression of the reversal-learning deficit in the cocaine-treated rats.

Associative encoding in posterior piriform cortex during odor discrimination and reversal learning

Recent proposals have conceptualized piriform cortex as an association cortex, capable of integrating incoming olfactory information with descending input from higher order associative regions such as orbitofrontal cortex and basolateral amygdala (ABL). If true, encoding in piriform cortex should reflect associative features prominent in these areas during associative learning involving olfactory cues. We recently reported that neurons in anterior piriform cortex (APC) in rats exhibited significant plasticity in their responses to odor cues during associative learning. Here, we have repeated this study, recording from neurons in posterior piriform cortex (PPC), a region of piriform cortex that receives much stronger input from ABL. If associative encoding in piriform cortex is driven by inputs from ABL, then we should see more plasticity in PPC neurons than we observed in APC. Consistent with this hypothesis, we found that PPC neurons were highly associative and appeared to be somewhat more likely than neurons recorded in APC to alter their responses to the odor cues after reversal of the odor-outcome associations in the task. Further, odor-selective PPC populations exhibited markedly different firing patterns based on the valence of the odor cue. These results suggest associative encoding in piriform cortex is represented in a topographical fashion, reflecting the stronger and more specific input from olfactory bulb concerning the sensory features of odors in anterior regions and stronger input from ABL concerning the meaning of odors in posterior regions.

Should I stay or should I go? Transformation of time-discounted rewards in orbitofrontal cortex and associated brain circuits

Animals prefer a small, immediate reward over a larger delayed reward (time discounting). Lesions of the orbitofrontal cortex (OFC) can either increase or decrease the breakpoint at which animals abandon the large delayed reward for the more immediate reward as the delay becomes longer. Here we argue that the varied effects of OFC lesions on delayed discounting reflect two different patterns of activity in OFC; one that bridges the gap between a response and an outcome and another that discounts delayed reward. These signals appear to reflect the spatial location of the reward and/or the action taken to obtain it, and are encoded independently from representations of absolute value. We suggest a dual role for output from OFC in both discounting delayed reward, while at the same time supporting new learning for them.

Withdrawal from cocaine self-administration produces long-lasting deficits in orbitofrontal-dependent reversal learning in rats

Drug addicts make poor decisions. These decision-making deficits have been modeled in addicts and laboratory animals using reversal-learning tasks. However, persistent reversal-learning impairments have been shown in rats and monkeys only after noncontingent cocaine injections. Current thinking holds that to represent the human condition effectively, animal models of addiction must utilize self-administration procedures in which drug is earned contingently; thus, it remains unclear whether reversal-learning deficits caused by noncontingent cocaine exposure are relevant to addiction. To test whether reversal learning deficits are caused by contingent cocaine exposure, we trained rats to self-administer cocaine, assessed cue-induced cocaine seeking in extinction tests after 1 and 30 d of withdrawal, and then tested for reversal learning more than a month later. We found robust time-dependent increases in cue-induced cocaine seeking in the two extinction tests (incubation of craving) and severe reversal-learning impairments.