Cannabinoid receptor-1 signaling contributions to sign-tracking and conditioned reinforcement in rats

Dorsomedial Striatum CB1R signaling is required for Pavlovian outcome devaluation in male Long Evans rats and reduces inhibitory synaptic transmission in both sexes

Cannabinoid-1 receptor (CB1R) signaling in the dorsal striatum regulates the shift from flexible to habitual behavior in instrumental outcome devaluation. Based on prior work establishing individual, sex, and experience-dependent differences in Pavlovian behaviors, we predicted a role for dorsomedial striatum CB1R signaling in driving rigid responding in Pavlovian autoshaping and outcome devaluation. We trained male and female Long Evans rats in Pavlovian Lever Autoshaping (PLA). We gave intra-dorsomedial striatum (DMS) infusions of the CB1R inverse agonist, rimonabant, before satiety-induced outcome devaluation test sessions, where we sated rats on training pellets or home cage chow and tested them in brief nonreinforced Pavlovian Lever Autoshaping sessions. Overall, inhibition of DMS CB1R signaling prevented Pavlovian outcome devaluation but did not affect behavior in reinforced PLA sessions. Males were sensitive to devaluation while females were not and DMS CB1R inhibition impaired devaluation sensitivity in males. We then investigated how DMS CB1R signaling impacts local inhibitory synaptic transmission in male and female Long Evans rats. We recorded spontaneous inhibitory postsynaptic currents (sIPSC) from DMS neurons at baseline and before and after application of a CB1R agonist, WIN 55,212-2. We found that male rats showed decreased sIPSC frequency compared to females, and that CB1R activation reduced DMS inhibitory transmission independent of sex. Altogether our results demonstrate that DMS CB1Rs regulate Pavlovian devaluation sensitivity and inhibitory synaptic transmission and suggest that basal sex differences in inhibitory synaptic transmission may underly sex differences in DMS function and behavioral flexibility.

The role of sex on sign-tracking acquisition and outcome devaluation sensitivity in Long Evans rats

Cues that predict rewards can trigger reward-seeking behaviors but also can, in some cases, become targets of motivation themselves. One behavioral phenomenon that captures this idea is sign-tracking in which animals, including humans, interact with reward-predictive cues even though it is not necessary to do so. Sign-tracking in rats has been studied in the domain of motivation and in how motivated behaviors can or cannot become excessive and habit-like over time. Many prior studies look at sign-tracking examine this behavior in male subjects, but there are few papers that look at this behavior in female subjects. Moreover, it is unknown where there might be sex-related variation in how flexible sign-tracking is when faced with changing reward values. Therefore, we asked if there were sex differences in the acquisition of sign-tracking behavior and if there were any sex differences in how sensitive animals were in their sign-tracking following reward devaluation. In contrast to previous reports, we found that males and females show no differences in how they acquire sign-tracking and in ultimate sign-tracking levels following training. Additionally, we found no difference in how quickly males and females learned to devalue the food reward, and we found no differences in sign-tracking levels by sex following outcome devaluation. We believe that this is primarily due to our experiment being performed in the Long Evans strain but also believe that there are many other factors contributing to differences between our study and previous work.

Decreased Ventral Tegmental Area CB1R Signaling Reduces Sign Tracking and Shifts Cue-Outcome Dynamics in Rat Nucleus Accumbens

Sign-tracking (ST) rats show enhanced cue sensitivity before drug experience that predicts greater discrete cue-induced drug seeking compared with goal-tracking or intermediate rats. Cue-evoked dopamine in the nucleus accumbens (NAc) is a neurobiological signature of sign-tracking behaviors. Here, we examine a critical regulator of the dopamine system, endocannabinoids, which bind the cannabinoid receptor-1 (CB1R) in the ventral tegmental area (VTA) to control cue-evoked striatal dopamine levels. We use cell type-specific optogenetics, intra-VTA pharmacology, and fiber photometry to test the hypothesis that VTA CB1R receptor signaling regulates NAc dopamine levels to control sign tracking. We trained male and female rats in a Pavlovian lever autoshaping (PLA) task to determine their tracking groups before testing the effect of VTA → NAc dopamine inhibition. We found that this circuit is critical for mediating the vigor of the ST response. Upstream of this circuit, intra-VTA infusions of rimonabant, a CB1R inverse agonist, during PLA decrease lever and increase food cup approach in sign-trackers. Using fiber photometry to measure fluorescent signals from a dopamine sensor, GRABDA (AAV9-hSyn-DA2m), we tested the effects of intra-VTA rimonabant on NAc dopamine dynamics during autoshaping in female rats. We found that intra-VTA rimonabant decreased sign-tracking behaviors, which was associated with increases in NAc shell, but not core, dopamine levels during reward delivery [unconditioned stimulus (US)]. Our results suggest that CB1R signaling in the VTA influences the balance between the conditioned stimulus-evoked and US-evoked dopamine responses in the NAc shell and biases behavioral responding to cues in sign-tracking rats.SIGNIFICANCE STATEMENT Substance use disorder (SUD) is a chronically relapsing psychological disorder that affects a subset of individuals who engage in drug use. Recent research suggests that there are individual behavioral and neurobiological differences before drug experience that predict SUD and relapse vulnerabilities. Here, we investigate how midbrain endocannabinoids regulate a brain pathway that is exclusively involved in driving cue-motivated behaviors of sign-tracking rats. This work contributes to our mechanistic understanding of individual vulnerabilities to cue-triggered natural reward seeking that have relevance for drug-motivated behaviors.

Dopamine in the dorsal bed nucleus of stria terminalis signals Pavlovian sign-tracking and reward violations

Midbrain and striatal dopamine signals have been extremely well characterized over the past several decades, yet novel dopamine signals and functions in reward learning and motivation continue to emerge. A similar characterization of real-time sub-second dopamine signals in areas outside of the striatum has been limited. Recent advances in fluorescent sensor technology and fiber photometry permit the measurement of dopamine binding correlates, which can divulge basic functions of dopamine signaling in non-striatal dopamine terminal regions, like the dorsal bed nucleus of the stria terminalis (dBNST). Here, we record GRABDA signals in the dBNST during a Pavlovian lever autoshaping task. We observe greater Pavlovian cue-evoked dBNST GRABDA signals in sign-tracking (ST) compared to goal-tracking/intermediate (GT/INT) rats and the magnitude of cue-evoked dBNST GRABDA signals decreases immediately following reinforcer-specific satiety. When we deliver unexpected rewards or omit expected rewards, we find that dBNST dopamine signals encode bidirectional reward prediction errors in GT/INT rats, but only positive prediction errors in ST rats. Since sign- and goal-tracking approach strategies are associated with distinct drug relapse vulnerabilities, we examined the effects of experimenter-administered fentanyl on dBNST dopamine associative encoding. Systemic fentanyl injections do not disrupt cue discrimination but generally potentiate dBNST dopamine signals. These results reveal multiple dBNST dopamine correlates of learning and motivation that depend on the Pavlovian approach strategy employed.

Discordant Effects of Cannabinoid 2 Receptor Antagonism/Inverse Agonism During Adolescence on Pavlovian and Instrumental Reward Learning in Adult Male Rats

The endocannabinoid system is responsible for regulating a spectrum of physiological activities and plays a critical role in the developing brain. During adolescence, the endocannabinoid system is particularly sensitive to external insults that may change the brain’s developmental trajectory. Cannabinoid receptor type 2 (CB2R) was initially thought to predominantly function in the peripheral nervous system, but more recent studies have implicated its role in the mesolimbic pathway, a network largely attributed to reward circuitry and reward motivated behavior, which undergoes extensive changes during adolescence. It is therefore important to understand how CB2R modulation during adolescence can impact reward-related behaviors in adulthood. In this study, adolescent male rats (postnatal days 28–41) were exposed to a low or high dose of the CB2R antagonist/inverse agonist SR144528 and Pavlovian autoshaping and instrumental conditional behavioral outcomes were measured in adulthood. SR144528-treated rats had significantly slower acquisition of the autoshaping task, seen by less lever pressing behavior over time [F_{(2, 19)} = 5.964, p = 0.010]. Conversely, there was no effect of adolescent SR144528 exposure on instrumental conditioning. These results suggest that modulation of the CB2R in adolescence differentially impacts reward-learning behaviors in adulthood.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

Orexin-A/Hypocretin-1 Controls the VTA-NAc Mesolimbic Pathway via Endocannabinoid-Mediated Disinhibition of Dopaminergic Neurons in Obese Mice

Disinhibition of orexin-A/hypocretin-1 (OX-A) release occurs to several output areas of the lateral hypothalamus (LH) in the brain of leptin knockout obese ob/ob mice. In this study, we have investigated whether a similar increase of OX-A release occurs to the ventral tegmental area (VTA), an orexinergic LH output area with functional effects on dopaminergic signaling at the mesolimbic circuit. By confocal and correlative light and electron microscopy (CLEM) morphological studies coupled to molecular, biochemical, and pharmacological approaches, we investigated OX-A-mediated dopaminergic signaling at the LH-VTA-nucleus accumbens (NAc) pathway in obese ob/ob mice compared to wild-type (wt) lean littermates. We found an elevation of OX-A trafficking and release to the VTA of ob/ob mice and consequent orexin receptor-1 (OX1R)-mediated over-activation of dopaminergic (DA) neurons via phospholipase C (PLC)/diacylglycerol lipase (DAGL-α)-induced biosynthesis of the endocannabinoid 2-arachidonoylglycerol (2-AG). In fact, by retrograde signaling to cannabinoid receptor type 1 (CB1R) at inhibitory inputs to DA neurons, 2-AG inhibited GABA release thus inducing an increase in DA concentration in the VTA and NAc of ob/ob mice. This effect was prevented by the OX1R antagonist SB-334867 (30 mg/Kg, i.p.), or the CB1R antagonist AM251 (10 mg/Kg, i.p.) and mimicked by OX-A injection (40 μg/Kg, i.p.) in wt lean mice. Enhanced DA signaling to the NAc in ob/ob mice, or in OX-A-injected wt mice, was accompanied by β-arrestin2-mediated desensitization of dopamine D2 receptor (D2R) in a manner prevented by SB-334867 or the D2R antagonist L741 (1.5 mg/Kg, i.p.). These results further support the role of OX-A signaling in the control of neuroadaptive responses, such as compulsive reward-seeking behavior or binge-like consumption of high palatable food, and suggest that aberrant OX-A trafficking to the DA neurons in the VTA of ob/ob mice influences the D2R response at NAc, a main target area of the mesolimbic pathway, via 2-AG/CB1-mediated retrograde signaling.

Behavioral determinants in the expression of the Kamin blocking effect: Implications for associative learning theory

Associative learning makes important contributions to our behavior and decisions. The Kamin blocking effect is an associative learning phenomenon that plays a central role in understanding of the psychological principles underlying associative learning. However, several recent failures to replicate the blocking effect suggest that the conditions necessary for blocking are poorly understood. To understand the conditions necessary for blocking, here we review studies into the expression of blocking in subjects that either approach and interact with the conditioned cue (sign trackers) or approach and interact with the reward location (goal trackers) during appetitive classical conditioning. Psychological theory and the neurophysiological correlates of appetitive classical conditioning make opposing predictions regarding the expression of blocking in sign and goal trackers. We reconcile these opposing predictions in a qualitative model using two parallel learning processes. Such models offer a better framework for understanding the psychological associative structures acquired during learning, their interactions contributing to the conditioned response, and how they affect subsequent learning and the expression of the Kamin blocking effect.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Effects of the cannabinoid receptor agonist CP-55,940 on incentive salience attribution

RATIONALE

Pavlovian conditioned approach paradigms are used to characterize the nature of motivational behaviors in response to stimuli as either directed toward the cue (i.e., sign-tracking) or the site of reward delivery (i.e., goal-tracking). Recent evidence has shown that activity of the endocannabinoid system increases dopaminergic activity in the mesocorticolimbic system, and other studies have shown that sign-tracking behaviors are dependent on dopamine.

OBJECTIVES

Therefore, we hypothesized that administration of a cannabinoid agonist would increase sign-tracking and decrease goal-tracking behaviors.

METHODS

Forty-seven adult male Sprague-Dawley rats were given a low, medium, or high dose of the cannabinoid agonist CP-55,940 (N = 12 per group) or saline (N = 11) before Pavlovian conditioned approach training. A separate group of rats (N = 32) were sacrificed after PCA training for measurement of cannabinoid receptor type 1 (CB1) and fatty acid amide hydrolase (FAAH) using in situ hybridization.

RESULTS

Contrary to our initial hypothesis, CP-55,940 dose-dependently decreased sign-tracking and increased goal-tracking behavior. CB1 expression was higher in sign-trackers compared with that in goal-trackers in the prelimbic cortex, but there were no significant differences in CB1 or FAAH expression in the infralimbic cortex, dorsal or ventral CA1, dorsal or ventral CA3, dorsal or ventral dentate gyrus, or amygdala.

CONCLUSIONS

These results demonstrate that cannabinoid signaling can specifically influence behavioral biases toward sign- or goal-tracking. Pre-existing differences in CB1 expression patterns, particularly in the prelimbic cortex, could contribute to individual differences in the tendency to attribute incentive salience to reward cues.

Effects of Limited and Extended Pavlovian Training on Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Individual differences in Pavlovian approach predict differences in devaluation sensitivity. Recent studies indicate goal-tracking (GT) rats are sensitive to outcome devaluation while sign-tracking (ST) rats are not. With extended training in Pavlovian lever autoshaping (PLA), GT rats display more lever-directed behavior, typical of ST rats, suggesting they may become insensitive to devaluation with more Pavlovian training experience. Here, we use a within-subject satiety-induced outcome devaluation procedure to test devaluation sensitivity after limited and extended PLA training in GT and ST rats. We trained rats in PLA to determine GT and ST groups. Then, we sated rats on either the training pellets (devalued condition) or homecage chow (valued condition) prior to brief non-reinforced test sessions after limited (sessions 5/6) and extended (sessions 17/18) PLA training. GT rats decreased conditioned responding under devalued relative to valued conditions after both limited and extended training, demonstrating they are sensitive to satiety devaluation regardless of the amount of PLA training. While ST rats were insensitive to satiety devaluation after limited training, their lever directed behavior became devaluation sensitive after extended training. To determine whether sign-tracking rats also displayed sensitivity to illness-induced outcome devaluation after extended training, we trained a separate cohort of rats in extended PLA and devalued the outcome with lithium chloride injections after pellet consumption in the homecage. ST rats failed to decrease conditioned responding after illness-induced outcome devaluation, while Non-ST rats (GT and intermediates) were sensitive to illness-induced outcome devaluation after extended training. Together, our results confirm devaluation sensitivity is stable in GT rats across training and devaluation approaches. Extended training unmasks devaluation sensitivity in ST rats after satiety, but not illness-induced devaluation, suggesting ST rats respond appropriately by decreasing responding to cues during state-dependent but not inference-based devaluation. The differences in behavioral flexibility across tracking groups and devaluation paradigms have translational relevance for the understanding state- vs. inference-based reward devaluation as it pertains to drug addiction vulnerability.

Stability of individual differences in sucralose taste preference

Outbred rats display variable preferences for bittersweet solutions, expressed as preference or avoidance of high concentrations of artificial sweeteners over water. This may reflect individual differences in appetitive/aversive conflict processing that may have predictive validity for disorders of motivation. Here we use a homecage two-bottle choice procedure to examine the test/retest stability and between-tastant consistency in sucralose preference to determine the reliability of bittersweet taste preference. Sucralose is a non-caloric artificial sweetener that is preferred by some rats and avoided by others. We sought to determine whether sucralose preference is consistent with preference of sucrose/quinine solutions that have known sweet and bitter taste qualities, respectively. We give fluid restricted rats 45-minutes homecage access to water and ascending concentrations of sucralose (SUCRA; 0.0025-10mM) or a compound solution of sucrose (116mM) + quinine (0.002-2mM) (SQ). We use a within-subject counterbalanced design (SUCRA or SQ testing) to determine preference of each bittersweet solution relative to water. We observed individual variability in preference for SUCRA and SQ, such that some rats preferred bittersweet solutions over water (preferring) while other rats preferred water over bittersweet solutions (avoiding). Within tastant, this preference remained stable across repeated testing. Between solutions, SUCRA preference scores correlated with SQ scores, suggesting consistent taste conflict processing for both bittersweet solutions. Population level analyses confirmed that preference generalizes across bittersweet solutions, and that rats' preferences for bittersweet solutions relative to water are stable over time. The test/retest and between-tastant reliability of this taste conflict screening procedure support the potential utility of this model for exploring individual variability in appetitive/aversive conflict processes mediating motivated behavior.