Evidence of structure and persistence in motivational attraction to serial Pavlovian cues

Sex-biased effects of outcome devaluation by sensory-specific satiety on Pavlovian-conditioned behavior

Goal-directed behavior relies on accurate mental representations of the value of expected outcomes. Disruptions to this process are a central feature of several neuropsychiatric disorders, including addiction. Goal-directed behavior is most frequently studied using instrumental paradigms paired with outcome devaluation, but cue-evoked behaviors in Pavlovian settings can also be goal-directed and therefore sensitive to changes in outcome value. Emerging literature suggests that male and female rats may differ in the degree to which their Pavlovian-conditioned responses are goal-directed, but interpretation of these findings is complicated by the tendency of female and male rats to engage in distinct types of Pavlovian responses when trained with localizable cues. Here, we used outcome devaluation via sensory-specific satiety to assess the behavioral responses in male and female Long Evans rats trained to respond to an auditory CS (conditioned stimulus) in a Pavlovian-conditioning paradigm. We found that satiety-induced devaluation led to a decrease in behavioral responding to the reward-predictive CS, with males showing an effect on both port entry latency and probability and females showing an effect only on port entry probability. Overall, our results suggest that outcome devaluation affects Pavlovian-conditioned responses in both male and female rats, but that females may be less sensitive to outcome devaluation.

Motivated to time: Effects of reinforcer devaluation and opportunity cost on interval timing

Prior research suggests that interval timing performance is sensitive to reinforcer devaluation effects and to the rate of competing sources of reinforcement. The present study sought to replicate and account for these findings in rats. A self-paced concurrent fixed-interval (FI) random-ratio (RR) schedule of reinforcement was implemented in which the FI requirement varied across training conditions (12, 24, 48 s). The RR requirement-which imposed an opportunity cost to responding on the FI component-was adjusted so that it took about twice the FI requirement, on average, to complete it. Probe reinforcer devaluation (prefeeding) sessions were conducted at the end of each condition. To assess the effect of contextual reinforcement on timing performance, the RR requirement was removed before the end of the experiment. Consistent with prior findings, performance on the FI component tracked schedule requirement and displayed scalar invariance; the removal of the RR component yielded more premature FI responses. For some rats, prefeeding reduced the number of trials initiated without affecting timing performance; for other rats, prefeeding delayed responding on the FI component but had a weaker effect on trial initiation. These results support the notion that timing and motivational processes are separable, suggesting novel explanations for ostensible motivational effects on timing performance.

Nucleus accumbens core acetylcholine receptors modulate the balance of flexible and inflexible cue-directed motivation

Sign-tracking is a conditioned response where animals interact with reward-predictive cues due to the cues having motivational value, or incentive salience. The nucleus accumbens core (NAc) has been implicated in mediating the sign-tracking response. Additionally, acetylcholine (ACh) transmission throughout the striatum has been attributed to both incentive motivation and behavioral flexibility. Here, we demonstrate a role for NAc ACh receptors in the flexibility of sign-tracking. Sign-tracking animals were exposed to an omission contingency, in which vigorous sign-tracking was punished by reward omission. Animals rapidly adjusted their behavior, but they maintained sign-tracking in a less vigorous manner that did not cancel reward. Within this context of sign-tracking being persistent yet flexible in structure, blockade of NAc nicotinic receptors (nAChRs) led to a persistence in the initial sign-tracking response during omission followed by a period of change in the makeup of sign-tracking, whereas blockade of muscarinic receptors (mAChRs) oppositely enhanced the omission-related development of the new sign-tracking behaviors. Later, once omission learning had occurred, nAChR blockade uniquely led to reduced sign-tracking and elevated reward-directed behaviors instead. These results indicate that NAc ACh receptors have opposing roles in maintaining learned patterns of sign-tracking, with nAChRs having a special involvement in regulating the structure of the sign-tracking response.

Outcome devaluation by sensory-specific satiety alters Pavlovian-conditioned behavior in male and female rats

Goal-directed behavior relies on accurate mental representations of the value of expected outcomes. Disruptions to this process are a central feature of several neuropsychiatric disorders, including addiction. Goal-directed behavior is most frequently studied using instrumental paradigms paired with outcome devaluation, but cue-evoked behaviors in Pavlovian settings can also be goal-directed and therefore sensitive to changes in outcome value. Emerging literature suggests that male and female rats may differ in the degree to which their Pavlovian-conditioned responses are goal-directed, but interpretation of these findings is complicated by the tendency of female and male rats to engage in distinct types of Pavlovian responses when trained with localizable cues. Here, we used outcome devaluation via sensory-specific satiety to assess the behavioral responses in male and female Long Evans rats trained to respond to an auditory CS (conditioned stimulus) in a Pavlovian-conditioning paradigm. We found that satiety-induced devaluation led to a decrease in behavioral responding to the reward-predictive CS, with males showing an effect on both port entry latency and probability and females showing an effect only on port entry probability. Overall, our results suggest that outcome devaluation affects Pavlovian-conditioned responses in both male and female rats, but that females may be less sensitive to outcome devaluation.

Dopamine in the dorsal bed nucleus of stria terminalis signals Pavlovian sign-tracking and reward violations

Midbrain and striatal dopamine signals have been extremely well characterized over the past several decades, yet novel dopamine signals and functions in reward learning and motivation continue to emerge. A similar characterization of real-time sub-second dopamine signals in areas outside of the striatum has been limited. Recent advances in fluorescent sensor technology and fiber photometry permit the measurement of dopamine binding correlates, which can divulge basic functions of dopamine signaling in non-striatal dopamine terminal regions, like the dorsal bed nucleus of the stria terminalis (dBNST). Here, we record GRABDA signals in the dBNST during a Pavlovian lever autoshaping task. We observe greater Pavlovian cue-evoked dBNST GRABDA signals in sign-tracking (ST) compared to goal-tracking/intermediate (GT/INT) rats and the magnitude of cue-evoked dBNST GRABDA signals decreases immediately following reinforcer-specific satiety. When we deliver unexpected rewards or omit expected rewards, we find that dBNST dopamine signals encode bidirectional reward prediction errors in GT/INT rats, but only positive prediction errors in ST rats. Since sign- and goal-tracking approach strategies are associated with distinct drug relapse vulnerabilities, we examined the effects of experimenter-administered fentanyl on dBNST dopamine associative encoding. Systemic fentanyl injections do not disrupt cue discrimination but generally potentiate dBNST dopamine signals. These results reveal multiple dBNST dopamine correlates of learning and motivation that depend on the Pavlovian approach strategy employed.

Inactivation of the Basolateral Amygdala to Insular Cortex Pathway Makes Sign-Tracking Sensitive to Outcome Devaluation

Goal-tracking (GT) rats are sensitive to Pavlovian outcome devaluation while sign-tracking (ST) rats are devaluation insensitive. During outcome devaluation, GT rats flexibly modify responding to cues based on the current value of the associated outcome. However, ST rats rigidly respond to cues regardless of the current outcome value. Prior work demonstrated disconnection of the basolateral amygdala (BLA) and anterior insular cortex (aIC) decreased both GT and ST behaviors. Given the role of these regions in appetitive motivation and behavioral flexibility, we predicted that disrupting BLA to aIC pathway during outcome devaluation would reduce flexibility in GT rats and reduce rigid appetitive motivation in ST rats. We inhibited the BLA to aIC pathway by infusing inhibitory DREADDs (hM4Di-mcherry) or control (mCherry) virus into the BLA and implanted cannulae into the aIC to inhibit BLA terminals using intracranial injections of clozapine N-oxide (CNO). After training, we used a within-subject satiety-induced outcome devaluation procedure in which we sated rats on training pellets (devalued condition) or homecage chow (valued condition). All rats received bilateral CNO infusions into the aIC before brief nonreinforced test sessions. Contrary to our hypothesis, BLA-IC inhibition did not interfere with devaluation sensitivity in GT rats but did make ST behaviors sensitive to devaluation. Intermediate rats showed the opposite effect, showing rigid responding to cues with BLA-aIC pathway inactivation. Together, these results demonstrate BLA-IC projections mediate tracking-specific Pavlovian devaluation sensitivity and highlights the importance of considering individual differences in Pavlovian approach when evaluating circuitry contributions to behavioral flexibility.

Sign Tracking in an Enriched Environment: A Potential Ecologically Relevant Animal Model of Adaptive Behavior Change

When an object conditioned stimulus (CS) is paired with a food unconditioned stimulus (US), anticipatory goal-directed action directed at the US location (goal tracking) is accompanied by behavior directed at the object CS (sign tracking). Sign-tracking behavior appears to be compulsive and habit-like and predicts increased vulnerability to the addictive potential of drugs in animal models. A large body of the literature also suggests that environmental enrichment protects against the development of addiction-prone phenotypes. Thus, we investigated whether compulsive-like sign tracking develops in environmentally enriched rats trained directly in their enriched home environment. We demonstrate that adolescent enriched-housed male Sprague-Dawley rats readily sign track a 5% ethanol bottle CS in their home environment and at a rate higher than adolescent standard-housed rats. We also show that enriched adolescent rats sign track less than enriched adult-trained rats and that acute isolation stress affects sign- and goal-tracking performance of adolescents and adults differently. Sign tracking increased more in the adult than the adolescent rats. Whereas the younger rats showed a decrease in goal tacking after the final stressor manipulation, the adults showed increased goal tracking. Our results are consistent with recent studies, which suggest that although sign tracking performance is compulsive-like, it is not as inflexible and habit-like as previously assumed. Testing in an enriched home environment with object CSs having greater affordance than "neutral" lever CSs may provide greater ecological relevance for investigating the development and expression of adaptive and compulsive-like behaviors in translational research.

Spatially restricted inhibition of cholinergic interneurons in the dorsolateral striatum encourages behavioral exploration

When pursuing desirable outcomes, one must make the decision between exploring possible actions to obtain those outcomes and exploiting known strategies to maximize efficiency. The dorsolateral striatum (DLS) has been extensively studied with respect to how actions can develop into habits and has also been implicated as an area involved in governing exploitative behavior. Surprisingly, prior work has shown that DLS cholinergic interneurons (ChIs) are not involved in the canonical habit formation function ascribed to the DLS but are instead modulators of behavioral flexibility after initial learning. To further probe this, we evaluated the role of DLS ChIs in behavioral exploration during a brief instrumental training experiment. Through designer receptors exclusively activated by designer drugs (DREADDs) in ChAT-Cre rats, ChIs in the DLS were inhibited during specific phases of the experiment: instrumental training, free-reward delivery, at both times, or never. Without ChI activity during instrumental training, animals biased their responding toward an "optimal" strategy while continuing to work efficiently. This effect was observed again when contingencies were removed as animals with ChIs offline during that phase, regardless of ChI inhibition previously, decreased responding more than animals with ChIs intact. These findings build upon a growing body of literature implicating ChIs in the striatum as gate-keepers of behavioral flexibility and exploration.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Monitoring in emotion regulation: behavioral decisions and neural consequences

Monitoring and deciding how to adjust an active regulatory strategy in order to maximize adaptive outcomes is an integral element of emotion regulation, yet existing evidence remains scarce. Filling this gap, the present study examined core factors that determine behavioral regulatory monitoring decisions and the neuro-affective consequences of these decisions. Using a novel paradigm, the initial implementation of central downregulation strategies (distraction, reappraisal) and the emotional intensity (high, low) were manipulated, prior to making a behavioral decision to maintain the initial implemented strategy or switch from it. Neuro-affective consequences of these behavioral decisions were evaluated using the Late Positive Potential (LPP), an electro-cortical measure of regulatory success. Confirming predictions, initial implementation of reappraisal in high intensity and distraction in low intensity (Strategy × Intensity combinations that were established in prior studies as non-preferred by individuals), resulted in increased behavioral switching frequency. Neurally, we expected and found that in high (but not low) emotional intensity, where distraction was more effective than reappraisal, maintaining distraction (relative to switching to reappraisal) and switching to distraction (relative to maintaining reappraisal) resulted in larger LPP modulation. These findings suggest that monitoring decisions are consistent with previously established regulatory preferences and are associated with adaptive short-term neural consequences.

From sign-tracking to attentional bias: Implications for gambling and substance use disorders

Sign-tracking behavior in Pavlovian autoshaping is known to be a relevant index of the incentive salience attributed to reward-related cues. Evidence has accumulated to suggest that animals that exhibit a sign-tracker phenotype are especially vulnerable to addiction and relapse due to their proneness to attribute incentive salience to drug cues, and their relatively weak cognitive and attentional control over their behavior. Interestingly, sign-tracking is also influenced by reward uncertainty in a way that may promote gambling disorder. Research indicates that reward uncertainty sensitizes sign-tracking responses and favors the development of a sign-tracker phenotype, compatible with the conditioned attractiveness of lights and sounds in casinos for problem gamblers. The study of attentional biases in humans (an effect akin to sign-tracking in animals) leads to similar observations, notably that the propensity to develop attraction for conditioned stimuli (CSs) is predictive of addictive behavior. Here we review the literature on drug addiction and gambling disorder, highlighting the similarities between studies of sign-tracking and attentional biases.

Sign-tracking behavior is sensitive to outcome devaluation in a devaluation context-dependent manner: implications for analyzing habitual behavior

Motivationally attractive cues can draw in behavior in a phenomenon termed incentive salience. Incentive cue attraction is an important model for animal models of drug seeking and relapse. One question of interest is the extent to which the pursuit of motivationally attractive cues is related to the value of the paired outcome or can become unrelated and habitual. We studied this question using a sign-tracking (ST) paradigm in rats, in which a lever stimulus preceding food reward comes to elicit conditioned lever-interaction behavior. We asked whether reinforcer devaluation by means of conditioned taste aversion, a classic test of habitual behavior, can modify ST to incentive cues, and whether this depends upon the manner in which reinforcer devaluation takes place. In contrast to several recent reports, we conclude that ST is indeed sensitive to reinforcer devaluation. However, this effect depends critically upon the congruence between the context in which taste aversion is learned and the context in which it is tested. When the taste aversion successfully transfers to the testing context, outcome value strongly influences ST behavior, both when the outcome is withheld (in extinction) and when animals can learn from outcome feedback (reacquisition). When taste aversion does not transfer to the testing context, ST remains high. In total, the extent to which ST persists after outcome devaluation is closely related to the extent to which that outcome is truly devalued in the task context. We believe this effect of context on devaluation can reconcile contradictory findings about the flexibility/inflexibility of ST. We discuss this literature and relate our findings to the study of habits generally.

Effects of Limited and Extended Pavlovian Training on Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Individual differences in Pavlovian approach predict differences in devaluation sensitivity. Recent studies indicate goal-tracking (GT) rats are sensitive to outcome devaluation while sign-tracking (ST) rats are not. With extended training in Pavlovian lever autoshaping (PLA), GT rats display more lever-directed behavior, typical of ST rats, suggesting they may become insensitive to devaluation with more Pavlovian training experience. Here, we use a within-subject satiety-induced outcome devaluation procedure to test devaluation sensitivity after limited and extended PLA training in GT and ST rats. We trained rats in PLA to determine GT and ST groups. Then, we sated rats on either the training pellets (devalued condition) or homecage chow (valued condition) prior to brief non-reinforced test sessions after limited (sessions 5/6) and extended (sessions 17/18) PLA training. GT rats decreased conditioned responding under devalued relative to valued conditions after both limited and extended training, demonstrating they are sensitive to satiety devaluation regardless of the amount of PLA training. While ST rats were insensitive to satiety devaluation after limited training, their lever directed behavior became devaluation sensitive after extended training. To determine whether sign-tracking rats also displayed sensitivity to illness-induced outcome devaluation after extended training, we trained a separate cohort of rats in extended PLA and devalued the outcome with lithium chloride injections after pellet consumption in the homecage. ST rats failed to decrease conditioned responding after illness-induced outcome devaluation, while Non-ST rats (GT and intermediates) were sensitive to illness-induced outcome devaluation after extended training. Together, our results confirm devaluation sensitivity is stable in GT rats across training and devaluation approaches. Extended training unmasks devaluation sensitivity in ST rats after satiety, but not illness-induced devaluation, suggesting ST rats respond appropriately by decreasing responding to cues during state-dependent but not inference-based devaluation. The differences in behavioral flexibility across tracking groups and devaluation paradigms have translational relevance for the understanding state- vs. inference-based reward devaluation as it pertains to drug addiction vulnerability.

Increased Goal Tracking in Adolescent Rats Is Goal-Directed and Not Habit-Like

When a cue is paired with reward in a different location, some animals will approach the site of reward during the cue, a behavior called goal tracking, while other animals will approach and interact with the cue itself: a behavior called sign tracking. Sign tracking is thought to reflect a tendency to transfer incentive salience from the reward to the cue. Adolescence is a time of heightened sensitivity to rewards, including environmental cues that have been associated with rewards, which may account for increased impulsivity and vulnerability to drug abuse. Surprisingly, however, studies have shown that adolescents are actually less likely to interact with the cue (i.e., sign track) than adult animals. We reasoned that adolescents might show decreased sign tracking, accompanied by increased apparent goal tracking, because they tend to attribute incentive salience to a more reward-proximal "cue": the food magazine. On the other hand, adolescence is also a time of enhanced exploratory behavior, novelty-seeking, and behavioral flexibility. Therefore, adolescents might truly express more goal-directed reward-seeking and less inflexible habit-like approach to a reward-associated cue. Using a reward devaluation procedure to distinguish between these two hypotheses, we found that adolescents indeed exhibit more goal tracking, and less sign tracking, than a comparable group of adults. Moreover, adolescents' goal tracking behavior is highly sensitive to reward devaluation and therefore goal-directed and not habit-like.

Circuit directionality for motivation: Lateral accumbens-pallidum, but not pallidum-accumbens, connections regulate motivational attraction to reward cues

Sign-tracking behavior, in which animals interact with a cue that predicts reward, provides an example of how incentive salience can be attributed to cues and elicit motivation. The nucleus accumbens (NAc) and ventral pallidum (VP) are two regions involved in cue-driven motivation. The VP, and NAc subregions including the medial shell and core, are critical for sign-tracking. Further, connections between the medial shell and VP are known to participate in sign-tracking and other motivated behaviors. The NAc lateral shell (NAcLSh) is a distinct and understudied subdivision of the NAc, and its contribution to the process by which reward cues acquire value remains unclear. The NAcLSh has been implicated in reward-directed behavior, and has reciprocal connections with the VP, suggesting that NAcLSh and VP interactions could be important mechanisms for incentive salience. Here, we use DREADDs (Designer Receptors Exclusively Activated by Designer Drugs) and an intersectional viral delivery strategy to produce a biased inhibition of NAcLSh neurons projecting to the VP, and vice versa. We find that disruption of connections from NAcLSh to VP reduces sign-tracking behavior while not affecting consumption of food rewards. In contrast, VP to NAcLSh disruption affected neither sign-tracking nor reward consumption, but did produce a greater shift in animals' behavior more towards the reward source when it was available. These findings indicate that the NAcLSh → VP pathway plays an important role in guiding animals towards reward cues, while VP → NAcLSh back-projections may not and may instead bias motivated behavior towards rewards.

Cannabinoid receptor-1 signaling contributions to sign-tracking and conditioned reinforcement in rats

RATIONALE

Endocannabinoids (eCBs) are critical gatekeepers of dopaminergic signaling, and disrupting cannabinoid receptor-1 (CB1) signaling alters DA dynamics to attenuate cue-motivated behaviors. Prior studies suggest that dopamine (DA) release plays a critical role in driving sign-tracking.

OBJECTIVES

Here, we determine whether systemic injections of rimonabant, a CB1 receptor inverse agonist, during Pavlovian lever autoshaping impair the expression of sign-tracking. We next examine whether rimonabant blocks the reinforcing properties of the Pavlovian lever cue in a conditioned reinforcement test.

METHODS

In Exp. 1, we trained rats in Pavlovian lever autoshaping prior to systemic rimonabant injections (0, 1, 3 mg/kg) during early and late Pavlovian lever autoshaping sessions. In Exp. 2, we trained rats in Pavlovian lever autoshaping prior to systemic rimonabant injections (0, 1 mg/kg) during a conditioned reinforcement test.

RESULTS

Rimonabant dose-dependently decreased lever contact and probability, and increased sign-tracker's latency to approach the lever cue early in Pavlovian training. With extended training, many previously goal-tracking and intermediate rats shifted to lever approach, which remained dose-dependently sensitive to rimonabant. Rimonabant attenuated cue-evoked food cup approach early, but not late, in conditioning, and did not affect pellet retrieval or consumption. The inserted lever cue served as a robust conditioned reinforcer after Pavlovian lever autoshaping, and 1 mg/kg rimonabant blocked conditioned reinforcement.

CONCLUSIONS

Together, our results suggest that CB1 signaling mediates two critical properties of incentive stimuli; their ability to attract (Exp. 1) and their ability to reinforce (Exp. 2) behavior.