The form of a conditioned stimulus can influence the degree to which it acquires incentive motivational properties

Conditioning- and reward-related dendritic and presynaptic plasticity of nucleus accumbens neurons in male and female sign-tracker rats

For a subset of individuals known as sign-trackers, discrete Pavlovian cues associated with rewarding stimuli can acquire incentive properties and exert control over behaviour. Because responsiveness to cues is a feature of various neuropsychiatric conditions, rodent models of sign-tracking may prove useful for exploring the neurobiology of individual variation in psychiatric vulnerabilities. Converging evidence points towards the involvement of dopaminergic neurotransmission in the nucleus accumbens core (NAc) in the development of sign-tracking, yet whether this phenotype is associated with specific accumbal postsynaptic properties is unknown. Here, we examined dendritic spine structural organisation, as well as presynaptic and postsynaptic markers of activity, in the NAc core of male and female rats following a Pavlovian-conditioned approach procedure. In contrast to our prediction that cue re-exposure would increase spine density, experiencing the discrete lever-cue without reward delivery resulted in lower spine density than control rats for which the lever was unpaired with reward during training; this effect was tempered in the most robust sign-trackers. Interestingly, this same behavioural test (lever presentation without reward) resulted in increased levels of a marker of presynaptic activity (synaptophysin), and this effect was greatest in female rats. Whilst some behavioural differences were observed in females during initial Pavlovian training, final conditioning scores did not differ from males and were unaffected by the oestrous cycle. This work provides novel insights into how conditioning impacts the neuronal plasticity of the NAc core, whilst highlighting the importance of studying the behaviour and neurobiology of both male and female rats.

Ventral pallidum neurons projecting to the ventral tegmental area reinforce but do not invigorate reward-seeking behavior

Reward-predictive cues acquire motivating and reinforcing properties that contribute to the escalation and relapse of drug use in addiction. The ventral pallidum (VP) and ventral tegmental area (VTA) are two key nodes in brain reward circuitry implicated in addiction and cue-driven behavior. In the current study, we use in vivo fiber photometry and optogenetics to record from and manipulate VP→VTA in rats performing a discriminative stimulus task to determine the role these neurons play in invigoration and reinforcement by reward cues. We find that VP→VTA neurons are active during reward consumption and that optogenetic stimulation of these neurons biases choice behavior and is reinforcing. Critically, we find no encoding of reward-seeking vigor, and optogenetic stimulation does not enhance the probability or vigor of reward seeking in response to cues. Our results suggest that VP→VTA activity is more important for reinforcement than for invigoration of reward seeking by cues.

Effects of predictive and incentive value manipulation on sign- and goal-tracking behavior

When a neutral stimulus is repeatedly paired with an appetitive reward, two different types of conditioned approach responses may develop: a sign-tracking response directed toward the neutral cue, or a goal-tracking response directed toward the location of impending reward delivery. Sign-tracking responses have been postulated to result from attribution of incentive value to conditioned cues, while goal-tracking reflects the assignment of only predictive value to the cue. We therefore hypothesized that sign-tracking rats would be more sensitive to manipulations of incentive value, while goal-tracking rats would be more responsive to changes in the predictive value of the cue. We tested sign- and goal-tracking before and after devaluation of a food reward using lithium chloride, and tested whether either response could be learned under negative contingency conditions that precluded any serendipitous reinforcement of the behavior that might support instrumental learning. We also tested the effects of blocking the predictive value of a cue using simultaneous presentation of a pre-conditioned cue. We found that sign-tracking was sensitive to outcome devaluation, while goal-tracking was not. We also confirmed that both responses are Pavlovian because they can be learned under negative contingency conditions. Goal-tracking was almost completely blocked by a pre-conditioned cue, while sign-tracking was much less sensitive to such interference. These results indicate that sign- and goal-tracking may follow different rules of reinforcement learning and suggest a need to revise current models of associative learning to account for these differences.

A mechanical task for measuring sign- and goal-tracking in humans: A proof-of-concept study

Cue-based associative learning (i.e., Pavlovian conditioning) is a foundational component of behavior in almost all forms of animal life and may provide insight into individual differences in addiction liability. Cues can take on incentive-motivational properties (i.e., incentive salience) through Pavlovian learning. Extensive testing with non-human animals (primarily rats) has demonstrated significant variation among individuals in the behaviors this type of learning evokes. So-named "sign-trackers" and "goal-trackers" have been examined in many studies of non-human animals, but this work in humans is still a nascent area of research. In the present proof-of-concept study, we used a Pavlovian conditioned approach task to investigate human sign- and goal-tracking in emerging adults. Conditioned behaviors that developed over the course of the task were directed toward the reward-cue and toward the reward location. Participants' eye-gaze and behavior during the task were submitted to a latent profile analysis, which revealed three groups defined as sign-trackers (n = 10), goal-trackers (n = 4), and intermediate responders (n = 36). Impulsivity was a significant predictor of the sign-tracking group relative to the goal-tracking group. The present study provides preliminary evidence that a simple procedure can produce learned Pavlovian conditioned approach behavior in humans. Though further investigation is required, findings provide a promising step toward the long-term goal of translating important insights gleaned from basic research into treatment strategies that can be applied to clinical populations.

Genetic characterization of outbred Sprague Dawley rats and utility for genome-wide association studies

Sprague Dawley (SD) rats are among the most widely used outbred laboratory rat populations. Despite this, the genetic characteristics of SD rats have not been clearly described, and SD rats are rarely used for experiments aimed at exploring genotype-phenotype relationships. In order to use SD rats to perform a genome-wide association study (GWAS), we collected behavioral data from 4,625 SD rats that were predominantly obtained from two commercial vendors, Charles River Laboratories and Harlan Sprague Dawley Inc. Using double-digest genotyping-by-sequencing (ddGBS), we obtained dense, high-quality genotypes at 291,438 SNPs across 4,061 rats. This genetic data allowed us to characterize the variation present in Charles River vs. Harlan SD rats. We found that the two populations are highly diverged (F_ST > 0.4). Furthermore, even for rats obtained from the same vendor, there was strong population structure across breeding facilities and even between rooms at the same facility. We performed multiple separate GWAS by fitting a linear mixed model that accounted for population structure and using meta-analysis to jointly analyze all cohorts. Our study examined Pavlovian conditioned approach (PavCA) behavior, which assesses the propensity for rats to attribute incentive salience to reward-associated cues. We identified 46 significant associations for the various metrics used to define PavCA. The surprising degree of population structure among SD rats from different sources has important implications for their use in both genetic and non-genetic studies.

Outbred Sprague Dawley rats are among the most commonly used rats for neuroscience, physiology and pharmacological research; in the year 2020, 4,188 publications contained the keyword “Sprague Dawley”. Rats identified as “Sprague Dawley” are sold by several commercial vendors, including Charles River Laboratories and Harlan Sprague Dawley Inc. (now Envigo). Despite their widespread use, little is known about the genetic diversity of SD. We genotyped more than 4,000 SD rats, which we used for a genome-wide association study (GWAS) and to characterize genetic differences between SD rats from Charles River Laboratories and Harlan. Our analysis revealed extensive population structure both between and within vendors. The GWAS for Pavlovian conditioned approach (PavCA) identified a number of genome-wide significant loci for that complex behavioral trait. Our results demonstrate that, despite sharing an identical name, SD rats that are obtained from different vendors are very different. Future studies should carefully define the exact source of SD rats being used and may exploit their genetic diversity for genetic studies of complex traits.

Cocaine cue-induced mesocorticolimbic activation in cocaine users: Effects of personality traits, lifetime drug use, and acute stimulant ingestion

Stimulant drug-paired cues can acquire the ability to activate mesocorticolimbic pathways and lead to new bouts of drug use. Studies in laboratory animals suggest that these effects are augmented by progressively greater drug use histories, impulsive personality traits, and acute drug ingestion. As a preliminary test of these hypotheses in humans, we exposed cocaine users (n = 14) and healthy volunteers (n = 10) to cocaine-related videos during two functional magnetic resonance imaging (fMRI) sessions, once following acute administration of placebo and once following d-amphetamine (0.3 mg/kg, p.o.). Across sessions, cocaine users showed larger cocaine cue-induced responses than healthy controls in the associative striatum and midbrain. Among the cocaine users, larger drug cue-induced responses during the placebo session were correlated with higher Barratt Impulsiveness Scale (BIS-11) nonplanning scores (associative striatum) and greater lifetime use of stimulant drugs (limbic, associative, and sensorimotor striatum). The administration of d-amphetamine did not augment the cue-induced activations, but, in cocaine users, drug cue-induced striatal activations were more widespread following prolonged cocaine cue exposure. Together, these effects of past and present drug use might aggravate the risk for stimulant drug use problems.

Quantifying the instrumental and noninstrumental underpinnings of Pavlovian responding with the Price equation

The Price equation is a mathematical expression of selectionist and non-selectionist pressures on biological, cultural, and behavioral change. We use it here to specify instrumental and noninstrumental behaviors as they arise within the context of the Pavlovian autoshaping procedure, for rats trained under reward certainty and reward uncertainty. The point of departure for this endeavor is that some portion of autoshaped behavior referred to as goal-tracking appears instrumental-a function of resource attainment (the individual approaches the location where the unconditioned stimulus is to be delivered). By contrast, some other portion of autoshaped behavior referred to as sign-tracking is noninstrumental-irrelevant to making contact with the to-be-delivered unconditioned stimulus. A Price equation model is proposed that unifies our understanding of Pavlovian autoshaping behavior by isolating operant and respondent influences on goal-tracking (instrumental) and sign-tracking (noninstrumental) behavior.

Nicotine Enhances Goal-Tracking in Ethanol and Food Pavlovian Conditioned Approach Paradigms

Rationale

Nicotine promotes alcohol intake through pharmacological and behavioral interactions. As an example of the latter, nicotine can facilitate approach toward food- and alcohol-associated stimuli ("sign-tracking") in lever-Pavlovian conditioned approach (PavCA) paradigms. However, we recently reported that nicotine can also enhance approach toward locations of reward delivery ("goal-tracking") triggered by ethanol-predictive stimuli when the location of ethanol delivery is non-static (i.e., a retractable sipper bottle).

Objective

To determine whether the non-static nature of the reward location could have biased the development of goal-tracking in our previous study (Loney et al., 2019); we assessed the effect of nicotine in a lever-PavCA paradigm wherein the location of ethanol delivery was static (i.e., a stationary liquid receptacle). Then, to determine whether nicotine's enhancement of goal-tracking is unique to ethanol-predictive stimuli, we assessed the effect of systemic nicotine on approach triggered by food-predictive stimuli in a lever-PavCA paradigm.

Methods

Long-Evans rats were used in two PavCA experiments wherein a lever predicted the receipt of ethanol (15% vol/vol; experiment 1) or food (experiment 2) into a stationary receptacle. Prior to testing, rats were administered nicotine (0.4 mg/kg subcutaneously) or saline systemically.

Results

In both experiments, nicotine increased measures of goal-tracking, but not sign-tracking.

Conclusion

Nicotine can facilitate approach to reward locations without facilitating approach to reward-predictive stimuli. As such, conceptualization of the mechanisms by which nicotine affects behavior must be expanded to explain an enhancement of goal-tracking by nicotine.

The pharmacological stressor yohimbine, but not U50,488, increases responding for conditioned reinforcers paired with ethanol or sucrose

RATIONALE

Environmental stimuli paired with alcohol can function as conditioned reinforcers (CRfs) and trigger relapse to alcohol-seeking. In animal models, pharmacological stressors can enhance alcohol consumption and reinstate alcohol-seeking, but it is unknown whether stress can potentiate the conditioned reinforcing properties of alcohol-paired stimuli.

OBJECTIVES

We examined whether the pharmacological stressors, the α-2 adrenoreceptor antagonist yohimbine (vehicle, 1.25, 2.5 mg/kg; IP) and the K-opioid receptor agonist U50,488 (vehicle, 1.25, 2.5 mg/kg; SC), increase responding for conditioned reinforcement, and if their effects interact with the nature of the reward (alcohol vs. sucrose). We subsequently examined the effects of yohimbine (vehicle, 1.25, 2.5 mg/kg; IP) on responding for sensory reinforcement.

METHODS

Male Long-Evans underwent Pavlovian conditioning, wherein a tone-light conditioned stimulus (CS) was repeatedly paired with 12% EtOH or 21.7% sucrose. Next, tests of responding for a CRf were conducted. Responding on the CRf lever delivered the CS, whereas responding on the other lever had no consequences. In a separate cohort of rats, the effects of yohimbine on responding just for the sensory reinforcer were examined.

RESULTS

Both doses of yohimbine, but not U50,488, increased responding for conditioned reinforcement. This enhancement occurred independently of the nature of the reward used during Pavlovian conditioning. Yohimbine-enhanced responding for a CRf was reproducible and stable over five tests; it even persisted when the CS was omitted. Both doses of yohimbine also increased responding for sensory reinforcement.

CONCLUSIONS

Yohimbine increases operant responding for a variety of sensory and conditioned reinforcers. This enhancement may be independent of its stress-like effects.

Rat Paraventricular Neurons Encode Predictive and Incentive Information of Reward Cues

The paraventricular nucleus of the thalamus (PVT) has been implicated in cue-induced motivated behaviors. Although reward-associated cues (conditioned stimuli, CSs) contain different types of information including predictive information of future reward delivery and incentive (motivational) value of the reward, it remains unknown whether PVT neurons represent predictive and incentive information of CSs. It is suggested that neural activity just after the onset of CSs (early activity) and that just before reward delivery (late activity) might more strongly represent predictive and incentive information, respectively. In this study, rats were trained to lick a tube, which was protruded close to their mouth just after a CS, to obtain a reward (sucrose or water) (cue-induced licking task). Auditory and visual CSs were used: each elemental cue (CS) predicted reward or non-reward outcome, while simultaneous presentation of the two elemental cues (configural cues) predicted the opposite reward outcome. We recorded PVT neurons in the cue-induced licking task, and report that half of the CS-responsive PVT neurons responded selectively to the CSs predicting reward outcome regardless of physical property of the cues (CS+-selective). In addition, the early activity of the CS+-selective neurons discriminated reward/non-reward association (predictive information) and was less sensitive to reward value and motivation reflected by lick latency (incentive information), while the late activity of the CS+-selective neurons was correlated with reward value and motivation rather than reward/non-reward association. Early and late population activity of the CS+-selective neurons also represented predictive and incentive information of the CSs, respectively. On the other hand, activity of more than half of the PVT neurons was correlated with individual licking during licking to acquire reward. Taken together, the results suggest that the PVT neurons engage in different neural processes involved in cue-induced motivated behaviors: CS encoding to determine reward availability and form motivation for reward-seeking behavior, and hedonic mouth movements during reward consumption.

Suboptimal choice by pigeons is eliminated when key-pecking behavior is replaced by treadle-pressing

In the study of suboptimal choice, a reliable result is that pigeons strongly prefer an alternative that signals whether a reinforcer will be delivered or not over another alternative without that information even if the first provides a lower probability of reinforcement. In the aforementioned research, key pecking has been the operant response and illuminated keys the discriminative stimuli. In the present study we modified both of these aspects of the procedure in order to analyze the generality of suboptimal preferences of pigeons and to investigate the effect of changes in the incentive salience of the discriminative stimuli. To accomplish this, we presented pigeons a choice situation with the same parameters of reinforcement than previous research, but with treadle pressing as the choice response and ambient lights as discriminative stimuli. Under these conditions, most of the pigeons showed optimal behavior and a high degree of discrimination of the stimuli associated with the discriminative alternative. A control condition with key pecking as choice response and keylights as discriminative stimuli showed that the same pigeons turned to be suboptimal, a result that discards the possibility that the optimality found in the main condition was a consequence of a particular characteristic of our sample of subjects or of our procedure. We discuss the influence that the attribution of incentive salience to the discriminative stimuli has on suboptimal choice in both pigeons and rats.

From sign-tracking to attentional bias: Implications for gambling and substance use disorders

Sign-tracking behavior in Pavlovian autoshaping is known to be a relevant index of the incentive salience attributed to reward-related cues. Evidence has accumulated to suggest that animals that exhibit a sign-tracker phenotype are especially vulnerable to addiction and relapse due to their proneness to attribute incentive salience to drug cues, and their relatively weak cognitive and attentional control over their behavior. Interestingly, sign-tracking is also influenced by reward uncertainty in a way that may promote gambling disorder. Research indicates that reward uncertainty sensitizes sign-tracking responses and favors the development of a sign-tracker phenotype, compatible with the conditioned attractiveness of lights and sounds in casinos for problem gamblers. The study of attentional biases in humans (an effect akin to sign-tracking in animals) leads to similar observations, notably that the propensity to develop attraction for conditioned stimuli (CSs) is predictive of addictive behavior. Here we review the literature on drug addiction and gambling disorder, highlighting the similarities between studies of sign-tracking and attentional biases.

Mapping sign-tracking and goal-tracking onto human behaviors

As evidenced through classic Pavlovian learning mechanisms, environmental cues can become incentivized and influence behavior. These stimulus-outcome associations are relevant in everyday life but may be particularly important for the development of impulse control disorders including addiction. Rodent studies have elucidated specific learning profiles termed 'sign-tracking' and 'goal-tracking' which map onto individual differences in impulsivity and other behaviors associated with impulse control disorders' etiology, course, and relapse. Whereas goal-trackers are biased toward the outcome, sign-trackers fixate on features that are associated with but not necessary for achieving an outcome; a pattern of behavior that often leads to escalation of reward-seeking that can be maladaptive. The vast majority of the sign- and goal-tracking research has been conducted using rodent models and very few have bridged this concept into the domain of human behavior. In this review, we discuss the attributes of sign- and goal-tracking profiles, how these are manifested neurobiologically, and how these distinct learning styles could be an important tool for clinical interventions in human addiction.

Reward uncertainty attributes incentive value to reward proximal cues, while amphetamine sensitization reverts attention to more predictive reward distal cues

Slot-machine gambling incorporates numerous audiovisual cues prior to and during reward delivery (e.g. spinning wheels, flashing lights, celebratory sounds). Over time, these cues may motivate playing and even elicit cravings and relapse in those affected by gambling disorder. Animal studies suggest a heightened attraction to these cues despite diminished predictive ability under reward uncertainty, as evidenced by sign-tracking behavior in rats. Repeated amphetamine administration may also enhance the incentive value attributed to cues. Here, we explored the impact of reward uncertainty and prior amphetamine sensitization on the relative attractiveness and conditioned reinforcing properties of serial Pavlovian cues with different degrees of predictive and incentive value in rats. Animals were sensitized through repeated injections of amphetamine (1-4 mg/kg) or saline and then trained in a Pavlovian autoshaping task involving two sequential lever-auditory cue combinations (CS1, CS2) under Certain (100%-1) or Uncertain (50%-1-2-3) reward conditions. Subsequently, we evaluated the impact of acute amphetamine exposure on cue attraction. Our results suggest that Uncertainty alone enhanced attraction towards the reward-proximal CS2. However, combined sensitization and Uncertainty reversed cue preference relative to Uncertainty alone, enhancing attraction towards the more predictive reward-distal CS1. Both cues acquired conditioned reinforcing properties, despite the CS2 being otherwise ignored in all groups besides Uncertainty. However, combined sensitization and Uncertainty increased the reinforcing value of both cues and doubled the amount of interaction with the CS1 lever per presentation. Our results imply competitive mechanisms for attributing incentive value to gambling-related cues between reward uncertainty, prior amphetamine sensitization, and acute amphetamine administration.

Occasion setters attain incentive motivational value: implications for contextual influences on reward-seeking

The context in which reward-paired cues are encountered can resolve ambiguity and set the occasion for appropriate reward-seeking. The psychological processes by which contexts regulate reward-seeking remain unclear as contexts are diffuse and difficult to isolate from other stimuli. To overcome this, we modeled a context as a phasic and discrete event—an occasion setter (OS)—which allowed for control over its presentation and influence on cue-driven reward-seeking. This allowed us to directly assess how OSs regulate the predictive and motivational significance of Pavlovian cues. Male rats (n = 50) were trained in a Pavlovian procedure where the presentation of an ambiguous conditioned stimulus (CS) was reinforced only if preceded by an occasion setting cue. We assessed the motivational value of the OS and CS alone or in combination using tests of conditioned reinforcement. Rats enhanced conditioned approach to the reward port during the CS when it was preceded by the OS. When allowed the opportunity, rats responded more to obtain presentations of the CS in combination with the OS than the CS alone. Critically, rats also worked to obtain presentations of the OS alone more than the CS alone, and this was resistant to manipulations of the value of the OS. We conclude that occasion setting can act via incentive motivational mechanisms and that, apart from resolving predictive information about ambiguous reward-paired cues, OSs themselves generate states of appetitive motivation that can facilitate reward-seeking.

Nicotine Produces a High-Approach, Low-Avoidance Phenotype in Response to Alcohol-Associated Cues in Male Rats

BACKGROUND

Nicotine and alcohol use are highly comorbid. Modulation of drug-paired extrinsic and intrinsic cues likely plays a role in this interaction, as cues can acquire motivational properties and augment drug seeking. The motivational properties of cues can be measured through Pavlovian conditioning paradigms, in which cues either elicit approach following pairing with the reinforcing properties of alcohol or elicit avoidance following pairing with the aversive consequences of alcohol. The present experiments tested whether nicotine would enhance the incentive properties of an appetitive ethanol (EtOH) cue and diminish the avoidance of an aversive EtOH cue in Pavlovian paradigms.

METHODS

In experiment 1, male Long-Evans rats with or without prior chronic intermittent access to EtOH were administered nicotine or saline injections prior to Pavlovian conditioned approach (PavCA) sessions, during which conditioned approach to the cue ("sign-tracking") or the EtOH delivery location ("goal-tracking") was measured. In experiment 2, male Long-Evans rats were administered nicotine or saline injections prior to pairing a flavor cue with increasing doses of EtOH (i.p.) in an adaptation of the conditioned taste avoidance (CTA) paradigm.

RESULTS

Results from PavCA indicate that, regardless of EtOH exposure, nicotine enhanced responding elicited by EtOH-paired cues with no effect on a similar cue not explicitly paired with EtOH. Furthermore, nicotine reduced sensitivity to EtOH-induced CTA, as indicated by a rightward shift in the dose-response curve of passively administered EtOH. The ED₅₀ , or the dose of EtOH that produced a 50% reduction in intake relative to baseline, was significantly higher in nicotine-treated rats compared to saline-treated rats.

CONCLUSIONS

We conclude that nicotine increases the approach and diminishes the avoidance elicited by Pavlovian cues paired, respectively, with the reinforcing and aversive properties of EtOH consumption in male rats. As such, nicotine may enhance alcoholism liability by engendering an attentional bias toward cues that predict the reinforcing outcomes of drinking.

Incentive hope: A default psychological response to multiple forms of uncertainty

Our target article proposes that a new concept - incentive hope - is necessary in the behavioral sciences to explain animal foraging under harsh environmental conditions. Incentive hope refers to a specific motivational mechanism in the brain - considered only in mammals and birds. But it can also be understood at a functional level, as an adaptive behavioral strategy that contributes to improve survival. Thus, this concept is an attempt to bridge across different research fields such as behavioral psychology, reward neuroscience, and behavioral ecology. Many commentaries suggest that incentive hope even could help understand phenomena beyond these research fields, including food wasting and food sharing, mental energy conservation, diverse psychopathologies, irrational decisions in invertebrates, and some aspects of evolution by means of sexual selection. We are favorable to such extensions because incentive hope denotes an unconscious process capable of working against many forms of adversity; organisms do not need to hope as a subjective feeling, but to behave as if they had this feeling. In our response, we carefully discuss each suggestion and criticism and reiterate the importance of having a theory accounting for motivation under reward uncertainty.

Differential involvement of dopamine receptor subtypes in the acquisition of Pavlovian sign-tracking and goal-tracking responses

RATIONALE

Previous work has identified that different forms of Pavlovian conditioned approach, sign-tracking and goal-tracking, are governed by distinct neurochemical mechanisms when compared in animals predisposed to learning one form vs. the other.

OBJECTIVES

The present study aimed to investigate whether these are also neurochemically distinct processes in a population of animals capable of developing either response when this is manipulated via the use of distinct conditioned stimuli (CS).

METHODS

Rats were trained on one of two Pavlovian conditioning procedures in which the CS was either a lever, which elicits sign-tracking, or an auditory click, which elicits goal-tracking. The differential involvement of dopamine D1- and D2-receptors (D1R; D2R) in the acquisition of approach types was investigated via systemic administration of antagonists selective to one or both receptor subtypes during Pavlovian training.

RESULTS

Results indicate that dopaminergic signalling is important for the acquisition of both sign-tracking and goal-tracking responses. However, whilst development of sign-tracking to a lever depends on activity at both D1R and D2R, development of goal-tracking in response to a click was shown to depend only on activity at D1R.

CONCLUSIONS

We suggest that the importance of D1R activity in both sign- and goal-tracking acquisition reflects a general role in learning Pavlovian associations, which aligns with data implicating dopamine in prediction error processes. In contrast, the selective involvement of D2R activity in sign-tracking acquisition may reflect its importance in motivational processes such as incentive salience attribution.

Disrupting reconsolidation: memory erasure or blunting of emotional/motivational value?

When memories are retrieved they become labile, and subject to alteration by a process known as reconsolidation. Disruption of memory reconsolidation decreases the performance of learned responses, which is often attributed to erasure of the memory; in the case of Pavlovian learning, to a loss of the association between a conditioned stimulus (CS) and unconditioned stimulus (US). However, an alternative interpretation is that disrupting reconsolidation does not erase memories, but blunts their emotional/motivational impact. It is difficult to parse the predictive vs. emotional/motivational value of CSs in non-human animals, but studies on variation in the form of conditioned responses (CRs) in a Pavlovian conditioned approach task suggest a way to do this. In this task a lever-CS paired with a food reward (US) acquires predictive value in all rats, but is attributed with emotional/motivational value to a greater extent in some rats (sign-trackers) than others (goal-trackers). We report that the post-retrieval administration of propranolol selectively attenuates a sign-tracking CR, and the associated neural activation of brain "motive circuits", while having no effect on conditioned orienting behavior in sign-trackers, or on goal-tracking CRs evoked by either a lever-CS or a tone-CS. We conclude that the disruption of reconsolidation by post-retrieval propranolol degrades the emotional/motivational impact of the CS, required for sign-tracking, but leaves the CS-US association intact. The possibility that post-retrieval interventions can reduce the emotional/motivational aspects of memories, without actually erasing them, has important implications for treating maladaptive memories that contribute to some psychiatric disorders.

Addiction vulnerability trait impacts complex movement control: Evidence from sign-trackers

Cognitive-motivational vulnerability traits are associated with increased risk for substance addiction and relapse. Sign-tracking (ST) behavior in rats is associated with poor attentional control, mediated by an unresponsive basal forebrain cholinergic system, and an increased risk for substance addiction/relapse. A separate literature links poor attentional control and cholinergic losses to increased fall risk in Parkinson's disease. Here we tested the hypothesis that the relatively inferior attentional control of STs extends to complex movement control and a propensity for falls. STs were found to fall more often than goal-trackers (GTs) while traversing a straight rotating rod and, similar to human fallers, when taxed by a secondary task. Furthermore, STs fell more often while traversing a rotating zig-zag rod. GTs exhibited fewer falls from this rod by avoiding entry to the rotating zig-zag sections when in, or rotating toward, a difficult traversal state. Goal-tracking rats approached risky movement situations using strategies indicative of superior top-down control. These results suggest that the impact of opponent cognitive-cholinergic traits extends to complex movement control, and that impairments in the cognitive-motor interface are likely to be comorbid with addiction vulnerability. Sign-tracking indexes an endophenotype that may increase the risk for a wide range of neurobehavioral disorders.

The hot 'n' cold of cue-induced drug relapse

Environmental cues associated with rewards can acquire motivational properties. However, there is considerable variation in the extent to which a reward cue gains motivational control over behavior, depending on the individual and the form of the cue. When a discrete cue is paired with food reward, it acquires greater control over motivated behavior in some rats (sign-trackers, STs) than others (goal-trackers, GTs) as indicated by the propensity to approach the cue, the willingness to work to obtain it, and its ability to reinstate reward-seeking behavior. Here, we review studies that employ this ST/GT animal model to investigate characteristics of individuals that are especially susceptible to reward cue-elicited behavior and the involvement of dopamine and acetylcholine neuromodulator systems in the susceptibility to cue-induced drug relapse. First, we discuss individual differences in the attribution of incentive salience to different forms of reward cues and the involvement of the mesolimbic dopamine system. We then discuss individual differences in cognitive/attentional control and the contributions of the cholinergic system in processing reward cues. It is suggested that in STs a propensity to attribute motivational properties to a drug cue is combined with poor attentional control in the face of these cues, making them particularly vulnerable to transition from casual/experimental patterns of drug use to addiction and to cue-induced relapse.

A glucocorticoid receptor antagonist reduces sign-tracking behavior in male Japanese quail

Addiction is characterized as a chronic debilitating disease. One devastating feature of addiction is the susceptibility of relapse (40-60%) after stretches of abstinence. One theory that may account for relapse suggests that drug cues (e.g., paraphernalia) may increase stress hormones, and this may prompt relapse. Repeatedly pairing a neutral cue with a reward is commonly utilized to measure what subjects learn about a cue that is predictive of reward. Research has shown that animals that attend to a cue more than to the reward (sign trackers) may be more vulnerable to drug addiction. Additionally, research has shown that sign tracking is associated with an increase in corticosterone, a primary stress hormone. PT150 is a novel glucocorticoid receptor antagonist that moderates the release of corticosterone. In the current experiment, it was hypothesized that subjects given repeated administration of PT150 would reduce sign tracking compared to subjects given placebo. Time spent (in seconds) near a cue that predicts reward (conditional stimulus) served as a measure of sign tracking, and PT150 or placebo was administered following sign tracking. An independent-samples t test revealed that subjects that received PT150 had reduced time spent near the conditioned stimulus compared to controls. Given the devastating effects of drug addiction, identification of a potential pharmacological intervention in the reduction of relapse would be of great value. Therefore, future research is needed to validate the use of PT150 in reducing behaviors associated with drug addiction. (PsycINFO Database Record

Dynamic Encoding of Incentive Salience in the Ventral Pallidum: Dependence on the Form of the Reward Cue

Some rats are especially prone to attribute incentive salience to a cue (conditioned stimulus, CS) paired with food reward (sign-trackers, STs), but the extent they do so varies as a function of the form of the CS. Other rats respond primarily to the predictive value of a cue (goal-trackers, GTs), regardless of its form. Sign-tracking is associated with greater cue-induced activation of mesolimbic structures than goal-tracking; however, it is unclear how the form of the CS itself influences activity in neural systems involved in incentive salience attribution. Thus, our goal was to determine how different cue modalities affect neural activity in the ventral pallidum (VP), which is known to encode incentive salience attribution, as rats performed a two-CS Pavlovian conditioned approach task in which both a lever-CS and a tone-CS predicted identical food reward. The lever-CS elicited sign-tracking in some rats (STs) and goal-tracking in others (GTs), whereas the tone-CS elicited only goal-tracking in all rats. The lever-CS elicited robust changes in neural activity (sustained tonic increases or decreases in firing) throughout the VP in STs, relative to GTs. These changes were not seen when STs were exposed to the tone-CS, and in GTs there were no differences in firing between the lever-CS and tone-CS. We conclude that neural activity throughout the VP encodes incentive signals and is especially responsive when a cue is of a form that promotes the attribution of incentive salience to it, especially in predisposed individuals.

Individual differences in incentive salience attribution are not related to suboptimal choice in rats

Previous research has shown great variation in the extent to which individual rats attribute incentive salience to stimuli that are predictors of reinforcement. When using the Pavlovian Conditioned Approach procedure, in which a discrete stimulus is presented contingently before the delivery of reinforcement, the attribution of incentive salience is demonstrated by sign-tracking behavior (responses directed toward the stimulus predictor of reinforcement), while an absence of this attribution is reflected by goal-tracking behavior (responses directed toward the source of reinforcement). It has been reported that sign-tracking subjects have a higher tendency to perform some maladaptive behaviors than goal-tracking subjects, and that in non-classified rats, increasing the incentive salience of the stimuli promotes suboptimal choice in the "suboptimal choice procedure". In this task, subjects are presented with two alternatives, one of them better in terms of the information provided by the discriminative stimuli, but worse in terms of probability of reinforcement (suboptimal alternative). Integrating these ideas, we hypothesized that sign-trackers would behave suboptimally, in contrast to goal-trackers. In the present study, 45 rats were classified according to their performance in the Pavlovian Conditioned Approach procedure and subjects with extreme values (sign-trackers, and goal-trackers) were evaluated in the suboptimal choice procedure. Both groups were found to behave optimally, with no differences between them. The difference between groups in capacity of attribution of incentive salience was preserved during the entire experiment, suggesting that this variable is not related to choice performance in the suboptimal choice procedure.

How foraging works: Uncertainty magnifies food-seeking motivation

Food uncertainty has the effect of invigorating food-related responses. Psychologists have noted that mammals and birds respond more to a conditioned stimulus that unreliably predicts food delivery, and ecologists have shown that animals (especially small passerines) consume and/or hoard more food and can get fatter when access to that resource is unpredictable. Are these phenomena related? We think they are. Psychologists have proposed several mechanistic interpretations, while ecologists have suggested a functional interpretation: The effect of unpredictability on fat reserves and hoarding behavior is an evolutionary strategy acting against the risk of starvation when food is in short supply. Both perspectives are complementary, and we argue that the psychology of incentive motivational processes can shed some light on the causal mechanisms leading animals to seek and consume more food under uncertainty in the wild. Our theoretical approach is in agreement with neuroscientific data relating to the role of dopamine, a neurotransmitter strongly involved in incentive motivation, and its plausibility has received some explanatory and predictive value with respect to Pavlovian phenomena. Overall, we argue that the occasional and unavoidable absence of food rewards has motivational effects (called incentive hope) that facilitate foraging effort. We show that this hypothesis is computationally tenable, leading foragers in an unpredictable environment to consume more food items and to have higher long-term energy storage than foragers in a predictable environment.

Adrenergic manipulation inhibits pavlovian conditioned approach behaviors

Environmental rewards and Pavlovian reward cues can acquire incentive salience, thereby eliciting incentive motivational states and instigate reward-seeking. In rats, the incentive salience of food cues can be measured during a Pavlovian conditioned approach paradigm, in which rats engage in cue-directed approach ("sign-tracking") or approach the food delivery location ("goal-tracking"). While it has been shown that dopamine signaling is necessary for sign-tracking, some studies have suggested that norepinephrine is involved in learning to sign-track as well. Thus, in order to investigate the influence of norepinephrine in Pavlovian conditioned approach, we administered three adrenergic drugs while rats learned that a food cue (an illuminated, retractable lever) preceded the delivery of banana-flavored food pellets into a food-cup. We found that pre-session injections of disulfiram (a dopamine-β-hydroxylase inhibitor) inhibited the development of sign-tracking, but goal-tracking was only affected at the high dose. In one experiment, post-session injections of disulfiram blocked the development of sign-tracking, although this effect was not replicated in a separate set of rats. Post-session injections of prazosin (an α1-adrenergic receptor antagonist) and propranolol (a β-adrenergic receptor antagonist) also blocked the development of sign-tracking but not goal-tracking. Taken together, these results suggest that adrenergic transmission mediates the acquisition of sign-tracking but not goal-tracking, and thus plays a selective role in the attribution of incentive salience food cues.

'Hot' vs. 'cold' behavioural-cognitive styles: motivational-dopaminergic vs. cognitive-cholinergic processing of a Pavlovian cocaine cue in sign- and goal-tracking rats

Discrete Pavlovian reward cues acquire more potent incentive motivational properties (incentive salience) in some animals (sign-trackers; STs) compared to others (goal-trackers; GTs). Conversely, GTs appear to be better than STs in processing more complex contextual cues, perhaps reflecting their relatively greater bias for goal-directed cue processing. Here, we investigated the activity of two major prefrontal neuromodulatory input systems, dopamine (DA) and acetylcholine (ACh), in response to a discrete Pavlovian cue that was previously paired with cocaine administration in STs and GTs. Rats underwent Pavlovian training in which light cue presentations were either paired or unpaired with an intravenous cocaine infusion. Following a 10-day abstinence period, prefrontal dialysates were collected in STs and GTs during cue presentations in the absence of cocaine. In STs, the cue previously paired with cocaine significantly increased prefrontal DA levels. DA levels remained elevated over baseline across multiple cue presentation blocks, and DA levels and approaches to the cue were significantly correlated. In STs, ACh levels were unaffected by cue presentations. In contrast, in GTs, presentations of the cocaine cue increased prefrontal ACh, but not DA, levels. GTs oriented towards the cue at rates similar to STs, but they did not approach it and elevated ACh levels did not correlate with conditioned orientation. The results indicate a double dissociation between the role of prefrontal DA and ACh in STs and GTs, and suggest that these phenotypes will be useful for studying the role of neuromodulator systems in mediating opponent behavioural-cognitive styles.

NMDA receptor blockade specifically impedes the acquisition of incentive salience attribution

Glutamatergic signaling plays an important role in learning and memory. Using Pavlovian conditioned approach procedures, the mechanisms that drive stimulus-reward learning and memory have been investigated. However, there are instances where reward-predictive stimuli can function beyond being solely predictive and can be attributed with "motivational value" or incentive salience. Using a Pavlovian conditioned approach procedure consisting of two different but equally predictive stimuli (lever vs. tone) we investigated the role NMDA receptor function has in the attribution of incentive salience. The results revealed that the administration of MK-801, an NMDA receptor antagonist, during acquisition of Pavlovian conditioned approach promoted goal-tracking to a lever stimulus, while control animals learned to sign-track. Moreover, within the same animals, the use of a tone stimulus elicited goal-tracking responses that were unaffected by MK-801 pretreatments. Furthermore, a lever CS that elicited sign-tracking served as a more robust conditioned reinforcer than a tone CS that elicited goal-tracking or a lever CS that elicited goal-tracking via MK-801 pretreatments. Collectively, these results demonstrate that NMDA receptor antagonism can alter the stimulus-reward relationship learned and prevent the attribution of incentive salience, rather than impede general learning.

Long-lasting contribution of dopamine in the nucleus accumbens core, but not dorsal lateral striatum, to sign-tracking

The attribution of incentive salience to reward-paired cues is dependent on dopamine release in the nucleus accumbens core (NAcC). These dopamine signals conform to traditional reward-prediction error signals and have been shown to diminish with time. Here we examined whether the diminishing dopamine signal in the NAcC has functional implications for the expression of sign-tracking, a Pavlovian conditioned response indicative of the attribution of incentive salience to reward-paired cues. Food-restricted male Sprague Dawley rats were trained in a Pavlovian paradigm in which an insertable lever predicted delivery of food reward in a nearby food cup. After 7 or 14 training sessions, rats received infusions of saline, the dopamine antagonist flupenthixol, or the GABA agonists baclofen and muscimol into the NAcC or the dorsal lateral striatum (DLS). Dopamine antagonism within the NAcC attenuated sign-tracking, whereas reversible inactivation did not affect sign-tracking but increased non-specific food cup checking behaviors. Neither drug in the DLS affected sign-tracking behavior. Critically, extended training did not alter these effects. Although extended experience with an incentive stimulus may reduce cue-evoked dopamine in the NAcC, this does not remove the dependence on dopamine in this region to promote Pavlovian cue approach nor result in the recruitment of dorsal lateral striatal systems for this behavior. These data support the notion that dopamine within the mesoaccumbal system, but not the nigrostriatal system, contributes critically to incentive motivational processes independent of the length of training.

Diverse Roads to Relapse: A Discriminative Cue Signaling Cocaine Availability Is More Effective in Renewing Cocaine Seeking in Goal Trackers Than Sign Trackers and Depends on Basal Forebrain Cholinergic Activity

Stimuli associated with taking drugs are notorious instigators of relapse. There is, however, considerable variation in the motivational properties of such stimuli, both as a function of the individual and the nature of the stimulus. The behavior of some individuals (sign trackers, STs) is especially influenced by cues paired with reward delivery, perhaps because they are prone to process information via dopamine-dependent, cue-driven, incentive salience systems. Other individuals (goal trackers, GTs) are better able to incorporate higher-order contextual information, perhaps because of better executive/attentional control over behavior, which requires frontal cortical cholinergic activity. We hypothesized, therefore, that a cue that "sets the occasion" for drug taking (a discriminative stimulus, DS) would reinstate cocaine seeking more readily in GTs than STs and that this would require intact cholinergic neurotransmission. To test this, male STs and GTs were trained to self-administer cocaine using an intermittent access schedule with periods of cocaine availability and unavailability signaled by a DS+ and a DS-, respectively. Thereafter, half of the rats received an immunotoxic lesion that destroyed 40-50% of basal forebrain cholinergic neurons and later, after extinction training, were tested for the ability of noncontingent presentations of the DS+ to reinstate cocaine seeking behavior. The DS+ was much more effective in reinstating cocaine seeking in GTs than STs and this effect was abolished by cholinergic losses despite the fact that all rats continued to orient to the DS+ We conclude that vulnerability to relapse involves interactions between individual cognitive-motivational biases and the form of the drug cue encountered.SIGNIFICANCE STATEMENT The most predictable outcome of a diagnosis of addiction is a high chance for relapse. When addicts encounter cues previously associated with drug, their attention may be unduly attracted to such cues and these cues can evoke motivational states that instigate and maintain drug-seeking behavior. Although sign-tracking rats were previously demonstrated to exhibit greater relapse vulnerability to Pavlovian drug cues paired with drug delivery, here, we demonstrate that their counterparts, the goal trackers, are more vulnerable if the drug cue acts to signal drug availability and that the forebrain cholinergic system mediates such vulnerability. Given the importance of contextual cues for triggering relapse and the human cognitive-cholinergic capacity for the processing of such cues, goal trackers model essential aspects of relapse vulnerability.

Rats behave optimally in a sunk cost task

The sunk cost effect has been defined as the tendency to persist in an alternative once an investment of effort, time or money has been made, even if better options are available. The goal of this study was to investigate in rats the relationship between sunk cost and the information about when it is optimal to leave the situation, which was studied by Navarro and Fantino (2005) with pigeons. They developed a procedure in which different fixed-ratio schedules were randomly presented, with the richest one being more likely; subjects could persist in the trial until they obtained the reinforcer, or start a new trial in which the most favorable option would be available with a high probability. The information about the expected number of responses needed to obtain the reinforcer was manipulated through the presence or absence of discriminative stimuli; also, they used different combinations of schedule values and their probabilities of presentation to generate escape-optimal and persistence- optimal conditions. They found optimal behavior in the conditions with presence of discriminative stimuli, but non-optimal behavior when they were absent. Unlike their results, we found optimal behavior in both conditions regardless of the absence of discriminative stimuli; rats seemed to use the number of responses already emitted in the trial as a criterion to escape. In contrast to pigeons, rats behaved optimally and the sunk cost effect was not observed.

Suboptimal choice in rats: Incentive salience attribution promotes maladaptive decision-making

Stimuli that are more predictive of subsequent reward also function as better conditioned reinforcers. Moreover, stimuli attributed with incentive salience function as more robust conditioned reinforcers. Some theories have suggested that conditioned reinforcement plays an important role in promoting suboptimal choice behavior, like gambling. The present experiments examined how different stimuli, those attributed with incentive salience versus those without, can function in tandem with stimulus-reward predictive utility to promote maladaptive decision-making in rats. One group of rats had lights associated with goal-tracking as the reward-predictive stimuli and another had levers associated with sign-tracking as the reward-predictive stimuli. All rats were first trained on a choice procedure in which the expected value across both alternatives was equivalent but differed in their stimulus-reward predictive utility. Next, the expected value across both alternatives was systematically changed so that the alternative with greater stimulus-reward predictive utility was suboptimal in regard to primary reinforcement. The results demonstrate that in order to obtain suboptimal choice behavior, incentive salience alongside strong stimulus-reward predictive utility may be necessary; thus, maladaptive decision-making can be driven more by the value attributed to stimuli imbued with incentive salience that reliably predict a reward rather than the reward itself.

Individual differences in food cue responsivity are associated with acute and repeated cocaine-induced vocalizations, but not cue-induced vocalizations

RATIONALE

Individuals prone to attribute incentive salience to food-associated stimuli ("cues") are also more sensitive to cues during drug seeking and drug taking. This may be due in part to a difference in sensitivity to the affective or other stimulus properties of the drug. In rats, these properties are associated with 50-kHz ultrasonic vocalizations (USVs), in that they are elicited during putative positive affective and motivational states, including in response to drugs of abuse.

OBJECTIVES

We sought to determine whether individual differences in the tendency to attribute incentive salience to a food cue (as measured by approach) were associated with differences in cocaine-induced USVs. We also tested whether the food cue would elicit USVs and if this response was related to approach to the food cue.

METHODS

In experiment 1, rats underwent Pavlovian conditioned approach (PavCA) training where they learned to associate a cue (an illuminated lever) with the delivery of a food pellet into a food cup. Subjects were categorized based on their approach to the cue ("sign-trackers") or to the food cup ("goal-trackers"). Rats subsequently underwent nine testing days in which they were given saline or cocaine (10 mg/kg i.p) and placed into a locomotor chamber. In experiment 2, rats were first tested in the locomotor chambers for one saline-treated day followed by one cocaine-treated day and then trained in PavCA. USVs were recorded from a subset of individuals during the last day of PavCA to determine if the food cue would elicit USVs.

RESULTS

Sign-trackers produced 5-24 times more cocaine-induced 50 kHz USVs compared to goal-trackers for all days of experiment 1, and this response sensitized with repeated cocaine, only in sign-trackers. Similarly in experiment 2, individuals that produced the most cocaine-induced USVs on a single exposure also showed the greatest tendency to sign-track during PavCA. Lastly, while sign-trackers produced more USVs during PavCA generally, the cue itself did not elicit additional USVs in sign- or goal-trackers.

CONCLUSIONS

These results indicate a robust and consistent relationship between approach to a food cue and cocaine-induced USV production. Thus, these USVs may index the neurobiological differences underlying the behavioral distinctions of sign- and goal-trackers.

Neurobiological Basis of Individual Variation in Stimulus-Reward Learning

Cues in the environment can guide behavior in adaptive ways, leading one towards valuable resources such as food, water, or a potential mate. However, cues in the environment may also serve as powerful motivators that lead to maladaptive patterns of behavior, such as addiction. Importantly, and central to this article, there is considerable individual variation in the extent to which reward cues gain motivational control over behavior. Here we describe an animal model that captures this individual variation, allowing us to better understand the psychological and neurobiological processes that contribute to cue-evoked behaviors. When a discrete cue is paired with a food reward in a Pavlovian manner it acquires greater control over motivated behavior in some rats ("sign-trackers, STs) than in others ("goal-trackers", GTs). We review studies that have exploited this animal model to parse the neurobiological mechanisms involved in learning associations between stimuli vs. those involved in attributing incentive salience to those same stimuli. The latter seems to be dependent on dopamine and subcortical circuits, whereas the former may engage more cortical "top-down" mechanisms.

The ability for cocaine and cocaine-associated cues to compete for attention

In humans, reward cues, including drug cues in individuals experiencing addiction, are especially effective in biasing attention towards them, so much so they can disrupt ongoing task performance. It is not known, however, whether this happens in rats. To address this question, we developed a behavioral paradigm to assess the capacity of an auditory drug (cocaine) cue to evoke cocaine-seeking behavior, thus distracting thirsty rats from performing a well-learned sustained attention task (SAT) to obtain a water reward. First, it was determined that an auditory cocaine cue (tone-CS) reinstated drug-seeking equally in sign-trackers (STs) and goal-trackers (GTs), which otherwise vary in the propensity to attribute incentive salience to a localizable drug cue. Next, we tested the ability of an auditory cocaine cue to disrupt performance on the SAT in STs and GTs. Rats were trained to self-administer cocaine intravenously using an Intermittent Access self-administration procedure known to produce a progressive increase in motivation for cocaine, escalation of intake, and strong discriminative stimulus control over drug-seeking behavior. When presented alone, the auditory discriminative stimulus elicited cocaine-seeking behavior while rats were performing the SAT, but it was not sufficiently disruptive to impair SAT performance. In contrast, if cocaine was available in the presence of the cue, or when administered non-contingently, SAT performance was severely disrupted. We suggest that performance on a relatively automatic, stimulus-driven task, such as the basic version of the SAT used here, may be difficult to disrupt with a drug cue alone. A task that requires more top-down cognitive control may be needed.

The sensory features of a food cue influence its ability to act as an incentive stimulus and evoke dopamine release in the nucleus accumbens core

The sensory properties of a reward-paired cue (a conditioned stimulus; CS) may impact the motivational value attributed to the cue, and in turn influence the form of the conditioned response (CR) that develops. A cue with multiple sensory qualities, such as a moving lever-CS, may activate numerous neural pathways that process auditory and visual information, resulting in CRs that vary both within and between individuals. For example, CRs include approach to the lever-CS itself (rats that “sign-track”; ST), approach to the location of reward delivery (rats that “goal-track”; GT), or an “intermediate” combination of these behaviors. We found that the multimodal sensory features of the lever-CS were important to the development and expression of sign-tracking. When the lever-CS was covered, and thus could only be heard moving, STs not only continued to approach the lever location but also started to approach the food cup during the CS period. While still predictive of reward, the auditory component of the lever-CS was a much weaker conditioned reinforcer than the visible lever-CS. This plasticity in behavioral responding observed in STs closely resembled behaviors normally seen in rats classified as “intermediates.” Furthermore, the ability of both the lever-CS and the reward-delivery to evoke dopamine release in the nucleus accumbens was also altered by covering the lever—dopamine signaling in STs resembled neurotransmission observed in rats that normally only GT. These data suggest that while the visible lever-CS was attractive, wanted, and had incentive value for STs, when presented in isolation, the auditory component of the cue was simply predictive of reward, lacking incentive salience. Therefore, the specific sensory features of cues may differentially contribute to responding and ensure behavioral flexibility.

The tendency to sign-track predicts cue-induced reinstatement during nicotine self-administration, and is enhanced by nicotine but not ethanol

RATIONALE

Some individuals are particularly responsive to reward-associated stimuli ("cues"), including the effects of these cues on craving and relapse to drug-seeking behavior. In the cases of nicotine and alcohol, cues may acquire these abilities via the incentive-enhancing properties of the drug.

OBJECTIVES

To determine the interaction between cue-responsivity and nicotine reinforcement, we studied the patterns of nicotine self-administration in rats categorized based on their tendency to approach a food-predictive cue ("sign-trackers") or a reward-delivery location ("goal-trackers"). In a second experiment, we determined whether nicotine and ethanol altered the incentive value of a food cue.

METHODS

Rats were classified as sign- or goal-trackers during a Pavlovian conditioned approach paradigm. Rats then self-administered intravenous nicotine (0.03 mg/kg infusions) followed by extinction and cue-induced reinstatement tests. We also tested the effects of nicotine (0.4 mg/kg base s.c.) or ethanol (0.7 g/kg i.p.) on the approach to, and reinforcing efficacy of, a food cue.

RESULTS

Sign-trackers showed greater reinstatement in response to a nicotine cue. Further, nicotine enhanced sign-tracking but not goal-tracking to a food cue and also enhanced responding for the food cue during the conditioned reinforcement test. Conversely, ethanol reduced sign-tracking and increased goal-tracking, but had no effect on conditioned reinforcement.

CONCLUSIONS

Our studies demonstrate that the tendency to attribute incentive value to a food cue predicts enhanced cue-induced reinstatement. Additionally, the incentive value of food cues is differentially modulated by nicotine and ethanol, which may be related to the reinforcing effects of these drugs.

Dopamine antagonism does not impair learning of Pavlovian conditioned approach to manipulable or non-manipulable cues but biases responding towards goal tracking

Dopamine's (DA) role in reward-processing is currently discussed as either providing a teaching signal to guide learning or mediating the transfer of incentive salience (i.e. motivational aspects) from unconditioned stimuli (US) to conditioned stimuli (CS). We used a Pavlovian conditioned approach (PCA) procedure to further investigate DAs contribution to these processes. Experiment 1 assessed the acquisition of PCA to a manipulable lever cue for 7days under DA-blockade with Flupenthixol (FLU; 225μg/kg) or Saline (SAL) treatment, followed by 6-days off-drug testing. FLU decreased the number of conditioned responses (CR) during the treatment phase, but cessation of treatment resulted in an immediate increase in CR to levels comparable to SAL controls; notably, CR in FLU-treated rats were restricted to goal tracking behaviour. During continued off-drug testing, rats from the FLU group developed sign tracking with a similar temporal pattern as controls. In experiment 2, acquisition of PCA to a non-manipulable auditory cue was investigated. FLU reduced the number of CR during treatment, and removing DA antagonism resulted in a similar rapid increase of CR as seen in experiment 1. These data complement other reports by demonstrating that, independently from the physical properties of the CS, DA is not required for learning predictive aspects of a CS-US relationship but for the development of behaviour (namely sign tracking) which is based on the motivational aspects of a CS-US relationship.

Neural Activity in the Ventral Pallidum Encodes Variation in the Incentive Value of a Reward Cue

There is considerable individual variation in the extent to which reward cues are attributed with incentive salience. For example, a food-predictive conditioned stimulus (CS; an illuminated lever) becomes attractive, eliciting approach toward it only in some rats ("sign trackers," STs), whereas others ("goal trackers," GTs) approach the food cup during the CS period. The purpose of this study was to determine how individual differences in Pavlovian approach responses are represented in neural firing patterns in the major output structure of the mesolimbic system, the ventral pallidum (VP). Single-unit in vivo electrophysiology was used to record neural activity in the caudal VP during the performance of ST and GT conditioned responses. All rats showed neural responses to both cue onset and reward delivery but, during the CS period, STs showed greater neural activity than GTs both in terms of the percentage of responsive neurons and the magnitude of the change in neural activity. Furthermore, neural activity was positively correlated with the degree of attraction to the cue. Given that the CS had equal predictive value in STs and GTs, we conclude that neural activity in the VP largely reflects the degree to which the CS was attributed with incentive salience.

SIGNIFICANCE STATEMENT

Cues associated with reward can acquire motivational properties (i.e., incentive salience) that cause them to have a powerful influence on desire and motivated behavior. There are individual differences in sensitivity to reward-paired cues, with some individuals attaching greater motivational value to cues than others. Here, we investigated the neural activity associated with these individual differences in incentive salience. We found that cue-evoked neural firing in the ventral pallidum (VP) reflected the strength of incentive motivation, with the greatest neural responses occurring in individuals that demonstrated the strongest attraction to the cue. This suggests that the VP plays an important role in the process by which cues gain control over motivation and behavior.

Low Impulsive Action, but not Impulsive Choice, Predicts Greater Conditioned Reinforcer Salience and Augmented Nucleus Accumbens Dopamine Release

Poor impulse control is associated with an increased propensity to develop an addiction and may contribute to relapse as high impulsive subjects appear to attribute greater salience toward drug-paired stimuli. In these studies, we determined whether trait impulsivity also predicts the desire to obtain natural reward-paired stimuli. Rats trained on the 5-choice serial reaction time task to measure impulsive action (Experiment 1) or a delay-discounting task to measure impulsive choice (Experiment 2) were separated into low, intermediate, or high impulsive action (L-IA, I-IA, H-IA) or choice (L-IC, I-IC, H-IC) groups. The motivation to obtain a conditioned stimulus (CS) paired with water-reward was subsequently determined by measuring responding for the CS as a conditioned reinforcer (CRf). Dopamine release in the nucleus accumbens was also measured using in vivo microdialysis. The effects of amphetamine were assessed on all tests. In Experiment 1, amphetamine increased impulsive action in all groups. L-IA rats initially demonstrated the highest responding for the CRf. Amphetamine increased responding for the CRf and this effect was augmented in L-IA rats. Dopamine release following amphetamine was greatest in L-IA subjects. In Experiment 2, amphetamine increased impulsive choice for L-IC and I-IC rats. However, all groups responded similarly for the CRf and dopamine release was moderately greater in L-IC rats. In conclusion, impulsive choice was unrelated to responding for a CRf. L-IA subjects initially attributed enhanced salience to a CS and exhibited greater dopamine release. Lower dopamine release in H-IA rats could result in reduced reinforcing properties of the CRf.

Toward isolating the role of dopamine in the acquisition of incentive salience attribution

Stimulus-reward learning has been heavily linked to the reward-prediction error learning hypothesis and dopaminergic function. However, some evidence suggests dopaminergic function may not strictly underlie reward-prediction error learning, but may be specific to incentive salience attribution. Utilizing a Pavlovian conditioned approach procedure consisting of two stimuli that were equally reward-predictive (both undergoing reward-prediction error learning) but functionally distinct in regard to incentive salience (levers that elicited sign-tracking and tones that elicited goal-tracking), we tested the differential role of D1 and D2 dopamine receptors and nucleus accumbens dopamine in the acquisition of sign- and goal-tracking behavior and their associated conditioned reinforcing value within individuals. Overall, the results revealed that both D1 and D2 inhibition disrupted performance of sign- and goal-tracking. However, D1 inhibition specifically prevented the acquisition of sign-tracking to a lever, instead promoting goal-tracking and decreasing its conditioned reinforcing value, while neither D1 nor D2 signaling was required for goal-tracking in response to a tone. Likewise, nucleus accumbens dopaminergic lesions disrupted acquisition of sign-tracking to a lever, while leaving goal-tracking in response to a tone unaffected. Collectively, these results are the first evidence of an intraindividual dissociation of dopaminergic function in incentive salience attribution from reward-prediction error learning, indicating that incentive salience, reward-prediction error, and their associated dopaminergic signaling exist within individuals and are stimulus-specific. Thus, individual differences in incentive salience attribution may be reflective of a differential balance in dopaminergic function that may bias toward the attribution of incentive salience, relative to reward-prediction error learning only.

Elicited versus emitted behavior: Time to abandon the distinction

The concept of emitted behavior was formulated as a part of the original argument for the validity of a new kind of learning called operant conditioning. The rationale for operant conditioning contrasted it with Pavlovian or classical conditioning, which was (and remains) fundamentally based on responses to conditioned and unconditioned stimuli. Classical conditioned responses were said to be elicited. In contrast, operant behavior was viewed as emitted and controlled primarily by response consequences rather than antecedents. I argue that the distinction between emitted and elicited behavior is no longer warranted for three major reasons. First, the distinction was based on a view of Pavlovian conditioning that is no longer viable. Second, the distinction is incompatible with both empirical data and contemporary conceptualizations of operant behavior. Third, the only way to overcome these problems is to define emitted and elicited in terms of the type of conditioning (operant and classical) that produces these behaviors, but that approach makes the definitions circular and does not avoid implications of the terms that are misleading and counterproductive in light of contemporary research and thinking.

Pavlovian Conditioned Approach Training in Rats

Cues that are contingently paired with unconditioned, rewarding stimuli can acquire rewarding properties themselves through a process known as the attribution of incentive salience, or the transformation of neutral stimuli into attractive, "wanted' stimuli capable of motivating behavior. Pavlovian conditioned approach (PCA) develops after the response-independent presentation of a conditioned stimulus (CS; e.g., a lever) that predicts the delivery of an unconditioned stimulus (US; e.g., a food pellet) and can be used to measure incentive salience. During training, three patterns of conditioned responses (CRs) can develop: sign-tracking behavior (CS-directed CR), goal-tracking behavior (US-directed CR), and an intermediate response (both CRs). Sign-trackers attribute incentive salience to reward-related cues and are more vulnerable to cue-induced reinstatement of drug-seeking as well as other addiction-related behaviors, making PCA a potentially valuable procedure for studying addiction vulnerability. Here, we describe materials and methods used to elicit PCA behavior from rats as well as analyze and interpret PCA behavior in individual experiments.

Individual variation in the motivational properties of a nicotine cue: sign-trackers vs. goal-trackers

RATIONALE

Individuals vary in the extent to which they attribute incentive salience to reward cues. Discrete food and drug (cocaine and opioid) cues become more attractive, eliciting approach toward them, and more "wanted," in that they serve as more effective conditioned reinforcers, in some rats (sign-trackers, STs), than in others (goal-trackers, GTs).

OBJECTIVES

We asked whether there is similar variation in the extent to which a cue associated with a drug from another class, nicotine, acquires incentive motivational properties.

METHODS

First, a Pavlovian conditioned approach procedure was used to identify rats that attribute incentive salience to a food cue (i.e., STs and GTs). We then measured the extent to which a cue (a light) paired with intravenous nicotine injections acquired two properties of an incentive stimulus: (1) the ability to elicit approach toward it, and (2) the ability to act as a conditioned reinforcer.

RESULTS

In contrast to previous findings with food, cocaine, and opioid cues, we found that the nicotine cue was equally attractive in STs and GTs, eliciting dose-dependent approach behavior in both. However, the nicotine cue was a more effective conditioned reinforcer in STs than in GTs.

CONCLUSIONS

We suggest the dissociation between these two measures of incentive salience attribution may be related to the fact that when present (as in the test of Pavlovian approach), nicotine can act as a potent "incentive amplifier," and by this action, nicotine may render cues especially salient for all animals.

Rats that sign-track are resistant to Pavlovian but not instrumental extinction

Individuals vary in the extent to which they attribute incentive salience to a discrete cue (conditioned stimulus; CS) that predicts reward delivery (unconditioned stimulus; US), which results in some individuals approaching and interacting with the CS (sign-trackers; STs) more than others (goal-trackers; GTs). Here we asked how periods of non-reinforcement influence conditioned responding in STs vs. GTs, in both Pavlovian and instrumental tasks. After classifying rats as STs or GTs by pairing a retractable lever (the CS) with the delivery of a food pellet (US), we introduced periods of non-reinforcement, first by simply withholding the US (i.e., extinction training; experiment 1), then by signaling alternating periods of reward (R) and non-reward (NR) within the same session (experiments 2 and 3). We also examined how alternating R and NR periods influenced instrumental responding for food (experiment 4). STs and GTs did not differ in their ability to discriminate between R and NR periods in the instrumental task. However, in Pavlovian settings STs and GTs responded to periods of non-reward very differently. Relative to STs, GTs very rapidly modified their behavior in response to periods of non-reward, showing much faster extinction and better and faster discrimination between R and NR conditions. These results highlight differences between Pavlovian and instrumental extinction learning, and suggest that if a Pavlovian CS is strongly attributed with incentive salience, as in STs, it may continue to bias attention toward it, and to facilitate persistent and relatively inflexible responding, even when it is no longer followed by reward.