Sign-tracking (autoshaping) in rats: a comparison of cocaine and food as unconditioned stimuli

Effects of an ethanol-paired conditioned stimulus on responding for ethanol suppressed by a conditioned-taste-aversion

Ethanol-Paired Conditioned Stimuli (CS) can increase ethanol-responding either in extinction or occurring at low rates late in a session. To examine the generality of CS-induced increases in ethanol-responding, we examined whether a CS could increase responding suppressed by Conditioned-Taste-Aversion (CTA), which presumably suppresses responding by changing ethanol's valence from positive to negative. Rats were trained to respond for ethanol under a Random Interval (RI) schedule. We then removed the lever and paired Random-Time ethanol deliveries with illumination of a stimulus light (i.e., CS) for 10 sessions. Results were compared with a Truly Random Control group, in which the light and ethanol deliveries occurred independently. In a subsequent experiment, rats were treated similarly, except the light served as a discriminative stimulus, as the lever was extended and ethanol deliveries were available under a RI during light presentations. After this training, the lever was returned and rats again responded for ethanol. Subsequently, sessions were followed by LiCl administration. When responding reached low levels, LiCl administration stopped and the light was occasionally illuminated during the session. Responding during the light presentation was compared to responding during the period preceding light presentation. Responding partially recovered across 10 sessions and was greater during light presentations than in the period before it in all three groups. Increases were not reliably different between the groups, indicating that explanations for these increases such as CS-induced increases in motivation or approach toward the light are unlikely to be correct. The most likely explanation for these light-induced increases is that during sessions in which the light had been presented previously, LiCl had never been presented and thus, the light had come to signal that ethanol was safe to drink.

Incentive motivation: 'wanting' roles of central amygdala circuitry

The central nucleus of amygdala (CeA) mediates positively-valenced reward motivation as well as negatively-valenced fear. Optogenetic or neurochemical stimulation of CeA circuitry can generate intense incentive motivation to pursue and consume a paired natural food, sex, or addictive drug reward, and even create maladaptive 'wanting what hurts' such as attraction to a shock rod. Evidence indicates CeA stimulations selectively amplify incentive motivation ('wanting') but not hedonic impact ('liking') of the same reward. Further, valence flips can occur for CeA contributions to motivational salience. That is, CeA stimulation can promote either incentive motivation or fearful motivation, even in the same individual, depending on situation. These findings may carry implications for understanding CeA roles in neuropsychiatric disorders involving aberrant motivational salience, ranging from addiction to paranoia and anxiety disorders.

Ethanol-paired stimuli can increase reinforced ethanol responding

While ethanol-paired stimuli are frequently postulated to increase drinking motivation and thus increase ethanol responding and precipitate relapse, no study has demonstrated increases in ethanol-reinforced responding following presentation of an ethanol-paired stimulus that had not previously been part of a contingent relationship. Previous studies have shown that food-paired stimuli can increase food responding that is at low rates and increase food consumption in food-sated rats. In Experiment 1, we show that an ethanol-paired stimulus can increase ethanol responding that is at low levels late in the experimental session, presumably due to satiation. However, these increases may have resulted from either associative or non-associative mechanisms. In Experiment 2, we compared the effects of an ethanol-paired stimulus to those of the same stimulus in a Truly-Random-Control group. In a Truly-Random-Control, the stimulus and ethanol each are presented on independent random schedules, and thus any differences between the effects of the stimulus in the experimental and control groups is likely attributable to the association between the stimulus and ethanol. The stimulus increased ethanol-reinforced responding in both the experimental and control groups, but these increases were greater in the experimental than the control group. Thus, both stimulus-change and the pairing of the stimulus with ethanol may result in increases in ethanol-reinforced responding.

Translations in Stimulus-Stimulus Pairing: Autoshaping of Learner Vocalizations

Stimulus-stimulus pairing (SSP) is a procedure used by behavior analysis practitioners that capitalizes on respondent conditioning processes to elicit vocalizations. These procedures usually are implemented only after other, more customary methods (e.g., standard echoic training via modeling) have been exhausted. Unfortunately, SSP itself has mixed research support, probably because certain as-yet-unidentified procedural variations are more effective than others. Even when SSP produces (or increases) vocalizations, its effects can be short-lived. Although specific features of SSP differ across published accounts, fundamental characteristics include presentation of a vocal stimulus proximal with presentation of a preferred item. In the present article, we draw parallels between SSP procedures and autoshaping, review factors shown to affect autoshaping, and interpret autoshaping research for suggested SSP tests and applications. We then call for extended use and reporting of SSP in behavior-analytic treatments. Finally, three bridges created by this article are identified: basic-applied, respondent-operant, and behavior analysis with other sciences.

Effects of rat strain and method of inducing ethanol drinking on Pavlovian-Instrumental-Transfer with ethanol-paired conditioned stimuli

Ethanol-paired conditioned stimuli (CSs) are widely thought to invigorate ethanol responding, and thus, precipitate relapse to drinking. However, preclinical studies investigating this issue using Pavlovian-Instrumental-Transfer (PIT) procedures have had mixed results, with some studies finding PIT while others did not. The studies failing to show PIT used Lewis rats and induced ethanol drinking using a post-prandial drinking procedure. The present experiments examined whether either of these two variables influenced the magnitude of PIT observed. In the first experiment, ethanol drinking in Lewis rats was induced using either sucrose fading or post-prandial drinking. In the second experiment, ethanol drinking was induced using post-prandial drinking in either Long-Evans Hooded or Lewis rats. In both experiments, rats were trained to respond for ethanol under a random interval schedule. Subsequently with the lever removed, 2-min light presentations were paired with ethanol deliveries. Finally, with the lever returned, the effect of light presentations on responding was tested while responding was in extinction. Light presentations similarly affected responding in Lewis rats regardless of the method of drinking induction. Likewise, light presentations similarly affected responding in both Lewis and Long-Evans Hooded rats. Neither ethanol induction method nor rat strain affected the magnitude of PIT observed, and thus, neither likely explains previous failures to observe PIT with ethanol-maintained behavior.

Remifentanil maintains lower initial delayed nonmatching-to-sample accuracy compared to food pellets in male rhesus monkeys

Emerging human laboratory and preclinical drug self-administration data suggest that a history of contingent abused drug exposure impairs performance in operant discrimination procedures, such as delayed nonmatching-to-sample (DNMTS), that are hypothesized to assess components of executive function. However, these preclinical discrimination studies have exclusively used food as the reinforcer and the effects of drugs as reinforcers in these operant procedures are unknown. The present study determined effects of contingent intravenous remifentanil injections on DNMTS performance hypothesized to assess 1 aspect of executive function, working memory. Daily behavioral sessions consisted of 2 components with sequential intravenous remifentanil (0, 0.01-1.0 μg/kg/injection) or food (0, 1-10 pellets) availability in nonopioid dependent male rhesus monkeys (n = 3). Remifentanil functioned as a reinforcer in the DNMTS procedure. Similar delay-dependent DNMTS accuracy was observed under both remifentanil- and food-maintained components, such that higher accuracies were maintained at shorter (0.1-1.0 s) delays and lower accuracies approaching chance performance were maintained at longer (10-32 s) delays. Remifentanil maintained significantly lower initial DNMTS accuracy compared to food. Reinforcer magnitude was not an important determinant of DNMTS accuracy for either remifentanil or food. These results extend the range of experimental procedures under which drugs function as reinforcers. Furthermore, the selective remifentanil-induced decrease in initial DNMTS accuracy is consistent with a selective impairment of attentional, but not memorial, processes. (PsycINFO Database Record

Optogenetic Central Amygdala Stimulation Intensifies and Narrows Motivation for Cocaine

Addiction is often characterized by intense motivation for a drug, which may be narrowly focused at the expense of other rewards. Here, we examined the role of amygdala-related circuitry in the amplification and narrowing of motivation focus for intravenous cocaine. We paired optogenetic channelrhodopsin (ChR2) stimulation in either central nucleus of amygdala (CeA) or basolateral amygdala (BLA) of female rats with one particular nose-poke porthole option for earning cocaine infusions (0.3 mg/kg, i.v.). A second alternative porthole earned identical cocaine but without ChR2 stimulation. Consequently, CeA rats quickly came to pursue their CeA ChR2-paired cocaine option intensely and exclusively, elevating cocaine intake while ignoring their alternative cocaine alone option. By comparison, BLA ChR2 pairing failed to enhance cocaine motivation. CeA rats also emitted consummatory bites toward their laser-paired porthole, suggesting that higher incentive salience made that cue more attractive. A separate progressive ratio test of incentive motivation confirmed that CeA ChR2 amplified rats' motivation, raising their breakpoint effort price for cocaine by 10-fold. However, CeA ChR2 laser on its own lacked any reinforcement value: laser by itself was never self-stimulated, not even by the same rats in which it amplified motivation for cocaine. Conversely, CeA inhibition by muscimol/baclofen microinjections prevented acquisition of cocaine self-administration and laser preference, whereas CeA inhibition by optogenetic halorhodopsin suppressed cocaine intake, indicating that CeA circuitry is needed for ordinary cocaine motivation. We conclude that CeA ChR2 excitation paired with a cocaine option specifically focuses and amplifies motivation to produce intense pursuit and consumption focused on that single target.SIGNIFICANCE STATEMENT In addiction, intense incentive motivation often becomes narrowly focused on a particular drug of abuse. Here we show that pairing central nucleus of amygdala (CeA) optogenetic stimulation with one option for earning intravenous cocaine makes that option almost the exclusive focus of intense pursuit and consumption. CeA stimulation also elevated the effort cost rats were willing to pay for cocaine and made associated cues become intensely attractive. However, we also show that CeA laser had no reinforcing properties at all when given alone for the same rats. Therefore, CeA laser pairing makes its associated cocaine option and cues become powerfully attractive in a nearly addictive fashion.

Conditioned Stimulus Form Does Not Explain Failures to See Pavlovian-Instrumental-Transfer With Ethanol-Paired Conditioned Stimuli

BACKGROUND

Pavlovian-Instrumental-Transfer (PIT) examines the effects of associative learning upon instrumental responding. Previous studies examining PIT with ethanol (EtOH)-maintained responding showed increases in responding following presentation of an EtOH-paired conditioned stimulus (CS). Recently, we conducted 2 studies examining PIT with an EtOH-paired CS. One of these found increases in responding, while the other did not. This less robust demonstration of PIT may have resulted from the form of the CS used, as we used a 120-second light stimulus as a CS, while the previous studies used either a 120-second auditory stimulus or a 10-second light stimulus. This study examined whether using conditions similar to our earlier study, but with either a 120-second auditory or a 10-second light stimulus as a CS, resulted in more robust PIT. We also examined the reliability of our previous failure to observe PIT.

METHODS

Three experiments were conducted examining whether PIT was obtained using (i) a 120-second light stimulus, (ii) a 10-second light stimulus, or (iii) a 120-second auditory stimulus as CSs.

RESULTS

We found PIT was not obtained using (i) a 120-second light stimulus as a CS, (ii) a 10-second light stimulus as a CS, or (iii) a 120-second auditory stimulus as a CS.

CONCLUSIONS

These results suggest that CS form does not account for our earlier failure to see PIT. Rather, factors like rat strain or how EtOH drinking is induced may account for when PIT is or is not observed.

Conditioned stimuli's role in relapse: preclinical research on Pavlovian-Instrumental-Transfer

RATIONALE AND OBJECTIVE

Pavlovian learning is central to many theories of addiction. In these theories, stimuli paired with drug ingestion become conditioned stimuli (CS) and subsequently elicit drug-seeking and drug-taking. However, in most relevant studies, Pavlovian and instrumental learning are confounded. This confound may be avoided in Pavlovian-Instrumental-Transfer (PIT) procedures. In PIT, Pavlovian and instrumental learning are established separately and then combined. In order to better understand the role of CSs in addiction, we review the relevant studies using PIT.

FINDINGS

We identified seven articles examining PIT effects of ethanol- or cocaine-paired CSs. Under at least one condition, six of these articles reported CS-elicited increases in responding previously maintained by drug. However, the only study using the optimal control condition failed to find a CS-elicited increase. Two studies examining CS specificity found the CS also increased responding maintained by a different reinforcer. Two studies examined if CSs elicit increases in actual drug-taking. Both failed to find CS-elicited increases, i.e., no study shows CS-elicited increases in actual drug-taking. Further, CS-elicited increases in extinguished responding are short-lived.

CONCLUSIONS

These findings are not entirely consistent with Pavlovian learning playing a central role in addiction. However, design issues can explain most of these inconsistencies. Studies without these design issues are needed. Additionally, existing theories hypothesize drug-paired CSs increase drug-taking by increasing motivation, by eliciting conditioned responses that make drug-seeking more probable, or by a combination of these. Work distinguishing between these mechanisms would also be useful.

Individual variability in behavioral flexibility predicts sign-tracking tendency

Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS-) and second-order (SOCS+, SOCS-) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.

Individual variation in the motivational properties of a nicotine cue: sign-trackers vs. goal-trackers

RATIONALE

Individuals vary in the extent to which they attribute incentive salience to reward cues. Discrete food and drug (cocaine and opioid) cues become more attractive, eliciting approach toward them, and more "wanted," in that they serve as more effective conditioned reinforcers, in some rats (sign-trackers, STs), than in others (goal-trackers, GTs).

OBJECTIVES

We asked whether there is similar variation in the extent to which a cue associated with a drug from another class, nicotine, acquires incentive motivational properties.

METHODS

First, a Pavlovian conditioned approach procedure was used to identify rats that attribute incentive salience to a food cue (i.e., STs and GTs). We then measured the extent to which a cue (a light) paired with intravenous nicotine injections acquired two properties of an incentive stimulus: (1) the ability to elicit approach toward it, and (2) the ability to act as a conditioned reinforcer.

RESULTS

In contrast to previous findings with food, cocaine, and opioid cues, we found that the nicotine cue was equally attractive in STs and GTs, eliciting dose-dependent approach behavior in both. However, the nicotine cue was a more effective conditioned reinforcer in STs than in GTs.

CONCLUSIONS

We suggest the dissociation between these two measures of incentive salience attribution may be related to the fact that when present (as in the test of Pavlovian approach), nicotine can act as a potent "incentive amplifier," and by this action, nicotine may render cues especially salient for all animals.

Drug versus sweet reward: greater attraction to and preference for sweet versus drug cues

Despite the unique ability of addictive drugs to directly activate brain reward circuits, recent evidence suggests that drugs induce reinforcing and incentive effects that are comparable to, or even lower than some nondrug rewards. In particular, when rats have a choice between pressing a lever associated with intravenous cocaine or heroin delivery and another lever associated with sweet water delivery, most respond on the latter. This outcome suggests that sweet water is more reinforcing and attractive than either drug. However, this outcome may also be due to the differential ability of sweet versus drug levers to elicit Pavlovian feeding-like conditioned responses that can cause involuntary lever pressing, such as pawing and biting the lever. To test this hypothesis, rats first underwent Pavlovian conditioning to associate one lever with sweet water (0.2% saccharin) and a different lever with intravenous cocaine (0.25 mg) or heroin (0.01 mg). Choice between these two levers was then assessed under two operant choice procedures: one that permitted the expression of Pavlovian-conditioned lever press responses during choice, the other not. During conditioning, Pavlovian-conditioned lever press responses were considerably higher on the sweet lever than on either drug lever, and slightly greater on the heroin lever than on the cocaine lever. Importantly, though these differences in Pavlovian-conditioned behavior predicted subsequent preference for sweet water during choice, they were not required for its expression. Overall, this study confirms that rats prefer the sweet lever because sweet water is more reinforcing and attractive than cocaine or heroin.

Sign-tracking predicts increased choice of cocaine over food in rats

The purpose of this study was to determine whether the tendency to sign-track to a food cue was predictive of rats' choice of cocaine over food. First, rats were trained on a procedure where insertion of a retractable lever was paired with food. A sub-group of rats - sign-trackers - primarily approached and contacted the lever, while another sub-group - goal-trackers - approached the site of food delivery. Rats were then trained on a choice task where they could choose between an infusion of cocaine (1.0 mg/kg) and a food pellet (45 mg). Sign-trackers chose cocaine over food significantly more often than did goal-trackers. These results support the incentive-salience theory of addiction and add to a growing number of studies which suggest that sign-trackers may model an addiction-prone phenotype.

Incentive salience attribution under reward uncertainty: A Pavlovian model

There is a vast literature on the behavioural effects of partial reinforcement in Pavlovian conditioning. Compared with animals receiving continuous reinforcement, partially rewarded animals typically show (a) a slower development of the conditioned response (CR) early in training and (b) a higher asymptotic level of the CR later in training. This phenomenon is known as the partial reinforcement acquisition effect (PRAE). Learning models of Pavlovian conditioning fail to account for it. In accordance with the incentive salience hypothesis, it is here argued that incentive motivation (or 'wanting') plays a more direct role in controlling behaviour than does learning, and reward uncertainty is shown to have an excitatory effect on incentive motivation. The psychological origin of that effect is discussed and a computational model integrating this new interpretation is developed. Many features of CRs under partial reinforcement emerge from this model.

Blocking in autoshaped lever-pressing procedures with rats

Rats will approach and contact a lever whose insertion into the chamber signals response-independent food delivery. This "autoshaping" or "sign-tracking" phenomenon has recently attracted considerable attention as a platform for studying individual differences in impulsivity, drug sensitization, and other traits associated with vulnerability to drug addiction. Here, we examined two basic stimulus selection phenomena-blocking and overshadowing-in the autoshaped lever pressing of rats. Blocking and overshadowing were decidedly asymmetrical. Previously reinforced lever-extension conditioned stimuli (CSs) completely blocked conditioning to auditory cues (Exps. 1 and 2), and previously nonreinforced lever-extension CSs overshadowed conditioning to auditory cues. By contrast, conditioning to lever-extension CSs was not blocked by either auditory (Exp. 3) or lever-insertion (Exp. 4) cues, and was not overshadowed by auditory cues. Conditioning to a lever-insertion cue was somewhat overshadowed by the presence of another lever, especially in terms of food cup behavior displayed after lever withdrawal. We discuss several frameworks in which the apparent immunity of autoshaped lever pressing to blocking might be understood. Given evidence that different brain systems are engaged when different kinds of cues are paired with food delivery, it is worth considering the possibility that interactions among them in learning and performance may follow different rules. In particular, it is intriguing to speculate that the roles of simple cue-reinforcer contiguity, as well as of individual and aggregate reinforcer prediction errors, may differ across stimulus classes.

Pavlovian conditioned approach, extinction, and spontaneous recovery to an audiovisual cue paired with an intravenous heroin infusion

RATIONALE

Novel stimuli paired with exposure to addictive drugs can elicit approach through Pavlovian learning. While such approach behavior, or sign tracking, has been documented for cocaine and alcohol, it has not been shown to occur with opiate drugs like heroin. Most Pavlovian conditioned approach paradigms use an operandum as the sign, so that sign tracking can be easily automated.

OBJECTIVES

We were interested in assessing whether approach behavior occurs to an audiovisual cue paired with an intravenous heroin infusion. If so, would this behavior exhibit characteristics of other Pavlovian conditioned behaviors, such as extinction and spontaneous recovery?

METHODS

Rats were repeatedly exposed to an audiovisual cue, similar to that used in standard self-administration models, along with an intravenous heroin infusion. Sign tracking was measured in an automated fashion by analyzing motion pixels within the cue zone during each cue presentation.

RESULTS

We were able to observe significant sign tracking after only five pairings of the conditioned stimulus (CS) with the unconditioned stimulus (US). This behavior rapidly extinguished over 2 days, but exhibited pronounced spontaneous recovery 3 weeks later.

CONCLUSIONS

We conclude that sign tracking measured by these methods exhibits all the characteristics of a classically conditioned behavior. This model can be used to examine the Pavlovian component of drug memories, alone, or in combination with self-administration methods.

Neural correlates of the formation and retention of cocaine-induced stimulus-reward associations

BACKGROUND

Cocaine can elicit drug-seeking behavior for drug-predicting stimuli, even after a single stimulus-cocaine pairing. Although orbitofrontal cortex is thought to be important during encoding and maintenance of stimulus-reward value, we still lack a comprehensive model of the neural circuitry underlying this cognitive process.

METHODS

We studied the conditioned effects of cocaine with monkey functional magnetic resonance imaging and classical conditioning by pairing a visual shape (conditioning stimulus [CS+]) with a noncontingent cocaine infusion; a control stimulus was never paired. We correlated the behavioral preference of the monkey for the CS+, as measured offline, with the activity induced by the CS+ relative to the control stimulus as function of time.

RESULTS

We observed that during formation of stimulus-cocaine associations strong CS+-induced functional magnetic resonance imaging activations emerged in frontal cortex that correlated significantly with behavioral CS+ preference. Afterward, CS+ preference correlated only with activity in early visual cortex. Control experiments suggest that these findings cannot be explained by increased familiarity for the CS+.

CONCLUSIONS

Our findings suggest a complex interaction between frontal and occipital cortex during cocaine conditioning. Frontal cortex is important for establishing novel representations of stimulus valence when cocaine is used as reinforcer, whereas early visual cortex is involved in retaining these cocaine-stimulus associations.

Quantifying individual variation in the propensity to attribute incentive salience to reward cues

If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience") to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers") vs. others that approached the location of reward delivery ("goal-trackers"). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.

A review of preclinical research demonstrating that drug and non-drug reinforcers differentially affect behavior

This review describes and summarizes current preclinical research revealing important differences between drug and non-drug reinforcers in terms of their effects on behavior. Despite research showing that drugs are not especially strong reinforcers in animals, a number of other behavioral differences potentially relevant to addiction have been reported in studies that have compared drug and non-drug reinforcers. Several of these effects appear only after long-term access to drugs. These include an escalation of drug intake, an increased persistence in responding for the drug, and a decreased sensitivity to the effects of punishers or other suppressors of drug seeking. Further differences between drug and non-drug reinforcers include the effects that reinforcer-paired stimuli have on behavior. Drug cues, as compared to food cues, have been shown to exert greater control over reinforcer-seeking behavior after periods of abstinence. Similarly, behavior previously reinforced by drugs, but not food, has been shown to be susceptible to stress-induced reinstatement after extinction. The behavioral differences between drug and non-drug reinforcers reviewed here may identify special features of drugs that lead to addiction.

Roles of nucleus accumbens and basolateral amygdala in autoshaped lever pressing

Initially-neutral cues paired with rewards are thought to acquire motivational significance, as if the incentive motivational value of the reward is transferred to the cue. Such cues may serve as secondary reinforcers to establish new learning, modulate the performance of instrumental action (Pavlovian-instrumental transfer, PIT), and be the targets of approach and other cue-directed behaviors. Here we examined the effects of lesions of the ventral striatal nucleus accumbens (ACb) and the basolateral amygdala (BLA) on the acquisition of discriminative autoshaped lever-pressing in rats. Insertion of one lever into the experimental chamber was reinforced by sucrose delivery, but insertion of another lever was not reinforced. Although sucrose was delivered independently of the rats' behavior, sham-lesioned rats rapidly came to press the reinforced but not the nonreinforced lever. Bilateral ACb lesions impaired the initial acquisition of sign-tracking but not its terminal levels. In contrast, BLA lesions produced substantial deficits in terminal levels of sign-tracking. Furthermore, whereas ACb lesions primarily affected the probability of lever press responses, BLA lesions mostly affected the rate of responding once it occurred. Finally, disconnection lesions that disrupted communication between ACb and BLA produced both sets of deficits. We suggest that ACb is important for initial acquisition of consummatory-like responses that incorporate hedonic aspects of the reward, while BLA serves to enhance such incentive salience once it is acquired.

Effects of lesions of the amygdala central nucleus on autoshaped lever pressing

Neutral cues paired with rewards often appear to acquire motivational significance, as if the incentive motivational value of the reward is transferred to the cue. Such cues have been reported to modulate the performance of instrumental action (Pavlovian-instrumental transfer, PIT), serve as conditioned reinforcers in the establishment of new learning, and be the targets of approach and other cue-directed behaviors. Here we examined the effects of lesions of the amygdala central nucleus (CeA) on the acquisition of discriminative autoshaped lever-pressing. Insertion of one lever into the experimental chamber was reinforced by sucrose delivery, but insertion of another lever was not reinforced. Although sucrose delivery was not contingent on lever pressing, both CeA- and sham-lesioned rats rapidly came to press the reinforced but not the nonreinforced lever. Despite their showing little evidence of impairments in autoshaped lever pressing, these same CeA-lesioned rats showed significant deficits in the expression of PIT in a subsequent phase of the experiment. The lack of impaired autoshaping in CeA-lesioned rats contrasts with effects previously reported for conditioned orienting responses (ORs) and for other putative measures of incentive learning including PIT and conditioned approach to visual cues.

Antagonism at NMDA receptors, but not β-adrenergic receptors, disrupts the reconsolidation of pavlovian conditioned approach and instrumental transfer for ethanol-associated conditioned stimuli

RATIONALE

Reconsolidation is the process by which memories require restabilisation following destabilisation at retrieval. Since even old, well-established memories become susceptible to disruption following reactivation, treatments based upon disrupting reconsolidation could provide a novel form of therapy for neuropsychiatric disorders based upon maladaptive memories, such as drug addiction. Pavlovian cues are potent precipitators of relapse to drug-seeking behaviour and influence instrumental drug seeking through at least three psychologically and neurobiologically distinct processes: conditioned reinforcement, conditioned approach (autoshaping) and conditioned motivation (pavlovian-instrumental transfer or PIT). We have previously demonstrated that the reconsolidation of memories underlying the conditioned reinforcing properties of drug cues depends upon NMDA receptor (NMDAR)- and β-adrenergic receptor (βAR)-mediated signalling. However, it is unknown whether the drug cue memory representations underlying conditioned approach and PIT depend upon the same mechanisms.

OBJECTIVES

Using orally self-administered ethanol as a reinforcer in two separate experiments, we investigated whether the reconsolidation of the memories underlying conditioned approach and PIT requires βAR- and NMDAR-dependent neurotransmission.

RESULTS

For ethanol self-administering but non-dependent rats, the memories underlying conditioned approach and PIT for a pavlovian drug cue were disrupted by the administration of the NMDAR antagonist MK-801, but not the administration of the βAR antagonist propranolol, when given in conjunction with memory reactivation.

CONCLUSIONS

As for natural reinforcers, NMDARs are required for the reconsolidation of all aspects of pavlovian drug memories, but βARs are only required for the memory representation underlying conditioned reinforcement. These results indicate the potential utility of treatments based upon disrupting cue-drug memory reconsolidation in preventing relapse.

The psychological and neurochemical mechanisms of drug memory reconsolidation: implications for the treatment of addiction

Memory reconsolidation is the process by which memories, destabilised at retrieval, require restabilisation to persist in the brain. It has been demonstrated that even old, well-established memories require reconsolidation following retrieval; therefore, memory reconsolidation could potentially be exploited to disrupt, or even erase, aberrant memories that underlie psychiatric disorders, thereby providing a novel therapeutic target. Drug addiction is one such disorder; it is both chronic and relapsing, and one prominent risk factor for a relapse episode is the presentation of environmental cues that have previously been associated with drugs of abuse. This 'cue-induced relapse' can be accounted for in psychological terms by reinforcing memories of the pavlovian association between the cue and the drug, which can thus influence behaviour through at least three psychologically and neurobiologically dissociable mechanisms: conditioned reinforcement, conditioned approach and conditioned motivation. As each of these psychological processes could contribute to the resumption of drug-seeking following abstinence, it is important to develop treatments that can reduce drug-seeking re-established via influences on each or all of these pavlovian processes, in order to minimise the risk of a subsequent relapse. Investigation of the memory reconsolidation mechanisms of the memories underlying conditioned reinforcement, conditioned approach and conditioned motivation indicate that they depend upon different neurochemical systems, including the glutamatergic and adrenergic systems within limbic corticostriatal circuitry. We also discuss here the subsequent translation to the clinic of this preclinical work.

Influence of conditioned reinforcement on the response-maintaining effects of quinpirole in rats

D2-like agonists, such as quinpirole, maintain responding in monkeys, rats, and mice when they are substituted for cocaine. This study examined the influence of operant history and cocaine-paired stimuli (CS) on quinpirole-maintained responding in rats trained to nose poke for cocaine. Upon acquisition of responding for cocaine, substitutions were performed in the presence or absence of injection-CS pairings. Although cocaine maintained responding regardless of whether injections were accompanied by CS, quinpirole maintained responding only when CS were paired with injections. To assess the influence of operant history, injections of cocaine, quinpirole, remifentanil, nicotine, or saline were made available on a previously inactive lever, while nose pokes continued to result in CS presentation. Although responding was reallocated from the nose poke to the lever when cocaine or remifentanil was available, lever presses remained low, and nose poking persisted when quinpirole or nicotine was made contingent upon lever presses. Finally, quinpirole pretreatments resulted in high rates of nose poking when nose pokes resulted in CS presentation alone, but failed to maintain nose poking when the CS was omitted. Taken together, these results suggest that the response-maintaining effects of quinpirole are primarily mediated by an enhancement of the conditioned reinforcing effects of earlier CS, rather than by a reinforcing effect of quinpirole.

Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction

In this review we discuss the evidence that drug addiction, defined as a maladaptive compulsive habit, results from the progressive subversion by addictive drugs of striatum-dependent operant and Pavlovian learning mechanisms that are usually involved in the control over behaviour by stimuli associated with natural reinforcement. Although mainly organized through segregated parallel cortico-striato-pallido-thalamo-cortical loops involved in motor or emotional functions, the basal ganglia, and especially the striatum, are key mediators of the modulation of behavioural responses, under the control of both action-outcome and stimulus-response mechanisms, by incentive motivational processes and Pavlovian associations. Here we suggest that protracted exposure to addictive drugs recruits serial and dopamine-dependent, striato-nigro-striatal ascending spirals from the nucleus accumbens to more dorsal regions of the striatum that underlie a shift from action-outcome to stimulus-response mechanisms in the control over drug seeking. When this progressive ventral to dorsal striatum shift is combined with drug-associated Pavlovian influences from limbic structures such as the amygdala and the orbitofrontal cortex, drug seeking behaviour becomes established as an incentive habit. This instantiation of implicit sub-cortical processing of drug-associated stimuli and instrumental responding might be a key mechanism underlying the development of compulsive drug seeking and the high vulnerability to relapse which are hallmarks of drug addiction.

Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction

Drugs of abuse acquire different degrees of control over thoughts and actions based not only on the effects of drugs themselves, but also on predispositions of the individual. Those individuals who become addicted are unable to shift their thoughts and actions away from drugs and drug-associated stimuli. Thus in addicts, exposure to places or things (cues) that has been previously associated with drug-taking often instigates renewed drug-taking. We and others have postulated that drug-associated cues acquire the ability to maintain and instigate drug-taking behavior in part because they acquire incentive motivational properties through Pavlovian (stimulus-stimulus) learning. In the case of compulsive behavioral disorders, including addiction, such cues may be attributed with pathological incentive value ("incentive salience"). For this reason, we have recently begun to explore individual differences in the tendency to attribute incentive salience to cues that predict rewards. When discrete cues are associated with the non-contingent delivery of food or drug rewards some animals come to quickly approach and engage the cue even if it is located at a distance from where the reward will be delivered. In these animals the reward-predictive cue itself becomes attractive, eliciting approach towards it, presumably because it is attributed with incentive salience. Animals that develop this type of conditional response are called "sign-trackers". Other animals, "goal-trackers", do not approach the reward-predictive cue, but upon cue presentation they immediately go to the location where food will be delivered (the "goal"). For goal-trackers the reward-predictive cue is not attractive, presumably because it is not attributed with incentive salience. We review here preliminary data suggesting that these individual differences in the tendency to attribute incentive salience to cues predictive of reward may confer vulnerability or resistance to compulsive behavioral disorders, including addiction. It will be important, therefore, to study how environmental, neurobiological and genetic interactions determine the extent to which individuals attribute incentive value to reward-predictive stimuli.

Behavioral characteristics and neurobiological substrates shared by Pavlovian sign-tracking and drug abuse

Drug abuse researchers have noted striking similarities between behaviors elicited by Pavlovian sign-tracking procedures and prominent symptoms of drug abuse. In Pavlovian sign-tracking procedures, repeated paired presentations of a small object (conditioned stimulus, CS) with a reward (unconditioned stimulus, US) elicits a conditioned response (CR) that typically consists of approaching the CS, contacting the CS, and expressing consummatory responses at the CS. Sign-tracking CR performance is poorly controlled and exhibits spontaneous recovery and long-term retention, effects that resemble relapse. Sign-tracking resembles psychomotor activation, a syndrome of behavioral responses evoked by addictive drugs, and the effects of sign-tracking on corticosterone levels and activation of dopamine pathways resemble the neurobiological effects of abused drugs. Finally, the neurobiological profile of individuals susceptible to sign-tracking resembles the pathophysiological profile of vulnerability to drug abuse, and vulnerability to sign-tracking predicts vulnerability to impulsive responding and alcohol self-administration. Implications of sign-tracking for models of drug addiction are considered.

Rapid induction of Pavlovian approach to an ethanol-paired visual cue in mice

RATIONALE

Although many studies have shown Pavlovian conditioned approach to cues paired with natural reinforcers, it has been quite difficult to induce such behavior with drug reinforcers.

OBJECTIVES

This experiment tested a novel Pavlovian procedure for inducing approach to a conditioned stimulus (CS) paired with ethanol.

METHODS

Mice (NZB/B1NJ, DBA/2J) received intraperitoneal injections of ethanol (2 g/kg) immediately before 10-min exposure to a rectangular chamber that contained a distinctive visual cue (star) at one end (Paired group, CS+ trials). On alternate days, saline injection preceded apparatus exposure with no distinctive cues (CS- trials). Unpaired control mice received ethanol in the home cage 60-75 min after each CS+ trial.

RESULTS

NZB/B1NJ Paired group mice spent increasing amounts of time (>85% of the session) in proximity to the star, whereas Unpaired group mice did not. DBA/2J Paired group mice spent slightly more time on the star side than Unpaired group mice but did not show an acquisition curve. Postconditioning tests showed a strong preference for the star side in Paired groups from both strains after saline injection. However, only NZB/B1NJ mice showed a preference after ethanol.

CONCLUSIONS

This study provides the first unambiguous demonstration of Pavlovian conditioned approach to an ethanol-paired visual stimulus in the absence of any contingency between the animal's behavior and drug exposure. This effect, which is remarkable both in terms of its magnitude and the rapidity with which it was produced (within 2-3 trials), may be related to the cue-associated craving that accompanies alcohol and drug addiction.

Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm

Environmental stimuli predictive of appetitive events can elicit Pavlovian approach responses that enhance an organism's ability to track and secure natural rewards, but may also contribute to the compulsive nature of drug addiction. Here, we examined the activity of individual nucleus accumbens (NAc) neurons during an autoshaping paradigm. One conditioned stimulus (CS+, a retractable lever presented for 10 s) was immediately followed by the delivery of a 45-mg sucrose pellet to a food receptacle, while another stimulus (CS-, a separate retractable lever presented for 10 s) was never followed by sucrose. Approach responses directed at the CS+ and CS- were recorded as lever presses and had no experimental consequence. Rats (n = 9) selectively approached the CS+ on more than 80% of trials and were surgically prepared for electrophysiological recording. Of 76 NAc neurons, 57 cells (75%) exhibited increases and/or decreases in firing rate (i.e. termed 'phasically active') during the CS+ presentation and corresponding approach response. Forty-seven percent of phasically active cells (27 out of 57) were characterized by time-locked but transient increases in cell firing, while 53% (30 out of 57) showed a significant reduction in firing for the duration of the CS+. In contrast, the same excitatory subpopulation exhibited smaller increases in activity relative to CS- onset, while the inhibitory subpopulation showed no change in firing during the CS- period. The magnitude and prevalence of cue-related neural responses reported here indicates that the NAc encodes biologically significant, repetitive approach responses that may model the compulsive nature of drug addiction in humans.

A comparison of Lewis and Fischer rat strains on autoshaping (sign-tracking), discrimination reversal learning and negative automaintenance

Lewis (LEW) and Fischer (F344) rat strains differ on a number of physiological characteristics, such as hypothalamic-pituitary-adrenal (HPA) axis activity, as well as on behavioral tasks, including those that measure impulsivity and drug reward. Since autoshaping, the phenomenon where animals approach and contact reward-paired conditioned stimuli, has been linked to HPA axis functioning, impulsivity and drug taking, the present study compared LEW and F344 rats on the rate of acquisition and performance of the autoshaping response. Rats were trained on an autoshaping procedure where insertions of one retractable lever (CS(+)) were paired response-independently with food, while insertions of another lever (CS(-)) were not paired with food. LEW rats acquired the autoshaping response more rapidly and also performed the autoshaping response at a higher rate than F344 rats. No differences between the strains were observed when rats were trained on a discrimination reversal where the CS(+) and CS(-) levers were reversed or during a negative auto-maintenance phase where CS(+) lever contacts cancelled food delivery. Potential physiological mechanisms that might mediate the present results, including strain differences in HPA axis and monoamine neurotransmitter activity, are discussed. The finding that LEW (as compared to F344 rats) more readily acquire autoshaping and perform more responses is consistent with research indicating that LEW rats behave more impulsively and more readily self-administer drugs of abuse.

The attribution of incentive salience to a stimulus that signals an intravenous injection of cocaine

A central premise of a number of theories of addiction is that discrete environmental stimuli repeatedly paired with drugs of abuse acquire incentive salience as a result of Pavlovian learning. There is, however, no unequivocal evidence supporting this assumption. Thus, we employed a Pavlovian conditioning procedure known to imbue non-drug reinforcers with incentive salience and extended it to study the effects of intravenous cocaine. Specifically, we examined whether a cue paired with intravenous cocaine administration would come to elicit approach towards it (sign-tracking), even if no behavioral response were required to receive the cue or drug. We found that when a cue was paired with intravenous cocaine delivery (but not when it was unpaired) rats came to approach and investigate the cue, and did so with increasing rapidity. We conclude that Pavlovian learning can imbue drug-paired cues with incentive salience, making them attractive and "wanted" stimuli. Delineating the neurobiological mechanisms responsible for this process will be important for understanding and treating drug addiction.

Behavioural assessment of drug reinforcement and addictive features in rodents: an overview

Some psychoactive drugs are abused because of their ability to act as reinforcers. As a consequence behavioural patterns (such as drug-seeking/drug-taking behaviours) are promoted that ensure further drug consumption. After prolonged drug self-administration, some individuals lose control over their behaviour so that these drug-seeking/taking behaviours become compulsive, pervading almost all life activities and precipitating the loss of social compatibility. Thus, the syndrome of addictive behaviour is qualitatively different from controlled drug consumption. Drug-induced reinforcement can be assessed directly in laboratory animals by either operant or non-operant self-administration methods, by classical conditioning-based paradigms such as conditioned place preference or sign tracking, by facilitation of intracranial electric self-stimulation, or, alternatively by drug-induced memory enhancement. In contrast, addiction cannot be modelled in animals, at least as a whole, within the constraints of the laboratory. However, various procedures have been proposed as possible rodent analogues of addiction's major elements including compulsive drug seeking, relapse, loss of control/impulsivity, and continued drug consumption despite negative consequences. This review provides an extensive overview and a critical evaluation of the methods currently used for studying drug-induced reinforcement as well as specific features of addictive behaviour. In addition, comic strips that illustrate behavioural methods used in the drug abuse field are provided given for free download under http://www.zi-mannheim/psychopharmacology.de.

Neural systems of reinforcement for drug addiction: from actions to habits to compulsion

Drug addiction is increasingly viewed as the endpoint of a series of transitions from initial drug use--when a drug is voluntarily taken because it has reinforcing, often hedonic, effects--through loss of control over this behavior, such that it becomes habitual and ultimately compulsive. Here we discuss evidence that these transitions depend on interactions between pavlovian and instrumental learning processes. We hypothesize that the change from voluntary drug use to more habitual and compulsive drug use represents a transition at the neural level from prefrontal cortical to striatal control over drug seeking and drug taking behavior as well as a progression from ventral to more dorsal domains of the striatum, involving its dopaminergic innervation. These neural transitions may themselves depend on the neuroplasticity in both cortical and striatal structures that is induced by chronic self-administration of drugs.