Outcome-specific conditioned inhibition in Pavlovian backward conditioning

Dopamine Release in the Nucleus Accumbens Core Encodes the General Excitatory Components of Learning

Dopamine release in the nucleus accumbens core (NAcC) is generally considered to be a proxy for phasic firing of the ventral tegmental area dopamine (VTADA) neurons. Thus, dopamine release in NAcC is hypothesized to reflect a unitary role in reward prediction error signaling. However, recent studies reveal more diverse roles of dopamine neurons, which support an emerging idea that dopamine regulates learning differently in distinct circuits. To understand whether the NAcC might regulate a unique component of learning, we recorded dopamine release in NAcC while male rats performed a backward conditioning task where a reward is followed by a neutral cue. We used this task because we can delineate different components of learning, which include sensory-specific inhibitory and general excitatory components. Furthermore, we have shown that VTADA neurons are necessary for both the specific and general components of backward associations. Here, we found that dopamine release in NAcC increased to the reward across learning while reducing to the cue that followed as it became more expected. This mirrors the dopamine prediction error signal seen during forward conditioning and cannot be accounted for temporal-difference reinforcement learning. Subsequent tests allowed us to dissociate these learning components and revealed that dopamine release in NAcC reflects the general excitatory component of backward associations, but not their sensory-specific component. These results emphasize the importance of examining distinct functions of different dopamine projections in reinforcement learning.

Prediction error in models of adaptive behavior

Pavlovian conditioning is evident in every species in which it has been assessed, and there is a consensus about its interpretation across behavioral,1,2 brain,3,4,5,6 and computational analyses7,8,9,10,11: conditioned behavior reflects the formation of a directional associative link from the memory of one stimulus (e.g., a visual stimulus) to another (e.g., food), with learning stopping when there is no error between the prediction generated by the visual stimulus and what happens next (e.g., food). This consensus fails to anticipate the results that we report here. In our experiments with rats, we find that arranging predictive (visual stimulus→food) and nonpredictive (food→visual stimulus) relationships produces marked and sustained changes in conditioned behaviors when the visual stimulus is presented alone. Moreover, the type of relationship affects (1) the distribution of conditioned behaviors related to the properties of both food (called goal-tracking) and the visual stimulus (called sign-tracking) and (2) when in the visual stimulus, these two behaviors are evident. These results represent an impetus for a fundamental shift in how Pavlovian conditioning is interpreted: animals learn about the relationship between two stimuli irrespective of the order in which they are presented, but they exhibit this knowledge in different ways. This interpretation and our new results are captured by a recent model of Pavlovian conditioning,12,13 HeiDI, and both are consistent with the need for animals to represent the fact that the impact of a cause (e.g., the ingestion of nutrients or the bite of a predator) can be felt before or after the cause has been perceived.

Learning to stop responding

Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this laboratory and others that has studied how animals learn to stop responding in operant extinction, punishment, and feature-negative learning. Extinction and punishment are shown to be similar in two fundamental ways. First, the response-suppressing effects of both are highly context-specific. Second, the response-suppressing effects of both can be remarkably response-specific: Inhibition of one response transfers little to other responses. Learning to inhibit the response so specifically may result from the correction of "response error," the difference between the level of responding and what the current reinforcer supports. In contrast, the inhibition of responding that develops in feature-negative learning, where the response is reinforced during one discriminative stimulus (A) but not in a compound of A and stimulus B, is less response-specific: The inhibition of responding by stimulus B transfers and inhibits a second response, especially if the second response has itself been inhibited before. The results thus indicate both response-specific and response-general forms of behavioral inhibition. One possibility is that response-specific inhibition is learned when the circumstances encourage the organism to pay attention to the response-to what it is actually doing-as behavioral suppression is learned.

Dopamine errors drive excitatory and inhibitory components of backward conditioning in an outcome-specific manner

For over two decades, phasic activity in midbrain dopamine neurons was considered synonymous with the prediction error in temporal-difference reinforcement learning.^1-4 Central to this proposal is the notion that reward-predictive stimuli become endowed with the scalar value of predicted rewards. When these cues are subsequently encountered, their predictive value is compared to the value of the actual reward received, allowing for the calculation of prediction errors.^5,6 Phasic firing of dopamine neurons was proposed to reflect this computation,^1,2 facilitating the backpropagation of value from the predicted reward to the reward-predictive stimulus, thus reducing future prediction errors. There are two critical assumptions of this proposal: (1) that dopamine errors can only facilitate learning about scalar value and not more complex features of predicted rewards, and (2) that the dopamine signal can only be involved in anticipatory cue-reward learning in which cues or actions precede rewards. Recent work^7-15 has challenged the first assumption, demonstrating that phasic dopamine signals across species are involved in learning about more complex features of the predicted outcomes, in a manner that transcends this value computation. Here, we tested the validity of the second assumption. Specifically, we examined whether phasic midbrain dopamine activity would be necessary for backward conditioning-when a neutral cue reliably follows a rewarding outcome.^16-20 Using a specific Pavlovian-to-instrumental transfer (PIT) procedure,^21-23 we show rats learn both excitatory and inhibitory components of a backward association, and that this association entails knowledge of the specific identity of the reward and cue. We demonstrate that brief optogenetic inhibition of VTA_DA neurons timed to the transition between the reward and cue reduces both of these components of backward conditioning. These findings suggest VTA_DA neurons are capable of facilitating associations between contiguously occurring events, regardless of the content of those events. We conclude that these data may be in line with suggestions that the VTA_DA error acts as a universal teaching signal. This may provide insight into why dopamine function has been implicated in myriad psychological disorders that are characterized by very distinct reinforcement-learning deficits.

Engagement of lateral habenula in the extinction of the appetitive conditioned responses

Following the association of a neutral stimulus (conditioned stimulus, CS) with a biologically significant stimulus (unconditioned stimulus, US), CS-alone presentations generate extinction: a decline in the conditioned response. Many studies have revealed the neural substrates of fear extinction; however, a few have identified the brain regions responsible for appetitive extinction. Midbrain dopamine neurons are activated by presenting a reward or predictable reward cue, whereas the cue signaling the absence of reward activates the lateral habenula (LHb) neurons. We examined the engagement of the LHb in appetitive extinction. In the first phase, rats first received pairings of a CS (light) with US delivery (food pellets). In the second phase, rats in the CS-alone group underwent four CS-alone presentations, whereas those in the paired-paired group received four pairings of light with food pellets. We also included a comparison group for CS-alone presentations: rats were placed in the training box without CS or US exposures in the first phase and received four CS-alone presentations in the second phase. Thirty minutes after the second phase, c-Fos levels in the ventral tegmental area (VTA), substantia nigra pars compacta (SNc), and LHb in these groups were measured. c-Fos levels in the LHb were higher in the paired-CS-alone group than in the paired-paired and comparison groups, while those in the VTA and SNc were significantly higher in the paired-paired group than in the other groups. On examination of LHb neurotoxic lesion effects on the decline of conditioned food-cup responses when a CS was repeatedly presented with no US, LHb lesions decelerated the decline in conditioned food-cup responses, suggesting a crucial role of LHb in appetitive extinction.

How predictive learning influences choice: Evidence for a GPCR-based memory process necessary for Pavlovian-instrumental transfer

Predictive learning endows stimuli with the capacity to signal both the sensory-specific and general motivational properties of their associated rewards or outcomes. These two signals can be distinguished behaviorally by their influence on the selection and performance of instrumental actions, respectively. This review focuses on how sensory-specific predictive learning guides choice between actions that earn otherwise equally desirable outcomes. We describe evidence that outcome-specific predictive learning is encoded in the basolateral amygdala and drives the accumulation of delta-opioid receptors on the surface of cholinergic interneurons located in the nucleus accumbens shell. This accumulation constitutes a novel form of cellular memory, not for outcome-specific predictive learning per se but for the selection of, and choice between, future instrumental actions. We describe recent evidence regarding the cascade of events necessary for the formation and expression of this cellular memory and point to open questions for future research into this process. Beyond these mechanistic considerations, the discovery of this new form of memory is consistent with recent evidence suggesting that intracellular rather than synaptic changes can mediate learning-related plasticity to modify brain circuitry to prepare for future significant events.

The role of the response-outcome association in the nature of inhibitory Pavlovian-instrumental transfer in rats

Inhibitory stimuli can reduce animals' reward seeking in an outcome-specific manner or outcome-general manner. However, we do not understand the factors that determine which of these effects are produced. To address this, we carried out three experiments which examined whether instrumental training with one or multiple outcomes determined the nature of subsequently observed Pavlovian-instrumental transfer (PIT). Rats underwent Pavlovian training to produce inhibitors and excitors for two outcomes using a feature-negative procedure. In Experiment 1, these stimuli were tested for their effects on a single response trained with one of those outcomes in a PIT procedure. Here, stimuli trained as inhibitors and excitors were found to produce outcome-general effects on reward seeking (in addition to an outcome-specific effect for excitors). In Experiment 2, we trained two responses, one for each of the Pavlovian outcomes, and tested the effect of the stimuli on each response individually. This design also produced outcome-general inhibitory and excitatory PIT effects. Experiment 3 followed the procedure of Experiment 2, except for implementation of a shorter Pavlovian training phase and an additional choice test, where both responses were concurrently available. This procedure produced putative inhibitory effects that were also outcome-general. However, outcome-specific excitatory effects were observed, indicating that the general inhibitory results may not be attributable to the duration of Pavlovian training. Overall, this study suggests that variations in the number of response-outcome contingencies experienced by animals do not readily determine the specificity of putative inhibitors.

The effect of conditioned inhibitors and preexposed cues on the outcome-specific Pavlovian-to-instrumental transfer effect in humans

Using a human Pavlovian-to-instrumental transfer (PIT) task, Alarcón and Bonardi showed that the selective elevation of instrumental responding produced by excitatory transfer cues was reduced when these cues were presented with a conditioned inhibitor (CI), relative to a control cue that was simply preexposed. However, previous research has shown that preexposed cues might also acquire inhibitor-like properties. This study aimed to contrast the inhibitory properties of CIs and preexposed cues, using novel stimuli as controls, in summation and PIT tests. Participants were trained to perform two actions, each reinforced with a distinct outcome (O₁ or O₂). Two images were trained as CIs, each signalling the absence of one of the outcomes, by presenting them with a cue that was otherwise followed by that outcome (e.g., A→O₁, AI→no O₁). In contrast, the preexposed cues were simply presented in the absence of the outcomes. In the summation test, participants rated the likelihood of the outcomes in the presence of two independently trained excitatory cues, each presented with a CI, a preexposed cue, or a novel stimulus. Similarly, in the PIT test, participants performed both actions in the presence and absence of these compounds. In the summation test, the CIs and the preexposed cues reduced participants' expectations of the outcomes more than the novel stimuli. However, in the PIT test, only the CIs reduced the selective elevation of responding produced by the transfer cues. These results might reflect distinct properties of stimuli trained as CIs and those simply preexposed.

Delay of reinforcement versus rate of reinforcement in Pavlovian conditioning

Conditioned stimulus (CS) duration is a determinant of conditioned responding, with increases in duration leading to reductions in response rates. The CS duration effect has been proposed to reflect sensitivity to the reinforcement rate across cumulative exposure to the CS, suggesting that the delay of reinforcement from the onset of the cue is not crucial. Here, we compared the effects of delay and rate of reinforcement on Pavlovian appetitive conditioning in mice. In Experiment 1, the influence of reinforcement delay on the timing of responding was removed by making the duration of cues variable across trials. Mice trained with variable duration cues were sensitive to differences in the rate of reinforcement to a similar extent as mice trained with fixed duration cues. Experiments 2 and 3 tested the independent effects of delay and reinforcement rate. In Experiment 2, food was presented at either the termination of the CS or during the CS. In Experiment 3, food occurred during the CS for all cues. The latter experiment demonstrated an effect of delay, but not reinforcement rate. Experiment 4 ruled out the possibility that the lack of effect of reinforcement rate in Experiment 3 was due to mice failing to learn about the nonreinforced CS exposure after the presentation of food within a trial. These results demonstrate that although the CS duration effect is not simply a consequence of timing of conditioned responses, it is dependent on the delay of reinforcement. The results provide a challenge to current associative and nonassociative, time-accumulation models of learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Associative mechanisms involved in specific Pavlovian-to-instrumental transfer in human learning tasks

Four experiments compared the effect of forward and backward conditioning procedures on the ability of conditioned stimuli (CS) to elevate instrumental responding in a Pavlovian-to-instrumental transfer (PIT) task. Two responses were each trained with one distinct outcome (R1->O1, R2->O2), either concurrently (Experiment 1) or separately (Experiments 2, 3 and 4). Then, in Experiments 1 and 2, four CSs were either followed or preceded by one outcome (A->O1, B->O2, O1->C, O2->D). In Experiment 3, each CS was preceded and followed by an outcome: for one group of participants, both outcomes were identical (e.g., O1->A->O1, O2->B->O2), but for the other, they were different (e.g., O1->A->O2, O2->B->O1). In Experiment 4, two CSs were preceded and followed by identical outcomes, and two CSs by different outcomes. In the PIT tests, participants performed R1 and R2 in the presence and absence of the CSs. In Experiments 1 and 2, only the CSs followed by outcomes in Pavlovian training elevated responding. In Experiments 3 and 4, all the CSs elevated responding but based on the outcome that followed them in training. These results support the stimulus-outcome-response (S-O-R) mechanism of specific PIT, according to which CSs elevate responding via activation of its associated outcome representation.

Factors that encourage generalization from extinction to test reduce resurgence of an extinguished operant response

Two experiments investigated methods that reduce the resurgence of an extinguished behavior (R1) that occurs when reinforcement for an alternative behavior (R2) is discontinued. In Experiment 1, R1 was first trained and then extinguished while R2 was reinforced during a 5- or 25-session treatment phase. For half the rats, sessions in which R2 was reinforced alternated with sessions in which R2 was extinguished. Controls received the same number of treatment sessions, but R2 was never extinguished. When reinforcement for R2 was discontinued, R1 resurged in the controls. However, the alternating groups showed reduced resurgence, and the magnitude of the resurgences observed during their R2 extinction sessions decreased systematically over Phase 2. In Experiment 2, R1 was first reinforced with one outcome (O1). The rats then had two types of double-alternating treatment sessions. In one type, R1 was extinguished and R2 produced O2. In the other, R1 was unavailable and R2 produced O3. R1 resurgence was weakened when O2, but not O3, was delivered freely during testing. Together, the results suggest that methods that encourage generalization between R1 extinction and resurgence testing weaken the resurgence effect. They are not consistent with an account of resurgence proposed by Shahan and Craig (2017).

The Lateral Habenula and Its Input to the Rostromedial Tegmental Nucleus Mediates Outcome-Specific Conditioned Inhibition

Animals can readily learn that stimuli predict the absence of specific appetitive outcomes; however, the neural substrates underlying such outcome-specific conditioned inhibition remain largely unexplored. Here, using female and male rats as subjects, we examined the involvement of the lateral habenula (LHb) and of its inputs onto the rostromedial tegmental nucleus (RMTg) in inhibitory learning. In these experiments, we used backward conditioning and contingency reversal to establish outcome-specific conditioned inhibitors for two distinct appetitive outcomes. Then, using the Pavlovian-instrumental transfer paradigm, we assessed the effects of manipulations of the LHb and the LHb-RMTg pathway on that inhibitory encoding. In control animals, we found that an outcome-specific conditioned inhibitor biased choice away from actions delivering that outcome and toward actions earning other outcomes. Importantly, this bias was abolished by both electrolytic lesions of the LHb and selective ablation of LHb neurons using Cre-dependent Caspase3 expression in Cre-expressing neurons projecting to the RMTg. This deficit was specific to conditioned inhibition; an excitatory predictor of a specific outcome-biased choice toward actions delivering the same outcome to a similar degree whether the LHb or the LHb-RMTg network was intact or not. LHb lesions also disrupted the ability of animals to inhibit previously encoded stimulus-outcome contingencies after their reversal, pointing to a critical role of the LHb and of its inputs onto the RMTg in outcome-specific conditioned inhibition in appetitive settings. These findings are consistent with the developing view that the LHb promotes a negative reward prediction error in Pavlovian conditioning.SIGNIFICANCE STATEMENT Stimuli that positively or negatively predict rewarding outcomes influence choice between actions that deliver those outcomes. Previous studies have found that a positive predictor of a specific outcome biases choice toward actions delivering that outcome. In contrast, a negative predictor of an outcome biases choice away from actions earning that outcome and toward other actions. Here we reveal that the lateral habenula is critical for negative predictors, but not positive predictors, to affect choice. Furthermore, these effects were found to require activation of lateral habenula inputs to the rostromedial tegmental nucleus. These results are consistent with the view that the lateral habenula establishes inhibitory relationships between stimuli and food outcomes and computes a negative prediction error in Pavlovian conditioning.

Renewal of extinguished instrumental responses: independence from Pavlovian processes and dependence on outcome value

The source of renewal of instrumental responding in rats was investigated. In Experiment 1, two responses (R1 and R2) were reinforced with one outcome (O1) in contexts A and B (i.e., R1→O1, R2→O1), and then R2 was extinguished in A and R1 was extinguished in B. At test, the rate of R1 was higher than that of R2 in context A, and the reverse was the case in context B: Renewed responding was independent of the Pavlovian context→O1 associations. In Experiment 2, all rats received R1→O1 and R2→O2 trials in A and then were placed in B, where they were sated on O2 and either did (Group Extinction) or did not (Group No Extinction) receive concurrent extinction of R1 and R2. At test, we found more responding in A than in B for Group Extinction, but not for Group No Extinction, and the renewed responding in A was as sensitive to the current value of the outcome as responding that had not been subject to extinction (i.e., the rate was higher for R1 than for R2). That is, the renewed responding was goal-directed. These results identify the removal of contextual inhibion of either the response or the response→outcome associaon as potenal bases for renewal, and the response→outcome associaon as the source of renewed responding.

A Closer Look at the Effects of Repeated Cocaine Exposure on Adaptive Decision-Making under Conditions That Promote Goal-Directed Control

It has been proposed that compulsive drug seeking reflects an underlying dysregulation in adaptive behavior that favors habitual (automatic and inflexible) over goal-directed (deliberative and highly flexible) action selection. Rodent studies have established that repeated exposure to cocaine or amphetamine facilitates the development of habits, producing behavior that becomes unusually insensitive to a reduction in the value of its outcome. The current study more directly investigated the effects of cocaine pre-exposure on goal-directed learning and action selection using an approach that discourages habitual performance. After undergoing a 15-day series of cocaine (15 or 30 mg/kg, i.p.) or saline injections and a drug withdrawal period, rats were trained to perform two different lever-press actions for distinct reward options. During a subsequent outcome devaluation test, both cocaine- and saline-treated rats showed a robust bias in their choice between the two actions, preferring whichever action had been trained with the reward that retained its value. Thus, it appears that the tendency for repeated cocaine exposure to promote habit formation does not extend to a more complex behavioral scenario that encourages goal-directed control. To further explore this issue, we assessed how prior cocaine treatment would affect the rats' ability to learn about a selective reduction in the predictive relationship between one of the two actions and its outcome, which is another fundamental feature of goal-directed behavior. Interestingly, we found that cocaine-treated rats showed enhanced, rather than diminished, sensitivity to this action-outcome contingency degradation manipulation. Given their mutual dependence on striatal dopamine signaling, we suggest that cocaine's effects on habit formation and contingency learning may stem from a common adaptation in this neurochemical system.

The effect of US signalling and the US-CS interval on backward conditioning in mice

The effect of US signalling and the US-CS interval in backward conditioning was assessed in mice. For one group of mice the presentation of food was signalled by a tone and for another group, food was unsignalled. For half of the mice, within each group, the presentation of food preceded a visual cue by 10 s. For the other half, food was presented at the start of the visual cue (0-s US-CS interval), resulting in simultaneous pairings of these events. A summation test and a subsequent retardation test were used to assess the inhibitory effects of backward conditioning in comparison to training with a non-reinforced visual cue that controlled for the possible effects of latent inhibition and conditioned inhibition caused as a consequence of differential conditioning. In the summation test unsignalled presentations of the US resulted in inhibition when the US-CS interval was 10 s, but not 0 s. Signalled presentations of the US resulted in inhibition, independent of the US-CS interval. In the retardation test, independent of US signalling, a US-CS interval of 10 s failed to result in inhibition, but an interval of 0 s resulted in greater conditioned responding to the backward CS than the control CS. A generalisation decrement account of the effect of signalling the US with a 0-s US-CS interval, which resulted in reduced responding in the summation test and faster acquisition in the retardation test, is discussed.

δ-Opioid receptors in the accumbens shell mediate the influence of both excitatory and inhibitory predictions on choice

BACKGROUND AND PURPOSE

Stimuli that predict rewarding events can control choice between future actions, and this control could be mediated by δ-opioid receptors in the nucleus accumbens shell (NAc-S). Stimuli predicting the absence of important events can also guide choice, although it remains unknown whether they do so via changes in an accumbal δ-opioid receptor-related process.

EXPERIMENTAL APPROACH

δ-opioid receptor-eGFP mice were trained to perform two instrumental actions that delivered different food outcomes. Choice between the two actions was then tested in the presence of stimuli paired with either the delivery or the non-delivery of each of the two outcomes. Bilateral infusions of the δ-opioid receptor antagonist naltrindole into the NAc-S were used to determine the role of these receptors at the time of choice and δ-opioid receptor expression in the NAc-S used to assess functional activity.

KEY RESULTS

A stimulus predicting a specific outcome biased choice performance towards the action previously earning that same outcome. In contrast, a stimulus signalling the absence of that outcome biased performance away from the action that delivered that outcome towards actions associated with the absence of that outcome. Both effects were associated with increased δ-opioid receptor expression on the membrane of cholinergic interneurons within the NAc-S. Furthermore, both effects were blocked by naltrindole infused into the NAc-S.

CONCLUSIONS AND IMPLICATIONS

These findings suggest that δ-opioid receptors in the NAc-S were involved in the effects of predictive learning on choice between actions, whether those predictions involve the presence or absence of specific rewarding events.

LINKED ARTICLES

This article is part of a themed section on Opioids: New Pathways to Functional Selectivity. To view the other articles in this section visit http://dx.doi.org/10.1111/bph.2015.172.issue-2.

Psychological and neural mechanisms of experimental extinction: a selective review

The present review examines key psychological concepts in the study of experimental extinction and implications these have for an understanding of the underlying neurobiology of extinction learning. We suggest that many of the signature characteristics of extinction learning (spontaneous recovery, renewal, reinstatement, rapid reacquisition) can be accommodated by the standard associative learning theory assumption that extinction results in partial erasure of the original learning together with new inhibitory learning. Moreover, we consider recent behavioral and neural evidence that supports the partial erasure view of extinction, but also note shortcomings in our understanding of extinction circuits as these relate to the negative prediction error concept. Recent work suggests that common prediction error and stimulus-specific prediction error terms both may be required to explain neural plasticity both in acquisition and extinction learning. In addition, we suggest that many issues in the content of extinction learning have not been fully addressed in current research, but that neurobiological approaches should be especially helpful in addressing such issues. These include questions about the nature of extinction learning (excitatory CS-No US, inhibitory CS-US learning, occasion setting processes), especially as this relates to studies of the micro-circuitry of extinction, as well as its representational content (sensory, motivational, response). An additional understudied problem in extinction research is the role played by attention processes and their underlying neural networks, although some research and theory converge on the idea that extinction is accompanied by attention decrements (i.e., habituation-like processes).

Outcome specificity in deepened extinction may limit treatment feasibility: co-presentation of a food cue interferes with extinction of cue-elicited cocaine seeking

BACKGROUND

We previously showed that presenting two cocaine cues simultaneously during extinction deepens the extinction of cue-elicited cocaine seeking (Kearns et al., 2012). The present study investigated whether compounding a non-drug appetitive cue with a cocaine cue would similarly deepen extinction.

METHODS

In Experiment 1, tone and click were each first established as discriminative stimuli for cocaine-reinforced responding and light was a cue for food-reinforced responding. In an initial extinction phase, all stimuli were presented individually. Then, during an additional compound extinction session, rats received 8 presentations of one of the cocaine cues (counterbalanced over subjects) simultaneously with light and 8 presentations of the other cue alone. A spontaneous recovery test was used to evaluate the effectiveness of the extinction treatments. Experiment 2 was performed under conditions designed to match those of Experiment 1, except food was the reinforcer in tone and click instead of cocaine.

RESULTS

In Experiment 1, the cocaine cue compounded with the food cue during extinction controlled greater spontaneous recovery of cocaine seeking than the cocaine cue always presented alone. In contrast, Experiment 2 demonstrated deepened extinction of responding to a food cue when both compounded cues were food cues.

CONCLUSIONS

Results suggest that deepened extinction depends on the compound presentation of cues associated with the same reinforcer. Compound presentation of cues associated with different reinforcers could lead to an enhancement of responding. Care is urged in attempts to deepen the extinction of cue-elicited drug seeking by compounding drug cues with non-drug cues.

The effects of amphetamine sensitization on conditioned inhibition during a Pavlovian-instrumental transfer task in rats

RATIONALE

Psychostimulant sensitization heightens behavioral and motivational responses to reward-associated stimuli; however, its effects on stimuli associated with reward absence are less understood.

OBJECTIVES

We examined whether amphetamine sensitization alters performance during Pavlovian-instrumental transfer (PIT) to conditioned excitors and inhibitors. We further sought to characterize the effects of amphetamine sensitization on learning versus performance by exposing rats to amphetamine prior to Pavlovian training or between training and test.

METHODS

Adult male Long-Evans rats were given conditioned inhibition (A+/AX-) and Pavlovian (B+) training, followed by variable-interval instrumental conditioning. Rats were sensitized to D-amphetamine (2 mg/kg daily injections for 7 days) or served as non-exposed controls. Rats were given a PIT test, in which they were presented with stimulus B alone or in compound with the conditioned inhibitor (BX).

RESULTS

During the PIT test, control rats significantly reduced instrumental responding on BX trials (to approximately 50 % of responding to B). Amphetamine sensitization prior to Pavlovian conditioning increased lever pressing on BX trials and reduced lever pressing on B trials compared to controls. Amphetamine sensitization between training and test increased lever pressing on B and BX trials compared to controls. No effects of sensitization were observed on conditioned food cup approach.

CONCLUSIONS

Amphetamine sensitization increases instrumental responding during PIT to a conditioned inhibitor by enhancing the excitation of conditioned stimuli and reducing the inhibition of conditioned inhibitors.

The effects of amphetamine exposure on outcome-selective Pavlovian-instrumental transfer in rats

RATIONALE

Repeated exposure to psychostimulants alters behavioral responses to reward-related cues; however, the motivational underpinnings of this effect have not been fully characterized.

OBJECTIVES

The following study was designed to examine how amphetamine sensitization affects performance in rats on a series of Pavlovian and operant tasks that distinguish between general-incentive and outcome-selective forms of conditioned responses.

METHODS

Adult male rats underwent Pavlovian and instrumental training for food pellet rewards. Following training, rats were sensitized to D-amphetamine (2 mg/kg for 7 days). Rats were subsequently tested on an outcome-selective Pavlovian-instrumental transfer (PIT) task, an outcome-reinstatement task, and an outcome devaluation task. Additionally, in a separate experiment, PIT was assessed in amphetamine-sensitized and control rats using a Pavlovian backward-conditioned stimulus.

RESULTS

Repeated amphetamine exposure sensitized locomotor activity to acute amphetamine challenge. Amphetamine altered responses to CS presentations by increasing conditioned approach. During tests of PIT, amphetamine-treated rats showed no outcome-selectivity in their responding, responding to a CS whether or not it shared a common outcome with the instrumental response. No effect of amphetamine sensitization was observed on tests of outcome-selective reinstatement by outcome delivery or action selection based on outcome value. Amphetamine-sensitized rats showed impaired outcome-selective PIT to a backward CS but were unaltered in conditioned approach.

CONCLUSIONS

Amphetamine sensitization prevents outcome-selective responding during PIT, which is dissociable from amphetamine's effects on conditioned approach. These data suggest fundamental alterations in how stimuli motivate action in addiction.

Importance of associative learning processes for one-trial behavioral sensitization of preweanling rats

During adulthood, associative learning is necessary for the expression of one-trial behavioral sensitization; however, it is uncertain whether the same associative processes are operative during the preweanling period. Two strategies were used to assess the importance of associative learning for one-trial behavioral sensitization of preweanling rats. In the initial experiments, we varied both the sequence and time interval between presentation of the conditioned stimulus (CS, novel environment) and unconditioned stimulus (US, cocaine). In the final experiment, we determined whether electroconvulsive shock-induced retrograde amnesia would disrupt one-trial behavioral sensitization. Results showed that robust-sensitized responding was apparent regardless of the sequence in which cocaine and the novel environment (the presumptive CS) were presented. Varying the time between CS and US presentation (0, 3, or 6 h) was also without effect. Results from experiment 3 showed that single or multiple electroconvulsive shock treatments did not alter the expression of the sensitized response. Therefore, these data indicated that one-trial behavioral sensitization of preweanling rats was exclusively mediated by nonassociative mechanisms and that associative processes did not modulate sensitized responding. These findings are in contrast to what is observed during adulthood, as adult rats exhibit one-trial behavioral sensitization only when associative processes are operative.

Dopamine receptor blockade attenuates the general incentive motivational effects of noncontingently delivered rewards and reward-paired cues without affecting their ability to bias action selection

Environmental cues affect our behavior in a variety of ways. Despite playing an invaluable role in guiding our daily activities, such cues also appear to trigger the harmful, compulsive behaviors that characterize addiction and other disorders of behavioral control. In instrumental conditioning, rewards and reward-paired cues bias action selection and invigorate reward-seeking behaviors, and appear to do so through distinct neurobehavioral processes. Although reward-paired cues are known to invigorate performance through a dopamine-dependent incentive motivational process, it is not known if dopamine also mediates the influence of rewards and reward-paired cues over action selection. The current study contrasted the effects of systemic administration of the nonspecific dopamine receptor antagonist flupentixol on response invigoration and action bias in Pavlovian-instrumental transfer, a test of cue-elicited responding, and in instrumental reinstatement, a test of noncontingent reward-elicited responding. Hungry rats were trained on two different stimulus-outcome relationships (eg, tone-grain pellets and noise-sucrose solution) and two different action-outcome relationships (eg, left press-grain and right press-sucrose). At test, we found that flupentixol pretreatment blocked the response invigoration generated by the cues but spared their ability to bias action selection to favor the action whose outcome was signaled by the cue being presented. The response-biasing influence of noncontingent reward deliveries was also unaffected by flupentixol. Interestingly, although flupentixol had a modest effect on the immediate response invigoration produced by those rewards, it was particularly potent in countering the lingering enhancement of responding produced by multiple reward deliveries. These findings indicate that dopamine mediates the general incentive motivational effects of noncontingent rewards and reward-paired cues but does not support their ability to bias action selection.

Differential outcome effects in pavlovian biconditional and ambiguous occasion setting tasks

Three experiments with rats explored the differential outcome effect (DOE) using a pavlovian magazine approach conditioning preparation. Experiment 1 compared groups trained on a biconditional discrimination (AX+, AY-, BX-, BY+) with differential or nondifferential outcomes, and Experiment 2 examined this using an ambiguous occasion setting task (e.g., AX+, X-, Y+, AY-). In both experiments, subjects trained with differential outcomes learned the tasks better than subjects trained with nondifferential outcomes. Furthermore, subjects given differential outcome training learned the positive occasion setting component of the ambiguous task more efficiently than the negative occasion setting component, although both were enhanced by differential outcome training. Experiment 3 demonstrated that the ambiguous occasion setting task was reversed more readily when the target-outcome relations (as opposed to the modulator-outcome relations) were maintained during the reversal phase. These data suggest that an acquired distinctiveness effect may be responsible for the DOE in pavlovian learning.

The basolateral amygdala mediates the effects of cues associated with meal interruption on feeding behavior

Considerable evidence shows that environmental cues that signal food delivery when rats are food-deprived can substantially potentiate feeding later when rats are food-sated. Similarly, cues associated with meal interruption, food removal or impending food scarcity may also induce increased eating. For example, after learning the association between a discrete "interruption" stimulus and the unexpected termination of food trials, sated rats show enhanced food consumption when exposed to that stimulus. In Experiment 1, unlike sham-lesioned controls, rats with bilateral excitotoxic lesions of the basolateral amygdala (BLA) failed to display such cue-potentiated feeding. In Experiment 2, potentiation of feeding by an interruption signal was found to be food-specific. That is, a stimulus that signaled interruption of trials with one food but not trials with a second food later only facilitated consumption of the first food. These studies extend our knowledge of the psychological and neural processes underlying cue-induced feeding. Understanding these mechanisms may contribute our understanding of the etiology and treatment of binge eating disorders.

The search for symmetry: 25 years in review

It has been 25 years since the publication of Sidman et al.'s (1982) report on the search for symmetry in nonhuman animals. They attributed their nonhuman subjects' failure to the absence of some critical experiences (e.g., exemplar training, control of location variables, and generalized identity matching). Since then, species ranging from rats to chimpanzees have been tested on symmetry, and the results have been equivocal. Twenty-four investigations of symmetry in nonhumans are reviewed to determine whether the underlying factors first addressed by Sidman et al. (1982) have been verified and whether new factors have been identified. The emergent picture shows that the standard procedures as typically implemented on a three-key apparatus are insufficient by themselves to produce emergent symmetry in nonhumans. Recent successful demonstrations of symmetry in sea lions and pigeons have clarified certain important stimulus control variables (i.e., select and reject control) and suggest avenues for future research. Reliable symmetry may be achievable with nonhumans if training and test procedures that encourage compatible stimulus-control topographies and relations are designed.

DIFFERENTIAL EFFECTS OF A FOOD-BASED CONDITIONED INHIBITOR ON FOOD- OR COCAINE-SEEKING BEHAVIOR

The present experiment compared the effects of a food-based conditioned inhibitor on food seeking vs. cocaine seeking behavior. In two groups of rats, the A+/AB- Pavlovian conditioned inhibition procedure was used to create a conditioned inhibitor for food. Then, for one group of rats (Food-Food Group), a click stimulus was established as an operant discriminative stimulus (S(D)) for food-reinforced lever pressing. In the other group (Food-Cocaine Group), the click was established as an S(D) for cocaine self-administration. In testing, the putative inhibitor for food was simultaneously presented with the click for the first time in both groups. In the Food-Food Group, the food-based inhibitor suppressed responding occasioned by the click significantly more than did a neutral control stimulus. In contrast, in the Food-Cocaine Group, there was no difference in the amount of suppression produced by the food-based inhibitor and the control stimulus. These results suggest that the effects of food-based Pavlovian conditioned inhibitors are specific for food-motivated behavior and do not easily transfer to cocaine-motivated behavior.

Formation of excitatory and inhibitory associations between absent events

Considerable evidence indicates that associations may be formed between two events even when one or both of them is absent at the time of learning. Previously, some researchers asserted that excitatory associations are formed when associatively activated representations for two events are paired, whereas others claimed that inhibitory associations are formed. In three experiments, the authors investigated the nature of tone-sucrose learning when associatively activated representations of those events were paired in the absence of either of the events themselves. Experiment 1 found substantial excitatory learning when the tone surrogate preceded the sucrose surrogate in training. Experiment 2 evaluated other accounts for the results of Experiment 1, and Experiment 3 found evidence for inhibitory tone-sucrose learning when the tone and sucrose surrogates were presented in simultaneous or backward order. The results indicated that the nature of representation-mediated learning is influenced by some of the same variables as more standard associative learning.

Differential involvement of the basolateral amygdala and mediodorsal thalamus in instrumental action selection

Although it has been shown that the basolateral amygdala (BLA) and the mediodorsal thalamus (MD) are critical for goal-directed instrumental performance, much remains unknown about the respective contributions of these structures to action selection. The current study assessed the effects of post-training BLA and MD lesions on several tests of instrumental action selection. We found that MD damage disrupted the influence of pavlovian cues over action selection but left intact rats' ability to select actions based on either the expected value or the discriminative stimulus properties of the outcome. In contrast, BLA lesions impaired performance on all three tests of action selection. Because both lesion types disrupted the influence of cues that signal reward over instrumental performance, we then investigated the involvement of these structures in pavlovian contingency learning using a task in which the predictive status of one of two cues is degraded by delivering its outcome noncontingently during the intertrial interval. As expected, the sham group selectively suppressed their conditioned approach performance to the cue that no longer signaled its outcome but continued to respond to the control stimulus. In contrast, both lesioned groups were impaired on this task. Interestingly, whereas the MD group displayed a nonspecific reduction in responding to both cues, the BLA group continued to show high levels of responding to both cues as if their performance was completely insensitive to this contingency manipulation. These findings demonstrate that the BLA and MD make important yet distinct contributions to instrumental action selection.

Excitatory and inhibitory learning with absent stimuli

Three experiments showed that 2 associatively activated stimulus representations may engage in excitatory or inhibitory learning, depending on their temporal relationship. Experiment 1a suggested that simultaneously activated stimulus representations show evidence of inhibitory learning in an acquisition test. Experiment 1b showed similar evidence of inhibition in a summation test. Experiment 2 found that activation of 2 stimulus representations in a serial compound resulted in excitatory learning between the antecedent and the subsequent (forward) and inhibitory learning between the subsequent and the antecedent (backward). The results show the dynamic influence of temporal contiguity on mediated learning.

Selective reinstatement of instrumental performance depends on the discriminative stimulus properties of the mediating outcome

We conducted three experiments to investigate the associative structure underlying the reinstatement of instrumental performance after extinction. In each experiment, rats were initially rewarded on two responses with different outcomes. At test, both responses were extinguished in order to assess the impact of a single noncontingent outcome delivery on response selection. Experiment 1 found evidence of outcome-selective reinstatement (i.e., more responses were performed on the lever that was trained with the reinstating outcome than on the other lever). Experiment 2 demonstrated that the outcome's capacity to reinstate performance was not affected by a reduction in its motivational value. Experiment 3 found evidence that the reinstating outcome selectively retrieved the response it signaled rather than the response it followed during training. Together, these findings are consistent with the view that instrumental reinstatement depends on the discriminative stimulus properties of the reinstating outcome.

Unconditioned stimulus devaluation effects in nutrient-conditioned flavor preferences

Experiments with different temporal relations between the conditioned stimulus (CS) and the unconditioned stimulus (US) in conditioning assessed whether US devaluation effects can be obtained after nutrient-conditioned flavor preference learning. One flavor (CScarb) was paired with a carbohydrate, Polycose; a 2nd flavor (CSprot) was paired with a protein, casein; and a 3rd flavor (CS-) was presented by itself. Following conditioning, one of the nutrients was devalued through pairings with lithium chloride in the absence of the CS flavors. In a subsequent 2-bottle test, rats preferred CScarb over CSprot; however, this preference was smaller when the carbohydrate was devalued than when the protein was devalued. Results suggest that CS flavors are able to form associations with the sensory features of nutrient USs under a wide variety of circumstances.