Encoding of the unconditioned stimulus in Pavlovian conditioning

Dopamine projections to the basolateral amygdala drive the encoding of identity-specific reward memories

To make adaptive decisions, we build an internal model of the associative relationships in an environment and use it to make predictions and inferences about specific available outcomes. Detailed, identity-specific cue-reward memories are a core feature of such cognitive maps. Here we used fiber photometry, cell-type and pathway-specific optogenetic manipulation, Pavlovian cue-reward conditioning and decision-making tests in male and female rats, to reveal that ventral tegmental area dopamine (VTADA) projections to the basolateral amygdala (BLA) drive the encoding of identity-specific cue-reward memories. Dopamine is released in the BLA during cue-reward pairing; VTADA→BLA activity is necessary and sufficient to link the identifying features of a reward to a predictive cue but does not assign general incentive properties to the cue or mediate reinforcement. These data reveal a dopaminergic pathway for the learning that supports adaptive decision-making and help explain how VTADA neurons achieve their emerging multifaceted role in learning.

The effects of extinction and an explicitly unpaired treatment on the reinforcing properties of a Pavlovian conditioned stimulus

This series of experiments examined the effects of extinction and an explicitly unpaired treatment on the ability of a conditioned stimulus (CS) to function as a reinforcer. Rats were trained to lever press for food, exposed to pairings of a noise CS and food, and, finally, tested for their willingness to lever press for the CS in the absence of the food. Experiment 1 provided a demonstration of conditioned reinforcement (using controls that were only exposed to unpaired presentations of the CS and food) and showed that it was equivalent after one or four sessions of CS-food pairings. Experiments 2 and 3 showed that, after one session of CS-food pairings, repeated presentations of the CS alone reduced its reinforcing properties; but after four sessions of CS-food pairings, repeated presentations of the CS alone had no effect on these properties. Experiment 4 showed that, after four sessions of CS-food pairings, explicitly unpaired presentations of the CS and food completely undermined conditioned reinforcement. Finally, Experiment 5 provided within-experiment evidence that, after four sessions of CS-food pairings, the reinforcing properties of the CS were disrupted by explicitly unpaired presentations of the CS and food but spared by repeated presentations of the CS alone. Together, these findings indicate that the effectiveness of extinction in undermining the reinforcing properties of a CS depends on its level of conditioning; and that, where extinction fails to disrupt these properties, they are successfully undermined by an explicitly unpaired treatment. They are discussed with respect to findings in the literature on Pavlovian-to-instrumental transfer; and the Rescorla-Wagner model, which anticipates that an explicitly unpaired treatment will be more effective than extinction in reversing the effects of conditioning.

Effects of predictive and incentive value manipulation on sign- and goal-tracking behavior

When a neutral stimulus is repeatedly paired with an appetitive reward, two different types of conditioned approach responses may develop: a sign-tracking response directed toward the neutral cue, or a goal-tracking response directed toward the location of impending reward delivery. Sign-tracking responses have been postulated to result from attribution of incentive value to conditioned cues, while goal-tracking reflects the assignment of only predictive value to the cue. We therefore hypothesized that sign-tracking rats would be more sensitive to manipulations of incentive value, while goal-tracking rats would be more responsive to changes in the predictive value of the cue. We tested sign- and goal-tracking before and after devaluation of a food reward using lithium chloride, and tested whether either response could be learned under negative contingency conditions that precluded any serendipitous reinforcement of the behavior that might support instrumental learning. We also tested the effects of blocking the predictive value of a cue using simultaneous presentation of a pre-conditioned cue. We found that sign-tracking was sensitive to outcome devaluation, while goal-tracking was not. We also confirmed that both responses are Pavlovian because they can be learned under negative contingency conditions. Goal-tracking was almost completely blocked by a pre-conditioned cue, while sign-tracking was much less sensitive to such interference. These results indicate that sign- and goal-tracking may follow different rules of reinforcement learning and suggest a need to revise current models of associative learning to account for these differences.

Outcome devaluation by specific satiety disrupts sensory-specific Pavlovian-to-instrumental transfer

Reward predictive cues can selectively motivate instrumental behaviors that predict the same rewarding outcomes, an effect known as specific Pavlovian-to-instrumental transfer (PIT). This selective effect is thought to be mediated by a representation of the sensory specific properties of an outcome, that has become associated with both the Pavlovian cue and the instrumental response during initial learning. Specific satiety is a common method of outcome devaluation that reduces an outcome's value but might also lead to the habituation of the outcome's sensory properties. Previous research has demonstrated that specific PIT is insensitive to changes in specific outcome value following taste aversion devaluation, as well as general satiety manipulations, and therefore specific satiety should not disrupt specific PIT by reducing outcome value. The present rodent experiments used a specific satiety devaluation procedure immediately prior to a specific PIT test to show that habituation of these outcome specific sensory representations can disrupt its efficacy as a stimulus and abolish the specific PIT effect. Experiment 1 employed a two-lever choice test to show that a non-devalued stimulus supports specific PIT, whereas a devalued stimulus abolished the specific PIT effect. Experiment 2 replicated this procedure while controlling for response competition by using a single-lever test to confirm that a devalued stimulus abolishes the specific PIT effect. These findings demonstrate that specific satiety can disrupt the ability of an outcome specific representation to support specific PIT. Given previous findings that specific PIT is insensitive to changes in outcome value by general satiety and taste aversion devaluation, this suggests that specific satiety devaluation might disrupt the use of sensory specific outcome representations to guide behavior via a mechanism that is independent of the outcome's current value.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

Mechanisms of higher-order learning in the amygdala

Adaptive behaviour is under the potent control of environmental cues. Such cues can acquire value by virtue of their associations with outcomes of motivational significance, be they appetitive or aversive. There are at least two ways through which an environmental cue can acquire value, through first-order and higher-order conditioning. In first-order conditioning, a neutral cue is directly paired with an outcome of motivational significance. In higher-order conditioning, a cue is indirectly associated with motivational events via a directly conditioned first-order stimulus. The present article reviews some of the associations that support learning in first- and higher-order conditioning, as well as the role of the BLA and the molecular mechanisms involved in these two types of learning.

General Pavlovian-instrumental transfer tests reveal selective inhibition of the response type - whether Pavlovian or instrumental - performed during extinction

The present experiments examined whether extinction of a stimulus predicting food affects the ability of that stimulus to energize instrumental performance to obtain food. We first used a general Pavlovian instrumental transfer (PIT) paradigm in which rats were first given Pavlovian conditioning with a stimulus predicting one type of food outcome and were then trained to lever press for a different food outcome. We found that the Pavlovian stimulus enhanced performance of the lever press response and that this enhancement was preserved after extinction of that stimulus (Experiment 1) even when the context was manipulated to favor the expression of extinction (Experiment 2). Next, we assessed whether extinction influenced the excitatory effect of a stimulus when it was trained as a discriminative stimulus. Extinction of this stimulus alone had no effect on its ability to control instrumental performance; however, when extinguished with its associated lever press response, discriminative control was lost (Experiments 3 and 4). Finally, after instrumental and Pavlovian training, we extinguished a Pavlovian stimulus predicting one food outcome with a lever press response that delivered a different outcome. In a general PIT test, we found this extinction abolished the ability of the Pavlovian stimulus to elevate responding on a lever trained with a different outcome, revealing for the first time that extinction can abolish the general PIT effect. We conclude that extinction can produce an inhibitory association between the stimulus and the general response type, whether Pavlovian or instrumental, performed during the extinction training.

A bidirectional corticoamygdala circuit for the encoding and retrieval of detailed reward memories

Adaptive reward-related decision making often requires accurate and detailed representation of potential available rewards. Environmental reward-predictive stimuli can facilitate these representations, allowing one to infer which specific rewards might be available and choose accordingly. This process relies on encoded relationships between the cues and the sensory-specific details of the rewards they predict. Here, we interrogated the function of the basolateral amygdala (BLA) and its interaction with the lateral orbitofrontal cortex (lOFC) in the ability to learn such stimulus-outcome associations and use these memories to guide decision making. Using optical recording and inhibition approaches, Pavlovian cue-reward conditioning, and the outcome-selective Pavlovian-to-instrumental transfer (PIT) test in male rats, we found that the BLA is robustly activated at the time of stimulus-outcome learning and that this activity is necessary for sensory-specific stimulus-outcome memories to be encoded, so they can subsequently influence reward choices. Direct input from the lOFC was found to support the BLA in this function. Based on prior work, activity in BLA projections back to the lOFC was known to support the use of stimulus-outcome memories to influence decision making. By multiplexing optogenetic and chemogenetic inhibition we performed a serial circuit disconnection and found that the lOFC→BLA and BLA→lOFC pathways form a functional circuit regulating the encoding (lOFC→BLA) and subsequent use (BLA→lOFC) of the stimulus-dependent, sensory-specific reward memories that are critical for adaptive, appetitive decision making.

How predictive learning influences choice: Evidence for a GPCR-based memory process necessary for Pavlovian-instrumental transfer

Predictive learning endows stimuli with the capacity to signal both the sensory-specific and general motivational properties of their associated rewards or outcomes. These two signals can be distinguished behaviorally by their influence on the selection and performance of instrumental actions, respectively. This review focuses on how sensory-specific predictive learning guides choice between actions that earn otherwise equally desirable outcomes. We describe evidence that outcome-specific predictive learning is encoded in the basolateral amygdala and drives the accumulation of delta-opioid receptors on the surface of cholinergic interneurons located in the nucleus accumbens shell. This accumulation constitutes a novel form of cellular memory, not for outcome-specific predictive learning per se but for the selection of, and choice between, future instrumental actions. We describe recent evidence regarding the cascade of events necessary for the formation and expression of this cellular memory and point to open questions for future research into this process. Beyond these mechanistic considerations, the discovery of this new form of memory is consistent with recent evidence suggesting that intracellular rather than synaptic changes can mediate learning-related plasticity to modify brain circuitry to prepare for future significant events.

Review of Orbitofrontal Cortex in Alcohol Dependence: A Disrupted Cognitive Map?

Alcoholism is a persistent worldwide problem associated with long-lasting impairments to decision making processes. Some aspects of dysfunction are thought to reflect alcohol-induced changes to relevant brain areas such as the orbitofrontal cortex (OFC). In this review, we will examine how chronic alcohol exposure alters OFC function to potentially contribute to maladaptive decision making, and explore experimental behavioral approaches that may be better suited to test whether alcohol dependence disrupts OFC's function. We argue that although past works suggest impairments in aspects of OFC function, more information may be gained by specifically targeting tasks to the broader function of OFC as put forth by the recent hypothesis of OFC as a "cognitive map" of task space. Overall, we suggest that such a focus could provide a better understanding of how OFC function changes in alcohol dependence, and could inform better assessment tools and treatment options for clinicians working with this population.

Degradation of an appetitive olfactory memory via devaluation of sugar reward is mediated by 5-HT signaling in the honey bee

Conditioned taste aversion (CTA) learning induces the devaluation of a preferred food through its pairing with a stimulus inducing internal illness. In invertebrates, it is still unclear how this aversive learning impairs the memories of stimuli that had been associated with the appetitive food prior to its devaluation. Here we studied this phenomenon in the honey bee and characterized its neural underpinnings. We first trained bees to associate an odorant (conditioned stimulus, CS) with appetitive fructose solution (unconditioned stimulus, US) using a Pavlovian olfactory conditioning. We then subjected the bees that learned the association to a CTA training during which the antennal taste of fructose solution was contingent or not to the ingestion of quinine solution, which induces malaise a few hours after ingestion. Only the group experiencing contingent fructose stimulation and quinine-based malaise exhibited a decrease in responses to the fructose and a concomitant decrease in odor-specific retention in tests performed 23 h after the original odor conditioning. Furthermore, injection of dopamine- and serotonin-receptor antagonists after CTA learning revealed that this long-term decrease was mediated by serotonergic signaling as its blockade rescued both the responses to fructose and the odor-specific memory 23 h after conditioning. The impairment of a prior CS memory by subsequent CTA conditioning confirms that bees retrieve a devaluated US representation when presented with the CS. Our findings further highlight the importance of serotonergic signaling in aversive learning in the bee and uncover mechanisms underlying aversive memories induced by internal illness in invertebrates.

Effects of hM4Di activation in CamKII basolateral amygdala neurons and CNO treatment on sensory-specific vs. general PIT: refining PIT circuits and considerations for using CNO

BACKGROUND

Pavlovian stimuli can influence instrumental behaviors via phenomena such as Pavlovian-to-instrumental transfer (PIT). PIT arises via dissociable processes as sensory-specific PIT (SS-PIT) and general PIT. The basolateral amygdala (BLA) mediates SS-PIT, but not general PIT. However, the specific BLA neuronal populations involved are unknown.

AIMS

To determine the contribution of glutamatergic BLA neurons to the expression of SS-PIT and to the recall of sensory-specific properties of stimulus-outcome associations.

METHODS

BLA neurons were transduced with virus containing either GFP or hM4Di, driven by the CamKII promoter. Rats were then tested for SS and general PIT and subsequently for expression of Pavlovian outcome devaluation effects and conditioned taste aversion following injections of vehicle or clozapine-N-oxide (CNO, the hM4Di agonist).

RESULTS

CNO selectively blocked SS-PIT in the hM4Di-expressing group, but not controls, without altering expression of Pavlovian outcome devaluation or sensory-specific taste aversion in either group. Unexpectedly, CNO disrupted general PIT in both groups.

CONCLUSIONS

CamKII BLA neurons mediate the expression of SS-PIT by enabling Pavlovian stimuli to trigger recall of the correct action-outcome associations rather than by mediating recall of the sensory-specific properties of the stimulus-outcome association. Separately, our data demonstrate that CNO alone is sufficient to disrupt affective, but not sensory-specific processes, an effect that was not due to generalized motor disruption. This non-specific effect on general PIT may be related to CNO-induced shifts in internal state. Together, these data identify BLA CamKII neurons as critical for the expression of SS-PIT and reveal important considerations for using CNO to study general affective motivation.

Roles of context in acquisition of human instrumental learning: Implications for the understanding of the mechanisms underlying context-switch effects

Four experiments in human instrumental learning explored the associations involving the context that develop after three trials of training on simple discriminations. Experiments 1 and 4 found a deleterious effect of switching the learning context that cannot be explained by the context-outcome binary associations commonly used to explain context-switch effects after short training in human predictive learning and in animal Pavlovian conditioning. Evidence for context-outcome (Experiment 2), context-discriminative stimulus (Experiment 3), and context-instrumental response (Experiment 4) binary associations was found within the same training paradigm, suggesting that contexts became associated with all the elements of the situation, regardless of whether those associations played a role in a specific context-switch effect detected on performance.

Triggering Avoidance: Dissociable Influences of Aversive Pavlovian Conditioned Stimuli on Human Instrumental Behavior

The present study investigates human aversive Pavlovian-to-Instrumental Transfer (PIT) and possible influences of outcome devaluation and instrumental overtraining on this effect. PIT measures the extent to which a Pavlovian conditioned stimulus (CS) can increase instrumental responses independently paired with the same (outcome-specific transfer) or a different (general transfer) reinforcer. Two measures of PIT were obtained: the percentage of instrumental responses and the vigor of such responses. Thirty-eight volunteers performed a standard PIT task sequence. Results showed a double dissociation between outcome-specific and general transfer: the first selectively expressed in the amount of responses, the second in the vigor measure solely. Furthermore, outcome-specific transfer was enhanced by overtraining, but not affected by devaluation. General transfer, on the other hand, was affected by neither overtraining, nor devaluation. A positive correlation between general transfer and sensitivity to punishments was found. Findings are discussed in terms of hypothetically different underlying neurobehavioral mechanisms and their relations to habits and goal-directed behavior.

Amygdala mu-opioid receptors mediate the motivating influence of cue-triggered reward expectations

Environmental reward-predictive stimuli can retrieve from memory a specific reward expectation that allows them to motivate action and guide choice. This process requires the basolateral amygdala (BLA), but little is known about the signaling systems necessary within this structure. Here we examined the role of the neuromodulatory opioid receptor system in the BLA in such cue-directed action using the outcome-specific Pavlovian-to-instrumental transfer (PIT) test in rats. Inactivation of BLA mu-, but not delta-opioid receptors was found to dose-dependently attenuate the ability of a reward-predictive cue to selectively invigorate the performance of actions directed at the same unique predicted reward (i.e. to express outcome-specific PIT). BLA mu-opioid receptor inactivation did not affect the ability of a reward itself to similarly motivate action (outcome-specific reinstatement), suggesting a more selective role for the BLA mu-opioid receptor in the motivating influence of currently unobservable rewarding events. These data reveal a new role for BLA mu-opioid receptor activation in the cued recall of precise reward memories and the use of this information to motivate specific action plans.

Nucleus accumbens core dopamine signaling tracks the need-based motivational value of food-paired cues

Environmental reward-predictive stimuli provide a major source of motivation for instrumental reward-seeking activity and this has been linked to dopamine signaling in the nucleus accumbens (NAc) core. This cue-induced incentive motivation can be quite general, not restricted to instrumental actions that earn the same unique reward, and is also typically regulated by one's current need state, such that cues only motivate actions when this is adaptive. But it remains unknown whether cue-evoked dopamine signaling is similarly regulated by need state. Here, we used fast-scan cyclic voltammetry to monitor dopamine concentration changes in the NAc core of rats during a Pavlovian-to-instrumental transfer task in which the motivating influence of two cues, each signaling a distinct food reward (sucrose or food pellets), over an action earning a third unique food reward (polycose) was assessed in a state of hunger and of satiety. Both cues elicited a robust NAc dopamine response when hungry. The magnitude of the sucrose cue-evoked dopamine response correlated with the Pavlovian-to-instrumental transfer effect that was selectively induced by this stimulus. Satiety attenuated these cue-evoked dopamine responses and behavioral responding, even though rats had never experienced the specific food rewards in this state. These data demonstrate that cue-evoked NAc core responses are sensitive to current need state, one critical variable that determines the current adaptive utility of cue-motivated behavior. Food-predictive stimuli motivate food-seeking behavior. Here, we show that food cues evoke a robust nucleus accumbens core dopamine response when hungry that correlates with the cue's ability to invigorate general food seeking. This response is attenuated when sated, demonstrating that food cue-evoked accumbens dopamine responses are sensitive to the need state information that determines the current adaptive utility of cue-motivated action.

Individual variability in behavioral flexibility predicts sign-tracking tendency

Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS-) and second-order (SOCS+, SOCS-) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.

The basolateral amygdala in reward learning and addiction

Sophisticated behavioral paradigms partnered with the emergence of increasingly selective techniques to target the basolateral amygdala (BLA) have resulted in an enhanced understanding of the role of this nucleus in learning and using reward information. Due to the wide variety of behavioral approaches many questions remain on the circumscribed role of BLA in appetitive behavior. In this review, we integrate conclusions of BLA function in reward-related behavior using traditional interference techniques (lesion, pharmacological inactivation) with those using newer methodological approaches in experimental animals that allow in vivo manipulation of cell type-specific populations and neural recordings. Secondly, from a review of appetitive behavioral tasks in rodents and monkeys and recent computational models of reward procurement, we derive evidence for BLA as a neural integrator of reward value, history, and cost parameters. Taken together, BLA codes specific and temporally dynamic outcome representations in a distributed network to orchestrate adaptive responses. We provide evidence that experiences with opiates and psychostimulants alter these outcome representations in BLA, resulting in long-term modified action.

Basolateral amygdala rapid glutamate release encodes an outcome-specific representation vital for reward-predictive cues to selectively invigorate reward-seeking actions

Environmental stimuli have the ability to generate specific representations of the rewards they predict and in so doing alter the selection and performance of reward-seeking actions. The basolateral amygdala participates in this process, but precisely how is unknown. To rectify this, we monitored, in near-real time, basolateral amygdala glutamate concentration changes during a test of the ability of reward-predictive cues to influence reward-seeking actions (Pavlovian-instrumental transfer). Glutamate concentration was found to be transiently elevated around instrumental reward seeking. During the Pavlovian-instrumental transfer test these glutamate transients were time-locked to and correlated with only those actions invigorated by outcome-specific motivational information provided by the reward-predictive stimulus (i.e., actions earning the same specific outcome as predicted by the presented CS). In addition, basolateral amygdala AMPA, but not NMDA glutamate receptor inactivation abolished the selective excitatory influence of reward-predictive cues over reward seeking. These data support [corrected] the hypothesis that transient glutamate release in the BLA can encode the outcome-specific motivational information provided by reward-predictive stimuli.

Learning and Motivational Processes Contributing to Pavlovian-Instrumental Transfer and Their Neural Bases: Dopamine and Beyond

Pavlovian stimuli exert a range of effects on behavior from simple conditioned reflexes, such as salivation, to altering the vigor and direction of instrumental actions. It is currently accepted that these distinct behavioral effects stem from two sources (i) the various associative connections between predictive stimuli and the component features of the events that these stimuli predict and (ii) the distinct motivational and cognitive functions served by cues, particularly their arousing and informational effects on the selection and performance of specific actions. Here, we describe studies that have assessed these latter phenomena using a paradigm that has come to be called Pavlovian-instrumental transfer. We focus first on behavioral experiments that have described distinct sources of stimulus control derived from the general affective and outcome-specific predictions of conditioned stimuli, referred to as general transfer and specific transfer, respectively. Subsequently, we describe research efforts attempting to establish the neural bases of these transfer effects, largely in the afferent and efferent connections of the nucleus accumbens (NAc) core and shell. Finally, we examine the role of predictive cues in examples of aberrant stimulus control associated with psychiatric disorders and addiction.

The problem with value

Neural correlates of value have been extensively reported in a diverse set of brain regions. However, in many cases it is difficult to determine whether a particular neural response pattern corresponds to a value-signal per se as opposed to an array of alternative non-value related processes, such as outcome-identity coding, informational coding, encoding of autonomic and skeletomotor consequences, alongside previously described "salience" or "attentional" effects. Here, I review a number of experimental manipulations that can be used to test for value, and I identify the challenges in ascertaining whether a particular neural response is or is not a value signal. Finally, I emphasize that some non-value related signals may be especially informative as a means of providing insight into the nature of the decision-making related computations that are being implemented in a particular brain region.

Associative relationships in human predictive learning

Two experiments were conducted to explore the associations involved in human predictive learning. Experiment 1 found that post-training devaluation of one of the outcomes by instructing participants that one of the attackers was indestructible, led to a significant decrease in participants' predictive responses to the cue that was initially followed by the devalued outcome, suggesting that cue-outcome associations play a major role in human simple predictive learning. In Experiment 2, immediately after receiving cue-outcome predictive judgment training, participants were instructed to destroy the attackers by using the same responses previously used to give predictive judgments, but they were not informed as to which response should be used on each attacker. During a test in which both attackers were present at the same time, when the cue was present, participants preferentially chose the instrumental response alternative that was previously used as a predictive judgment about the relationship between the present cue and its outcome. In the absence of cues, participants equally chose either response alternative. This transfer of control shows that participants also establish judgment-outcome associations during predictive training.

Resolution of outcome-induced response conflict by humans after extended training

Studies of incongruent discrimination learning, where the outcome event of one response acts as the discriminative stimulus for the opposite response, suggest that humans rely on habitual stimulus-response (S-R) associations when outcome-response (O-R) associations would cause response conflict. Here, two experiments were conducted to investigate the robustness of this habitual strategy. In Experiment 1, we found that extensive instrumental discrimination training supported learning about the incongruent R → O contingencies, as assessed by an outcome devaluation test. Differential representations of the stimulus and the (associatively retrieved) outcome may have allowed for goal-directed incongruent performance. Experiment 2 failed to provide evidence for this possibility; direct presentation as well as associative retrieval of the incongruent events (by Pavlovian stimuli) activated the response that was associated with each event in its role of stimulus as opposed to outcome. We did find that participants successfully acquired explicit knowledge of the incongruent contingencies, which raises the possibility that propositional encoding allowed them to overcome the response conflict caused by O-R associations. Alternative associative and propositional accounts of successful goal-directed incongruent performance with extensive training will be discussed.

The effects of amphetamine exposure on outcome-selective Pavlovian-instrumental transfer in rats

RATIONALE

Repeated exposure to psychostimulants alters behavioral responses to reward-related cues; however, the motivational underpinnings of this effect have not been fully characterized.

OBJECTIVES

The following study was designed to examine how amphetamine sensitization affects performance in rats on a series of Pavlovian and operant tasks that distinguish between general-incentive and outcome-selective forms of conditioned responses.

METHODS

Adult male rats underwent Pavlovian and instrumental training for food pellet rewards. Following training, rats were sensitized to D-amphetamine (2 mg/kg for 7 days). Rats were subsequently tested on an outcome-selective Pavlovian-instrumental transfer (PIT) task, an outcome-reinstatement task, and an outcome devaluation task. Additionally, in a separate experiment, PIT was assessed in amphetamine-sensitized and control rats using a Pavlovian backward-conditioned stimulus.

RESULTS

Repeated amphetamine exposure sensitized locomotor activity to acute amphetamine challenge. Amphetamine altered responses to CS presentations by increasing conditioned approach. During tests of PIT, amphetamine-treated rats showed no outcome-selectivity in their responding, responding to a CS whether or not it shared a common outcome with the instrumental response. No effect of amphetamine sensitization was observed on tests of outcome-selective reinstatement by outcome delivery or action selection based on outcome value. Amphetamine-sensitized rats showed impaired outcome-selective PIT to a backward CS but were unaltered in conditioned approach.

CONCLUSIONS

Amphetamine sensitization prevents outcome-selective responding during PIT, which is dissociable from amphetamine's effects on conditioned approach. These data suggest fundamental alterations in how stimuli motivate action in addiction.

On the nature of CS and US representations in Pavlovian learning

A significant problem in the study of Pavlovian conditioning is characterizing the nature of the representations of events that enter into learning. This issue has been explored extensively with regard to the question of what features of the unconditioned stimulus enter into learning, but considerably less work has been directed to the question of characterizing the nature of the conditioned stimulus. This article introduces a multilayered connectionist network approach to understanding how "perceptual" or "conceptual" representations of the conditioned stimulus might emerge from conditioning and participate in various learning phenomena. The model is applied to acquired equivalence/distinctiveness of cue effects, as well as a variety of conditional discrimination learning tasks (patterning, biconditional, ambiguous occasion setting, feature discriminations). In addition, studies that have examined what aspects of the unconditioned stimulus enter into learning are also reviewed. Ultimately, it is concluded that adopting a multilayered connectionist network perspective of Pavlovian learning provides us with a richer way in which to view basic learning processes, but a number of key theoretical problems remain to be solved, particularly as they relate to the integration of what we know about the nature of the representations of conditioned and unconditioned stimuli.

Effects of sensitization on the detection of an instrumental contingency

While prior exposure to drugs of abuse permanently changes many behaviors, the underlying psychological mechanisms are relatively obscure. Here, the effects of sensitization on the detection of an action-outcome relationship were assessed, using a particularly stringent contingency degradation procedure. Rats were trained to leverpress until the probability of reinforcement for a response on one lever, or alternative reinforcement for a response on a second lever was reduced to 0.05 per second. Sensitization was then carried out (1mg/kg d-amphetamine/day for 7 days). Then, one reinforcer was also made available for a lack of response on either lever (p=0.05/s), maintaining its contiguity with the original response but eliminating its contingent relationship. Sensitized animals were more active, particularly early in the contingency degradation phase, but reduced responding directed at the degraded action-outcome contingency at a similar rate as controls. However, controls also reduced responding directed at the nondegraded contingency until very late in training, while sensitized animals maintained nondegraded responding at baseline levels. It was suggested that the relatively specific response shown by sensitized animals may reflect either improved action-outcome utilization or discrimination of relevant task features.

Role of the mesoamygdaloid dopamine projection in emotional learning

RATIONALE

Amygdala dopamine is crucially involved in the acquisition of Pavlovian associations, as measured via conditioned approach to the location of the unconditioned stimulus (US). However, learning begins before skeletomotor output, so this study assessed whether amygdala dopamine is also involved in earlier 'emotional' learning.

OBJECTIVES

A variant of the conditioned reinforcement (CR) procedure was validated where training was restricted to curtail the development of selective conditioned approach to the US location, and effects of amygdala dopamine manipulations before training or later CR testing assessed.

METHODS

Experiment 1a presented a light paired (CS+ group) or unpaired (CS- group) with a US. There were 1, 2 or 10 sessions, 4 trials per session. Then, the US was removed, and two novel levers presented. One lever (CR+) presented the light, and lever pressing was recorded. Experiment 1b also included a tone stimulus. Experiment 2 applied intra-amygdala R(+) 7-OH-DPAT (10 nmol/1.0 microl/side) before two training sessions (Experiment 2a) or a CR session (Experiment 2b).

RESULTS

For Experiments 1a and 1b, the CS+ group preferred the CR+ lever across all sessions. Conditioned alcove approach during 1 or 2 training sessions or associated CR tests was low and nonspecific. In Experiment 2a, R(+) 7-OH-DPAT before training greatly diminished lever pressing during a subsequent CR test, preferentially on the CR+ lever. For Experiment 2b, R(+) 7-OH-DPAT infusions before the CR test also reduced lever pressing.

CONCLUSIONS

Manipulations of amygdala dopamine impact the earliest stage of learning in which emotional reactions may be most prevalent.

Pavlovian to instrumental transfer: a neurobehavioural perspective

Pavlovian-to-instrumental transfer (PIT) is a key concept in developing our understanding of cue-controlled behaviours. Here we have reviewed the literature on behavioural and neurobiological factors that influence PIT. Meta-analyses of the data for individual groups in PIT studies revealed that PIT is related to both the order and amounts of instrumental and Pavlovian training, and that it is critically determined by competition between instrumental and Pavlovian responses. We directly addressed the role of response competition in PIT in two experiments which showed that extensive Pavlovian conditioning produced more Pavlovian magazine visits and weaker PIT than moderate Pavlovian conditioning (Experiment 1); and that PIT lost after extensive Pavlovian conditioning was restored by Pavlovian extinction training (Experiment 2). These findings confirm that response competition is indeed an important determinant of PIT. This has significant implications for lesion and inactivation studies that assess the neurobiological substrates of PIT, as well as attempts to demonstrate PIT in the drug self-administration paradigm where the effect is yet to be reliably shown.

Associative theories of goal-directed behaviour: a case for animal-human translational models

Associative accounts of goal-directed action, developed in the fields of human ideomotor action and that of animal learning, can capture cognitive belief-desire psychology of human decision-making. Whereas outcome-response accounts can account for the fact that the thought of a goal can call to mind the action that has previously procured this goal, response-outcome accounts capture decision-making processes that start out with the consideration of possible response alternatives followed only in the second instance by evaluation of their consequences. We argue that while the outcome-response mechanism plays a crucial role in response priming effects, the response-outcome mechanism is particularly important for action selection on the basis of current needs and desires. We therefore develop an integrative account that encapsulates these two routes of action selection within the framework of the associative-cybernetic model. This model has the additional benefit of providing mechanisms for the incentive modulation of goal-directed action and for the development of behavioural autonomy, and therefore provides a promising account of the multi-faceted process of animal as well as human instrumental decision-making.

Instrumental outcome devaluation with representation-mediated conditioning

In three experiments, rats were trained to perform two instrumental behaviours (R1 and R2) in the presence of discriminative stimuli (Sd1 and Sd2, respectively) to obtain a common food outcome (O1). Acquisition of the two discriminations was followed by switching the outcome accompanying R2 performance from O1 to a new one (O2). Experiment 1 showed paired presentations of O2 with a lithium chloride (LiCl) injection resulted in a reduction in the R2 performance. In the subsequent two experiments, each Sd was paired with LiCl injection and its effects on outcome consumption and instrumental performance were investigated. A reduction in the O2 consumption subsequent to the Sd devaluation was found in Experiments 2 and 3. Experiment 3 revealed a reduced R2 performance in an extinction test, following the animals' consummatory access to the outcomes in training context. These results demonstrate representation-mediated outcome devaluation in the course of the Sd devaluation.

Basolateral amygdala lesions and sensitivity to reinforcer magnitude in concurrent chains schedules

Previous studies show that the basolateral amygdala (BLA) is required for behavior to adjust when the value of a reinforcer decreases after satiation or pairing with gastric distress. This study evaluated the effect of pre- or post-training excitotoxic lesions of the BLA on changes in preference with another type of contingency change, reinforcer magnitude reversal. Rats were trained to press left and right levers during a variable-interval choice phase for 50 microl or 150 microl sucrose delivered to consistent locations after a 16-s delay. Tones were presented during the first and last 2s of the delay to reinforcement. The tone frequency predicted the magnitude of sucrose reinforcement in baseline conditions. All groups acquired stable preference for the lever on the large (150 microl) reinforcer side. However, nose poking during the delay to large reinforcement was highly accurate (i.e., to the reinforced side) for all groups except the rats with BLA lesions induced before training, suggesting impaired control of behavior by the tone. After the acquisition of stable preference, the locations of the reinforcer magnitudes were unpredictably reversed for a single session. Pre-training lesions blunted changes in preference when the reinforcer magnitudes were reversed. Lesions induced after stable preference was acquired, but prior to reversal, did not disrupt changes in preference. The data suggest that the BLA contributes to the adaptation of choice behavior following changes in reinforcer magnitude. Impaired learning about the tone-reinforcer magnitude relationships may have disrupted discrimination of the reinforcer magnitude reversal.

General and outcome-specific forms of Pavlovian-instrumental transfer: the effect of shifts in motivational state and inactivation of the ventral tegmental area

This study compared the contribution of the general activating and specific cueing properties of Pavlovian stimuli to Pavlovian-instrumental transfer (PIT) and the role of the ventral tegmental area (VTA) in mediating these effects. In Experiment 1, hungry rats initially received Pavlovian training, in which three distinct auditory stimuli predicted the delivery of three different food outcomes. Next, the rats were trained to perform two instrumental actions, each earning a unique outcome selected from the three used in Pavlovian conditioning. Finally, the effects of the three stimuli on performance of the two actions were assessed in extinction. Presentation of a stimulus that had been paired with the same outcome as an action increased its performance relative to the other action, demonstrating that PIT effects can be outcome selective. In contrast, presentation of the stimulus that predicted the outcome that was not earned during instrumental training facilitated the performance of both actions indiscriminately. This effect, but not the outcome-selective effect, was abolished by a shift from a hungry to a relatively sated state. Experiment 2 examined the effects of inactivation of the VTA on these two forms of PIT. VTA inactivation was found to attenuate PIT but, unlike satiety, did not appear to differentially affect the general or the outcome-selective forms of PIT. The VTA appears therefore to play an important but general role in the initiation of instrumental actions, enabling cues to influence performance whether they enhance responding by changes in arousal or by retrieving particular actions based on their consequences.

Outcome expectations drive learned behaviour in larval Drosophila

Why does Pavlov's dog salivate? In response to the tone, or in expectation of food? While in vertebrates behaviour can be driven by expected outcomes, it is unknown whether this is true for non-vertebrates as well. We find that, in the Drosophila larva, odour memories are expressed behaviourally only if animals can expect a positive outcome from doing so. The expected outcome of tracking down an odour is determined by comparing the value of the current situation with the value of the memory for that odour. Memory is expressed behaviourally only if the expected outcome is positive. This uncovers a hitherto unrecognized evaluative processing step between an activated memory trace and behaviour control, and argues that learned behaviour reflects the pursuit of its expected outcome. Shown in a system with a simple brain, an apparently cognitive process like representing the expected outcome of behaviour seems to be a basic feature of behaviour control.

Reinforcer-specificity of appetitive and consummatory behavior of rats after Pavlovian conditioning with food reinforcers

We examined the reinforcer-specificity of Pavlovian conditioning in the control of appetitive and consummatory behaviors in Pavlovian-to-instrumental transfer, cue-potentiated eating, and devaluation procedures. Rats received pairings of one conditioned stimulus with sucrose and another conditioned stimulus with maltodextrin. In Experiment 1, rats were also trained to earn sucrose for one instrumental response and maltodextrin for another. In a transfer test, the Pavlovian cues enhanced the rate of instrumental responding more when the food reinforcer predicted by the instrumental response and the Pavlovian cue were consistent than when they were inconsistent, but both cues enhanced both responses. In Experiment 2, sated rats' consumption of each food was potentiated in the presence of a cue for that food, but not in the presence of a cue for the other food. In Experiment 3, one food was devalued by pairing it with lithium chloride, prior to testing food consumption and food-cup directed behaviors. The food cues selectively controlled food-cup related behaviors, regardless of the presence of the devalued or nondevalued foods in the food cup. Together, these results are consistent with the view that conditioned cues modulate appetitive and consummatory behaviors with increasing levels of specificity. The closer an action comes to ingestion, the more it is controlled by sensory properties conveyed by learned cues. These data are discussed in the context of allostatic regulation of food foraging and intake.

Ethanol-associated cues produce general pavlovian-instrumental transfer

BACKGROUND

Conditioned stimuli are thought to play an important role in maintaining ethanol use and inducing relapse. Therefore, understanding the mechanisms through which such stimuli trigger ethanol seeking is of interest in the study and treatment of alcoholism.

METHODS

This series of experiments examined the impact of ethanol-associated cues on ethanol-seeking behavior using a Pavlovian-instrumental transfer design. Rats received Pavlovian training in which an auditory stimulus predicted ethanol (10%) delivery. In a separate instrumental training phase, animals were trained to press a lever for ethanol. In the test phase, the impact of the stimulus on instrumental performance was assessed in extinction by presenting the stimulus while animals were free to perform the lever-press response. Experiment 2 assessed the selectivity of the transfer effect; rats received training with 2 auditory stimuli which predicted either ethanol or sucrose (2%) delivery and were trained to perform 2 instrumental responses, one earning ethanol and the other earning sucrose. Finally, Experiment 3 examined the selectivity of PIT using 2 natural rewards (sucrose and polycose).

RESULTS

The results from Experiment 1 show that ethanol supports excitatory conditioning and that ethanol-associated cues facilitate instrumental performance for ethanol. When the selectivity of the transfer effect was examined in Experiment 2, the ethanol-paired stimulus was found to have a general excitatory effect on reward-seeking behavior, affecting both ethanol-directed and sucrose-directed responding equally. In contrast, the sucrose-paired stimulus had a selective effect, elevating sucrose-directed responding only. Experiment 3 confirms that selective transfer is observed when 2 natural rewards are used to reinforce responding.

CONCLUSIONS

These data provide further evidence that ethanol-associated cues can drive ethanol-seeking behaviors. Because ethanol-associated cues also enhanced seeking behavior for a nonalcohol reward, these results additionally suggest that the modulation of reward-directed behaviors by cues associated with ethanol versus natural rewards may rely on different behavioral and neural mechanisms.

Amount of training effects in representation-mediated food aversion learning: no evidence of a role for associability changes

Rats acquired aversions to food pellets when a previously trained signal for that food was paired with a toxin, but only after minimal signal-food training. After extensive signal-food training, signal-toxin pairings had no effect on food consumption even after manipulations that enhanced the associability of the signal. By contrast, conditioned responding to the signal retained its sensitivity to devaluation of the food reinforcer by food-toxin pairings after extensive training. These results suggest that the nature of associatively activated event representations changes over the course of training.

Effects of a systemic AMPA/KA and NMDA receptor blockade on pavlovian-instrumental transfer

RATIONALE

Pavlovian stimuli can markedly elevate instrumental responding directed toward a common reward, an effect known as pavlovian-instrumental transfer (PIT). PIT critically depends on the amygdala and nucleus accumbens (ACB); however, little is known yet about its neurochemical basis.

OBJECTIVE

Here we examined the role of ionotropic glutamate receptors in PIT.

METHODS

The effects of a systemic blockade of alpha-amino-3-hydroxy-5-methyl-4-isoxazole propionate/kainate (AMPA/KA) and N-methyl-D: -aspartate (NMDA) receptors on PIT and locomotor activity were investigated in rats.

RESULTS

The competitive AMPA/KA receptor antagonist 6-cyano-7-nitroquinoxaline-2,3-dione disodium salt (CNQX) (1.5 mg/kg i.p.) did not alter the overall rate of lever pressing and left PIT intact, i.e. presentation of a pavlovian stimulus significantly enhanced instrumental responding. Furthermore, CNQX did not affect horizontal and vertical activity in an open field. The non-competitive NMDA receptor antagonist (+)-5-methyl-10,11-dihydro-5H-dibenzo[a,d]cyclohepten-5,10-imine hydrogen maleate (dizocilpine) (0.08 mg/kg i.p.) elevated the overall rate of lever pressing but did not affect PIT. In addition, dizocilpine increased horizontal and decreased vertical activity in an open field.

CONCLUSIONS

Previous studies imply that the training protocol used here induced a general, not outcome-specific, form of PIT which is mediated by the central nucleus of the amygdala (CeN) through modulation of mesoaccumbens dopamine transmission. Thus, we suggest that an AMPA/KA and NMDA receptor blockade did not affect PIT here because the general motivational influence of pavlovian stimuli to induce PIT is conveyed by GABAergic projections from the CeN to midbrain dopaminergic neurons.

Motivating operations in appetite research

Appetite research frequently employs principles derived from behaviour analysis. However, it has yet to utilise the more recent theoretical advances in this field. This paper describes the concept of the motivating operation (MO)--a behaviour analytic formulation of motivation. An MO is an environmental event that (a) establishes or abolishes the reinforcing or punishing effect of another event and (b) evokes or abates behaviours associated with that event. The paper describes both unconditioned and conditioned MOs and the ways in which they may help account for a variety of eating behaviours. It then goes on to highlight the main ways in which the MO account differs from other theories of motivation employed in appetite research. These relate to (1) the ways in which they account for non-regulatory feeding, (2) the extent to which they address cognitive variables and (3) their underlying philosophical assumptions and subsequent relation to intervention.

On the relationship between anticipatory behaviour in a Pavlovian paradigm and Pavlovian-to-Instrumental Transfer in rats (Rattus norvegicus)

The present rat study assessed the relationship between, and the sensitivity of, two different tests for appetitive conditioned responding to differences in the contingency between a conditioned stimulus (CS) and an unconditioned stimulus (US), and to differences in US magnitude. The first test used a Pavlovian-to-Instrumental Transfer (PIT) paradigm, assessing the capacity of the CS to enhance instrumental responding for food. The second test employed a Pavlovian conditioning paradigm with an extended CS-US interval, and total number of behavioural elements in this interval as a dependent measure. The PIT test proved to be sensitive to contingency but not reward magnitude differences, whereas the reverse was true for the Pavlovian test. Although there was a significant correlation between tests in the magnitude of the CS-induced increase of food-magazine entries, the main dependent measure from PIT (number of lever presses) and that from the Pavlovian test (total number of behavioural elements) did not correlate. It is suggested that in the PIT procedure, the CS induces a chain of behavioural responses of which lever pressing is just a single element and that the Pavlovian test, in principle, is more sensitive.

Relations Between Pavlovian-Instrumental Transfer and Reinforcer Devaluation

Relations between posttraining reinforcer devaluation and Pavlovian-instrumental transfer were examined in 2 experiments. When a single reinforcer was used, extended training of the instrumental response increased transfer but reduced devaluation effects. When multiple instrumental reinforcers were used, both reinforcer-specific transfer and devaluation effects were less influenced by the amount of instrumental training. Finally, although reinforcer devaluation decreased both Pavlovian conditioned responses and baseline instrumental responding, it had no effect on either single-reinforcer or reinforcer-specific transfer. These results indicate that transfer and reinforcer devaluation can reflect different aspects of associative learning and that the nature of associative learning can be influenced by parameters such as the amount of training and the use of multiple reinforcers.

Outcome-specific conditioned inhibition in Pavlovian backward conditioning

In the present experiments, the outcome specificity of learning was explored in an appetitive Pavlovian backward conditioning procedure with rats. The rats initially were administered Pavlovian backward training with two qualitatively different unconditioned stimulus conditioned-stimulus (US-CS) pairs of stimuli (e.g., pellet --> noise or sucrose --> light), and then the effects of this training were assessed in Pavlovian-to-instrumental transfer (Experiment 1) and retardation-of-learning (Experiment 2) tests. In the transfer test, it was shown that during the last 10-sec interval, the CSs selectively reduced the rate of the instrumental responses with which they shared a US, relative to the instrumental responses with which they did not share a US. The opposite result was obtained when the USs (in the absence of the CSs) were presented noncontingently. In the retardation test, conditioned magazine approach, responding to the CSs was acquired more slowly when the stimulus-outcome combinations in the backward and the forward conditioning phases were the same, as compared with when they were reversed. These results are collectively in accord with the view that Pavlovian backward conditioning can result in the formation of outcome-specific inhibitory associations. Alternative views of backward conditioning are also examined.

Facilitation of appetitive pavlovian conditioning by d-amphetamine in the shell, but not the core, of the nucleus accumbens

The effects of postsession d-amphetamine within subregions of the ventral and dorsal striatum on appetitive Pavlovian learning were assessed. Rats acquired a conditioned approach response on presentation of a stimulus predictive of 10% sucrose solution (unconditioned stimulus [US]), but not during equally frequent presentations of a stimulus uncorrelated with the US. In Experiment 1, postsession d-amphetamine infusions enhanced acquisition of conditioned responding, with no effect on control measures. In Experiment 2, rats received postsession d-amphetamine in the accumbens shell or core. Shell infusions facilitated conditioning; core infusions did not. In Experiment 3, dorsomedial striatal infusions of d-amphetamine also were ineffective. In sum, dopaminergic activation within the shell, but not the core, of the nucleus accumbens facilitates the acquisition of a Pavlovian association.

Immunohistochemical assessment of mesotelencephalic dopamine activity during the acquisition and expression of Pavlovian versus instrumental behaviours

Dopaminergic activity during Pavlovian or instrumental learning in key target regions of the mesotelencephalic dopamine system was investigated immunohistochemically using antibodies raised against glutaraldehyde-conjugated dopamine. Experiment 1 examined dopamine immunoreactivity during acquisition of a Pavlovian conditioned-approach response. Observations were taken at three stages of learning: initial, intermediate and asymptotic; each with a conditioned stimulus+ (CS+) group for whom visual or auditory stimuli immediately preceded an unconditioned stimulus (sucrose), and a conditioned stimulus- (CS-) group for whom stimuli and the unconditioned stimulus were unpaired. Animals learned to approach the alcove during CS+ presentations, whilst approach behaviour of the CS- group remained low. In general, target regions exhibiting a dopaminergic reaction responded maximally during the intermediate stage of acquisition, and were less responsive initially, and not responsive at all at asymptote. Specifically, the pattern of dopaminergic response was: shell more than core of the nucleus accumbens; prefrontal cortex, central and basolateral nuclei of the amygdala also significantly responsive. Mediodorsal and laterodorsal striatal regions were reactive only very early in training. Experiment 2 examined dopaminergic reaction following acquisition of a novel conditioned instrumental response. The conditioned response+ (CR+) group responded at a much higher rate on the lever for which unconditioned stimulus-associated stimuli were presented, than on the control lever. The conditioned response- (CR-) group responded at a low rate on both levers. In contrast with experiment 1, the most responsive regions were the core of the nucleus accumbens, medial prefrontal cortex and basolateral area of the amygdala. Thus, the acquisition, but not expression of Pavlovian associations activated dopamine within several key target regions of the mesotelencephalic dopamine system, and preferentially within the shell rather than core of the nucleus accumbens. By contrast, acquisition of a novel instrumental response preferentially activated the core of the nucleus accumbens, and basolateral area of the amygdala. These data carry significant implications for the potential role of these regions in learning and memory.

Instrumental and Pavlovian incentive processes have dissociable effects on components of a heterogeneous instrumental chain

Two experiments compared the effects of Pavlovian stimuli and incentive learning on the performance of a heterogeneous chain of instrumental actions. Using a Pavlovian-instrumental transfer design, the authors found that only a stimulus paired with the same outcome as that earned by performance of the chain produced positive transfer, an effect that was restricted to the action in the chain most proximal to reward delivery. In contrast, after a shift to a nondeprived state, only animals that had previously consumed the instrumental outcome when they were nondeprived decreased instrumental performance. Furthermore, this effect of the incentive learning treatment was limited to performance of the distal action. Together these data suggest that Pavlovian and instrumental incentive manipulations have dissociable effects on instrumental performance.

Reward, motivation, and reinforcement learning

There is substantial evidence that dopamine is involved in reward learning and appetitive conditioning. However, the major reinforcement learning-based theoretical models of classical conditioning (crudely, prediction learning) are actually based on rules designed to explain instrumental conditioning (action learning). Extensive anatomical, pharmacological, and psychological data, particularly concerning the impact of motivational manipulations, show that these models are unreasonable. We review the data and consider the involvement of a rich collection of different neural systems in various aspects of these forms of conditioning. Dopamine plays a pivotal, but complicated, role.

Transfer of control between causal predictive judgments and instrumental responding

Four experiments were conducted to explore outcome-specific transfer from causal predictive judgments to instrumental responding. A video game was designed in which participants had to defend Andalusia from navy and air force attacks. First, they learned the relationship between two instrumental responses (two key on a standard keyboard) and two different outcomes (destruction of the ships or destruction of the planes). Then they learned to predict which of two different stimuli predicted which outcome. Finally, they had the opportunity of making either of the two instrumental responses in the presence of either stimulus. Transfer was shown as a preference for the response that shared an outcome with the current stimulus. The presentation of the stimulus during the test produced a decrease in the overall rate of response. Responding to a neutral stimulus in Experiments 2 and 3 suggested that this overall decrease in responding was due to a combination of the time needed to process the meaning of the stimulus and the activation of the representation of the outcome in the presence of the stimulus during the test. Transfer between predictive judgments and instrumental responding mirrors the outcome-specific Pavlovian instrumental transfer observed in conditioning studies with rats.