Outcome-specific transfer between predictive and instrumental learning is unaffected by extinction but reversed by counterconditioning in human participants

General Pavlovian-to-instrumental transfer in humans: Evidence from Bayesian inference

When repeatedly paired with rewarding outcomes (i.e., Pavlovian conditioning), environmental cues may acquire predictive and motivational significance and later enhance instrumental responding for the same (i.e., outcome-specific transfer) or motivationally similar (i.e., general transfer) outcomes. Although outcome-specific and general Pavlovian-to-Instrumental Transfer (PIT) are characterized by different neural substrates and behavioral mechanisms, general transfer has never been studied in isolation from outcome-specific transfer in humans. The first aim of the present study was to test whether the general transfer effect could emerge in isolation and independently of outcome-specific transfer. Our results showed that general transfer can be elicited without the concurrent presence of outcome-specific transfer, supporting the idea that outcome-specific and general transfer can be studied independently of each other. The second aim of the present study was to clarify whether the affordance-like properties of the outcomes can affect the general transfer. In fact, a critical difference in current studies on general transfer concerns the use of cues associated with outcomes for which an action was previously learned (or not) during the instrumental training. This apparently minor difference affects the affordance-like properties of the outcome and may also be transferred to the cue, in turn impacting general transfer. Results revealed a general transfer of the same magnitude regardless of whether cues were associated with reward earned or not during instrumental conditioning. These findings increase the current knowledge on the incentive motivational mechanism behind general transfer, indicating that it is independent of the motor features of the outcome.

Susceptibility to interference between Pavlovian and instrumental control is associated with early hazardous alcohol use

Pavlovian-to-instrumental transfer (PIT) tasks examine the influence of Pavlovian stimuli on ongoing instrumental behaviour. Previous studies reported associations between a strong PIT effect, high-risk drinking and alcohol use disorder. This study investigated whether susceptibility to interference between Pavlovian and instrumental control is linked to risky alcohol use in a community sample of 18-year-old male adults. Participants (N = 191) were instructed to 'collect good shells' and 'leave bad shells' during the presentation of appetitive (monetary reward), aversive (monetary loss) or neutral Pavlovian stimuli. We compared instrumental error rates (ER) and functional magnetic resonance imaging (fMRI) brain responses between the congruent and incongruent conditions, as well as among high-risk and low-risk drinking groups. On average, individuals showed a substantial PIT effect, that is, increased ER when Pavlovian cues and instrumental stimuli were in conflict compared with congruent trials. Neural PIT correlates were found in the ventral striatum and the dorsomedial and lateral prefrontal cortices (lPFC). Importantly, high-risk drinking was associated with a stronger behavioural PIT effect, a decreased lPFC response and an increased neural response in the ventral striatum on the trend level. Moreover, high-risk drinkers showed weaker connectivity from the ventral striatum to the lPFC during incongruent trials. Our study links interference during PIT to drinking behaviour in healthy, young adults. High-risk drinkers showed higher susceptibility to Pavlovian cues, especially when they conflicted with instrumental behaviour, indicating lower interference control abilities. Increased activity in the ventral striatum (bottom-up), decreased lPFC response (top-down), and their altered interplay may contribute to poor interference control in the high-risk drinkers.

General Pavlovian-instrumental transfer tests reveal selective inhibition of the response type - whether Pavlovian or instrumental - performed during extinction

The present experiments examined whether extinction of a stimulus predicting food affects the ability of that stimulus to energize instrumental performance to obtain food. We first used a general Pavlovian instrumental transfer (PIT) paradigm in which rats were first given Pavlovian conditioning with a stimulus predicting one type of food outcome and were then trained to lever press for a different food outcome. We found that the Pavlovian stimulus enhanced performance of the lever press response and that this enhancement was preserved after extinction of that stimulus (Experiment 1) even when the context was manipulated to favor the expression of extinction (Experiment 2). Next, we assessed whether extinction influenced the excitatory effect of a stimulus when it was trained as a discriminative stimulus. Extinction of this stimulus alone had no effect on its ability to control instrumental performance; however, when extinguished with its associated lever press response, discriminative control was lost (Experiments 3 and 4). Finally, after instrumental and Pavlovian training, we extinguished a Pavlovian stimulus predicting one food outcome with a lever press response that delivered a different outcome. In a general PIT test, we found this extinction abolished the ability of the Pavlovian stimulus to elevate responding on a lever trained with a different outcome, revealing for the first time that extinction can abolish the general PIT effect. We conclude that extinction can produce an inhibitory association between the stimulus and the general response type, whether Pavlovian or instrumental, performed during the extinction training.

Motivational sensitivity of outcome-response priming: Experimental research and theoretical models

Outcome-response (O-R) priming is at the core of various associative theories of human intentional action. This is a simple and parsimonious mechanism by which activation of outcome representations (e.g. thinking about the light coming on) leads to activation of the associated motor patterns required to achieve it (e.g. pushing the light switch). In the current manuscript, we review the evidence for such O-R associative links demonstrated by converging (yet until now, separate) strands of research. While there is a wealth of evidence that both the perceptual and motivational properties of an outcome can be encoded in the O-R association and mediate O-R priming, we critically examine the integration of these mechanisms and the conditions under which motivational factors constrain the sensory O-R priming effect. We discuss the clinical relevance of this O-R priming mechanism, whether it can satisfactorily account for human goal-directed behaviour, and the implications for theories of human action control.

Associative mechanisms involved in specific Pavlovian-to-instrumental transfer in human learning tasks

Four experiments compared the effect of forward and backward conditioning procedures on the ability of conditioned stimuli (CS) to elevate instrumental responding in a Pavlovian-to-instrumental transfer (PIT) task. Two responses were each trained with one distinct outcome (R1->O1, R2->O2), either concurrently (Experiment 1) or separately (Experiments 2, 3 and 4). Then, in Experiments 1 and 2, four CSs were either followed or preceded by one outcome (A->O1, B->O2, O1->C, O2->D). In Experiment 3, each CS was preceded and followed by an outcome: for one group of participants, both outcomes were identical (e.g., O1->A->O1, O2->B->O2), but for the other, they were different (e.g., O1->A->O2, O2->B->O1). In Experiment 4, two CSs were preceded and followed by identical outcomes, and two CSs by different outcomes. In the PIT tests, participants performed R1 and R2 in the presence and absence of the CSs. In Experiments 1 and 2, only the CSs followed by outcomes in Pavlovian training elevated responding. In Experiments 3 and 4, all the CSs elevated responding but based on the outcome that followed them in training. These results support the stimulus-outcome-response (S-O-R) mechanism of specific PIT, according to which CSs elevate responding via activation of its associated outcome representation.

Drug cue reactivity involves hierarchical instrumental learning: evidence from a biconditional Pavlovian to instrumental transfer task

RATIONALE

Drug cue reactivity plays a crucial role in addiction, yet the underlying mechanisms are poorly understood. According to the binary associative account, drug stimuli retrieve an expectation of the drug outcome, which, in turn, elicits the associated drug-seeking response (S-O-R). By contrast, according to the hierarchical account, drug stimuli retrieve an expectation that the contingency between the drug-seeking response and the drug outcome is currently more effective, promoting performance of the drug-seeking response (S:R-O).

METHODS

The current study discriminated between these two accounts using a biconditional Pavlovian-to-instrumental transfer (PIT) task with 128 alcohol drinkers. A biconditional discrimination was first trained in which two responses produced alcohol and food outcomes, respectively, and these response-outcome contingencies were reversed across two discriminative stimuli (SDs). In the PIT test, alcohol and food cues were compounded with the two SDs to examine their impact on percent alcohol choice in extinction.

RESULTS

It was found that alcohol and food cues selectively primed choice of the response that earned that outcome in each SD (p < .001), and this effect was associated with participants' belief that cues signalled greater effectiveness of that response (p < .0001).

CONCLUSIONS

The alcohol stimulus could not have selectively primed the alcohol-seeking response through binary S-O-R associations because the drug outcome was equally associated with both responses. Rather, the alcohol stimulus must have retrieved an expectation that the response-alcohol contingency available in the current context was more likely to be effective (S:R-O), which primed performance of the alcohol-seeking response.

Extinction Can Reduce the Impact of Reward Cues on Reward-Seeking Behavior

Reward-associated cues are thought to promote relapse after treatment of appetitive disorders such as drug-taking, binge eating, and gambling. This process has been modelled in the laboratory using a Pavlovian-instrumental transfer (PIT) design in which Pavlovian cues facilitate instrumental reward-directed action. Attempts to reduce facilitation by cue exposure (extinction) have produced mixed results. We tested the effect of extinction in a recently developed PIT procedure using a natural reward, chocolate, in human participants. Facilitation of instrumental responding was only observed in participants who were aware of the Pavlovian contingencies. Pavlovian extinction successfully reduced, but did not completely eliminate, expectancy of reward and facilitation of instrumental responding. The results indicate that exposure can reduce the ability of cues to promote reward-directed behavior in the laboratory. However, the residual potency of extinguished cues means that additional active strategies may be needed in clinical practice to train patients to resist the impact of these cues in their environment.

A hierarchical instrumental decision theory of nicotine dependence

It is important to characterize the learning processes governing tobacco-seeking in order to understand how best to treat this behavior. Most drug learning theories have adopted a Pavlovian framework wherein the conditioned response is the main motivational process. We favor instead a hierarchical instrumental decision account, wherein expectations about the instrumental contingency between voluntary tobacco-seeking and the receipt of nicotine reward determines the probability of executing this behavior. To support this view, we review titration and nicotine discrimination research showing that internal signals for deprivation/satiation modulate expectations about the current incentive value of smoking, thereby modulating the propensity of this behavior. We also review research on cue-reactivity which has shown that external smoking cues modulate expectations about the probability of the tobacco-seeking response being effective, thereby modulating the propensity of this behavior. Economic decision theory is then considered to elucidate how expectations about the value and probability of response-nicotine contingency are integrated to form an overall utility estimate for that option for comparison with qualitatively different, nonsubstitute reinforcers, to determine response selection. As an applied test for this hierarchical instrumental decision framework, we consider how well it accounts for individual liability to smoking uptake and perseveration, pharmacotherapy, cue-extinction therapies, and plain packaging. We conclude that the hierarchical instrumental account is successful in reconciling this broad range of phenomenon precisely because it accepts that multiple diverse sources of internal and external information must be integrated to shape the decision to smoke.

Plain cigarette packs do not exert Pavlovian to instrumental transfer of control over tobacco-seeking

AIMS

To gain insight into the potential impact of plain tobacco packaging policy, two experiments were undertaken to test whether 'prototype' plain compared with branded UK cigarette pack stimuli would differentially elicit instrumental tobacco-seeking in a nominal Pavlovian to instrumental transfer (PIT) procedure.

DESIGN, SETTING AND PARTICIPANTS

Two experiments were undertaken at the University of Bristol UK, with a convenience sample of adult smokers (experiment 1, n = 23, experiment 2, n = 121).

MEASUREMENT

In both experiments, smokers were trained on a concurrent choice procedure in which two responses earned points for cigarettes and chocolate, respectively, before images of branded and plain packs were tested for capacity to elicit the tobacco-seeking response in extinction. The primary outcome was percentage choice of the tobacco- over the chocolate-seeking response in plain pack, branded pack and no-stimulus conditions.

FINDINGS

Both experiments found that branded packs primed a greater percentage of tobacco-seeking (overall mean = 62%) than plain packs (overall mean = 53%) and the no-stimulus condition (overall mean = 52%; Ps ≤ 0.01, ŋp (2) s ≥ 0.16), and that there was no difference in percentage tobacco-seeking between plain packs and the no-stimulus condition (Ps ≥ 0.17, ŋp (2) s ≤ 0.04). Plain tobacco packs showed an overall 9% reduction in the priming of a tobacco choice response compared to branded tobacco packs.

CONCLUSIONS

Plain packaging may reduce smoking in current smokers by degrading cue-elicited tobacco-seeking.

How food cues can enhance and inhibit motivation to obtain and consume food

Learning may play an important role in over-eating. One example is Pavlovian-to-instrumental transfer (PIT), whereby reward cues facilitate responding to obtain that reward. Whilst there is increasing research indicating PIT for food in humans, these studies have exclusively tested PIT under instrumental extinction (i.e. when the food is no longer available), which may reduce their ecological validity. To address this, we conducted two experiments exploring PIT for food in humans when tested under instrumental reinforcement. Participants first underwent Pavlovian discrimination training with an auditory cue paired with a chocolate reward (CS+) and another auditory cue unpaired (CS-). In instrumental training participants learnt to press a button to receive the chocolate reward on a VR10 schedule. In the test phase, each CS was presented whilst participants maintained the opportunity to press the button to receive chocolate. In Experiment 1, the PIT test was implemented after up to 20 min of instrumental training (satiation) whereas in Experiment 2 it was implemented after only 4 min of instrumental training. In both experiments there was evidence for differential PIT, but the pattern differed according to the rate of responding at the time of the PIT test. In low baseline responders the CS+ facilitated both button press responding and consumption, whereas in high baseline responders the CS- suppressed responding. These findings suggest that both excitatory and inhibitory associations may be learnt during PIT training and that the expression of these associations depends on motivation levels at the time the cues are encountered. Particularly concerning is that a food-paired cue can elicit increased motivation to obtain and consume food even when the participant is highly satiated and no longer actively seeking food, as this may be one mechanism by which over-consumption is maintained.

Extinction of cue-evoked drug-seeking relies on degrading hierarchical instrumental expectancies

There has long been need for a behavioural intervention that attenuates cue-evoked drug-seeking, but the optimal method remains obscure. To address this, we report three approaches to extinguish cue-evoked drug-seeking measured in a Pavlovian to instrumental transfer design, in non-treatment seeking adult smokers and alcohol drinkers. The results showed that the ability of a drug stimulus to transfer control over a separately trained drug-seeking response was not affected by the stimulus undergoing Pavlovian extinction training in experiment 1, but was abolished by the stimulus undergoing discriminative extinction training in experiment 2, and was abolished by explicit verbal instructions stating that the stimulus did not signal a more effective response-drug contingency in experiment 3. These data suggest that cue-evoked drug-seeking is mediated by a propositional hierarchical instrumental expectancy that the drug-seeking response is more likely to be rewarded in that stimulus. Methods which degraded this hierarchical expectancy were effective in the laboratory, and so may have therapeutic potential.

Facilitation of Voluntary Goal-Directed Action by Reward Cues

Reward-associated cues are known to influence motivation to approach both natural and man-made rewards, such as food and drugs. However, the mechanisms underlying these effects are not well understood. To model these processes in the laboratory with humans, we developed an appetitive Pavlovian-instrumental transfer procedure with a chocolate reward. We used a single unconstrained response that led to an actual rather than symbolic reward to assess the strength of reward motivation. Presentation of a chocolate-paired cue, but not an unpaired cue, markedly enhanced instrumental responding over a 30-s period. The same pattern was observed with 10-s and 30-s cues, showing that close cue-reward contiguity is not necessary for facilitation of reward-directed action. The results confirm that reward-related cues can instigate voluntary action to obtain that reward. The effectiveness of long-duration cues suggests that in clinical settings, attention should be directed to both proximal and distal cues for reward.

Associative relationships in human predictive learning

Two experiments were conducted to explore the associations involved in human predictive learning. Experiment 1 found that post-training devaluation of one of the outcomes by instructing participants that one of the attackers was indestructible, led to a significant decrease in participants' predictive responses to the cue that was initially followed by the devalued outcome, suggesting that cue-outcome associations play a major role in human simple predictive learning. In Experiment 2, immediately after receiving cue-outcome predictive judgment training, participants were instructed to destroy the attackers by using the same responses previously used to give predictive judgments, but they were not informed as to which response should be used on each attacker. During a test in which both attackers were present at the same time, when the cue was present, participants preferentially chose the instrumental response alternative that was previously used as a predictive judgment about the relationship between the present cue and its outcome. In the absence of cues, participants equally chose either response alternative. This transfer of control shows that participants also establish judgment-outcome associations during predictive training.