Human appetitive Pavlovian-to-instrumental transfer: a goal-directed account

Pupil dilation reflects effortful action invigoration in overcoming aversive Pavlovian biases

"Pavlovian" or "motivational" biases describe the phenomenon that the valence of prospective outcomes modulates action invigoration: Reward prospect invigorates action, whereas punishment prospect suppresses it. The adaptive role of these biases in decision-making is still unclear. One idea is that they constitute a fast-and-frugal decision strategy in situations characterized by high arousal, e.g., in presence of a predator, which demand a quick response. In this pre-registered study (N = 35), we tested whether such a situation-induced via subliminally presented angry versus neutral faces-leads to increased reliance on Pavlovian biases. We measured trial-by-trial arousal by tracking pupil diameter while participants performed an orthogonalized Motivational Go/NoGo Task. Pavlovian biases were present in responses, reaction times, and even gaze, with lower gaze dispersion under aversive cues reflecting "freezing of gaze." The subliminally presented faces did not affect responses, reaction times, or pupil diameter, suggesting that the arousal manipulation was ineffective. However, pupil dilations reflected facets of bias suppression, specifically the physical (but not cognitive) effort needed to overcome aversive inhibition: Particularly strong and sustained dilations occurred when participants managed to perform Go responses to aversive cues. Conversely, no such dilations occurred when they managed to inhibit responses to Win cues. These results suggest that pupil diameter does not reflect response conflict per se nor the inhibition of prepotent responses, but specifically effortful action invigoration as needed to overcome aversive inhibition. We discuss our results in the context of the "value of work" theory of striatal dopamine.

Effects of sleep deprivation on food-related Pavlovian-instrumental transfer: a randomized crossover experiment

Recent research suggests that insufficient sleep elevates the risk of obesity. Although the mechanisms underlying the relationship between insufficient sleep and obesity are not fully understood, preliminary evidence suggests that insufficient sleep may intensify habitual control of behavior, leading to greater cue-elicited food-seeking behavior that is insensitive to satiation. The present study tested this hypothesis using a within-individual, randomized, crossover experiment. Ninety-six adults underwent a one-night normal sleep duration (NSD) condition and a one-night total sleep deprivation (TSD) condition. They also completed the Pavlovian-instrumental transfer paradigm in which their instrumental responses for food in the presence and absence of conditioned cues were recorded. The sleep × cue × satiation interaction was significant, indicating that the enhancing effect of conditioned cues on food-seeking responses significantly differed across sleep × satiation conditions. However, this effect was observed in NSD but not TSD, and it disappeared after satiation. This finding contradicted the hypothesis but aligned with previous literature on the effect of sleep disruption on appetitive conditioning in animals-sleep disruption following learning impaired the expression of appetitive behavior. The present finding is the first evidence for the role of sleep in Pavlovian-instrumental transfer effects. Future research is needed to further disentangle how sleep influences motivational mechanisms underlying eating.

Investigating habits in humans with a symmetrical outcome-revaluation task

The translation of the outcome-devaluation paradigm to study habit in humans has yielded interesting insights but proven to be challenging. We present a novel, outcome-revaluation task with a symmetrical design, in the sense that half of the available outcomes are always valuable and the other half not-valuable. In the present studies, during the instrumental learning phase, participants learned to respond (Go) to certain stimuli to collect valuable outcomes (and points) while refraining to respond (NoGo) to stimuli signaling not-valuable outcomes. Half of the stimuli were short-trained, while the other half were long-trained. Subsequently, in the test phase, the signaled outcomes were either value-congruent with training (still-valuable and still-not-valuable), or value-incongruent (devalued and upvalued). The change in outcome value on value-incongruent trials meant that participants had to flexibly adjust their behavior. At the end of the training phase, participants completed the self-report behavioral automaticity index - providing an automaticity score for each stimulus-response association. We conducted two experiments using this task, that both provided evidence for stimulus-driven habits as reflected in poorer performance on devalued and upvalued trials relative to still-not-valuable trials and still-valuable trials, respectively. While self-reported automaticity increased with longer training, behavioral flexibility was not affected. After extended training (Experiment 2), higher levels of self-reported automaticity when responding to stimuli signaling valuable outcomes were related to more 'slips of action' when the associated outcome was subsequently devalued. We conclude that the symmetrical outcome-revaluation task provides a promising paradigm for the experimental investigation of habits in humans.

Modifying Pavlovian-to-instrumental transfer by approach avoidance training in healthy subjects: a proof of concept study

The modulation of instrumental action by conditioned Pavlovian cues is hypothesized to play a role in the emergence and maintenance of maladaptive behavior. The Pavlovian to Instrumental transfer task (PIT) is designed to examine the magnitude of the influence of cues on behavior and we aim to manipulate the motivational value of Pavlovian cues to reduce their effect on instrumental responding. To this end, we utilized a joystick-based modification of approach and avoidance propensities that has shown success in clinical populations. To examine changes in PIT, we subjected 35 healthy participants to a series of experimental procedures: (1) Instrumental training was followed by (2) Pavlovian conditioning of neutral stimuli that were associated with monetary reward or loss. (3) In a subsequent joystick task, approach and avoidance tendencies toward conditioned cues were assessed. (4) In a transfer test, the PIT effect as the impact of conditioned cues on instrumental behavior was measured. (5) The explicit knowledge of cue-reward contingencies was assessed in a forced-choice phase. (6, 7) systematic joystick training was followed by a posttest (8) the transfer task and forced-choice test were repeated. We found no effect of training on approach-avoidance propensities in the context of this proof of concept study. A higher response rate towards negative stimuli during PIT after systematic training compared to sham training was seen. On the other hand, we saw an increased PIT effect after sham training. These results contribute to the understanding of the strength of the influence of cues on instrumental behavior. Our findings further stress the importance of context, instructions and operationalization of instrumental behavior in the framework of transfer effects.

How the environment evokes actions that lead to different goals: the role of object multi-functionality in pavlovian-to-instrumental transfer

Research shows that stimuli in the environment can trigger behavior via the activation of goal representations. This process can be tested in the Pavlovian-to-Instrumental Transfer (PIT) paradigm, where stimuli can only affect behavior through the activation of the representation of its desired outcome (i.e., the PIT effect). Previous research has demonstrated that the PIT effect is stronger when the goal is more desirable. While this research only looked at actions that have single outcomes (e.g., obtaining a snack to satisfy appetite), in the present paper, we reason that actions that are instrumental in obtaining outcomes that are desirable in multiple ways (e.g., obtaining a snack to satisfy one's appetite, giving it to a friend, trading it for money) should produce stronger PIT effects. In two experiments, participants learned to perform left and right key presses to earn a snack, either framed as having a single function or multiple functions. Participants also learned to associate the two differently framed snacks with two cues. In a PIT test, they were required to press the keys as fast as possible upon exposure to the cues (i.e., the PIT effect). We found that cues associated with the multi-functional snack facilitated the actions that earned those snacks before, while cues associated with the single-functional snack did not facilitate such actions. We discuss these findings in the context of research on free choice and personal autonomy and how people appreciate the multi-functional nature of their goal-directed behavior in the environment.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12144-023-04612-2.

A value accumulation account of unhealthy food choices: testing the influence of outcome salience under varying time constraints

People often engage in unhealthy eating despite having an explicit goal to follow a healthy diet, especially under certain conditions such as a lack of time. A promising explanation from the value accumulation account is that food choices are based on the sequential consideration of the values of multiple outcomes, such as health and taste outcomes. Unhealthy choices may result if taste is considered before health. We examined whether making a health outcome more salient could alter this order, thereby leading to more healthy choices even under time pressure. Two studies examined the time-dependent effect of outcome values and salience on food choices. Participants first completed priming trials on which they rated food items on healthiness (health condition), tastiness (taste condition), or both healthiness and tastiness (control condition). They then completed blocks of binary choice trials between healthy and tasty items. The available response time was manipulated continuously in Study 1 (N = 161) and categorically in Study 2 (N = 318). As predicted, results showed that the values of health and taste outcomes influenced choices and that priming led to more choices in line with the primed outcomes even when time was scarce. We did not obtain support for the prediction that the priming effect is time-dependent in the sense that primed outcomes are considered before non-primed outcomes. Together, these findings suggest that increasing the value and salience of a health outcome may be effective ways to increase healthy choices, even under poor conditions such as time pressure.

Effects of a brief mindfulness meditation practice on Pavlovian-to-instrumental transfer in alcohol use disorder - a pilot study

Introduction

Pavlovian conditioned contextual cues have been suggested to modulate instrumental action and might explain maladaptive behavior such as relapse in participants suffering from alcohol use disorder (AUD). Pavlovian-to-Instrumental transfer (PIT) experimentally assesses the magnitude of this context-dependent effect and studies have shown a larger PIT effect in AUD populations. Taken this into account, a reduction of the influence of cues on behavior seems warranted and one approach that could alter such cue reactivity is mindfulness. Mindfulness-based interventions have been shown to be efficient in the treatment of AUD, but underlying mechanisms are yet to be elucidated. Therefore, we aim at investigating the effect of a brief mindful body scan meditation on the magnitude of the PIT effect in AUD subjects and matched controls.

Methods

Using a randomized within-subjects design, we compared the effect of a short audio guided body scan meditation against a control condition (audio of nature sounds) on PIT in healthy (n = 35) and AUD (n = 27) participants.

Results

We found no differences in PIT effect between healthy and AUD participants as well as between conditions. However, a significant interaction effect points to a decreased PIT effect after body scan meditation in AUD subjects only.

Discussion

These pilot results suggest that AUD might be susceptible to mindfulness-induced changes in PIT, with these findings contributing to entangling the underlying mechanisms of the efficacy of mindfulness-based interventions in AUD. However, further investigation should confirm these preliminary results and the efficacy of mindfulness meditation practice in decreasing the PIT effect.

Environmental control of social goals: using Pavlovian-to-instrumental transfer to test cue-based pro-self and pro-social outcome responses

A large amount of literature demonstrates that social behaviour can be triggered by environmental cues. A long-standing debate involves the question of whether such stimuli trigger behaviour directly (i.e. habits) or whether these effects mediate goals. As studies on automatic goal pursuit typically use real-world cues that are already associated with the behaviour and potentially the goal, it is impossible to make strong claims about the nature of the effects. In the present paper, we use a paradigm inspired by the Pavlovian-to-instrumental transfer (PIT) literature to examine how the environment can trigger goal-directed behaviour. Building on the essence of pro-self and pro-social motives in humans, two experiments explored the PIT effect when the outcomes were framed in terms of self- versus other-interest. Participants performed actions to earn money for themselves or a charity. Each outcome was linked to a different cue. The results showed that a cue predictive of self-interest outcomes facilitated responses instrumental in gaining the outcome, while such specific PIT effect for other-interest outcomes only emerged when participants were free to donate the money. We briefly discuss these findings reflecting on whether the PIT effect in our paradigm is indeed sensitive to the value of social goals.

Pavlovian-to-Instrumental Transfer across Mental Disorders: A Review

A mechanism known as Pavlovian-to-instrumental transfer (PIT) describes a phenomenon by which the values of environmental cues acquired through Pavlovian conditioning can motivate instrumental behavior. PIT may be one basic mechanism of action control that can characterize mental disorders on a dimensional level beyond current classification systems. Therefore, we review human PIT studies investigating subclinical and clinical mental syndromes. The literature prevails an inhomogeneous picture concerning PIT. While enhanced PIT effects seem to be present in non-substance-related disorders, overweight people, and most studies with AUD patients, no altered PIT effects were reported in tobacco use disorder and obesity. Regarding AUD and relapsing alcohol-dependent patients, there is mixed evidence of enhanced or no PIT effects. Additionally, there is evidence for aberrant corticostriatal activation and genetic risk, e.g., in association with high-risk alcohol consumption and relapse after alcohol detoxification. In patients with anorexia nervosa, stronger PIT effects elicited by low caloric stimuli were associated with increased disease severity. In patients with depression, enhanced aversive PIT effects and a loss of action-specificity associated with poorer treatment outcomes were reported. Schizophrenic patients showed disrupted specific but intact general PIT effects. Patients with chronic back pain showed reduced PIT effects. We provide possible reasons to understand heterogeneity in PIT effects within and across mental disorders. Further, we strengthen the importance of reliable experimental tasks and provide test-retest data of a PIT task showing moderate to good reliability. Finally, we point toward stress as a possible underlying factor that may explain stronger PIT effects in mental disorders, as there is some evidence that stress per se interacts with the impact of environmental cues on behavior by selectively increasing cue-triggered wanting. To conclude, we discuss the results of the literature review in the light of Research Domain Criteria, suggesting future studies that comprehensively assess PIT across psychopathological dimensions.

Obsessive-compulsive disorder is characterized by decreased Pavlovian influence on instrumental behavior

Obsessive-compulsive disorder (OCD) is characterized by uncontrollable repetitive actions thought to rely on abnormalities within fundamental instrumental learning systems. We investigated cognitive and computational mechanisms underlying Pavlovian biases on instrumental behavior in both clinical OCD patients and healthy controls using a Pavlovian-Instrumental Transfer (PIT) task. PIT is typically evidenced by increased responding in the presence of a positive (previously rewarded) Pavlovian cue, and reduced responding in the presence of a negative cue. Thirty OCD patients and thirty-one healthy controls completed the Pavlovian Instrumental Transfer test, which included instrumental training, Pavlovian training for positive, negative and neutral cues, and a PIT phase in which participants performed the instrumental task in the presence of the Pavlovian cues. Modified Rescorla-Wagner models were fitted to trial-by-trial data of participants to estimate underlying computational mechanism and quantify individual differences during training and transfer stages. Bayesian hierarchical methods were used to estimate free parameters and compare the models. Behavioral and computational results indicated a weaker Pavlovian influence on instrumental behavior in OCD patients than in HC, especially for negative Pavlovian cues. Our results contrast with the increased PIT effects reported for another set of disorders characterized by compulsivity, substance use disorders, in which PIT is enhanced. A possible reason for the reduced PIT in OCD may be impairment in using the contextual information provided by the cues to appropriately adjust behavior, especially when inhibiting responding when a negative cue is present. This study provides deeper insight into our understanding of deficits in OCD from the perspective of Pavlovian influences on instrumental behavior and may have implications for OCD treatment modalities focused on reducing compulsive behaviors.

Dysfunction of goal-directed control in patients with depression and nonsuicidal self-injury

BACKGROUND

Non-suicidal self-injury (NSSI) is a common problem associated with dangerous outcomes. Dysfunction of goal-directed behavioral control may contribute to NSSI. To test this, we used a novel experimental paradigm (Pavlovian-to-Instrumental Transfer, PIT) to test whether patients with NSSI utilize Pavlovian conditioned stimuli (CSs) during goal-directed control of ongoing behavior.

METHODS

Thirty-five depressed patients with NSSI (D-NSSI) and thirty-four healthy controls performed a PIT task. We measured the influence of positive and negative background CSs on instrumental responses for rewards.

RESULTS

The results showed that D-NSSI performed significantly lower PIT than controls, and PIT measures were negatively correlated with NSSI frequency. Furthermore, in a subset of patients exhibiting high levels of compulsivity, PIT positively moderated the relationship between compulsivity and NSSI frequency.

CONCLUSIONS

These results indicate that D-NSSI patients have difficulties in using different CSs to control ongoing behavior in a goal-directed manner, and the dysfunction of goal-directed control may contribute to NSSI.

Effects of Motivational Downshifts on Specific Pavlovian-Instrumental Transfer in Rats

BACKGROUND

Pavlovian stimuli predictive of appetitive outcomes can exert a powerful influence on the selection and initiation of action, a phenomenon termed outcome-selective Pavlovian-instrumental transfer (sPIT). Rodent studies suggest that sPIT is insensitive to motivational downshift induced by outcome devaluation, an effect that is, however, relatively underexplored.

METHODS

Here we examined in detail the effects of distinct shifts in motivation from hunger to a state of relative satiety on sPIT in rats.

RESULTS

A motivational downshift by outcome-specific devaluation immediately prior to testing markedly reduced overall lever responding and magazine entries but left intact the sPIT effect. A motivational downshift prior testing by (1) giving ad libitum rather than restricted access to maintenance diet in the home cage for 24 hours or by (2) a systemic blockade of hormone secretagogue receptor subtype 1A receptors to inhibit orexigenic actions of ghrelin both reduced overall lever responding and magazine entries. Moreover, these latter motivational downshifts reduced the sPIT effect; however, the sizes of the sPIT effects were still large.

CONCLUSIONS

Collectively, our rodent findings indicate that major effects of various motivational downshifts are overall inhibition of lever pressing and magazine approach, possibly reflecting reduced general motivation. The observed effects of motivational downshifts on sPIT have implications with regard to the role of general motivating effects in sPIT and to the contribution of Pavlovian-instrumental interactions to excessive food seeking as well as obesity in humans.

Volume and Connectivity Differences in Brain Networks Associated with Cognitive Constructs of Binge Eating

Bulimia nervosa (BN) and binge eating disorder (BED) are characterized by episodes of eating large amounts of food while experiencing a loss of control. Recent studies suggest that the underlying causes of BN/BED consist of a complex system of environmental cues, atypical processing of food stimuli, altered behavioral responding, and structural/functional brain differences compared with healthy controls (HC). In this narrative review, we provide an integrative account of the brain networks associated with the three cognitive constructs most integral to BN and BED, namely increased reward sensitivity, decreased cognitive control, and altered negative affect and stress responding. We show altered activity in BED/BN within several brain networks, specifically in the striatum, insula, prefrontal cortex (PFC) and orbitofrontal cortex (OFC), and cingulate gyrus. Numerous key nodes in these networks also differ in volume and connectivity compared with HC. We provide suggestions for how this integration may guide future research into these brain networks and cognitive constructs.

Metabotropic group II glutamate receptors in the basolateral amygdala mediate cue-triggered increases in incentive motivation

RATIONALE

Reward-associated cues can trigger incentive motivation for reward and invigorate reward-seeking behaviour via Pavlovian-to-instrumental transfer (PIT). Glutamate signaling within the basolateral amygdala (BLA) modulates cue-triggered increases in incentive motivation. However, the role of BLA metabotropic group II glutamate (mGlu_2/3) receptors is largely unknown.

OBJECTIVES

In Experiment 1, we characterized cue-triggered increases in incentive motivation for water reward using the PIT paradigm. In Experiment 2, we assessed the influence of intra-BLA microinjections of the mGlu_2/3 receptor agonist LY379268 on this effect.

METHODS

Water-restricted male Sprague-Dawley rats learned to press a lever for water. Separately, they learned to associate one of two auditory cues with free water. On test days, rats could lever press under extinction conditions (no water), with intermittent, non-contingent CS+ and CS- presentations. In Experiment 1, rats were tested under baseline conditions. In Experiment 2, rats received intra-BLA microinjections of LY379268 (0, 3 and 6 [Formula: see text]g/hemisphere) before testing.

RESULTS

Across experiments, CS+, but not CS-, presentations increased water-associated lever pressing during testing, even though responding was reinforced neither by water nor the CS+. Intra-BLA LY379268 abolished both CS+ potentiated pressing on the water-associated lever and CS+ evoked conditioned approach to the site of water delivery. LY379268 did not influence locomotion or instrumental and Pavlovian response rates during intervals between CS presentations or during the CS-, indicating no motor effects.

CONCLUSIONS

mGlu_2/3 receptor activity in the BLA mediates cue-triggered potentiation of incentive motivation for reward, suppressing both cue-induced increases in instrumental pursuit of the reward and anticipatory approach behaviour.

Is the habit system altered in individuals with obesity? A systematic review

Habit-like eating behavior is repeatedly pointed to as a key cognitive mechanism contributing to the emergence and maintenance of obesity. Here, we conducted a systematic review of the literature to assess the existent behavioral evidence for the Habit Hypothesis for Overeating (HHO) which states that obesity is the consequence of an imbalance between the habit and goal-directed reward learning systems, leading to overconsumption of food. We found a total of 19 studies implementing a variety of experimental protocols (i.e., free operant paradigm, slips-of-action test, two-step task, Pavlovian-to-Instrumental paradigm, probabilistic learning task) and manipulations. Taken together, the studies on clinical (binge eating disorder) and non-clinical individuals with overweight or obesity do not support the HHO conclusively. While the scientific literature on HHO is still in its infancy, the heterogeneity of the extant studies makes it difficult to evaluate the degree of convergence of these findings. Uncovering the role of reward learning systems in eating behaviors might have a transformative impact on public health.

Robust valence-induced biases on motor response and confidence in human reinforcement learning

In simple instrumental-learning tasks, humans learn to seek gains and to avoid losses equally well. Yet, two effects of valence are observed. First, decisions in loss-contexts are slower. Second, loss contexts decrease individuals’ confidence in their choices. Whether these two effects are two manifestations of a single mechanism or whether they can be partially dissociated is unknown. Across six experiments, we attempted to disrupt the valence-induced motor bias effects by manipulating the mapping between decisions and actions and imposing constraints on response times (RTs). Our goal was to assess the presence of the valence-induced confidence bias in the absence of the RT bias. We observed both motor and confidence biases despite our disruption attempts, establishing that the effects of valence on motor and metacognitive responses are very robust and replicable. Nonetheless, within- and between-individual inferences reveal that the confidence bias resists the disruption of the RT bias. Therefore, although concomitant in most cases, valence-induced motor and confidence biases seem to be partly dissociable. These results highlight new important mechanistic constraints that should be incorporated in learning models to jointly explain choice, reaction times and confidence.