Sign-tracking is an expectancy-mediated behavior that relies on prediction error mechanisms

Understanding the shift to compulsion in addiction: insights from personality traits, social factors, and neurobiology

Compulsion stands as a central symptom of drug addiction; however, only a small fraction of individuals who use drugs exhibit compulsive characteristics. Differences observed in Sign-trackers (ST) and Goal-trackers (GT) during Pavlovian conditioning may shed light on individual variances in drug addiction. Here, we focus on the behavioral attributes, formation processes, and neural mechanisms underlying ST and how they drive addiction toward compulsivity in humans. We will explore addiction from three interconnected levels: individual personality traits, social factors, and neurobiology. Furthermore, we distinguish between the processes of sensitization and habituation within ST. These nuanced distinctions across various aspects of addiction will contribute to our understanding of the addiction development process and the formulation of targeted preventive strategies.

Variation in the effectiveness of reinforcement and nonreinforcement in generating different conditioned behaviors

Rat autoshaping procedures generate two readily measurable conditioned responses: During lever presentations that have previously signaled food, rats approach the food well (called goal-tracking) and interact with the lever itself (called sign-tracking). We investigated how reinforced and nonreinforced trials affect the overall and temporal distributions of these two responses across 10-second lever presentations. In two experiments, reinforced trials generated more goal-tracking than sign-tracking, and nonreinforced trials resulted in a larger reduction in goal-tracking than sign-tracking. The effect of reinforced trials was evident as an increase in goal-tracking and reduction in sign-tracking across the duration of the lever presentations, and nonreinforced trials resulted in this pattern transiently reversing and then becoming less evident with further training. These dissociations are consistent with a recent elaboration of the Rescorla-Wagner model, HeiDI (Honey, R.C., Dwyer, D.M., & Iliescu, A.F. (2020a). HeiDI: A model for Pavlovian learning and performance with reciprocal associations. Psychological Review, 127, 829-852.), a model in which responses related to the nature of the unconditioned stimulus (e.g., goal-tracking) have a different origin than those related to the nature of the conditioned stimulus (e.g., sign-tracking).

Effects of outcome revaluation on attentional prioritisation of reward-related stimuli

Stimuli associated with rewards can acquire the ability to capture our attention independently of our goals and intentions. Here, we examined whether attentional prioritisation of reward-related cues is sensitive to changes in the value of the reward itself. To this end, we incorporated an instructed outcome devaluation (Experiment 1a), "super-valuation" (Experiment 1b), or value switch (Experiment 2) into a visual search task, using eye-tracking to examine attentional prioritisation of stimuli signalling high- and low-value rewards. In Experiments 1a and 1b, we found that prioritisation of high- and low-value stimuli was insensitive to devaluation of a previously high-value outcome, and super-valuation of a previously low-value outcome, even when participants were provided with further experience of receiving that outcome. In Experiment 2, following a value-switch manipulation, we found that prioritisation of a high-value stimulus could not be overcome with knowledge of the new values of outcomes alone. Only when provided with further experience of receiving the outcomes did patterns of attentional prioritisation of high- and low-value stimuli switch, in line with the updated values of the outcomes they signalled. To reconcile these findings, we suggest that participants were motivated to engage in effortful updating of attentional control settings when there was a relative difference between reward values at test (Experiment 2) but that previous settings were allowed to persist when both outcomes had the same value at test (Experiments 1a and 1b). These findings provide a novel framework to further understand the role of cognitive control in driving reward-modulated attention and behaviour.

The role of sex on sign-tracking acquisition and outcome devaluation sensitivity in Long Evans rats

Cues that predict rewards can trigger reward-seeking behaviors but also can, in some cases, become targets of motivation themselves. One behavioral phenomenon that captures this idea is sign-tracking in which animals, including humans, interact with reward-predictive cues even though it is not necessary to do so. Sign-tracking in rats has been studied in the domain of motivation and in how motivated behaviors can or cannot become excessive and habit-like over time. Many prior studies look at sign-tracking examine this behavior in male subjects, but there are few papers that look at this behavior in female subjects. Moreover, it is unknown where there might be sex-related variation in how flexible sign-tracking is when faced with changing reward values. Therefore, we asked if there were sex differences in the acquisition of sign-tracking behavior and if there were any sex differences in how sensitive animals were in their sign-tracking following reward devaluation. In contrast to previous reports, we found that males and females show no differences in how they acquire sign-tracking and in ultimate sign-tracking levels following training. Additionally, we found no difference in how quickly males and females learned to devalue the food reward, and we found no differences in sign-tracking levels by sex following outcome devaluation. We believe that this is primarily due to our experiment being performed in the Long Evans strain but also believe that there are many other factors contributing to differences between our study and previous work.

Sex-biased effects of outcome devaluation by sensory-specific satiety on Pavlovian-conditioned behavior

Goal-directed behavior relies on accurate mental representations of the value of expected outcomes. Disruptions to this process are a central feature of several neuropsychiatric disorders, including addiction. Goal-directed behavior is most frequently studied using instrumental paradigms paired with outcome devaluation, but cue-evoked behaviors in Pavlovian settings can also be goal-directed and therefore sensitive to changes in outcome value. Emerging literature suggests that male and female rats may differ in the degree to which their Pavlovian-conditioned responses are goal-directed, but interpretation of these findings is complicated by the tendency of female and male rats to engage in distinct types of Pavlovian responses when trained with localizable cues. Here, we used outcome devaluation via sensory-specific satiety to assess the behavioral responses in male and female Long Evans rats trained to respond to an auditory CS (conditioned stimulus) in a Pavlovian-conditioning paradigm. We found that satiety-induced devaluation led to a decrease in behavioral responding to the reward-predictive CS, with males showing an effect on both port entry latency and probability and females showing an effect only on port entry probability. Overall, our results suggest that outcome devaluation affects Pavlovian-conditioned responses in both male and female rats, but that females may be less sensitive to outcome devaluation.

Excitatory and Inhibitory Signaling in the Nucleus Accumbens Encode Different Aspects of a Pavlovian Cue in Sign Tracking and Goal Tracking Rats

When a Pavlovian cue is presented separately from its associated reward, some animals will acquire a sign tracking (ST) response - approach and/or interaction with the cue - while others will acquire a goal tracking response - approach to the site of reward. We have previously shown that cue-evoked excitations in the nucleus accumbens (NAc) encode the vigor of both behaviors; in contrast, reward-related responses diverge over the course of training, possibly reflecting neurochemical differences between sign tracker and goal tracker individuals. However, a substantial subset of neurons in the NAc exhibit inhibitory, rather than excitatory, cue-evoked responses, and the evolution of their signaling during Pavlovian conditioning remains unknown. Using single-neuron recordings in behaving rats, we show that NAc neurons with cue-evoked inhibitions have distinct coding properties from neurons with cue-evoked excitations. Cue-evoked inhibitions become more numerous over the course of training and, like excitations, may encode the vigor of sign tracking and goal tracking behavior. However, the responses of cue-inhibited neurons do not evolve differently between sign tracker and goal tracker individuals. Moreover, cue-evoked inhibitions, unlike excitations, are insensitive to extinction of the cue-reward relationship. Finally, we show that cue-evoked excitations are greatly diminished by reward devaluation, while inhibitory cue responses are virtually unaffected. Overall, these findings converge with existing evidence that cue-excited neurons in NAc, but not cue-inhibited neurons, are profoundly sensitive to the same behavior variations that are often associated with changes in dopamine release.

Effects of predictive and incentive value manipulation on sign- and goal-tracking behavior

When a neutral stimulus is repeatedly paired with an appetitive reward, two different types of conditioned approach responses may develop: a sign-tracking response directed toward the neutral cue, or a goal-tracking response directed toward the location of impending reward delivery. Sign-tracking responses have been postulated to result from attribution of incentive value to conditioned cues, while goal-tracking reflects the assignment of only predictive value to the cue. We therefore hypothesized that sign-tracking rats would be more sensitive to manipulations of incentive value, while goal-tracking rats would be more responsive to changes in the predictive value of the cue. We tested sign- and goal-tracking before and after devaluation of a food reward using lithium chloride, and tested whether either response could be learned under negative contingency conditions that precluded any serendipitous reinforcement of the behavior that might support instrumental learning. We also tested the effects of blocking the predictive value of a cue using simultaneous presentation of a pre-conditioned cue. We found that sign-tracking was sensitive to outcome devaluation, while goal-tracking was not. We also confirmed that both responses are Pavlovian because they can be learned under negative contingency conditions. Goal-tracking was almost completely blocked by a pre-conditioned cue, while sign-tracking was much less sensitive to such interference. These results indicate that sign- and goal-tracking may follow different rules of reinforcement learning and suggest a need to revise current models of associative learning to account for these differences.

Nucleus accumbens core acetylcholine receptors modulate the balance of flexible and inflexible cue-directed motivation

Sign-tracking is a conditioned response where animals interact with reward-predictive cues due to the cues having motivational value, or incentive salience. The nucleus accumbens core (NAc) has been implicated in mediating the sign-tracking response. Additionally, acetylcholine (ACh) transmission throughout the striatum has been attributed to both incentive motivation and behavioral flexibility. Here, we demonstrate a role for NAc ACh receptors in the flexibility of sign-tracking. Sign-tracking animals were exposed to an omission contingency, in which vigorous sign-tracking was punished by reward omission. Animals rapidly adjusted their behavior, but they maintained sign-tracking in a less vigorous manner that did not cancel reward. Within this context of sign-tracking being persistent yet flexible in structure, blockade of NAc nicotinic receptors (nAChRs) led to a persistence in the initial sign-tracking response during omission followed by a period of change in the makeup of sign-tracking, whereas blockade of muscarinic receptors (mAChRs) oppositely enhanced the omission-related development of the new sign-tracking behaviors. Later, once omission learning had occurred, nAChR blockade uniquely led to reduced sign-tracking and elevated reward-directed behaviors instead. These results indicate that NAc ACh receptors have opposing roles in maintaining learned patterns of sign-tracking, with nAChRs having a special involvement in regulating the structure of the sign-tracking response.

Outcome devaluation by sensory-specific satiety alters Pavlovian-conditioned behavior in male and female rats

Goal-directed behavior relies on accurate mental representations of the value of expected outcomes. Disruptions to this process are a central feature of several neuropsychiatric disorders, including addiction. Goal-directed behavior is most frequently studied using instrumental paradigms paired with outcome devaluation, but cue-evoked behaviors in Pavlovian settings can also be goal-directed and therefore sensitive to changes in outcome value. Emerging literature suggests that male and female rats may differ in the degree to which their Pavlovian-conditioned responses are goal-directed, but interpretation of these findings is complicated by the tendency of female and male rats to engage in distinct types of Pavlovian responses when trained with localizable cues. Here, we used outcome devaluation via sensory-specific satiety to assess the behavioral responses in male and female Long Evans rats trained to respond to an auditory CS (conditioned stimulus) in a Pavlovian-conditioning paradigm. We found that satiety-induced devaluation led to a decrease in behavioral responding to the reward-predictive CS, with males showing an effect on both port entry latency and probability and females showing an effect only on port entry probability. Overall, our results suggest that outcome devaluation affects Pavlovian-conditioned responses in both male and female rats, but that females may be less sensitive to outcome devaluation.

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

A mechanical task for measuring sign- and goal-tracking in humans: A proof-of-concept study

Cue-based associative learning (i.e., Pavlovian conditioning) is a foundational component of behavior in almost all forms of animal life and may provide insight into individual differences in addiction liability. Cues can take on incentive-motivational properties (i.e., incentive salience) through Pavlovian learning. Extensive testing with non-human animals (primarily rats) has demonstrated significant variation among individuals in the behaviors this type of learning evokes. So-named "sign-trackers" and "goal-trackers" have been examined in many studies of non-human animals, but this work in humans is still a nascent area of research. In the present proof-of-concept study, we used a Pavlovian conditioned approach task to investigate human sign- and goal-tracking in emerging adults. Conditioned behaviors that developed over the course of the task were directed toward the reward-cue and toward the reward location. Participants' eye-gaze and behavior during the task were submitted to a latent profile analysis, which revealed three groups defined as sign-trackers (n = 10), goal-trackers (n = 4), and intermediate responders (n = 36). Impulsivity was a significant predictor of the sign-tracking group relative to the goal-tracking group. The present study provides preliminary evidence that a simple procedure can produce learned Pavlovian conditioned approach behavior in humans. Though further investigation is required, findings provide a promising step toward the long-term goal of translating important insights gleaned from basic research into treatment strategies that can be applied to clinical populations.

Flexible control of Pavlovian-instrumental transfer based on expected reward value

The Pavlovian-instrumental transfer (PIT) paradigm is widely used to assay the motivational influence of reward-predictive cues, reflected by their ability to invigorate instrumental behavior. Leading theories assume that a cue's motivational properties are tied to predicted reward value. We outline an alternative view that recognizes that reward-predictive cues may suppress rather than motivate instrumental behavior under certain conditions, an effect termed positive conditioned suppression. We posit that cues signaling imminent reward delivery tend to inhibit instrumental behavior, which is exploratory by nature, in order to facilitate efficient retrieval of the expected reward. According to this view, the motivation to engage in instrumental behavior during a cue should be inversely related to the value of the predicted reward, since there is more to lose by failing to secure a high-value reward than a low-value reward. We tested this hypothesis in rats using a PIT protocol known to induce positive conditioned suppression. In Experiment 1, cues signaling different reward magnitudes elicited distinct response patterns. Whereas the one-pellet cue increased instrumental behavior, cues signaling three or nine pellets suppressed instrumental behavior and elicited high levels of food-port activity. Experiment 2 found that reward-predictive cues suppressed instrumental behavior and increased food-port activity in a flexible manner that was disrupted by post-training reward devaluation. Further analyses suggest that these findings were not driven by overt competition between the instrumental and food-port responses. We discuss how the PIT task may provide a useful tool for studying cognitive control over cue-motivated behavior in rodents. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Attentional capture by signals of reward persists following outcome devaluation

Attention, the mechanism that prioritizes stimuli in the environment for further processing, plays an important role in behavioral choice. In the present study, we investigated the automatic orienting of attention to cues that signal reward. Such attentional capture occurs despite negative consequences, and we investigated whether this counterproductive and reflexive behavior would persist following outcome devaluation. Thirsty participants completed a visual search task in which the color of a distractor stimulus in the search display signaled whether participants would earn water or potato chips for making a rapid eye movement to a diamond target, but looking at the colored distractor was punished by omission of the signaled reward. Nevertheless, participants looked at the water-signaling distractor more frequently than the chip-signaling distractor. Half the participants then drank water ad libitum before continuing with the visual search task. Although the water was now significantly less desirable for half of the participants, there was no difference between groups in the tendency for the water-signaling distractor to capture attention. These findings suggest that once established, counterproductive attentional bias to signals of reward persists even when those outcomes are no longer valuable. This suggests a "habit-like" attentional mechanism that prioritizes reward stimuli in the environment for further action, regardless of whether those stimuli are aligned with current goals or currently desired.

Planning in the brain

Recent breakthroughs in artificial intelligence (AI) have enabled machines to plan in tasks previously thought to be uniquely human. Meanwhile, the planning algorithms implemented by the brain itself remain largely unknown. Here, we review neural and behavioral data in sequential decision-making tasks that elucidate the ways in which the brain does-and does not-plan. To systematically review available biological data, we create a taxonomy of planning algorithms by summarizing the relevant design choices for such algorithms in AI. Across species, recording techniques, and task paradigms, we find converging evidence that the brain represents future states consistent with a class of planning algorithms within our taxonomy-focused, depth-limited, and serial. However, we argue that current data are insufficient for addressing more detailed algorithmic questions. We propose a new approach leveraging AI advances to drive experiments that can adjudicate between competing candidate algorithms.

Quantifying the instrumental and noninstrumental underpinnings of Pavlovian responding with the Price equation

The Price equation is a mathematical expression of selectionist and non-selectionist pressures on biological, cultural, and behavioral change. We use it here to specify instrumental and noninstrumental behaviors as they arise within the context of the Pavlovian autoshaping procedure, for rats trained under reward certainty and reward uncertainty. The point of departure for this endeavor is that some portion of autoshaped behavior referred to as goal-tracking appears instrumental-a function of resource attainment (the individual approaches the location where the unconditioned stimulus is to be delivered). By contrast, some other portion of autoshaped behavior referred to as sign-tracking is noninstrumental-irrelevant to making contact with the to-be-delivered unconditioned stimulus. A Price equation model is proposed that unifies our understanding of Pavlovian autoshaping behavior by isolating operant and respondent influences on goal-tracking (instrumental) and sign-tracking (noninstrumental) behavior.

Sign Tracking in an Enriched Environment: A Potential Ecologically Relevant Animal Model of Adaptive Behavior Change

When an object conditioned stimulus (CS) is paired with a food unconditioned stimulus (US), anticipatory goal-directed action directed at the US location (goal tracking) is accompanied by behavior directed at the object CS (sign tracking). Sign-tracking behavior appears to be compulsive and habit-like and predicts increased vulnerability to the addictive potential of drugs in animal models. A large body of the literature also suggests that environmental enrichment protects against the development of addiction-prone phenotypes. Thus, we investigated whether compulsive-like sign tracking develops in environmentally enriched rats trained directly in their enriched home environment. We demonstrate that adolescent enriched-housed male Sprague-Dawley rats readily sign track a 5% ethanol bottle CS in their home environment and at a rate higher than adolescent standard-housed rats. We also show that enriched adolescent rats sign track less than enriched adult-trained rats and that acute isolation stress affects sign- and goal-tracking performance of adolescents and adults differently. Sign tracking increased more in the adult than the adolescent rats. Whereas the younger rats showed a decrease in goal tacking after the final stressor manipulation, the adults showed increased goal tracking. Our results are consistent with recent studies, which suggest that although sign tracking performance is compulsive-like, it is not as inflexible and habit-like as previously assumed. Testing in an enriched home environment with object CSs having greater affordance than "neutral" lever CSs may provide greater ecological relevance for investigating the development and expression of adaptive and compulsive-like behaviors in translational research.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

Behavioral determinants in the expression of the Kamin blocking effect: Implications for associative learning theory

Associative learning makes important contributions to our behavior and decisions. The Kamin blocking effect is an associative learning phenomenon that plays a central role in understanding of the psychological principles underlying associative learning. However, several recent failures to replicate the blocking effect suggest that the conditions necessary for blocking are poorly understood. To understand the conditions necessary for blocking, here we review studies into the expression of blocking in subjects that either approach and interact with the conditioned cue (sign trackers) or approach and interact with the reward location (goal trackers) during appetitive classical conditioning. Psychological theory and the neurophysiological correlates of appetitive classical conditioning make opposing predictions regarding the expression of blocking in sign and goal trackers. We reconcile these opposing predictions in a qualitative model using two parallel learning processes. Such models offer a better framework for understanding the psychological associative structures acquired during learning, their interactions contributing to the conditioned response, and how they affect subsequent learning and the expression of the Kamin blocking effect.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Individual variation in the vigor and form of Pavlovian conditioned responses: Analysis of a model system

Pavlovian conditioning results in individual variation in the vigor and form of acquired behaviors. Here, we describe a general-process model of associative learning (HeiDI; How excitation and inhibition determine ideo-motion) that provides an analysis for such variation together with a range of other important group-level phenomena. The model takes as its starting point the idea that pairings of a conditioned stimulus (CS) and an unconditioned stimulus (US) result in the formation of reciprocal associations between their central representations. The asymptotic values of these associations and the rate at which these are reached are held to be influenced by the perceived salience of the CS (αCS) and US (βUS). Importantly, whether this associative knowledge is exhibited in behavior that reflects the properties of the CS (e.g., sign-tracking) or US (e.g., goal-tracking) is also influenced by the relative values of αCS and βUS. In this way, HeiDI provides an analysis for both quantitative and qualitative individual differences generated by Pavlovian conditioning procedures.

Mapping sign-tracking and goal-tracking onto human behaviors

As evidenced through classic Pavlovian learning mechanisms, environmental cues can become incentivized and influence behavior. These stimulus-outcome associations are relevant in everyday life but may be particularly important for the development of impulse control disorders including addiction. Rodent studies have elucidated specific learning profiles termed 'sign-tracking' and 'goal-tracking' which map onto individual differences in impulsivity and other behaviors associated with impulse control disorders' etiology, course, and relapse. Whereas goal-trackers are biased toward the outcome, sign-trackers fixate on features that are associated with but not necessary for achieving an outcome; a pattern of behavior that often leads to escalation of reward-seeking that can be maladaptive. The vast majority of the sign- and goal-tracking research has been conducted using rodent models and very few have bridged this concept into the domain of human behavior. In this review, we discuss the attributes of sign- and goal-tracking profiles, how these are manifested neurobiologically, and how these distinct learning styles could be an important tool for clinical interventions in human addiction.

Sign-tracking behavior is sensitive to outcome devaluation in a devaluation context-dependent manner: implications for analyzing habitual behavior

Motivationally attractive cues can draw in behavior in a phenomenon termed incentive salience. Incentive cue attraction is an important model for animal models of drug seeking and relapse. One question of interest is the extent to which the pursuit of motivationally attractive cues is related to the value of the paired outcome or can become unrelated and habitual. We studied this question using a sign-tracking (ST) paradigm in rats, in which a lever stimulus preceding food reward comes to elicit conditioned lever-interaction behavior. We asked whether reinforcer devaluation by means of conditioned taste aversion, a classic test of habitual behavior, can modify ST to incentive cues, and whether this depends upon the manner in which reinforcer devaluation takes place. In contrast to several recent reports, we conclude that ST is indeed sensitive to reinforcer devaluation. However, this effect depends critically upon the congruence between the context in which taste aversion is learned and the context in which it is tested. When the taste aversion successfully transfers to the testing context, outcome value strongly influences ST behavior, both when the outcome is withheld (in extinction) and when animals can learn from outcome feedback (reacquisition). When taste aversion does not transfer to the testing context, ST remains high. In total, the extent to which ST persists after outcome devaluation is closely related to the extent to which that outcome is truly devalued in the task context. We believe this effect of context on devaluation can reconcile contradictory findings about the flexibility/inflexibility of ST. We discuss this literature and relate our findings to the study of habits generally.

Effects of Limited and Extended Pavlovian Training on Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Individual differences in Pavlovian approach predict differences in devaluation sensitivity. Recent studies indicate goal-tracking (GT) rats are sensitive to outcome devaluation while sign-tracking (ST) rats are not. With extended training in Pavlovian lever autoshaping (PLA), GT rats display more lever-directed behavior, typical of ST rats, suggesting they may become insensitive to devaluation with more Pavlovian training experience. Here, we use a within-subject satiety-induced outcome devaluation procedure to test devaluation sensitivity after limited and extended PLA training in GT and ST rats. We trained rats in PLA to determine GT and ST groups. Then, we sated rats on either the training pellets (devalued condition) or homecage chow (valued condition) prior to brief non-reinforced test sessions after limited (sessions 5/6) and extended (sessions 17/18) PLA training. GT rats decreased conditioned responding under devalued relative to valued conditions after both limited and extended training, demonstrating they are sensitive to satiety devaluation regardless of the amount of PLA training. While ST rats were insensitive to satiety devaluation after limited training, their lever directed behavior became devaluation sensitive after extended training. To determine whether sign-tracking rats also displayed sensitivity to illness-induced outcome devaluation after extended training, we trained a separate cohort of rats in extended PLA and devalued the outcome with lithium chloride injections after pellet consumption in the homecage. ST rats failed to decrease conditioned responding after illness-induced outcome devaluation, while Non-ST rats (GT and intermediates) were sensitive to illness-induced outcome devaluation after extended training. Together, our results confirm devaluation sensitivity is stable in GT rats across training and devaluation approaches. Extended training unmasks devaluation sensitivity in ST rats after satiety, but not illness-induced devaluation, suggesting ST rats respond appropriately by decreasing responding to cues during state-dependent but not inference-based devaluation. The differences in behavioral flexibility across tracking groups and devaluation paradigms have translational relevance for the understanding state- vs. inference-based reward devaluation as it pertains to drug addiction vulnerability.

Increased Goal Tracking in Adolescent Rats Is Goal-Directed and Not Habit-Like

When a cue is paired with reward in a different location, some animals will approach the site of reward during the cue, a behavior called goal tracking, while other animals will approach and interact with the cue itself: a behavior called sign tracking. Sign tracking is thought to reflect a tendency to transfer incentive salience from the reward to the cue. Adolescence is a time of heightened sensitivity to rewards, including environmental cues that have been associated with rewards, which may account for increased impulsivity and vulnerability to drug abuse. Surprisingly, however, studies have shown that adolescents are actually less likely to interact with the cue (i.e., sign track) than adult animals. We reasoned that adolescents might show decreased sign tracking, accompanied by increased apparent goal tracking, because they tend to attribute incentive salience to a more reward-proximal "cue": the food magazine. On the other hand, adolescence is also a time of enhanced exploratory behavior, novelty-seeking, and behavioral flexibility. Therefore, adolescents might truly express more goal-directed reward-seeking and less inflexible habit-like approach to a reward-associated cue. Using a reward devaluation procedure to distinguish between these two hypotheses, we found that adolescents indeed exhibit more goal tracking, and less sign tracking, than a comparable group of adults. Moreover, adolescents' goal tracking behavior is highly sensitive to reward devaluation and therefore goal-directed and not habit-like.

Sign- and goal-tracking rats show differences in various executive functions: Authors

Sign tracking (ST) is a complex Pavlovian trait that is known to impact instrumental behaviour. Recent work suggests that this trait also correlates with altered top-down executive control relative to goal tracking (GT) rats. This raises the question as to the extent to which both phenotypes differ in executive functions. Moreover, it is unclear which cognitive processes might cause potential differences between ST and GT rats. We therefore compared the behaviour of ST and GT rats in several assays, such as outcome devaluation, attentional set shifting and reversal learning, conditional responding, as well as delayed alternation to measure different aspects of executive functioning. Goal-directed behaviour per se was not different between ST and GT rats in the outcome devaluation task. ST rats performed slightly better than GT rats in one condition of the set shifting task (place->cue shift) and the delayed alternation task, but did not perform as well in the conditional responding task. Thus, differential behavioural performance between ST and GT rats was dependent on the specific task context. Further, we found evidence that the differences in executive functions are likely related to increased incentive salience attribution and impulsive action in ST rats.

Sign Tracking and Goal Tracking Are Characterized by Distinct Patterns of Nucleus Accumbens Activity

During Pavlovian conditioning, if a cue (e.g., lever extension) predicts reward delivery in a different location (e.g., a food magazine), some individuals will come to approach and interact with the cue, a behavior known as sign tracking (ST), and others will approach the site of reward, a behavior known as goal tracking (GT). In rats, the acquisition of ST versus GT behavior is associated with distinct profiles of dopamine release in the nucleus accumbens (NAc), but it is unknown whether it is associated with different patterns of accumbens neural activity. Therefore, we recorded from individual neurons in the NAc core during the acquisition, maintenance, and extinction of ST and GT behavior. Even though NAc dopamine is specifically important for the acquisition and expression of ST, we found that cue-evoked excitatory responses encode the vigor of both ST and GT behavior. In contrast, among sign trackers only, there was a prominent decrease in reward-related activity over the course of training, which may reflect the decreasing reward prediction error encoded by phasic dopamine. Finally, both behavior and cue-evoked activity were relatively resistant to extinction in sign trackers, as compared with goal trackers, although a subset of neurons in both groups retained their cue-evoked responses. Overall, the results point to the convergence of multiple forms of reward learning in the NAc.

Impacts of inter-trial interval duration on a computational model of sign-tracking vs. goal-tracking behaviour

In the context of Pavlovian conditioning, two types of behaviour may emerge within the population (Flagel et al. Nature, 469(7328): 53-57, 2011). Animals may choose to engage either with the conditioned stimulus (CS), a behaviour known as sign-tracking (ST) which is sensitive to dopamine inhibition for its acquisition, or with the food cup in which the reward or unconditioned stimulus (US) will eventually be delivered, a behaviour known as goal-tracking (GT) which is dependent on dopamine for its expression only. Previous work by Lesaint et al. (PLoS Comput Biol, 10(2), 2014) offered a computational explanation for these phenomena and led to the prediction that varying the duration of the inter-trial interval (ITI) would change the relative ST-GT proportion in the population as well as phasic dopamine responses. A recent study verified this prediction, but also found a rich variance of ST and GT behaviours within the trial which goes beyond the original computational model. In this paper, we provide a computational perspective on these novel results.