Dissociating neural learning signals in human sign- and goal-trackers

Pupil dilation reflects effortful action invigoration in overcoming aversive Pavlovian biases

"Pavlovian" or "motivational" biases describe the phenomenon that the valence of prospective outcomes modulates action invigoration: Reward prospect invigorates action, whereas punishment prospect suppresses it. The adaptive role of these biases in decision-making is still unclear. One idea is that they constitute a fast-and-frugal decision strategy in situations characterized by high arousal, e.g., in presence of a predator, which demand a quick response. In this pre-registered study (N = 35), we tested whether such a situation-induced via subliminally presented angry versus neutral faces-leads to increased reliance on Pavlovian biases. We measured trial-by-trial arousal by tracking pupil diameter while participants performed an orthogonalized Motivational Go/NoGo Task. Pavlovian biases were present in responses, reaction times, and even gaze, with lower gaze dispersion under aversive cues reflecting "freezing of gaze." The subliminally presented faces did not affect responses, reaction times, or pupil diameter, suggesting that the arousal manipulation was ineffective. However, pupil dilations reflected facets of bias suppression, specifically the physical (but not cognitive) effort needed to overcome aversive inhibition: Particularly strong and sustained dilations occurred when participants managed to perform Go responses to aversive cues. Conversely, no such dilations occurred when they managed to inhibit responses to Win cues. These results suggest that pupil diameter does not reflect response conflict per se nor the inhibition of prepotent responses, but specifically effortful action invigoration as needed to overcome aversive inhibition. We discuss our results in the context of the "value of work" theory of striatal dopamine.

The Continuing Challenges of Studying Parallel Behaviours in Humans and Animal Models

The use of animal models continues to be essential for carrying out research into clinical phenomena, including addiction. However, the complexity of the clinical condition inevitably means that even the best animal models are inadequate, and this may go some way to account for the apparent failures of discoveries from animal models, including the identification of potential novel therapies, to translate to the clinic. We argue here that it is overambitious and misguided in the first place to attempt to model complex, multifacetted human disorders such as addiction in animals, and especially in rodents, and that all too frequently "validity" of such models is limited to superficial similarities, referred to as "face validity", that reflect quite different underlying phenomena and biological processes from the clinical situation. Instead, a more profitable approach is to identify (a) well-defined intermediate human behavioural phenotypes that reflect defined, limited aspects of, or contributors to, the human clinical disorder, and (b) to develop animal models that are homologous with those discrete human behavioural phenotypes in terms of psychological processes, and underlying neurobiological mechanisms. Examples of past and continuing weaknesses and suggestions for more limited approaches that may allow better homology between the test animal and human condition are made.

Risk-promoting effects of reward-paired cues in human sign- and goal-trackers

Animal research suggests trait-like individual variation in the degree of incentive salience attribution to reward-predictive cues, defined phenotypically as sign-tracking (high) and goal-tracking (low incentive salience attribution). While these phenotypes have been linked to addiction features in rodents, their translational validity is less clear. Here, we examined whether sign- and goal-tracking in healthy human volunteers modulates the effects of reward-paired cues on decision making. Sign-tracking was measured in a Pavlovian conditioning paradigm as the amount of eye gaze fixation on the reward-predictive cue versus the location of impending reward delivery. In Study 1 (Cherkasova et al., 2018), participants were randomly assigned to perform a binary choice task in which rewards were either accompanied (cued, n = 63) or unaccompanied (uncued, n = 68) by money images and casino jingles. In Study 2, participants (n = 58) performed cued and uncued versions of the task in a within-subjects design. Across both studies, cues promoted riskier choice. Sign-tracking was not associated with risky choice in either study. Goal-tracking rather than sign-tracking was significantly associated with greater risk-promoting effects of cues in Study 1 but not in Study 2, although the direction of findings was consistent across both studies. These findings are at odds with the notion of sign-trackers being preferentially susceptible to the influence of reward cues on behavior and point to the role of mechanisms besides incentive salience in mediating such influences.

Sign-tracking and goal-tracking in humans: Utilising eye-tracking in clinical and non-clinical populations

BACKGROUND

In Pavlovian conditioning, learned behaviour varies according to the perceived value of environmental cues. For goal-trackers (GT), the cue merely predicts a reward, whilst for sign-trackers (ST), the cue holds incentive value. The sign-tracking/goal-tracking model is well-validated in animals, but translational work is lacking. Despite the model's relevance to several conditions, including attention deficit hyperactivity disorder (ADHD), we are unaware of any studies that have examined the model in clinical populations.

METHODS

The current study used an eye-tracking Pavlovian conditioning paradigm to identify ST and GT in non-clinical (N = 54) and ADHD (N = 57) participants. Eye movements were recorded whilst performing the task. Dwell time was measured for two areas of interest: sign (i.e., cue) and goal (i.e., reward), and an eye-gaze index (EGI) was computed based on the dwell time sign-to-goal ratio. Higher EGI values indicate sign-tracking behaviour. ST and GT were determined using median and tertiary split approaches in both samples.

RESULTS

Despite greater propensity for sign-tracking in those with ADHD, there was no significant difference between groups. The oculomotor conditioned response was reward-specific (CS+) and present, at least partly, from the start of the task indicating dispositional and learned components. There were no differences in externalising behaviours between ST and GT for either sample.

CONCLUSIONS

Sign-tracking is associated with CS+ trials only. There may be both dispositional and learned components to sign-tracking, potentially more common in those with ADHD. This holds translational potential for understanding individual differences in reward-learning.

Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans

We aim to differentiate the brain regions involved in the learning and encoding of Pavlovian associations sensitive to changes in outcome value from those that are not sensitive to such changes by combining a learning task with outcome devaluation, eye-tracking, and functional magnetic resonance imaging in humans. Contrary to theoretical expectation, voxels correlating with reward prediction errors in the ventral striatum and subgenual cingulate appear to be sensitive to devaluation. Moreover, regions encoding state prediction errors appear to be devaluation insensitive. We can also distinguish regions encoding predictions about outcome taste identity from predictions about expected spatial location. Regions encoding predictions about taste identity seem devaluation sensitive while those encoding predictions about an outcome's spatial location seem devaluation insensitive. These findings suggest the existence of multiple and distinct associative mechanisms in the brain and help identify putative neural correlates for the parallel expression of both devaluation sensitive and insensitive conditioned behaviors.

Propensity for risky choices despite lower cue reactivity in adolescent rats

Adolescence is a time of heightened risk-taking across species. Salient audiovisual cues associated with rewards are a common feature of gambling environments and have been connected to increased risky decision-making. We have previously shown that, in adult male rats, sign tracking - a behavioral measure of cue reactivity - predicts an individual's propensity for suboptimal risky choices in a rodent gambling task (rGT) with win-paired cues. However, adolescents perform less sign tracking than adult animals, suggesting that they are less cue-reactive than adults in some circumstances. Therefore, we investigated the performance of adolescent male rats on the rGT with win cues and examined its relationship with their sign-tracking behavior. We found that adolescents make more risky choices and fewer optimal choices on the rGT compared with adults, evidence of the validity of the rGT as a model of adolescent gambling behavior. We also confirmed that adolescents perform less sign tracking than adults, and we found that, unlike in adults, adolescents' sign tracking was unrelated to their risk-taking in the rGT. This implies that adolescent risk-taking is less likely than that of adults to be driven by reward-related cues. Finally, we found that adults trained on the rGT as adolescents retained an adolescent-like propensity toward risky choices, suggesting that early exposure to a gambling environment may have a long-lasting impact on risk-taking behavior.

Individual differences in late positive potential amplitude and theta power predict cue-induced eating

Cue-induced reward-seeking behaviors are regulated by both the affective and cognitive control systems of the brain. This study aimed at investigating how individual differences in affective and cognitive responses to cues predicting food rewards contribute to the regulation of cue-induced eating. We recorded electroencephalogram (EEG) from 59 adults while they viewed emotional and food-related images that preceded the delivery of food rewards (candies) or non-food objects (beads). We measured the amplitude of the late positive potential (LPP) in response to a variety of motivationally relevant images and power in the theta (4-8 Hz) frequency band after candies or beads were dispensed to the participants. We found that individuals with larger LPP responses to food images than to pleasant images (C>P group) ate significantly more during the experiment than those with the opposite response pattern (P>C group, p < 0.001). Furthermore, we found that individuals with higher theta power after dispensation of the candy than of the bead (θCA>θBE) ate significantly more than those with the opposite response pattern (θBE>θCA, p < 0.001). Finally, we found that the crossed P>C and θBE>θCA group ate less (p < 0.001) than did the other three groups formed by crossing the LPP and theta group assignments, who exhibited similar eating behavior on average (p = 0.662). These findings demonstrate that individual differences in both affective and cognitive responses to reward-related cues underlie vulnerability to cue-induced behaviors, underscoring the need for individualized treatments to mitigate maladaptive behaviors.

The propensity to sign-track is associated with externalizing behavior and distinct patterns of reward-related brain activation in youth

Externalizing behaviors in childhood often predict impulse control disorders in adulthood; however, the underlying bio-behavioral risk factors are incompletely understood. In animals, the propensity to sign-track, or the degree to which incentive motivational value is attributed to reward cues, is associated with externalizing-type behaviors and deficits in executive control. Using a Pavlovian conditioned approach paradigm, we quantified sign-tracking in 40 healthy 9-12-year-olds. We also measured parent-reported externalizing behaviors and anticipatory neural activations to outcome-predicting cues using the monetary incentive delay fMRI task. Sign-tracking was associated with attentional and inhibitory control deficits and the degree of amygdala, but not cortical, activation during reward anticipation. These findings support the hypothesis that youth with a propensity to sign-track are prone to externalizing tendencies, with an over-reliance on subcortical cue-reactive brain systems. This research highlights sign-tracking as a promising experimental approach delineating the behavioral and neural circuitry of individuals at risk for externalizing disorders.

Reward Processing in Alcohol-Dependent Patients and First-Degree Relatives: Functional Brain Activity During Anticipation of Monetary Gains and Losses

BACKGROUND

According to the reward deficiency syndrome and allostatic hypotheses, hyposensitivity of mesocorticolimbic regions to non-alcohol-related stimuli predisposes to dependence or is long-lastingly enhanced by chronic substance use. To date, no study has directly compared mesocorticolimbic brain activity during non-drug reward anticipation between alcohol-dependent, at risk, and healthy subjects.

METHODS

Seventy-five abstinent alcohol-dependent human subjects (mean abstinence duration 957.66 days), 62 healthy first-degree relatives of alcohol-dependent individuals, and 76 healthy control subjects without family history of alcohol dependence performed a monetary incentive delay task. Functional magnetic resonance imaging data of the anticipation phase were analyzed, during which visual cues predicted that fast response to a target would result in monetary gain, avoidance of monetary loss, or a neutral outcome.

RESULTS

During gain anticipation, there were no significant group differences. During loss anticipation, abstinent alcohol-dependent subjects showed lower activity in the left anterior insula compared with healthy control subjects without family history of alcohol dependence only (Montreal Neurological Institute [MNI] -25 19 -5; t₂₀₆ = 4.17, familywise error corrected p = .009). However, this effect was no longer significant when age was included as a covariate. There were no group differences between abstinent alcohol-dependent subjects and healthy first-degree relatives or between healthy first-degree relatives and healthy control subjects during loss anticipation, respectively.

CONCLUSIONS

Neither the neural reward deficiency syndrome nor the allostatic hypotheses are supported by the results. Future studies should investigate whether the incentive salience hypothesis allows for more accurate predictions regarding mesocorticolimbic brain activity of subjects with alcohol dependence and healthy individuals during reward and loss anticipation and further examine the neural substrates underlying a predisposition to dependence.

A mechanical task for measuring sign- and goal-tracking in humans: A proof-of-concept study

Cue-based associative learning (i.e., Pavlovian conditioning) is a foundational component of behavior in almost all forms of animal life and may provide insight into individual differences in addiction liability. Cues can take on incentive-motivational properties (i.e., incentive salience) through Pavlovian learning. Extensive testing with non-human animals (primarily rats) has demonstrated significant variation among individuals in the behaviors this type of learning evokes. So-named "sign-trackers" and "goal-trackers" have been examined in many studies of non-human animals, but this work in humans is still a nascent area of research. In the present proof-of-concept study, we used a Pavlovian conditioned approach task to investigate human sign- and goal-tracking in emerging adults. Conditioned behaviors that developed over the course of the task were directed toward the reward-cue and toward the reward location. Participants' eye-gaze and behavior during the task were submitted to a latent profile analysis, which revealed three groups defined as sign-trackers (n = 10), goal-trackers (n = 4), and intermediate responders (n = 36). Impulsivity was a significant predictor of the sign-tracking group relative to the goal-tracking group. The present study provides preliminary evidence that a simple procedure can produce learned Pavlovian conditioned approach behavior in humans. Though further investigation is required, findings provide a promising step toward the long-term goal of translating important insights gleaned from basic research into treatment strategies that can be applied to clinical populations.

Basal Forebrain Chemogenetic Inhibition Converts the Attentional Control Mode of Goal-Trackers to That of Sign-Trackers

Sign tracking versus goal tracking in rats indicate vulnerability and resistance, respectively, to Pavlovian cue-evoked addictive drug taking and relapse. Here, we tested hypotheses predicting that the opponent cognitive-behavioral styles indexed by sign tracking versus goal tracking include variations in attentional performance which differentially depend on basal forebrain projection systems. Pavlovian Conditioned Approach (PCA) testing was used to identify male and female sign-trackers (STs) and goal-trackers (GTs), as well as rats with an intermediate phenotype (INTs). Upon reaching asymptotic performance in an operant task requiring the detection of visual signals (hits) as well as the reporting of signal absence for 40 min per session, GTs scored more hits than STs, and hit rates across all phenotypes correlated with PCA scores. STs missed relatively more signals than GTs specifically during the last 15 min of a session. Chemogenetic inhibition of the basal forebrain decreased hit rates in GTs but was without effect in STs. Moreover, the decrease in hits in GTs manifested solely during the last 15 min of a session. Transfection efficacy in the horizontal limb of the diagonal band (HDB), but not substantia innominate (SI) or nucleus basalis of Meynert (nbM), predicted the behavioral efficacy of chemogenetic inhibition in GTs. Furthermore, the total subregional transfection space, not transfection of just cholinergic neurons, correlated with performance effects. These results indicate that the cognitive-behavioral phenotype indexed by goal tracking, but not sign tracking, depends on activation of the basal forebrain-frontal cortical projection system and associated biases toward top-down or model-based performance.

The role of reinforcement learning and value-based decision-making frameworks in understanding food choice and eating behaviors

The obesogenic food environment includes easy access to highly-palatable, energy-dense, "ultra-processed" foods that are heavily marketed to consumers; therefore, it is critical to understand the neurocognitive processes the underlie overeating in response to environmental food-cues (e.g., food images, food branding/advertisements). Eating habits are learned through reinforcement, which is the process through which environmental food cues become valued and influence behavior. This process is supported by multiple behavioral control systems (e.g., Pavlovian, Habitual, Goal-Directed). Therefore, using neurocognitive frameworks for reinforcement learning and value-based decision-making can improve our understanding of food-choice and eating behaviors. Specifically, the role of reinforcement learning in eating behaviors was considered using the frameworks of (1) Sign-versus Goal-Tracking Phenotypes; (2) Model-Free versus Model-Based; and (3) the Utility or Value-Based Model. The sign-and goal-tracking phenotypes may contribute a mechanistic insight on the role of food-cue incentive salience in two prevailing models of overconsumption-the Extended Behavioral Susceptibility Theory and the Reactivity to Embedded Food Cues in Advertising Model. Similarly, the model-free versus model-based framework may contribute insight to the Extended Behavioral Susceptibility Theory and the Healthy Food Promotion Model. Finally, the value-based model provides a framework for understanding how all three learning systems are integrated to influence food choice. Together, these frameworks can provide mechanistic insight to existing models of food choice and overconsumption and may contribute to the development of future prevention and treatment efforts.

Electrophysiological responses to emotional and cocaine cues reveal individual neuroaffective profiles in cocaine users

Smokers with stronger neuroaffective responses to drug-related cues compared to nondrug-related pleasant images (C > P) are more vulnerable to compulsive smoking than individuals with the opposite brain reactivity profile (P > C). However, it is unknown if these neurobehavioral profiles exist in individuals abusing other drugs. We tested whether individuals with cocaine use disorder (CUD) show similar neuroaffective profiles to smokers. We also monitored eye movements to assess attentional bias toward cues and we further performed exploratory analyses on demographics, personality, and drug use between profiles. Participants with CUD (n = 43) viewed pleasant, unpleasant, cocaine, and neutral images while we recorded electroencephalogram. For each picture category, we computed the amplitude of the late positive potential (LPP), an event-related potential component that reflects motivational relevance. k-means clustering classified participants based on their LPP responses. In line with what has been observed in smokers, clustering participants using LPP responses revealed the presence of two groups: one with larger LPPs to pleasant images compared to cocaine images (P > C) and one group with larger LPPs to cocaine images compared to pleasant images (C > P). Individuals with the C > P reactivity profile also had higher attentional bias toward drug cues. The two groups did not differ on demographic and drug use characteristics, however individuals with the C > P profile reported lower distress tolerance, higher anhedonia, and higher posttraumatic stress symptoms compared to the P > C group. This is the first study to report the presence of these neuroaffective profiles in individuals with CUD, indicating that this pattern may cut across addiction populations. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Multistability, perceptual value, and internal foraging

Substantial experimental, theoretical, and computational insights into sensory processing have been derived from the phenomena of perceptual multistability-when two or more percepts alternate or switch in response to a single sensory input. Here, we review a range of findings suggesting that alternations can be seen as internal choices by the brain responding to values. We discuss how elements of external, experimenter-controlled values and internal, uncertainty- and aesthetics-dependent values influence multistability. We then consider the implications for the involvement in switching of regions, such as the anterior cingulate cortex, which are more conventionally tied to value-dependent operations such as cognitive control and foraging.

From skinner box to daily life: Sign-tracker phenotype co-segregates with impulsivity, compulsivity, and addiction tendencies in humans

Pavlovian conditioning holds the potential to incentivize environmental cues, leading to approach behavior toward them, even outside our awareness. Animal models suggest that this is particularly true for the so-called sign-tracker (ST) phenotype, which is considered to reflect a predisposition toward developing addiction-related behaviours. Despite its potential clinical relevance, few studies have demonstrated the translational validity of this model, likely due to difficulties in studying Pavlovian processes in humans. To fill this gap, we combined an ecological momentary assessment with ambulatory peripheral autonomic monitoring to test the hypothesis that traits associated with ST in preclinical studies would be associated with attribution of high incentive salience to reward-related cues. Several times for 2 days, participants were asked to rate the attractiveness of several preselected ecological rewards (e.g., coffee) and the preceding cues (the smell of coffee) while their electrocardiogram was recorded. While no absolute difference in subjective and physiological measures of motivational approach to daily cues compared with rewards emerged, individuals with high levels of impulsivity, obsessive-compulsive, and addiction-prone behaviors rated as more attractive and showed a greater increase in sympathetic arousal to cues versus rewards. The opposite pattern emerged for those with low levels in those dispositional traits, who responded more (both subjectively and physiologically) to rewards compared with their preceding cues. This study represents an attempt to answer the call to parcel complex behaviors into smaller constructs, improving the early detection of those who are vulnerable to develop psychopathological disorders, particularly in the domain of impulse control such as addiction.

Brain signals of a Surprise-Actor-Critic model: Evidence for multiple learning modules in human decision making

Learning how to reach a reward over long series of actions is a remarkable capability of humans, and potentially guided by multiple parallel learning modules. Current brain imaging of learning modules is limited by (i) simple experimental paradigms, (ii) entanglement of brain signals of different learning modules, and (iii) a limited number of computational models considered as candidates for explaining behavior. Here, we address these three limitations and (i) introduce a complex sequential decision making task with surprising events that allows us to (ii) dissociate correlates of reward prediction errors from those of surprise in functional magnetic resonance imaging (fMRI); and (iii) we test behavior against a large repertoire of model-free, model-based, and hybrid reinforcement learning algorithms, including a novel surprise-modulated actor-critic algorithm. Surprise, derived from an approximate Bayesian approach for learning the world-model, is extracted in our algorithm from a state prediction error. Surprise is then used to modulate the learning rate of a model-free actor, which itself learns via the reward prediction error from model-free value estimation by the critic. We find that action choices are well explained by pure model-free policy gradient, but reaction times and neural data are not. We identify signatures of both model-free and surprise-based learning signals in blood oxygen level dependent (BOLD) responses, supporting the existence of multiple parallel learning modules in the brain. Our results extend previous fMRI findings to a multi-step setting and emphasize the role of policy gradient and surprise signalling in human learning.

Pupil dilation as an index of Pavlovian conditioning. A systematic review and meta-analysis

The use of pupillometry to track emotional learning processes in humans is generating an increasing interest. Here, we provide a first systematic review and meta-analysis on the value of pupil dilation as a marker of Pavlovian conditioning, focusing on the roles of UCS valence (aversive vs. appetitive), the time course across trials and response intervals within trials. Based on data from 39 independent samples (total n = 1303), our results revealed strong evidence for the overall validity of conditioned pupil responses, with a trend for larger effects in aversive (average g = 0.73) vs. appetitive conditioning (g = 0.39). Response differentiation increased over the course of acquisition. Substantial differentiation effects were found in both early and late response windows. Moderator analyses revealed a consistent influence of UCS modality on differential conditioning, while evidence for moderation by contingency instructions and length of acquisition phase was mixed. The results highlight pupil dilation as a sensitive and reliable index of Pavlovian conditioning across valence categories and stimulus modalities. Important implications regarding methodological considerations and research goals are discussed.

Sign tracking predicts suboptimal behavior in a rodent gambling task

RATIONALE

Reward-associated cues can promote maladaptive behavior, including risky decision-making in a gambling setting. A propensity for sign tracking over goal tracking-i.e., interaction with a reward-predictive cue rather than the site of reward-demonstrates an individual's tendency to transfer motivational value to a cue. However, the relationship of sign tracking to risky decision-making remains unclear.

OBJECTIVES

To determine whether sign tracking predicts risky choice, we used a Pavlovian conditioned approach task to evaluate the tendency of male rats to sign track to a lever cue and then trained rats on a rodent gambling task (rGT) with win-associated cues. We also tested the effects of D-amphetamine, quinpirole (a D₂/D₃ receptor agonist), and PD128907 (a D₃ receptor agonist) on gambling behavior in sign tracker and goal tracker individuals.

RESULTS

Increased sign tracking relative to goal tracking was associated with suboptimal performance on the rGT, including decreased selection of the optimal choice, increased selection of a high-risk/high-reward option, and increased impulsive premature choices. Amphetamine increased choices of a low-risk/low-reward option at the expense of optimal and high-risk choices, whereas quinpirole and PD128907 had little effect on choice allocation, but reduced impulsivity. Drug effects were similar across sign tracker and goal tracker individuals.

CONCLUSIONS

Cue reactivity, as measured by sign tracking, is predictive and may be an important driver of risky and impulsive choices in a gambling setting laden with salient audiovisual cues. Evaluating an individual's sign tracking behavior may be an avenue to predict vulnerability to pathological gambling and the efficacy of treatments.

Sign- and goal-tracking score does not correlate with addiction-like behavior following prolonged cocaine self-administration

RATIONALE

In classical conditioning, sign-tracking reflects behavior directed toward a conditioned stimulus (CS) in expectation of a reward (unconditioned stimulus, US); in contrast, goal-tracking describes behavior directed toward the location of delivery of a US. As cues previously paired with drugs of abuse promote drug-seeking and drug-taking behavior in both animals and humans and thus contribute to the severity of substance abuse, sign-tracking may represent a maladaptive cue-focused behavior that may increase addiction vulnerability as compared to goal-tracking. Recent studies do, in fact, support this possibility. Previous work in this area has focused primarily on paradigms using relatively limited exposure to drug rather than extended drug intake.

OBJECTIVES

Here, we used the DSM-IV-based 3-criteria (3-CRIT) model and examined whether a relationship exists between sign- or goal-tracking phenotypes and the prevalence of criteria associated with addiction-like behavior following extended cocaine self-administration as measured in this model.

METHODS

Forty-six male Sprague Dawley rats underwent a Pavlovian conditioned approach (PCA) procedure and were characterized along a continuum as goal-trackers (GTs), intermediates (INTs), or sign-trackers (STs). The animals were subsequently trained to intravenous self-administer cocaine during 45 self-administration (SA) sessions and characterized for the 3 criteria outlined in the model: persistence of drug-seeking, motivation for cocaine-taking, and resistance to punishment.

RESULTS

We performed correlational analyses on the traits measured, finding no relationships between PCA score and addiction-like characteristics measured using the 3-CRIT model of addiction. However, STs showed significantly greater resistance to punishment than GTs.

CONCLUSIONS

Phenotyping along a continuum of PCA scores may not be a valid predictor for identifying vulnerability to the addiction-like behaviors examined using the 3-CRIT model. However, PCA phenotype may predict a single feature of the 3-CRIT model, resistance to punishment, among those rats classified as either STs or GTs.

Fronto-striatal structures related with model-based control as an endophenotype for obsessive-compulsive disorder

Recent theories suggest a shift from model-based goal-directed to model-free habitual decision-making in obsessive–compulsive disorder (OCD). However, it is yet unclear, whether this shift in the decision process is heritable. We investigated 32 patients with OCD, 27 unaffected siblings (SIBs) and 31 healthy controls (HCs) using the two-step task. We computed behavioral and reaction time analyses and fitted a computational model to assess the balance between model-based and model-free control. 80 subjects also underwent structural imaging. We observed a significant ordered effect for the shift towards model-free control in the direction OCD > SIB > HC in our computational parameter of interest. However less directed analyses revealed no shift towards model-free control in OCDs. Nonetheless, we found evidence for reduced model-based control in OCDs compared to HCs and SIBs via 2nd stage reaction time analyses. In this measure SIBs also showed higher levels of model-based control than HCs. Across all subjects these effects were associated with the surface area of the left medial/right dorsolateral prefrontal cortex. Moreover, correlations between bilateral putamen/right caudate volumes and these effects varied as a function of group: they were negative in SIBs and OCDs, but positive in HCs. Associations between fronto-striatal regions and model-based reaction time effects point to a potential endophenotype for OCD.

Association of the OPRM1 A118G polymorphism and Pavlovian-to-instrumental transfer: Clinical relevance for alcohol dependence

BACKGROUND

Pavlovian-to-instrumental transfer (PIT) quantifies the extent to which a stimulus that has been associated with reward or punishment alters operant behaviour. In alcohol dependence (AD), the PIT effect serves as a paradigmatic model of cue-induced relapse. Preclinical studies have suggested a critical role of the opioid system in modulating Pavlovian-instrumental interactions. The A118G polymorphism of the OPRM1 gene affects opioid receptor availability and function. Furthermore, this polymorphism interacts with cue-induced approach behaviour and is a potential biomarker for pharmacological treatment response in AD. In this study, we tested whether the OPRM1 polymorphism is associated with the PIT effect and relapse in AD.

METHODS

Using a PIT task, we examined three independent samples: young healthy subjects (N = 161), detoxified alcohol-dependent patients (N = 186) and age-matched healthy controls (N = 105). We used data from a larger study designed to assess the role of learning mechanisms in the development and maintenance of AD. Subjects were genotyped for the A118G (rs1799971) polymorphism of the OPRM1 gene. Relapse was assessed after three months.

RESULTS

In all three samples, participants with the minor OPRM1 G-Allele (G+ carriers) showed increased expression of the PIT effect in the absence of learning differences. Relapse was not associated with the OPRM1 polymorphism. Instead, G+ carriers displaying increased PIT effects were particularly prone to relapse.

CONCLUSION

These results support a role for the opioid system in incentive salience motivation. Furthermore, they inform a mechanistic model of aberrant salience processing and are in line with the pharmacological potential of opioid receptor targets in the treatment of AD.

Development of a Classical Conditioning Task for Humans Examining Phasic Heart Rate Responses to Signaled Appetitive Stimuli: A Pilot Study

Cardiac responses to appetitive stimuli have been studied as indices of motivational states and attentional processes, the former being associated with cardiac acceleration and latter deceleration. Very few studies have examined heart rate changes in appetitive classical conditioning in humans. The current study describes the development and pilot testing of a classical conditioning task to assess cardiac responses to appetitive stimuli and cues that reliably precede them. Data from 18 adults were examined. They were shown initially neutral visual stimuli (putative CS) on a computer screen followed by pictures of high-caloric food (US). Phasic cardiac deceleration to food images was observed, consistent with an orienting response to motivationally significant stimuli. Similar responses were observed to non-appetitive stimuli when they were preceded by the cue associated with the food images, suggesting that attentional processes were engaged by conditioned stimuli. These autonomic changes provide significant information about classical conditioning effects in humans.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

Behavioral determinants in the expression of the Kamin blocking effect: Implications for associative learning theory

Associative learning makes important contributions to our behavior and decisions. The Kamin blocking effect is an associative learning phenomenon that plays a central role in understanding of the psychological principles underlying associative learning. However, several recent failures to replicate the blocking effect suggest that the conditions necessary for blocking are poorly understood. To understand the conditions necessary for blocking, here we review studies into the expression of blocking in subjects that either approach and interact with the conditioned cue (sign trackers) or approach and interact with the reward location (goal trackers) during appetitive classical conditioning. Psychological theory and the neurophysiological correlates of appetitive classical conditioning make opposing predictions regarding the expression of blocking in sign and goal trackers. We reconcile these opposing predictions in a qualitative model using two parallel learning processes. Such models offer a better framework for understanding the psychological associative structures acquired during learning, their interactions contributing to the conditioned response, and how they affect subsequent learning and the expression of the Kamin blocking effect.

Reward-Related Attentional Capture Moderates the Association between Fear-Driven Motives and Heavy Drinking

BACKGROUND

To date, there has been little investigation on how motivational and cognitive mechanisms interact to influence problematic drinking behaviours. Towards this aim, the current study examined whether reward-related attentional capture is associated with reward, fear (relief), and habit drinking motives, and further, whether it interacts with these motives in relation to problematic drinking patterns.

METHODS

Ninety participants (mean age = 34.8 years, SD = 9.1, 54% male) who reported having consumed alcohol in the past month completed an online visual search task that measured reward-related attentional capture as well as the Habit Reward Fear Scale, a measure of drinking motives. Participants also completed measures of psychological distress, impulsivity, compulsive drinking, and consumption items of Alcohol Use Disorders Identification Test. Regression analyses examined the associations between motives for alcohol consumption and reward-related attentional capture, as well as the associations between reward-related attentional capture, motives, and their interaction, with alcohol consumption and problems.

RESULTS

Greater reward-related attentional capture was associated with greater reward motives. Further, reward-related attentional capture also interacted with fear motives in relation to alcohol consumption. Follow-up analyses showed that this interaction was driven by greater fear motives being associated with heavier drinking among those with lower reward-related attentional capture (i.e., "goal-trackers").

CONCLUSION

These findings have implications for understanding how cognition may interact with motives in association with problematic drinking. Specifically, the findings highlight different potential pathways to problematic drinking according to an individual's cognitive-motivational profile and may inform tailored interventions to target profile-specific mechanisms. Finally, these findings offer support for contemporary models of addiction that view excessive goal-directed behaviour under negative affect as a critical contributor to addictive behaviours.

Advances in the computational understanding of mental illness

Computational psychiatry is a rapidly growing field attempting to translate advances in computational neuroscience and machine learning into improved outcomes for patients suffering from mental illness. It encompasses both data-driven and theory-driven efforts. Here, recent advances in theory-driven work are reviewed. We argue that the brain is a computational organ. As such, an understanding of the illnesses arising from it will require a computational framework. The review divides work up into three theoretical approaches that have deep mathematical connections: dynamical systems, Bayesian inference and reinforcement learning. We discuss both general and specific challenges for the field, and suggest ways forward.

Mapping sign-tracking and goal-tracking onto human behaviors

As evidenced through classic Pavlovian learning mechanisms, environmental cues can become incentivized and influence behavior. These stimulus-outcome associations are relevant in everyday life but may be particularly important for the development of impulse control disorders including addiction. Rodent studies have elucidated specific learning profiles termed 'sign-tracking' and 'goal-tracking' which map onto individual differences in impulsivity and other behaviors associated with impulse control disorders' etiology, course, and relapse. Whereas goal-trackers are biased toward the outcome, sign-trackers fixate on features that are associated with but not necessary for achieving an outcome; a pattern of behavior that often leads to escalation of reward-seeking that can be maladaptive. The vast majority of the sign- and goal-tracking research has been conducted using rodent models and very few have bridged this concept into the domain of human behavior. In this review, we discuss the attributes of sign- and goal-tracking profiles, how these are manifested neurobiologically, and how these distinct learning styles could be an important tool for clinical interventions in human addiction.

Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models

Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.

Understanding Addiction Using Animal Models

Drug addiction is a neuropsychiatric disorder with grave personal consequences that has an extraordinary global economic impact. Despite decades of research, the options available to treat addiction are often ineffective because our rudimentary understanding of drug-induced pathology in brain circuits and synaptic physiology inhibits the rational design of successful therapies. This understanding will arise first from animal models of addiction where experimentation at the level of circuits and molecular biology is possible. We will review the most common preclinical models of addictive behavior and discuss the advantages and disadvantages of each. This includes non-contingent models in which animals are passively exposed to rewarding substances, as well as widely used contingent models such as drug self-administration and relapse. For the latter, we elaborate on the different ways of mimicking craving and relapse, which include using acute stress, drug administration or exposure to cues and contexts previously paired with drug self-administration. We further describe paradigms where drug-taking is challenged by alternative rewards, such as appetitive foods or social interaction. In an attempt to better model the individual vulnerability to drug abuse that characterizes human addiction, the field has also established preclinical paradigms in which drug-induced behaviors are ranked by various criteria of drug use in the presence of negative consequences. Separation of more vulnerable animals according to these criteria, along with other innate predispositions including goal- or sign-tracking, sensation-seeking behavior or impulsivity, has established individual genetic susceptibilities to developing drug addiction and relapse vulnerability. We further examine current models of behavioral addictions such as gambling, a disorder included in the DSM-5, and exercise, mentioned in the DSM-5 but not included yet due to insufficient peer-reviewed evidence. Finally, after reviewing the face validity of the aforementioned models, we consider the most common standardized tests used by pharmaceutical companies to assess the addictive potential of a drug during clinical trials.