Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists

Ghrelin receptor antagonism and satiety attenuate Pavlovian-instrumental transfer

Animals rely on learned cues to guide their behaviour for rewards such as food. The Pavlovian-instrumental transfer (PIT) task can be used to investigate the influence of Pavlovian stimuli on instrumental responding. Ghrelin, an orexigenic peptide, and its receptor, growth hormone secretagogue receptor 1A (GHS-R1A), has received growing interest for its role in reward-motivated learning and behaviours. A significant population of GHS-R1A have been identified within the ventral tegmental area (VTA), a critical node in the mesolimbic reward circuit that is necessary for the expression of PIT. As ghrelin has been found to increase dopaminergic activity in the VTA, we predicted that GHS-R1A antagonism with JMV-2959 would attenuate PIT. Further, given the relationship between hunger levels and changes in ghrelin signalling, we sought to compare the effects GHS-R1A antagonism with those of satiety, hypothesizing parallel effects, with each attenuating PIT. Rats received daily sessions of Pavlovian and then instrumental training over 3 weeks. Across three experiments, we examined the effects of a shift to satiety, or treatment with the GHS-R1A antagonist JMV-2959, either peripherally or directly into the VTA. We found that presentations of a stimulus paired with food reward enhanced responding for food across all conditions, thus demonstrating the expected PIT effect. Further, GHS-R1A antagonism, both peripherally and within the VTA, as well as satiety significantly reduced the magnitude of the PIT effect compared to control conditions. These results clarify our understanding of ghrelin signalling in PIT and begin to elucidate the role of feeding-related peptides in the modulation of reward-related responding.

Separating desire from prediction of outcome value

Individuals typically want what they expect to like, often based on memories of previous positive experiences. However, in some situations desire can decouple completely from memories and from learned predictions of outcome value. The potential for desire to separate from prediction arises from independent operating rules that control motivational incentive salience. Incentive salience, or 'wanting', is a type of mesolimbic desire that evolved for adaptive goals, but can also generate maladaptive addictions. Two proof-of-principle examples are presented here to show how motivational 'wanting' can soar above memory-based predictions of outcome value: (i) 'wanting what is remembered to be disgusting', and (ii) 'wanting what is predicted to hurt'. Consequently, even outcomes remembered and predicted to be negatively aversive can become positively 'wanted'. Similarly, in human addictions, people may experience powerful cue-triggered cravings for outcomes that are not predicted to be enjoyable.

The Dopamine System in Mediating Alcohol Effects in Humans

Brain-imaging studies show that the development and maintenance of alcohol use disorder (AUD) is determined by a complex interaction of different neurotransmitter systems and multiple psychological factors. In this context, the dopaminergic reinforcement system appears to be of fundamental importance. We focus on the excitatory and depressant effects of acute versus chronic alcohol intake and its impact on dopaminergic neurotransmission. Furthermore, we describe alterations in dopaminergic neurotransmission as associated with symptoms of alcohol dependence. We specifically focus on neuroadaptations to chronic alcohol consumption and their effect on central processing of alcohol-associated and reward-related stimuli. Altered reward processing, complex conditioning processes, impaired reinforcement learning, and increased salience attribution to alcohol-associated stimuli enable alcohol cues to drive alcohol seeking and consumption. Finally, we will discuss how the neurobiological and neurochemical mechanisms of alcohol-associated alterations in reward processing and learning can interact with stress, cognition, and emotion processing.

Unraveling the mysteries of dendritic spine dynamics: Five key principles shaping memory and cognition

Recent research extends our understanding of brain processes beyond just action potentials and chemical transmissions within neural circuits, emphasizing the mechanical forces generated by excitatory synapses on dendritic spines to modulate presynaptic function. From in vivo and in vitro studies, we outline five central principles of synaptic mechanics in brain function: P1: Stability - Underpinning the integral relationship between the structure and function of the spine synapses. P2: Extrinsic dynamics - Highlighting synapse-selective structural plasticity which plays a crucial role in Hebbian associative learning, distinct from pathway-selective long-term potentiation (LTP) and depression (LTD). P3: Neuromodulation - Analyzing the role of G-protein-coupled receptors, particularly dopamine receptors, in time-sensitive modulation of associative learning frameworks such as Pavlovian classical conditioning and Thorndike's reinforcement learning (RL). P4: Instability - Addressing the intrinsic dynamics crucial to memory management during continual learning, spotlighting their role in "spine dysgenesis" associated with mental disorders. P5: Mechanics - Exploring how synaptic mechanics influence both sides of synapses to establish structural traces of short- and long-term memory, thereby aiding the integration of mental functions. We also delve into the historical background and foresee impending challenges.

The Role of Impulsivity and Reward Deficiency in "Liking" and "Wanting" of Potentially Problematic Behaviors and Substance Uses

A few studies have examined the changes in substance- and behavior-related "wanting" and "liking" of human subjects, the key properties of Incentive Sensitization Theory (IST). The aim of this study was to examine the dissociation between "wanting" and "liking" as a function of usage frequency, intensity, and subjective severity in individuals across four substances (alcohol, nicotine, cannabis, and other drugs) and ten behaviors (gambling, overeating, gaming, pornography use, sex, social media use, Internet use, TV-series watching, shopping, and work). Also, the potential roles of impulsivity and reward deficiency were investigated in "wanting," "liking," and wellbeing. The sex differences between "wanting" and "liking" were also examined. Based on our findings using structural equation modeling with 749 participants (503 women, M age = 35.7 years, SD = 11.84), who completed self-report questionnaires, "wanting" increased with the severity, frequency, and intensity of potentially problematic use, while "liking" did not change. Impulsivity positively predicted "wanting," and "wanting" positively predicted problem uses/behaviors. Reward deficiency positively predicted problem uses/behaviors, and both impulsivity and problem uses/behaviors negatively predicted wellbeing. Finally, women showed higher levels of "wanting," compared to men. These findings demonstrate the potential roles of incentive sensitization in both potentially problematic substance uses and behaviors.

Anticipation: An Essential Feature of Anhedonia

The following essay addresses the evolution of the term "anhedonia" as a key construct in biological psychiatry, especially as it pertains to positive emotional and motivational states central to mental health and well-being. In its strictest definition, anhedonia was intended to convey an inability to experience "pleasure" derived from ingestion of sweet tastes or the experience of pleasant odors and tactile sensations, among a host of positive sensations. However, this definition has proved to be too restrictive to capture the complexity of key psychological factors linked to major depression, schizophrenia, and substance use disorders it was originally intended to address. Despite the appeal of the elegant simplicity of the term anhedonia, its limitations soon became apparent when used to explain psychological constructs including aspects of learning, memory, and incentive motivation that are major determinants of success in securing the necessities of life. Accordingly, the definition of anhedonia has morphed into a much broader term that includes key roles in the disturbance of motivation in the form of anergia, impaired incentive motivation, along with deficits in associative learning and key aspects of memory, on which the ability to predict the consequences of one's actions are based. Here we argue that it is this latter capacity, namely predicting the likely consequences of motivated behavior, which can be termed "anticipation," that is especially important in the key deficits implied by the general term anhedonia in the context of neuropsychiatric conditions.

Quantifying the instrumental and noninstrumental underpinnings of Pavlovian responding with the Price equation

The Price equation is a mathematical expression of selectionist and non-selectionist pressures on biological, cultural, and behavioral change. We use it here to specify instrumental and noninstrumental behaviors as they arise within the context of the Pavlovian autoshaping procedure, for rats trained under reward certainty and reward uncertainty. The point of departure for this endeavor is that some portion of autoshaped behavior referred to as goal-tracking appears instrumental-a function of resource attainment (the individual approaches the location where the unconditioned stimulus is to be delivered). By contrast, some other portion of autoshaped behavior referred to as sign-tracking is noninstrumental-irrelevant to making contact with the to-be-delivered unconditioned stimulus. A Price equation model is proposed that unifies our understanding of Pavlovian autoshaping behavior by isolating operant and respondent influences on goal-tracking (instrumental) and sign-tracking (noninstrumental) behavior.

Rewarding Effects of Nicotine Self-administration Increase Over Time in Male and Female Rats

INTRODUCTION

Smoking and the use of other nicotine-containing products is rewarding in humans. The self-administration of nicotine is also rewarding in male rats. However, it is unknown if there are sex differences in the reward-enhancing effects of nicotine self-administration and if the rewarding effects of nicotine change over time.

METHODS

Rats were prepared with catheters and intracranial self-stimulation (ICSS) electrodes to investigate the effects of nicotine and saline self-administration on reward function. A decrease in thresholds in the ICSS procedure reflects an enhancement of reward function. The ICSS parameters were determined before and after the self-administration sessions from days 1 to 10, and after the self-administration sessions from days 11 to 15.

RESULTS

During the first 10 days, there was no sex difference in nicotine intake, but during the last 5 days, the females took more nicotine than the males. During the first 10 days, nicotine self-administration did not lower the brain reward thresholds but decreased the response latencies. During the last 5 days, nicotine lowered the reward thresholds and decreased the response latencies. An analysis with the 5-day averages (days 1-5, 6-10, and 11-15) showed that the reward enhancing and stimulatory effects of nicotine increased over time. There were no sex differences in the reward-enhancing and stimulatory effects of nicotine. The nicotinic receptor antagonist mecamylamine diminished the reward-enhancing and stimulatory effects of nicotine.

CONCLUSION

These findings indicate that the rewarding effects of nicotine self-administration increase over time, and there are no sex differences in the reward-enhancing effects of nicotine self-administration in rats.

IMPLICATIONS

This study investigated the rewarding effect of nicotine and saline self-administration in male and female rats. The self-administration of nicotine, but not saline, enhanced brain reward function and had stimulatory effects. The rewarding effects of nicotine increased over time in the males and the females. Despite that the females had a higher level of nicotine intake than the males, the reward-enhancing effects of nicotine self-administration were the same. These findings suggest that in new tobacco and e-cigarette users, nicotine's rewarding effects might increase quickly, and a higher level of nicotine use in females might not translate into greater rewarding effects.

The effects of the dopamine D2/3 agonist quinpirole on incentive value and palatability-based choice in a rodent model of attention-deficit/hyperactivity disorder

RATIONALE

Palatability and incentive value influence animal food choice. Dopamine D2/3 receptor signaling may mediate the effects of palatability and incentive value on choice. Dopamine signaling is disrupted in attention-deficit hyperactivity disorder (ADHD). Investigating behavioral choice processes under D2/3 receptor agonists will help elucidate behavioral and pharmacological correlates of ADHD.

OBJECTIVES

To determine (1) how changes in incentive value affects choice of actions for outcomes that differ in palatability; (2) the effects of the D2/3 agonist quinpirole on choice based on palatability and incentive value; (3) how choice differs in spontaneously hypertensive rats (SHR; ADHD model) compared with control strains.

METHODS

Rats responded instrumentally for two food outcomes (chocolate and grain pellets) that differed in palatability. Following specific satiety of one outcome, rats underwent a choice test. Prior to the choice test, rats were given intra-peritoneal quinpirole (0.01-0.1 mg/kg) body weight. These manipulations were conducted in three strains of rats: SHR rats; the normotensive Wistar-Kyoto (WKY) controls; and Wistar outbred (WIS) controls.

RESULTS

All rat strains responded more vigorously for chocolate pellets compared with grain pellets. Quinpirole reduced the effects of palatability and dose-dependently increased the effects of incentive value on choice. SHR rats were the least influenced by incentive value, whereas WKY rats were the least influenced by palatability.

CONCLUSIONS

These results show that D2/3 signaling modulates choice based on palatability and incentive value. Disruption of this process in SHR rats may mirror motivational impairments observed in ADHD.

Dopamine D1 and D2 Receptors Are Important for Learning About Neutral-Valence Relationships in Sensory Preconditioning

Dopamine neurotransmission has been ascribed multiple functions with respect to both motivational and associative processes in reward-based learning, though these have proven difficult to tease apart. In order to better describe the role of dopamine in associative learning, this series of experiments examined the potential of dopamine D1- and D2-receptor antagonism (or combined antagonism) to influence the ability of rats to learn neutral valence stimulus-stimulus associations. Using a sensory preconditioning task, rats were first exposed to pairings of two neutral stimuli (S2-S1). Subsequently, S1 was paired with a mild foot-shock and resulting fear to both S1 (directly conditioned) and S2 (preconditioned) was examined. Initial experiments demonstrated the validity of the procedure in that measures of sensory preconditioning were shown to be contingent on pairings of the two sensory stimuli. Subsequent experiments indicated that systemic administration of dopamine D1- or D2-receptor antagonists attenuated learning when administered prior to S2-S1 pairings. However, the administration of a more generic D1R/D2R antagonist was without effect. These effects remained constant regardless of the affective valence of the conditioning environment and did not differ between male and female rats. The results are discussed in the context of recent suggestions that dopaminergic systems encode more than a simple reward prediction error, and provide potential avenues for future investigation.

Association of the OPRM1 A118G polymorphism and Pavlovian-to-instrumental transfer: Clinical relevance for alcohol dependence

BACKGROUND

Pavlovian-to-instrumental transfer (PIT) quantifies the extent to which a stimulus that has been associated with reward or punishment alters operant behaviour. In alcohol dependence (AD), the PIT effect serves as a paradigmatic model of cue-induced relapse. Preclinical studies have suggested a critical role of the opioid system in modulating Pavlovian-instrumental interactions. The A118G polymorphism of the OPRM1 gene affects opioid receptor availability and function. Furthermore, this polymorphism interacts with cue-induced approach behaviour and is a potential biomarker for pharmacological treatment response in AD. In this study, we tested whether the OPRM1 polymorphism is associated with the PIT effect and relapse in AD.

METHODS

Using a PIT task, we examined three independent samples: young healthy subjects (N = 161), detoxified alcohol-dependent patients (N = 186) and age-matched healthy controls (N = 105). We used data from a larger study designed to assess the role of learning mechanisms in the development and maintenance of AD. Subjects were genotyped for the A118G (rs1799971) polymorphism of the OPRM1 gene. Relapse was assessed after three months.

RESULTS

In all three samples, participants with the minor OPRM1 G-Allele (G+ carriers) showed increased expression of the PIT effect in the absence of learning differences. Relapse was not associated with the OPRM1 polymorphism. Instead, G+ carriers displaying increased PIT effects were particularly prone to relapse.

CONCLUSION

These results support a role for the opioid system in incentive salience motivation. Furthermore, they inform a mechanistic model of aberrant salience processing and are in line with the pharmacological potential of opioid receptor targets in the treatment of AD.

Adaptive Changes in the Central Control of Energy Homeostasis Occur in Response to Variations in Energy Status

Energy homeostasis is regulated in coordinate fashion by the brain-gut axis, the homeostatic energy balance circuitry in the hypothalamus and the hedonic energy balance circuitry comprising the mesolimbcortical A10 dopamine pathway. Collectively, these systems convey and integrate information regarding nutrient status and the rewarding properties of ingested food, and formulate it into a behavioral response that attempts to balance fluctuations in consumption and food-seeking behavior. In this review we start with a functional overview of the homeostatic and hedonic energy balance circuitries; identifying the salient neural, hormonal and humoral components involved. We then delve into how the function of these circuits differs in males and females. Finally, we turn our attention to the ever-emerging roles of nociceptin/orphanin FQ (N/OFQ) and pituitary adenylate cyclase-activating polypeptide (PACAP)-two neuropeptides that have garnered increased recognition for their regulatory impact in energy homeostasis-to further probe how the imposed regulation of energy balance circuitry by these peptides is affected by sex and altered under positive (e.g., obesity) and negative (e.g., fasting) energy balance states. It is hoped that this work will impart a newfound appreciation for the intricate regulatory processes that govern energy homeostasis, as well as how recent insights into the N/OFQ and PACAP systems can be leveraged in the treatment of conditions ranging from obesity to anorexia.

Sensory cue reactivity: Sensitization in alcohol use disorder and obesity

Neuroimaging techniques to measure the function of the human brain such as electroencephalography (EEG), positron emission tomography (PET), and functional magnetic resonance imaging (fMRI), are powerful tools for understanding the underlying neural circuitry associated with alcohol use disorder (AUD) and obesity. The sensory (visual, taste and smell) paradigms used in neuroimaging studies represent an ideal platform to investigate the connection between the different neural circuits subserving the reward/executive control systems in these disorders, which may offer a translational mechanism for novel intervention predictions. Thus, the current review provides an integrated summary of the recent neuroimaging studies that have applied cue-reactivity paradigms and neuromodulation strategies to explore underlying alterations in neural circuitry as well in treatment strategies in AUD and obesity. Finally, we discuss literature on mechanisms associated with increased alcohol sensitivity post-bariatric surgery (BS) which offers guidance for future research to use sensory percepts in elucidating the relation of reward signaling in AUD development post-BS.

Modulation of cue-guided choices by transcranial direct current stimulation

Environmental cues may anticipate the availability of rewards, thus acting as a guide towards a specific choice (i.e., cue-guided choices). Despite the lateral prefrontal cortex having a critical role in using past learning and flexibly selecting relevant information to guide behavior, the literature on the neural basis of human cue-guided choice mainly focused on the subcortical brain structures implicated, while the specific role of cortical areas remained unclear. The present study aimed to provide causal evidence for the involvement of the lateral prefrontal cortex in two forms of human cue-guided choice, namely outcome-specific and general. To do this, 2 mA cathodal, anodal or sham transcranial direct current stimulation was applied over the lateral prefrontal cortex (with the posterior parietal cortex serving as control region) in three separate groups performing a Pavlovian-to-Instrumental Transfer task. Results showed, for the first time, a dissociation in the cortical structures involved in human cue-guided choice. Cathodal stimulation of the lateral prefrontal cortex reduced the outcome-specific transfer. In striking contrast, there was no influence on the general transfer. These results argue in favor of the presence of at least two possible neural pathways underlying cue-guided choices.

Motivational factors underlying aversive Pavlovian-instrumental transfer

While interest in active avoidance has recently been resurgent, many concerns relating to the nature of this form of learning remain unresolved. By separating stimulus and response acquisition, aversive Pavlovian-instrumental transfer can be used to measure the effect of avoidance learning on threat processing with more control than typical avoidance procedures. However, the motivational substrates that contribute to the aversive transfer effect have not been thoroughly examined. In three studies using rodents, the impact of a variety of aversive signals on shock-avoidance responding (i.e., two-way shuttling) was evaluated. Fox urine, as well as a tone paired with the delivery of the predator odor were insufficient modulatory stimuli for the avoidance response. Similarly, a signal for the absence of food did not generate appropriate aversive motivation to enhance shuttling. Only conditioned Pavlovian stimuli that had been paired with unconditioned threats were capable of augmenting shock-avoidance responding. This was true whether the signaled outcome was the same (e.g., shock) or different (e.g., klaxon) from the avoidance outcome (i.e., shock). These findings help to characterize the aversive transfer effect and provide a more thorough analysis of its generalization to warning signals for different kinds of threats. This feature of aversive motivation has not been demonstrated using conventional avoidance procedures and could be potentially useful for applying avoidance in treatment settings.

Reward-predictive cues elicit excessive reward seeking in adolescent rats

Impulsive behavior during adolescence may stem from developmental imbalances between motivational and cognitive-control systems, producing greater urges to pursue reward and weakened capacities to inhibit such actions. Here, we developed a Pavlovian-instrumental transfer (PIT) protocol to assay rats' ability to suppress cue-motivated reward seeking based on changes in reward expectancy. Traditionally, PIT studies focus on how reward-predictive cues motivate instrumental reward-seeking behavior (lever pressing). However, cues signaling imminent reward delivery also elicit countervailing focal-search responses (food-port entry). We first examined how reward expectancy (cue-reward probability) influences expression of these competing behaviors. Adult male rats increased rates of lever pressing when presented with cues signaling lower probabilities of reward but focused their activity at the food cup on trials with cues that signaled higher probabilities of reward. We then compared adolescent and adult male rats in their responsivity to cues signaling different reward probabilities. In contrast to adults, adolescent rats did not flexibly adjust patterns of responding based on the expected likelihood of reward delivery but increased their rate of lever pressing for both weak and strong cues. These findings indicate that control over cue-motivated behavior is fundamentally dysregulated during adolescence, providing a model for studying neurobiological mechanisms of adolescent impulsivity.

Subliminal determinants of cue-guided choice

By anticipating potential rewards, external cues can guide behavior to achieve a goal. Whether the conscious elaboration of these cues is necessary to elicit cue-guided choices is still unknown. The goal of the present study is to test whether the subliminal presentation of a visual cue previously paired with a reward is sufficient to bias responses that can lead to the same or a similar reward. To this aim, three experiments compared the subliminal and supraliminal presentation of reward-associated cues during a Pavlovian-to-Instrumental Transfer task. In line with previous evidence, results showed that the supraliminal presentation of reward-associated Pavlovian cues biased participant’s choice towards motivationally similar rewards (general transfer) as well as towards rewards sharing the precise sensory-specific properties of the cue (outcome-specific transfer). In striking contrast, subliminal cues biased choice only towards motivationally similar rewards (general transfer). Taken together, these findings suggest that cue-guided choices are modulated by the level of perceptual threshold (i.e., subliminal vs supraliminal) of reward-associated cues. Although conscious elaboration of the cue is necessary to guide choice towards a specific reward, subliminal processing is still sufficient to push towards choices sharing the motivational properties of the cue. Implications for everyday life, clinical conditions, and theoretical accounts of cue-guided choices are discussed.

Preadolescent dopamine receptor antagonism increases postadolescent reward-related operant behaviors that may depend on dopamine receptor hypersensitivity

The dopaminergic system has a long history of being associated with reward-related activities but the developmental consequences of blocking dopamine receptor function on reward-based associative learning has been less studied. To this end, male, Long Evans rats were systemically (i.p.) treated with the dopamine receptor (DAr) antagonist, flupenthixol (0.25 mg/kg), or saline, from postnatal day (P)18 - 24 (preadolescence) then trained on an operant conditioning task from P41 - P45 (postadolescent) without drug treatment. The preadolescent flupenthixol group showed elevated active lever responses and locomotor activity during the drug-free test. Another group of rats was given flupenthixol prior to each acquisition session from P41 - 45 which significantly suppressed both active lever presses and locomotor activity. Separate groups of rats were treated with flupenthixol or saline from P18 - 24 then treated with apomorphine or saline on P41 followed by assessment of c-Fos labeling in the nucleus accumbens. Early flupenthixol treatment was associated with more apomorphine-induced c-Fos labeling in the nucleus accumbens shell than the early saline-apomorphine group, indicating a sensitized response. These findings suggest that preadolescent dopamine receptor blockade may lead to a sensitized postadolescent dopaminergic response that underlies enhanced behavioral responses in the presence of rewarding stimuli.

Affect-driven impulsivity impairs human action control and selection, as measured through Pavlovian instrumental transfer and outcome devaluation

Two experiments were designed to study the role of emotional impulsiveness in action control and selection, involving healthy young women participants. In Experiment 1, the effects of both outcome devaluation and Pavlovian-to-instrumental transfer (PIT) were assessed on instrumental responding. In Experiment 2, we further explored the effect of outcome devaluation on outcome-specific PIT. The role of emotional impulsivity, specifically negative urgency (NU), was also evaluated in both experiments using a self-reported measure (UPPS-P scale, Spanish short version). Experiment 1 showed both outcome devaluation and outcome-specific PIT effects, which were positively inter-correlated and negatively correlated with scores in NU. Experiment 2 found an effect of outcome devaluation on outcome-specific PIT, which was negatively correlated with scores on NU. These results highlight the relevance of considering individual differences in affect-driven impulsivity, specifically NU, when addressing failures in action control and selection (proneness to habit). Moreover, these findings suggest that, at least with the procedure used in these experiments, outcome-specific PIT may be based on a goal-directed process that is under the participant's control.

Alterations in functional networks during cue-reactivity in Internet gaming disorder

BACKGROUND

Cue-induced brain reactivity has been suggested to be a fundamental and important mechanism explaining the development, maintenance, and relapse of addiction, including Internet gaming disorder (IGD). Altered activity in addiction-related brain regions has been found during cue-reactivity in IGD using functional magnetic resonance imaging (fMRI), but less is known regarding the alterations of coordinated whole brain activity patterns in IGD.

METHODS

To investigate the activity of temporally coherent, large-scale functional brain networks (FNs) during cue-reactivity in IGD, independent component analysis was applied to fMRI data from 29 male subjects with IGD and 23 matched healthy controls (HC) performing a cue-reactivity task involving Internet gaming stimuli (i.e., game cues) and general Internet surfing-related stimuli (i.e., control cues).

RESULTS

Four FNs were identified that were related to the response to game cues relative to control cues and that showed altered engagement/disengagement in IGD compared with HC. These FNs included temporo-occipital and temporo-insula networks associated with sensory processing, a frontoparietal network involved in memory and executive functioning, and a dorsal-limbic network implicated in reward and motivation processing. Within IGD, game versus control engagement of the temporo-occipital and frontoparietal networks were positively correlated with IGD severity. Similarly, disengagement of temporo-insula network was negatively correlated with higher game-craving.

DISCUSSION

These findings are consistent with altered cue-reactivity brain regions reported in substance-related addictions, providing evidence that IGD may represent a type of addiction. The identification of the networks might shed light on the mechanisms of the cue-induced craving and addictive Internet gaming behaviors.

Sign- and goal-tracking rats show differences in various executive functions: Authors

Sign tracking (ST) is a complex Pavlovian trait that is known to impact instrumental behaviour. Recent work suggests that this trait also correlates with altered top-down executive control relative to goal tracking (GT) rats. This raises the question as to the extent to which both phenotypes differ in executive functions. Moreover, it is unclear which cognitive processes might cause potential differences between ST and GT rats. We therefore compared the behaviour of ST and GT rats in several assays, such as outcome devaluation, attentional set shifting and reversal learning, conditional responding, as well as delayed alternation to measure different aspects of executive functioning. Goal-directed behaviour per se was not different between ST and GT rats in the outcome devaluation task. ST rats performed slightly better than GT rats in one condition of the set shifting task (place->cue shift) and the delayed alternation task, but did not perform as well in the conditional responding task. Thus, differential behavioural performance between ST and GT rats was dependent on the specific task context. Further, we found evidence that the differences in executive functions are likely related to increased incentive salience attribution and impulsive action in ST rats.

Mesolimbic dopamine projections mediate cue-motivated reward seeking but not reward retrieval in rats

Efficient foraging requires an ability to coordinate discrete reward-seeking and reward-retrieval behaviors. We used pathway-specific chemogenetic inhibition to investigate how rats’ mesolimbic and mesocortical dopamine circuits contribute to the expression and modulation of reward seeking and retrieval. Inhibiting ventral tegmental area dopamine neurons disrupted the tendency for reward-paired cues to motivate reward seeking, but spared their ability to increase attempts to retrieve reward. Similar effects were produced by inhibiting dopamine inputs to nucleus accumbens, but not medial prefrontal cortex. Inhibiting dopamine neurons spared the suppressive effect of reward devaluation on reward seeking, an assay of goal-directed behavior. Attempts to retrieve reward persisted after devaluation, indicating they were habitually performed as part of a fixed action sequence. Our findings show that complete bouts of reward seeking and retrieval are behaviorally and neurally dissociable from bouts of reward seeking without retrieval. This dichotomy may prove useful for uncovering mechanisms of maladaptive behavior.

Dopamine Signaling Is Critical for Supporting Cue-Driven Behavioral Control

Mesolimbic dopamine has been implicated in reward learning. Fischbach-Weiss and Janak (this issue) use optogenetics to attenuate dopamine signaling and study its role in cue-driven motivated behavior.

Individual differences in working memory capacity and cue-guided behavior in humans

Information gathered via Pavlovian and Instrumental learning can be integrated to guide behavior, in a phenomenon experimentally known as Pavlovian-to-Instrumental Transfer (PIT). In particular, in appetitive PIT, a reward-associated cue is able to enhance the instrumental response previously associated with the same (outcome-specific PIT), or a similar (general PIT), reward. The PIT effect is increasingly investigated for its numerous implications in clinical contexts as well as daily life situations. Nevertheless, the precise mechanism behind it is not yet clear. The relation between the PIT effect and high-level cognitive abilities - like working memory - is still unknown, but potentially relevant to unveil its functioning. The present study aims to examine the precise relationship between individual differences in working memory and the two forms of PIT effect, namely outcome-specific and general. For this purpose, 100 participants underwent a classical PIT paradigm. Results showed a relationship between individual working memory and outcome-specific PIT, but not general PIT. Importantly, the role of working memory was not related to the acquisition of the learning contingencies, but rather linked to an imbalance between congruent and incongruent choices. The results are discussed in terms of the adaptive and maladaptive implications for human behavior.

Dopamine blockade impairs the exploration-exploitation trade-off in rats

In a volatile environment where rewards are uncertain, successful performance requires a delicate balance between exploitation of the best option and exploration of alternative choices. It has theoretically been proposed that dopamine contributes to the control of this exploration-exploitation trade-off, specifically that the higher the level of tonic dopamine, the more exploitation is favored. We demonstrate here that there is a formal relationship between the rescaling of dopamine positive reward prediction errors and the exploration-exploitation trade-off in simple non-stationary multi-armed bandit tasks. We further show in rats performing such a task that systemically antagonizing dopamine receptors greatly increases the number of random choices without affecting learning capacities. Simulations and comparison of a set of different computational models (an extended Q-learning model, a directed exploration model, and a meta-learning model) fitted on each individual confirm that, independently of the model, decreasing dopaminergic activity does not affect learning rate but is equivalent to an increase in random exploration rate. This study shows that dopamine could adapt the exploration-exploitation trade-off in decision-making when facing changing environmental contingencies.

Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning

The brain’s dopaminergic pathways are crucially important for adaptive behavior. They are thought to enable us to approach rewards and stay away from punishments. During learning, dopaminergic reward prediction errors are thought to reinforce previously rewarded actions, so they become easier to repeat. This dopaminergic activity could lead to a systematic bias by which rewarded actions are more readily learned than rewarded inactions. We present two findings. First, dopamine receptors in cortex, dorsal striatum, and nucleus accumbens provide distinct sources of variance in the human brain. Second, the boost in an individual’s learning rate from previously rewarded actions is dependent on the dopamine receptor density in dorsal striatum, a central structure in the dopaminergic circuit.

Learning to act to obtain reward and inhibit to avoid punishment is easier compared with learning the opposite contingencies. This coupling of action and valence is often thought of as a Pavlovian bias, although recent research has shown it may also emerge through instrumental mechanisms. We measured this learning bias with a rewarded go/no-go task in 60 adults of different ages. Using computational modeling, we characterized the bias as being instrumental. To assess the role of endogenous dopamine (DA) in the expression of this bias, we quantified DA D1 receptor availability using positron emission tomography (PET) with the radioligand [¹¹C]SCH23390. Using principal-component analysis on the binding potentials in a number of cortical and striatal regions of interest, we demonstrated that cortical, dorsal striatal, and ventral striatal areas provide independent sources of variance in DA D1 receptor availability. Interindividual variation in the dorsal striatal component was related to the strength of the instrumental bias during learning. These data suggest at least three anatomical sources of variance in DA D1 receptor availability separable using PET in humans, and we provide evidence that human dorsal striatal DA D1 receptors are involved in the modulation of instrumental learning biases.

Dopaminergic Modulation of Goal-Directed Behavior in a Rodent Model of Attention-Deficit/Hyperactivity Disorder

Aside from its clinical symptoms of inattention, impulsivity and hyperactivity, patients with Attention/Deficit-Hyperactivity Disorder (ADHD) display reward and motivational impairments. These impairments may reflect a deficit in action control, that is, an inability to flexibly adapt behavior to changing consequences. We previously showed that spontaneously hypertensive rats (SHR), an inbred rodent model of ADHD, show impairments in goal-directed action control, and instead are predominated by habits. In this study, we examined the effects of specific dopamine receptor sub-type (D1 and D2) agonists and antagonists on goal-directed behavior in SHR and the normotensive inbred control strain Wistar-Kyoto (WKY) rats. Rats acquired an instrumental response for different-flavored food rewards. A selective-satiety outcome devaluation procedure followed by a choice test in extinction revealed outcome-insensitive habitual behavior in SHR rats. Outcome-sensitive goal-directed behavior was restored in SHR rats following injection prior to the choice test of the dopamine D2 receptor agonist Quinpirole or dopamine D1 receptor antagonist SCH23390, whereas WKY rats showed habitual responding following exposure to these drugs. This novel finding indicates that the core behavioral deficit in ADHD might not be a consequence of dopamine hypofunction, but rather is due to a misbalance between activation of dopamine D1 and D2 receptor pathways that govern action control.

Oxygen responses within the nucleus accumbens are associated with individual differences in effort exertion in rats

Goal‐directed motivated behaviour is crucial for everyday life. Such behaviour is often measured, in rodents, under a progressive ratio (PR) schedule of reinforcement. Previous studies have identified a few brain structures critical for supporting PR performance. However, the association between neural activity within these regions and individual differences in effort‐related behaviour is not known. Presently, we used constant potential in vivo oxygen amperometry, a surrogate for functional resonance imaging in rodents, to assess changes in tissue oxygen levels within the nucleus accumbens (NAc) and orbitofrontal cortex (OFC) in male Wistar rats performing a PR task. Within both regions, oxygen responses to rewards increased as the effort exerted to obtain the rewards was larger. Furthermore, higher individual breakpoints were associated with greater magnitude NAc oxygen responses. This association could not be explained by temporal confounds and remained significant when controlling for the different number of completed trials. Animals with higher breakpoints also showed greater magnitude NAc oxygen responses to rewards delivered independently of any behaviour. In contrast, OFC oxygen responses were not associated with individual differences in behavioural performance. The present results suggest that greater NAc oxygen responses following rewards, through a process of incentive motivation, may allow organisms to remain on task for longer and to overcome greater effort costs.

The impact of drugs of abuse on executive function: characterizing long-term changes in neural correlates following chronic drug exposure and withdrawal in rats

Addiction has long been characterized by diminished executive function, control, and impulsivity management. In particular, these deficits often manifest themselves as impairments in reversal learning, delay discounting, and response inhibition. Understanding the neurobiological substrates of these behavioral deficits is of paramount importance to our understanding of addiction. Within the cycle of addiction, periods during and after withdrawal represent a particularly difficult point of intervention in that the negative physical symptoms associated with drug removal and drug craving increase the likelihood that the patient will relapse and return to drug use in order to abate these symptoms. Moreover, it is often during this time that drug induced deficits in executive function hinder the ability of the patient to refrain from drug use. Thus, it is necessary to understand the physiological and behavioral changes associated with withdrawal and drug craving-largely manifesting as deficits in executive control-to develop more effective treatment strategies. In this review, we address the long-term impact that drugs of abuse have on the behavioral and neural correlates that give rise to executive control as measured by reversal learning, delay discounting, and stop-signal tasks, focusing particularly on our work using rats as a model system.

Repeated cocaine exposure dysregulates cognitive control over cue-evoked reward-seeking behavior during Pavlovian-to-instrumental transfer

Drug-paired cues acquire powerful motivational properties, but only lead to active drug-seeking behavior if they are potent enough to overwhelm the cognitive control processes that serve to suppress such urges. Studies using the Pavlovian-to-instrumental transfer (PIT) task have shown that rats pretreated with cocaine or amphetamine exhibit heightened levels of cue-motivated food-seeking behavior, suggesting that exposure to these drugs sensitizes the incentive motivational system. However, the PIT testing protocol can also create conflict between two competing behavioral responses to the reward-paired cue: active reward seeking (e.g., lever pressing) and passive conditioned food-cup approach behavior. We therefore investigated whether repeated cocaine exposure alters the way in which rats use cue-based reward expectations to resolve such conflict. In-depth analysis of previously published and new research confirmed that when drug-naïve rats are given a cue that signals the timing of a delayed noncontingent reward, they adaptively transition from reward seeking to conditioned approach behavior, facilitating efficient collection of the predicted reward. In contrast, cocaine-exposed rats exhibit pronounced behavioral dysregulation, increasing, rather than suppressing, their reward-seeking behavior over time, disrupting their ability to passively collect reward. Such findings speak to the important and sometimes overlooked role that cognitive control plays in determining the motivational impact of cues associated with drug and nondrug rewards.

Contributions of Pavlovian incentive motivation to cue-potentiated feeding

Cues signaling the availability of palatable food acquire the ability to potentiate food seeking and consumption. The current study employed a combination of behavioral, pharmacological, and analytical techniques to probe the role of Pavlovian incentive motivation in cue-potentiated feeding. We show that a cue paired with sucrose solution (CS+) can transfer its control over feeding to stimulate sucrose consumption at a new receptacle, and that this effect depends on activation of D1 dopamine receptors, which is known to modulate other forms of cue-motivated behavior but not taste palatability. Microstructural analyses of sucrose-licking behavior revealed that the CS+ tended to increase the frequency with which rats engaged in active bouts of licking behavior without having a reliable effect on the duration of those licking bouts, a measure that was instead associated with sucrose palatability. Furthermore, we found that individual differences in CS+ elicited increases in bout frequency were associated with total sucrose intake at test, supporting the view that this process was related to meaningful dysregulation of eating behavior. The current study, therefore, (1) demonstrates that a dopamine-dependent Pavlovian incentive motivational process can mediate cue-potentiated feeding, and (2) lays out an experimental and analytical approach for parsing this aspect of behavior.

Junk Food Exposure Disrupts Selection of Food-Seeking Actions in Rats

There is growing evidence that repeated consumption of highly palatable, nutritionally poor "junk food" diets can produce deficits in cognition and behavioral control. We explored whether long-term junk-food diet exposure disrupts rats' ability to make adaptive choices about which foods to pursue based on (1) expected reward value (outcome devaluation test) and (2) cue-evoked reward expectations (Pavlovian-to-instrumental test). Rats were initially food restricted and trained on two distinct response-outcome contingencies (e.g., left press chocolate pellets, and right press sweetened condensed milk) and stimulus-outcome contingencies (e.g., white noise chocolate pellets, and clicker sweetened condensed milk). They were then given 6 weeks of unrestricted access to regular chow alone (controls) or chow and either 1 or 24 h access to junk food per day. Subsequent tests of decision making revealed that rats in both junk-food diet groups were impaired in selecting actions based on either expected food value or the presence of food-paired cues. These data demonstrate that chronic junk food consumption can disrupt the processes underlying adaptive control over food-seeking behavior. We suggest that the resulting dysregulation of food seeking may contribute to overeating and obesity.

Learning Contextual Reward Expectations for Value Adaptation

Substantial evidence indicates that subjective value is adapted to the statistics of reward expected within a given temporal context. However, how these contextual expectations are learned is poorly understood. To examine such learning, we exploited a recent observation that participants performing a gambling task adjust their preferences as a function of context. We show that, in the absence of contextual cues providing reward information, an average reward expectation was learned from recent past experience. Learning dependent on contextual cues emerged when two contexts alternated at a fast rate, whereas both cue-independent and cue-dependent forms of learning were apparent when two contexts alternated at a slower rate. Motivated by these behavioral findings, we reanalyzed a previous fMRI data set to probe the neural substrates of learning contextual reward expectations. We observed a form of reward prediction error related to average reward such that, at option presentation, activity in ventral tegmental area/substantia nigra and ventral striatum correlated positively and negatively, respectively, with the actual and predicted value of options. Moreover, an inverse correlation between activity in ventral tegmental area/substantia nigra (but not striatum) and predicted option value was greater in participants showing enhanced choice adaptation to context. The findings help understanding the mechanisms underlying learning of contextual reward expectation.

Catecholaminergic challenge uncovers distinct Pavlovian and instrumental mechanisms of motivated (in)action

Catecholamines modulate the impact of motivational cues on action. Such motivational biases have been proposed to reflect cue-based, 'Pavlovian' effects. Here, we assess whether motivational biases may also arise from asymmetrical instrumental learning of active and passive responses following reward and punishment outcomes. We present a novel paradigm, allowing us to disentangle the impact of reward and punishment on instrumental learning from Pavlovian response biasing. Computational analyses showed that motivational biases reflect both Pavlovian and instrumental effects: reward and punishment cues promoted generalized (in)action in a Pavlovian manner, whereas outcomes enhanced instrumental (un)learning of chosen actions. These cue- and outcome-based biases were altered independently by the catecholamine enhancer melthylphenidate. Methylphenidate's effect varied across individuals with a putative proxy of baseline dopamine synthesis capacity, working memory span. Our study uncovers two distinct mechanisms by which motivation impacts behaviour, and helps refine current models of catecholaminergic modulation of motivated action.

Repeated MDMA administration increases MDMA-produced locomotor activity and facilitates the acquisition of MDMA self-administration: role of dopamine D2 receptor mechanisms

RATIONALE

Repeated exposure to ±3, 4-methylenedioxymethamphetamine (MDMA) produces sensitization to MDMA-produced hyperactivity, but the mechanisms underlying the development of this sensitized response or the relationship to the reinforcing effects of MDMA is unknown.

OBJECTIVES

This study determined the effect of a sensitizing regimen of MDMA exposure on the acquisition of MDMA self-administration and investigated the role of dopamine D₂ receptor mechanisms.

METHODS

Rats received the selective D₂ antagonist, eticlopride (0.0 or 0.3 mg/kg, i.p.) and MDMA (0.0 or 10.0 mg/kg, i.p.) during a five-day pretreatment regimen. Two days following the final session, the locomotor activating effects of MDMA (5 mg/kg, i.p.) and the latency to acquisition of MDMA self-administration were determined.

RESULTS

Pretreatment with MDMA enhanced the locomotor activating effects of MDMA and facilitated the acquisition of MDMA self-administration. Administration of eticlopride during MDMA pretreatment completely blocked the development of sensitization to MDMA-produced hyperactivity but failed to significantly alter the facilitated acquisition of MDMA self-administration. Pretreatment with eticlopride alone facilitated the acquisition of self-administration.

CONCLUSIONS

These data suggest that repeated MDMA exposure sensitized both the locomotor activating and reinforcing effects of MDMA. Activation of D₂ receptors during MDMA pretreatment appears critical for the development of sensitization to MDMA-produced hyperactivity. The role of D₂ receptor mechanisms in the development of sensitization to the reinforcing effects of MDMA is equivocal.

Modulation of cue-triggered reward seeking by cholinergic signaling in the dorsomedial striatum

The dorsomedial striatum (DMS) has been strongly implicated in flexible, outcome-based decision making, including the outcome-specific Pavlovian-to-instrumental transfer effect (PIT), which measures the tendency for a reward-predictive cue to preferentially motivate actions that have been associated with the predicted reward over actions associated with different rewards. Although the neurochemical underpinnings of this effect are not well understood, there is growing evidence that striatal acetylcholine signaling may play an important role. This study investigated this hypothesis by assessing the effects of intra-DMS infusions of the nicotinic antagonist mecamylamine or the muscarinic antagonist scopolamine on expression of specific PIT in rats. These treatments produced dissociable behavioral effects. Mecamylamine infusions enhanced rats' tendency to use specific cue-elicited outcome expectations to select whichever action was trained with the predicted outcome, relative to their performance when tested after vehicle infusions. In contrast, scopolamine infusions appeared to render instrumental performance insensitive to this motivational influence of reward-paired cues. These drug treatments had no detectable effect on conditioned food cup approach behavior, indicating that they selectively perturbed cue-guided action selection without producing more wide-ranging alterations in behavioral control. Our findings reveal an important role for DMS acetylcholine signaling in modulating the impact of cue-evoked reward expectations on instrumental action selection.

Effects of sex and remifentanil dose on rats' acquisition of responding for a remifentanil-conditioned reinforcer

Opioid-conditioned reinforcement is thought to exacerbate opioid abuse and dependence. Sex/gender can influence opioid abuse behaviors, but the effects of sex/gender on opioid-conditioned reinforcement, specifically, are unclear. In this study, we compared new-response acquisition with opioid-conditioned reinforcement in male and female rats. First, separate groups received response-independent remifentanil injections (0.0-32.0 μg/kg, intravenous) and presentations of a light-noise stimulus. In the experimental groups, injections and stimulus presentations always co-occurred [paired Pavlovian conditioning (PAV)]; in the control groups, the two occurred independently of each other (random PAV). Next, in the instrumental acquisition (ACQ) sessions, two novel nose-poke manipulanda were introduced. All animals (regardless of sex, dose, and PAV type) could respond in the active nose-poke, which produced the stimulus alone, or in the inactive nose-poke. Both males and females dose-dependently acquired nose-poke responding (active>inactive) after paired PAV, but not after random PAV. Therefore, the stimulus was a conditioned reinforcer. We identified three sex differences. First, only females acquired responding after paired PAV with 32.0 μg/kg remifentanil. Second, using a progressive ratio schedule for ACQ, both sexes acquired responding, but females made significantly more active responses. Third, when a single session of PAV was conducted, only males acquired responding. Thus, rats' sex interacts with pharmacological and environmental factors to determine opioid-conditioned reinforcement.

Dopamine D2/3- and μ-opioid receptor antagonists reduce cue-induced responding and reward impulsivity in humans

Increased responding to drug-associated stimuli (cue reactivity) and an inability to tolerate delayed gratification (reward impulsivity) have been implicated in the development and maintenance of drug addiction. Whereas data from animal studies suggest that both the dopamine and opioid system are involved in these two reward-related processes, their role in humans is less clear. Moreover, dopaminergic and opioidergic drugs have not been directly compared with regard to these functions, even though a deeper understanding of the underlying mechanisms might inform the development of specific treatments for elevated cue reactivity and reward impulsivity. In a randomized, double-blind, between-subject design we administered the selective dopamine D2/D3 receptor antagonist amisulpride (400 mg, n=41), the unspecific opioid receptor antagonist naltrexone (50 mg, n=40) or placebo (n=40) to healthy humans and measured cue-induced responding with a Pavlovian-instrumental transfer task and reward impulsivity with a delay discounting task. Mood was assessed using a visual analogue scale. Compared with placebo, amisulpride significantly suppressed cue-induced responding and reward impulsivity. The effects of naltrexone were similar, although less pronounced. Both amisulpride and naltrexone decreased average mood ratings compared with placebo. Our results demonstrate that a selective blockade of dopamine D2/D3 receptors reduces cue-induced responding and reward impulsivity in healthy humans. Antagonizing μ-opioid receptors has similar effects for cue-induced responding and to a lesser extent for reward impulsivity.

Toward isolating the role of dopamine in the acquisition of incentive salience attribution

Stimulus-reward learning has been heavily linked to the reward-prediction error learning hypothesis and dopaminergic function. However, some evidence suggests dopaminergic function may not strictly underlie reward-prediction error learning, but may be specific to incentive salience attribution. Utilizing a Pavlovian conditioned approach procedure consisting of two stimuli that were equally reward-predictive (both undergoing reward-prediction error learning) but functionally distinct in regard to incentive salience (levers that elicited sign-tracking and tones that elicited goal-tracking), we tested the differential role of D1 and D2 dopamine receptors and nucleus accumbens dopamine in the acquisition of sign- and goal-tracking behavior and their associated conditioned reinforcing value within individuals. Overall, the results revealed that both D1 and D2 inhibition disrupted performance of sign- and goal-tracking. However, D1 inhibition specifically prevented the acquisition of sign-tracking to a lever, instead promoting goal-tracking and decreasing its conditioned reinforcing value, while neither D1 nor D2 signaling was required for goal-tracking in response to a tone. Likewise, nucleus accumbens dopaminergic lesions disrupted acquisition of sign-tracking to a lever, while leaving goal-tracking in response to a tone unaffected. Collectively, these results are the first evidence of an intraindividual dissociation of dopaminergic function in incentive salience attribution from reward-prediction error learning, indicating that incentive salience, reward-prediction error, and their associated dopaminergic signaling exist within individuals and are stimulus-specific. Thus, individual differences in incentive salience attribution may be reflective of a differential balance in dopaminergic function that may bias toward the attribution of incentive salience, relative to reward-prediction error learning only.

Reappraising striatal D1- and D2-neurons in reward and aversion

The striatum has been involved in complex behaviors such as motor control, learning, decision-making, reward and aversion. The striatum is mainly composed of medium spiny neurons (MSNs), typically divided into those expressing dopamine receptor D1, forming the so-called direct pathway, and those expressing D2 receptor (indirect pathway). For decades it has been proposed that these two populations exhibit opposing control over motor output, and recently, the same dichotomy has been proposed for valenced behaviors. Whereas D1-MSNs mediate reinforcement and reward, D2-MSNs have been associated with punishment and aversion. In this review we will discuss pharmacological, genetic and optogenetic studies that indicate that there is still controversy to what concerns the role of striatal D1- and D2-MSNs in this type of behaviors, highlighting the need to reconsider the early view that they mediate solely opposing aspects of valenced behaviour.

Safety out of control: dopamine and defence

We enjoy a sophisticated understanding of how animals learn to predict appetitive outcomes and direct their behaviour accordingly. This encompasses well-defined learning algorithms and details of how these might be implemented in the brain. Dopamine has played an important part in this unfolding story, appearing to embody a learning signal for predicting rewards and stamping in useful actions, while also being a modulator of behavioural vigour. By contrast, although choosing correct actions and executing them vigorously in the face of adversity is at least as important, our understanding of learning and behaviour in aversive settings is less well developed. We examine aversive processing through the medium of the role of dopamine and targets such as D2 receptors in the striatum. We consider critical factors such as the degree of control that an animal believes it exerts over key aspects of its environment, the distinction between 'better' and 'good' actual or predicted future states, and the potential requirement for a particular form of opponent to dopamine to ensure proper calibration of state values.

Transitionality in addiction: A "temporal continuum" hypotheses involving the aberrant motivation, the hedonic dysregulation, and the aberrant learning

Addiction is a chronic compulsion and relapsing disorder. It involves several brain areas and circuits, which encode vary functions such as reward, motivation, and memory. Drug addiction is defined as a "pathological pattern of use of a substance", characterized by the loss of control on drug-taking-related behaviors, the pursuance of those behaviors even in the presence of negative consequences, and a strong motivated activity to assume substances. Three different theories guide experimental research on drug addiction. Each of these theories consider singles features, such as an aberrant motivation, a hedonic dysregulation, and an aberrant habit learning as the main actor to explain the entire process of the addictive behaviors. The major goal of this study is to present a new hypotheses of transitionality from a controlled use to abuse of addictive substances trough the overview of the three different theories, considering all the single features of each single theory together on the same "temporal continuum" from use to abuse of addictive substances. Recently, it has been suggested that common neural systems may be activated by natural and pharmacological stimuli, raising the hypotheses that binge-eating disorders could be considered as addictive behaviors. The second goal of this study is to present evidences in order to highlight a possible psycho-bio-physiological superimposition between drug and "food addiction". Finally, interesting questions are brought up starting from last findings about a theoretical/psycho-bio-physiological superimposition between drug and "food addiction" and their possibly same transitionality along the same "temporal continuum" from use to abuse of addictive substances in order to investigate new therapeutic strategies based on new therapeutic strategies based on the individual moments characterizing the transition from the voluntary intake of substances to the maladaptive addictive behavior.

The Dopaminergic Midbrain Mediates an Effect of Average Reward on Pavlovian Vigor

Dopamine plays a key role in motivation. Phasic dopamine response reflects a reinforcement prediction error (RPE), whereas tonic dopamine activity is postulated to represent an average reward that mediates motivational vigor. However, it has been hard to find evidence concerning the neural encoding of average reward that is uncorrupted by influences of RPEs. We circumvented this difficulty in a novel visual search task where we measured participants' button pressing vigor in a context where information (underlying an RPE) about future average reward was provided well before the average reward itself. Despite no instrumental consequence, participants' pressing force increased for greater current average reward, consistent with a form of Pavlovian effect on motivational vigor. We recorded participants' brain activity during task performance with fMRI. Greater average reward was associated with enhanced activity in dopaminergic midbrain to a degree that correlated with the relationship between average reward and pressing vigor. Interestingly, an opposite pattern was observed in subgenual cingulate cortex, a region implicated in negative mood and motivational inhibition. These findings highlight a crucial role for dopaminergic midbrain in representing aspects of average reward and motivational vigor.