Amphetamine-induced sensitization and reward uncertainty similarly enhance incentive salience for conditioned cues

Decomposition of Reinforcement Learning Deficits in Disordered Gambling via Drift Diffusion Modeling and Functional Magnetic Resonance Imaging

Gambling disorder is associated with deficits in reward-based learning, but the underlying computational mechanisms are still poorly understood. Here, we examined this issue using a stationary reinforcement learning task in combination with computational modeling and functional resonance imaging (fMRI) in individuals that regular participate in gambling (n = 23, seven fulfilled one to three DSM 5 criteria for gambling disorder, sixteen fulfilled four or more) and matched controls (n = 23). As predicted, the gambling group exhibited substantially reduced accuracy, whereas overall response times (RTs) were not reliably different between groups. We then used comprehensive modeling using reinforcement learning drift diffusion models (RLDDMs) in combination with hierarchical Bayesian parameter estimation to shed light on the computational underpinnings of this performance deficit. In both groups, an RLDDM in which both non-decision time and decision threshold (boundary separation) changed over the course of the experiment accounted for the data best. The model showed good parameter and model recovery, and posterior predictive checks revealed that, in both groups, the model accurately reproduced the evolution of accuracies and RTs over time. Modeling revealed that, compared to controls, the learning impairment in the gambling group was linked to a more rapid reduction in decision thresholds over time, and a reduced impact of value-differences on the drift rate. The gambling group also showed shorter non-decision times. FMRI analyses replicated effects of prediction error coding in the ventral striatum and value coding in the ventro-medial prefrontal cortex, but there was no credible evidence for group differences in these effects. Taken together, our findings show that reinforcement learning impairments in disordered gambling are linked to both maladaptive decision threshold adjustments and a reduced consideration of option values in the choice process.

Win-Paired Cues Modulate the Effect of Dopamine Neuron Sensitization on Decision Making and Cocaine Self-administration: Divergent Effects Across Sex

BACKGROUND

Both psychostimulant use and engagement with probabilistic schedules of reward sensitize the mesocorticolimbic dopamine (DA) system. Such behaviors may act synergistically to explain the high comorbidity between stimulant use and gambling disorder. The salient audiovisual stimuli of modern electronic gambling may exacerbate the situation.

METHODS

To probe these interactions, we sensitized ventral tegmental area DA neurons via chronic chemogenetic stimulation while rats (n = 134) learned a rat gambling task in the presence or absence of casino-like cues. The same rats then learned to self-administer cocaine. In a separate cohort (n = 25), we confirmed that our chemogenetic methods sensitized the locomotor response to cocaine and potentiated phasic excitability of ventral tegmental area DA neurons through in vivo electrophysiological recordings.

RESULTS

In the absence of cues, sensitization promoted risk taking in both sexes. When rewards were cued, sensitization expedited the development of a risk-preferring phenotype in males while attenuating cue-induced risk taking in females.

CONCLUSIONS

While these results provide further confirmation that ventral tegmental area DA neurons critically modulate risky decision making, they also reveal stark sex differences in the decisional impact that dopaminergic signals exert when winning outcomes are cued. As previously observed, risky decision making on the cued rat gambling task increased as both males and females learned to self-administer cocaine. The combination of DA sensitization and win-paired cues while gambling led to significantly greater cocaine taking, but these rats did not show any increase in risky choice as a result. Therefore, cocaine and heavily cued gambles may partially substitute for each other once the DA system has been rendered labile through sensitization, thereby compounding addiction risk across modalities.

Unpredictable, intermittent access to sucrose or water promotes increased reward pursuit in rats

Reward uncertainty can sensitize reward pathways, promoting increased reward-seeking and -taking behaviours. This is relevant to human conditions such as pathological gambling, eating disorders and drug addiction. In the context of addiction, preclinical self-administration procedures have been developed to model the intermittency of human drug use. These intermittent-access (IntA) procedures involve intermittent but predictable access to drug during self-administration sessions. However, human drug use typically involves intermittent and unpredictable drug access. We introduce a new procedure modeling unpredictable, intermittent access (UIntA) to a reinforcer, and we compare it to procedures that provide predictable reinforcer availability; continuous (ContA) or intermittent (IntA) access. Female rats self-administered water or liquid sucrose in daily hour-long sessions. IntA and ContA rats had access to a fixed volume of water or sucrose (0.1 ml), under a fixed ratio 3 schedule of reinforcement. IntA rats had predictable 5-min reinforcer ON and 25-min reinforcer OFF periods. ContA rats had 60 min of reinforcer access during each session. For UIntA rats, variation in the length of ON and OFF periods (1, 5 or 9 min/period), response requirement (variable ratio 3 schedule of reinforcement), reinforcer probability (50%) and quantity (0, 0.1 or 0.2 ml) introduced reward uncertainty. Following 14 daily self-administration sessions, UIntA rats showed the highest levels of responding for water or sucrose under progressive ratio conditions, responding under extinction conditions, and cue-induced reinstatement of sucrose seeking. Thus, unpredictable, intermittent reward access promotes increased reward pursuit. This has implications for modeling addiction and other disorders of increased reward seeking.

Quantifying the instrumental and noninstrumental underpinnings of Pavlovian responding with the Price equation

The Price equation is a mathematical expression of selectionist and non-selectionist pressures on biological, cultural, and behavioral change. We use it here to specify instrumental and noninstrumental behaviors as they arise within the context of the Pavlovian autoshaping procedure, for rats trained under reward certainty and reward uncertainty. The point of departure for this endeavor is that some portion of autoshaped behavior referred to as goal-tracking appears instrumental-a function of resource attainment (the individual approaches the location where the unconditioned stimulus is to be delivered). By contrast, some other portion of autoshaped behavior referred to as sign-tracking is noninstrumental-irrelevant to making contact with the to-be-delivered unconditioned stimulus. A Price equation model is proposed that unifies our understanding of Pavlovian autoshaping behavior by isolating operant and respondent influences on goal-tracking (instrumental) and sign-tracking (noninstrumental) behavior.

Effort-motivated behavior resolves paradoxes in appetitive conditioning

Motivated behavior has long been studied by psychologists, ethologists, and neuroscientists. To date, many scientists agree with the view that cue and reward attraction is the product of a dopamine-dependent unconscious process called incentive salience or "wanting". This process allows the influence of multiple factors such as hunger and odors on motivational attraction. In some cases, however, the resulting motivated behavior differs from what the incentive salience hypothesis would predict. I argue that seeking behavior under reward uncertainty illustrates this situation: Organisms do not just "want" (appetite-based attraction) cues that are inconsistent or associated with reward occasionally, they "hope" that those cues will consistently predict reward procurement in the ongoing trial. Said otherwise, they become motivated to invest time and energy to find consistent cue-reward associations despite no guarantee of success (effort-based attraction). A multi-test comparison of performance between individuals trained under uncertainty and certainty reveals behavioral paradoxes suggesting that the concept of incentive salience cannot fully account for responding to inconsistent cues. A mathematical model explains how appetite-based and effort-based attractions might combine their effects.

Effects of dopamine receptor antagonism and amphetamine-induced psychomotor sensitization on sign- and goal-tracking after extended training

The dopamine system is important for incentive salience attribution, where motivational value is assigned to conditioned cues that predict appetitive reinforcers. However, the role of dopamine in this process may change with extended training. We tested the effects of dopamine D1-like and D2-like receptor antagonism on the expression of sign-tracking and goal-tracking conditioned responses following extended Pavlovian conditioned approach (PCA) training. We also tested if amphetamine-induced psychomotor sensitization accelerates the enhanced acquisition of sign-tracking that is observed with extended training. In experiment 1, 24 male Long-Evans rats received 20 PCA sessions in which one lever (CS+, 10 s) predicted 0.2 ml sucrose (10 %, w/v) delivery and the other lever (CS-) did not. SCH-23390 (D1-like antagonist) or eticlopride (D2-like antagonist) were administered before non-reinforced behavioural tests at doses of 0, 0.01, and 0.1 mg/kg (s.c.). In experiment 2, rats received vehicle or 2 mg/kg amphetamine (i.p.) for 7 days (n = 12/group). Ten days later, they received 16 PCA training sessions. Both doses of SCH-23390 reduced sign- and goal-tracking, but also reduced locomotor behaviour. A low dose of eticlopride (0.01 mg/kg) selectively reduced goal-tracking, without affecting sign-tracking or locomotor behaviour. Amphetamine produced psychomotor sensitization, and this did not affect the acquisition of sign- or goal-tracking. Following extended PCA training, dopamine D2-like receptor activity is required for the expression of goal-tracking but not sign-tracking. Psychomotor sensitization to amphetamine did not impact incentive salience attribution; however, more selective manipulations of the dopamine system may be needed.

[Neurogenetics of schizophrenia: findings from studies based on data sharing and global partnerships]

Schizophrenic psychoses are the result of a multifactorial process in which not only environmental influences but also genetic factors play an important role. These factors are based on a complex mode of inheritance that involves a large number of genetic variants. In the last three decades, biological psychiatric research has focused closely on molecular genetic aspects of the hereditary basis of schizophrenic psychoses. In particular, international consortia are combining cohorts from individual researchers, creating continuously increasing sample sizes and thus increased statistical power. As part of the Psychiatric Genomics Consortium (PGC), genome-wide association studies with tens of thousands of patients and controls have for the first time found robustly replicable markers for schizophrenic psychoses. Through intensive phenotyping, first approaches to a transdiagnostic clinical reclassification of severe mental illnesses have been established in the longitudinal PsyCourse study of the UMG Göttingen and the LMU Munich, allowing new biologically validated disease subgroups with prognostic value to be identified. For the first time environmental factors could even be examined in an African cohort that contribute to the development of the psychosis. In the coming years, the enormous technical progress in the area of genomic high-throughput technologies (next-generation sequencing) is expected to provide new knowledge not only about the influence of frequently occurring single nucleotide polymorphisms but also about rare variants. For the successful use of this technological revolution an exchange of data between research groups is essential.

Directing Technology Addiction Research in Information Systems

Technology-related addictions have become common in many societies. Consequently, IS research has started examining such issues. In these embryonic stages of research, this line of work has already shown some promise in terms of understanding and tackling technology addiction problems. Nevertheless, there is a need to step back and understand the roots of technology-related addictions and how their foundations evolved in reference disciplines in order to be able to conduct more scientifically informed research on such issues. This study, therefore, explains the concept of behavioral addictions (the family of addictions to which technology-related addictions presumably belong), reviews the field's history and evolution, explains the relevant brain circuitry, and discusses similarities and differences between behavioral and substance addictions. A synthesis of this information provides eight key observations and recommendations that should help the field move forward.

Unpredictability as a modulator of drug self-administration: Relevance for substance-use disorders

Drug self-administration has been regarded as a gold-standard preclinical model of addiction and substance-use disorder (SUD). However, investigators are becoming increasingly aware, that certain aspects of addiction or SUDs experienced by humans are not accurately captured in our preclinical self-administration models. The current review will focus on two such aspects of current preclinical drug self-administration models: 1) Predictable vs. unpredictable drug access in terms of the time and effort put into obtaining drugs (i.e., response requirement) and drug quality (i.e., amount) and 2) rich vs. lean access to drugs. Some behavioral and neurobiological mechanisms that could contribute to excessive allocation of behavior toward drug-seeking and drug-taking at the expense of engaging in nondrug-related activities are discussed, and some directions for future research are identified. Based on the experiments reviewed, lean and unpredictable drug access could worsen drug-seeking and drug-taking behavior in individuals with SUDs. Once more fully explored, this area of research will help determine whether and how unpredictable and lean cost requirements affect drug self-administration in preclinical laboratory studies with nonhuman subjects and will help determine whether incorporating these conditions in current self-administration models will increase their predictive validity.

From sign-tracking to attentional bias: Implications for gambling and substance use disorders

Sign-tracking behavior in Pavlovian autoshaping is known to be a relevant index of the incentive salience attributed to reward-related cues. Evidence has accumulated to suggest that animals that exhibit a sign-tracker phenotype are especially vulnerable to addiction and relapse due to their proneness to attribute incentive salience to drug cues, and their relatively weak cognitive and attentional control over their behavior. Interestingly, sign-tracking is also influenced by reward uncertainty in a way that may promote gambling disorder. Research indicates that reward uncertainty sensitizes sign-tracking responses and favors the development of a sign-tracker phenotype, compatible with the conditioned attractiveness of lights and sounds in casinos for problem gamblers. The study of attentional biases in humans (an effect akin to sign-tracking in animals) leads to similar observations, notably that the propensity to develop attraction for conditioned stimuli (CSs) is predictive of addictive behavior. Here we review the literature on drug addiction and gambling disorder, highlighting the similarities between studies of sign-tracking and attentional biases.

Maladaptive consequences of repeated intermittent exposure to uncertainty

Recently we reported that nucleus accumbens (NAcc) dopamine (DA) tracks uncertainty during operant responding for non-caloric saccharin. We also showed that repeated intermittent exposure to this uncertainty, like exposure to drugs of abuse, leads to sensitization of the locomotor and NAcc DA effects of amphetamine and promotes the subsequent self-administration of the drug. Here we review these findings together with others showing that NAcc glutamate signaling is similarly affected by uncertainty. Extracellular levels of glutamate in this site also track uncertainty in a task in which nose poking for saccharin on an escalating variable ratio schedule of reinforcement is associated with progressively increasing variance between performance of the operant and payout. Furthermore, sensitized behavioral responding to and for amphetamine following exposure to uncertainty is accompanied by increased levels of Ca2+/calmodulin-dependent protein kinase II (CaMKII) and protein kinase C (PKC) phosphorylation as well as altered protein levels of the transcription factor ∆FosB (increased) and glutamate transporter 1 (GLT1; decreased) in NAcc tissues. Notably, phosphorylation by CaMKII and PKC regulates AMPA receptor trafficking and function in this site, is elevated following psychostimulant exposure, and is necessary for the expression of enhanced drug taking. Increased ∆FosB and decreased GLT1 levels are observed following psychostimulant exposure, are associated with increased drug taking and seeking, and are known to modulate AMPA receptors and extracellular glutamate levels respectively. These adaptations in glutamate transmission as well as those observed with DA following repeated intermittent exposure to uncertainty are similar to those produced by exposure to abused drugs. Together, they point to the recruitment of both DA and glutamate signaling pathways in the NAcc in both drug and behavioral addictions. As uncertainty is central to games of chance, these findings have particular relevance for gambling disorders known to exhibit comorbidity with drug abuse.

Reward uncertainty attributes incentive value to reward proximal cues, while amphetamine sensitization reverts attention to more predictive reward distal cues

Slot-machine gambling incorporates numerous audiovisual cues prior to and during reward delivery (e.g. spinning wheels, flashing lights, celebratory sounds). Over time, these cues may motivate playing and even elicit cravings and relapse in those affected by gambling disorder. Animal studies suggest a heightened attraction to these cues despite diminished predictive ability under reward uncertainty, as evidenced by sign-tracking behavior in rats. Repeated amphetamine administration may also enhance the incentive value attributed to cues. Here, we explored the impact of reward uncertainty and prior amphetamine sensitization on the relative attractiveness and conditioned reinforcing properties of serial Pavlovian cues with different degrees of predictive and incentive value in rats. Animals were sensitized through repeated injections of amphetamine (1-4 mg/kg) or saline and then trained in a Pavlovian autoshaping task involving two sequential lever-auditory cue combinations (CS1, CS2) under Certain (100%-1) or Uncertain (50%-1-2-3) reward conditions. Subsequently, we evaluated the impact of acute amphetamine exposure on cue attraction. Our results suggest that Uncertainty alone enhanced attraction towards the reward-proximal CS2. However, combined sensitization and Uncertainty reversed cue preference relative to Uncertainty alone, enhancing attraction towards the more predictive reward-distal CS1. Both cues acquired conditioned reinforcing properties, despite the CS2 being otherwise ignored in all groups besides Uncertainty. However, combined sensitization and Uncertainty increased the reinforcing value of both cues and doubled the amount of interaction with the CS1 lever per presentation. Our results imply competitive mechanisms for attributing incentive value to gambling-related cues between reward uncertainty, prior amphetamine sensitization, and acute amphetamine administration.

Using rodent models to understand interactions between gambling and substance use

Substance use is strongly associated with gambling, but the nature of this association can be difficult to determine. Rodents offer the opportunity to test causal models of these relationships through isolation of individual variables of interest. This review describes recent research in rodents showing: a) predisposing factors for both gambling-like behavior and substance use; b) exposure to drugs of abuse increasing gambling-like behavior; c) experience with gambling-like behavior increasing substance use; and d) links between gambling-like behavior and substance use in models of Parkinson's disease therapies. These findings reveal novel relationships between gambling and substance use, and highlight the utility of rodent models for future work in this area.

Genotype-phenotype feasibility studies on khat abuse, traumatic experiences and psychosis in Ethiopia

Studying the relationship between mental illnesses and their environmental and genetic risk factors in low-income countries holds excellent promises. These studies will improve our understanding of how risk factors identified predominantly in high-income countries also apply to other settings and will identify new, sometimes population-specific risk factors. Here we report the successful completion of two intertwined pilot studies on khat abuse, trauma, and psychosis at the Gilgel Gibe Field Research Center in Ethiopia. We found that the Gilgel Gibe Field Research Center offers a unique opportunity to collect well-characterized samples for mental health research and to perform genetic studies that, at this scale, have not been undertaken in Ethiopia yet. We also supported service development, education, and research for strengthening the professional profile of psychiatry at the site.

Cued for risk: Evidence for an incentive sensitization framework to explain the interplay between stress and anxiety, substance abuse, and reward uncertainty in disordered gambling behavior

Gambling disorder is an impairing condition confounded by psychiatric co-morbidity, particularly with substance use and anxiety disorders. Yet, our knowledge of the mechanisms that cause these disorders to coalesce remains limited. The Incentive Sensitization Theory suggests that sensitization of neural "wanting" pathways, which attribute incentive salience to rewards and their cues, is responsible for the excessive desire for drugs and cue-triggered craving. The resulting hyper-reactivity of the "wanting' system is believed to heavily influence compulsive drug use and relapse. Notably, evidence for sensitization of the mesolimbic dopamine pathway has been seen across gambling and substance use, as well as anxiety and stress-related pathology, with stress playing a major role in relapse. Together, this evidence highlights a phenomenon known as cross-sensitization, whereby sensitization to stress, drugs, or gambling behaviors enhance the sensitivity and dopaminergic response to any of those stimuli. Here, we review the literature on how cue attraction and reward uncertainty may underlie gambling pathology, and examine how this framework may advance our understanding of co-mordidity with substance-use disorders (e.g., alcohol, nicotine) and anxiety disorders. We argue that reward uncertainty, as seen in slot machines and games of chance, increases dopaminergic activity in the mesolimbic pathway and enhances the incentive value of reward cues. We propose that incentive sensitization by reward uncertainty may interact with and predispose individuals to drug abuse and stress, creating a mechanism through which co-mordidity of these disorders may emerge.

Incentive hope: A default psychological response to multiple forms of uncertainty

Our target article proposes that a new concept - incentive hope - is necessary in the behavioral sciences to explain animal foraging under harsh environmental conditions. Incentive hope refers to a specific motivational mechanism in the brain - considered only in mammals and birds. But it can also be understood at a functional level, as an adaptive behavioral strategy that contributes to improve survival. Thus, this concept is an attempt to bridge across different research fields such as behavioral psychology, reward neuroscience, and behavioral ecology. Many commentaries suggest that incentive hope even could help understand phenomena beyond these research fields, including food wasting and food sharing, mental energy conservation, diverse psychopathologies, irrational decisions in invertebrates, and some aspects of evolution by means of sexual selection. We are favorable to such extensions because incentive hope denotes an unconscious process capable of working against many forms of adversity; organisms do not need to hope as a subjective feeling, but to behave as if they had this feeling. In our response, we carefully discuss each suggestion and criticism and reiterate the importance of having a theory accounting for motivation under reward uncertainty.

Exposure to conditions of uncertainty promotes the pursuit of amphetamine

Prior exposure to abused drugs leads to long-lasting neuroadaptations culminating in excessive drug intake. Given the comorbidity between substance use and gambling disorders, surprisingly little is known about the effects of exposure to reinforcement contingencies experienced during games of chance. As it is a central feature of these games, we characterized the effects of exposure to uncertainty on biochemical and behavioral effects normally observed in rats exposed to amphetamine. Rats in different groups were trained to nose-poke for saccharin under certain [fixed-ratio (FR)] or uncertain conditions [variable-ratio (VR)] for 55 1-h sessions. Ratios were escalated on successive sessions and rats maintained on the last ratio (FR/VR 20) for 20-25 days. Two to three weeks later, rats were tested for their locomotor or nucleus accumbens dopamine (NAcc DA) response to amphetamine or self-administration of the drug using a lever press operant. NAcc DA overflow was also assessed in additional rats during the saccharin sessions. Rats exposed to uncertainty subsequently showed a higher locomotor and NAcc DA response to amphetamine and self-administered more drug infusions relative to rats exposed to predictable reinforcement. NAcc DA levels during the saccharin sessions tracked the variance of the scheduled ratios (a measure of uncertainty). VR rats showed escalating DA overflow with increasing ratios. Exposure to uncertainty triggered neuroadaptations similar to those produced by exposure to abused drugs. As these were produced in drug naive rats both during and after exposure to uncertainty, they provide a novel common pathway to drug and behavioral addictions.

Nicotine pre-treatment reduces sensitivity to the interoceptive stimulus effects of commonly abused drugs as assessed with taste conditioning paradigms

BACKGROUND

Drug pre-exposure attenuates sensitivity to the interoceptive stimulus properties of additional subsequently administered drugs in drug-induced conditioned taste avoidance (CTA) and conditioned place preference (CPP) paradigms. Specifically, nicotine, commonly used in conjunction with other addictive substances, attenuates acquisition of ethanol and caffeine CTAs and morphine-induced CPP.

METHODS

Because nicotine use is comorbid with a number of substance use disorders, we systematically examined the effects of nicotine pre-exposure on two different conditioning paradigms involving integration of the interoceptive stimulus properties of multiple commonly abused drugs, in male and female rats, designed to examine both the aversive and reinforcing properties of these drugs.

RESULTS

Nicotine dose-dependently interfered with acquisition of CTA to passively administered morphine, ethanol, and cocaine, but not lithium chloride, demonstrating that the effects of nicotine are not simply a matter of reduced orosensory processing or an inability to learn such associations. Moreover, nicotine-treated rats required higher doses of drug in order to develop CTA and did not show increased acceptance of the taste of self-administered ethanol compared with saline-treated rats.

CONCLUSIONS

These data demonstrate that nicotine pre-exposure attenuates sensitivity to the stimulus effects of multiple drugs in two conditioning paradigms, in a manner which is consistent with a reduced ability to integrate the interoceptive properties of abused drugs. Through reducing these stimulus properties of drugs of abuse, concomitant nicotine use may result in a need to increase either the frequency or strength of doses during drug-taking, thus likely contributing to enhanced addiction liability in smokers.

Sign-tracking is an expectancy-mediated behavior that relies on prediction error mechanisms

When discrete localizable stimuli are used during appetitive Pavlovian conditioning, "sign-tracking" and "goal-tracking" responses emerge. Sign-tracking is observed when conditioned responding is directed toward the CS, whereas goal-tracking manifests as responding directed to the site of expected reward delivery. These behaviors seem to rely on distinct, though overlapping neural circuitries, and, possibly, distinct psychological processes as well, and are thought to be related to addiction vulnerability. One currently popular view is that sign-tracking reflects an incentive motivational process, whereas goal-tracking reflects the influence of more top-down cognitive processes. To test these ideas, we used illness-induced outcome-devaluation and Kamin blocking procedures to determine whether these behaviors rely on similar or distinct underlying associative mechanisms. In Experiments 1 and 2 we showed that outcome-devaluation reduced sign-tracking responses, demonstrating that sign-tracking is controlled by reward expectancies. We also observed that post-CS goal-tracking in these animals is also devaluation sensitive. To test whether these two types of behaviors rely on similar or different prediction error mechanisms, we next tested whether Kamin blocking effects could be observed across these two classes of behaviors. In Experiment 3 we asked if sign-tracking to a lever CS could block the development of goal-tracking to a tone CS; whereas in Experiment 4, we examined whether goal-tracking to a tone CS could block sign-tracking to a lever CS. In both experiments blocking effects were observed suggesting that both sign- and goal-tracking emerge via a common prediction error mechanism. Collectively, the studies reported here suggest that the psychological mechanisms mediating sign- and goal-tracking are more similar than is commonly acknowledged.

Heterogeneous dopamine signals support distinct features of motivated actions: implications for learning and addiction

Despite decades of research, investigations into effective neural and pharmacological therapies for many drugs of abuse, such as cocaine, have produced no FDA-approved approaches. This difficulty derives from the complexity of substance use disorders, which encompass a variety of behavioral, psychological, and neural circuit-based changes that occur as a result of repeated experience with the drug. Dopamine signaling has been demonstrated to play a key role in several aspects of drug abuse—from mediating its reinforcing properties and drug-seeking to triggering relapse—while also mediating a number of important aspects of normal (nondrug related) motivated behaviors and actions. Real-time recording methods such as in vivo voltammetry, electrophysiology, and calcium imaging demonstrate that the signaling properties of dopamine for motivationally relevant stimuli are highly dynamic and spatiotemporally circumscribed within afferent target regions. In this review, we identify the origins and functional consequences of heterogeneous dopamine release in the limbic system, and how these properties are persistently altered in the drug-experienced brain. We propose that these spatiotemporally parallel dopaminergic signals are simultaneously available to the animal, but that these circuits are impaired following prolonged drug experience by disrupting the location and content of dopamine signals in afferent target regions. These findings are discussed in the context of relapse and pathways to discovering new treatments for addiction disorders.

Repeated cocaine exposure dysregulates cognitive control over cue-evoked reward-seeking behavior during Pavlovian-to-instrumental transfer

Drug-paired cues acquire powerful motivational properties, but only lead to active drug-seeking behavior if they are potent enough to overwhelm the cognitive control processes that serve to suppress such urges. Studies using the Pavlovian-to-instrumental transfer (PIT) task have shown that rats pretreated with cocaine or amphetamine exhibit heightened levels of cue-motivated food-seeking behavior, suggesting that exposure to these drugs sensitizes the incentive motivational system. However, the PIT testing protocol can also create conflict between two competing behavioral responses to the reward-paired cue: active reward seeking (e.g., lever pressing) and passive conditioned food-cup approach behavior. We therefore investigated whether repeated cocaine exposure alters the way in which rats use cue-based reward expectations to resolve such conflict. In-depth analysis of previously published and new research confirmed that when drug-naïve rats are given a cue that signals the timing of a delayed noncontingent reward, they adaptively transition from reward seeking to conditioned approach behavior, facilitating efficient collection of the predicted reward. In contrast, cocaine-exposed rats exhibit pronounced behavioral dysregulation, increasing, rather than suppressing, their reward-seeking behavior over time, disrupting their ability to passively collect reward. Such findings speak to the important and sometimes overlooked role that cognitive control plays in determining the motivational impact of cues associated with drug and nondrug rewards.

The β-adrenoceptor blocker propranolol ameliorates compulsive-like gambling behaviour in a rodent slot machine task: implications for iatrogenic gambling disorder

Previous work has shown that chronic administration of the dopamine D_2/3 receptor agonist ropinirole invigorates performance on a rodent slot machine task (rSMT). This behavioural change appears superficially similar to the iatrogenic gambling disorder (GD) observed in a sub-set of patients with Parkinson's disease (PD), and has been associated with increased activation of the intra-cellular signalling proteins GSK3β and CREB in the striatum. Here, we wanted to determine whether this response to ropinirole could be attenuated by targeting these signalling proteins, and if the loss of dopaminergic innervation characteristic of PD would alter ropinirole's effects on the rSMT. Male Long Evans rats were trained on the rSMT. Dopaminergic terminals innervating the dorsolateral striatum were then lesioned bilaterally using the neurotoxin 6-hydroxydopamine hydrochloride (6-OHDA). Subsequently animals were implanted with osmotic mini-pumps delivering ropinirole. Lastly, animals were given dietary lithium (Li⁺ ), to inhibit the activation of GSK3β, or injections of the ß-adrenoceptor antagonist propranolol, which potently inhibits CREB as a secondary mechanism of action, and any changes in ropinirole-induced increases in compulsive-like engagement in the rSMT evaluated. Chronic ropinirole increased the number of trials animals completed, reproducing our original finding. This increase in task engagement was not altered in animals with 6-OHDA lesions, a putative model of early PD. In addition, the effects of ropinirole were not attenuated by administration of Li⁺ , but were ameliorated by propranolol. These data suggest that propranolol may represent a potential pharmacotherapy for the treatment of iatrogenic gambling.

Current perspectives on incentive salience and applications to clinical disorders

Affective neuroscience research has revealed that reward contains separable components of 'liking', 'wanting', and learning. Here we focus on current 'liking' and 'wanting' findings and applications to clinical disorders. 'Liking' is the hedonic impact derived from a pleasant experience, and is amplified by opioid and related signals in discrete sites located in limbic-related brain areas. 'Wanting' refers to incentive salience, a motivation process for reward, and is mediated by larger systems involving mesocorticolimbic dopamine. Deficits in incentive salience may contribute to avolitional features of depression and related disorders, whereas deficits in hedonic impact may produce true anhedonia. Excesses in incentive salience, on the other hand, can lead to addiction, especially when narrowly focused on a particular target. Finally, a fearful form of motivational salience may even contribute to some paranoia symptoms of schizophrenia and related disorders.

Dynamic Encoding of Incentive Salience in the Ventral Pallidum: Dependence on the Form of the Reward Cue

Some rats are especially prone to attribute incentive salience to a cue (conditioned stimulus, CS) paired with food reward (sign-trackers, STs), but the extent they do so varies as a function of the form of the CS. Other rats respond primarily to the predictive value of a cue (goal-trackers, GTs), regardless of its form. Sign-tracking is associated with greater cue-induced activation of mesolimbic structures than goal-tracking; however, it is unclear how the form of the CS itself influences activity in neural systems involved in incentive salience attribution. Thus, our goal was to determine how different cue modalities affect neural activity in the ventral pallidum (VP), which is known to encode incentive salience attribution, as rats performed a two-CS Pavlovian conditioned approach task in which both a lever-CS and a tone-CS predicted identical food reward. The lever-CS elicited sign-tracking in some rats (STs) and goal-tracking in others (GTs), whereas the tone-CS elicited only goal-tracking in all rats. The lever-CS elicited robust changes in neural activity (sustained tonic increases or decreases in firing) throughout the VP in STs, relative to GTs. These changes were not seen when STs were exposed to the tone-CS, and in GTs there were no differences in firing between the lever-CS and tone-CS. We conclude that neural activity throughout the VP encodes incentive signals and is especially responsive when a cue is of a form that promotes the attribution of incentive salience to it, especially in predisposed individuals.

Ethanol Exposure History and Alcoholic Reward Differentially Alter Dopamine Release in the Nucleus Accumbens to a Reward-Predictive Cue

BACKGROUND

Conditioned stimuli (CS) that predict reward delivery acquire the ability to induce phasic dopamine release in the nucleus accumbens (NAc). This dopamine release may facilitate conditioned approach behavior, which often manifests as approach to the site of reward delivery (called "goal-tracking") or to the CS itself (called "sign-tracking"). Previous research has linked sign-tracking in particular to impulsivity and drug self-administration, and addictive drugs may promote the expression of sign-tracking. Ethanol (EtOH) acutely promotes phasic release of dopamine in the accumbens, but it is unknown whether an alcoholic reward alters dopamine release to a CS. We hypothesized that Pavlovian conditioning with an alcoholic reward would increase dopamine release triggered by the CS and subsequent sign-tracking behavior. Moreover, we predicted that chronic intermittent EtOH (CIE) exposure would promote sign-tracking while acute administration of naltrexone (NTX) would reduce it.

METHODS

Rats received 14 doses of EtOH (3 to 5 g/kg, intragastric) or water followed by 6 days of Pavlovian conditioning training. Rewards were a chocolate solution with or without 10% (w/v) alcohol. We used fast-scan cyclic voltammetry to measure phasic dopamine release in the NAc core in response to the CS and the rewards. We also determined the effect of NTX (1 mg/kg, subcutaneous) on conditioned approach.

RESULTS

Both CIE and alcoholic reward, individually but not together, associated with greater dopamine to the CS than control conditions. However, this increase in dopamine release was not linked to greater sign-tracking, as both CIE and alcoholic reward shifted conditioned approach from sign-tracking behavior to goal-tracking behavior. However, they both also increased sensitivity to NTX, which reduced goal-tracking behavior.

CONCLUSIONS

While a history of EtOH exposure or alcoholic reward enhanced dopamine release to a CS, they did not promote sign-tracking under the current conditions. These findings are consistent with the interpretation that EtOH can stimulate conditioned approach, but indicate that the conditioned response may manifest as goal-tracking.

How foraging works: Uncertainty magnifies food-seeking motivation

Food uncertainty has the effect of invigorating food-related responses. Psychologists have noted that mammals and birds respond more to a conditioned stimulus that unreliably predicts food delivery, and ecologists have shown that animals (especially small passerines) consume and/or hoard more food and can get fatter when access to that resource is unpredictable. Are these phenomena related? We think they are. Psychologists have proposed several mechanistic interpretations, while ecologists have suggested a functional interpretation: The effect of unpredictability on fat reserves and hoarding behavior is an evolutionary strategy acting against the risk of starvation when food is in short supply. Both perspectives are complementary, and we argue that the psychology of incentive motivational processes can shed some light on the causal mechanisms leading animals to seek and consume more food under uncertainty in the wild. Our theoretical approach is in agreement with neuroscientific data relating to the role of dopamine, a neurotransmitter strongly involved in incentive motivation, and its plausibility has received some explanatory and predictive value with respect to Pavlovian phenomena. Overall, we argue that the occasional and unavoidable absence of food rewards has motivational effects (called incentive hope) that facilitate foraging effort. We show that this hypothesis is computationally tenable, leading foragers in an unpredictable environment to consume more food items and to have higher long-term energy storage than foragers in a predictable environment.

Multiple reward-cue contingencies favor expectancy over uncertainty in shaping the reward-cue attentional salience

Reward-predicting cues attract attention because of their motivational value. A debated question regards the conditions under which the cue's attentional salience is governed more by reward expectancy rather than by reward uncertainty. To help shedding light on this relevant issue, here, we manipulated expectancy and uncertainty using three levels of reward-cue contingency, so that, for example, a high level of reward expectancy (p = .8) was compared with the highest level of reward uncertainty (p = .5). In Experiment 1, the best reward-cue during conditioning was preferentially attended in a subsequent visual search task. This result was replicated in Experiment 2, in which the cues were matched in terms of response history. In Experiment 3, we implemented a hybrid procedure consisting of two phases: an omission contingency procedure during conditioning, followed by a visual search task as in the previous experiments. Crucially, during both phases, the reward-cues were never task relevant. Results confirmed that, when multiple reward-cue contingencies are explored by a human observer, expectancy is the major factor controlling both the attentional and the oculomotor salience of the reward-cue.

Interaction between trait and housing condition produces differential decision-making toward risk choice in a rat gambling task

Poor decision-making is a core problem in psychiatric disorders such as pathological gambling and substance abuse. Both trait and environmental factors are considerably important to affect decision-making. However, it has not yet been systematically shown how they interact to affect risk preference in animal models evaluating decision-making. Here, we trained rats, housed in pairs or in isolation, in a touch screen chamber to detect the association between four different light signals on the screen and accompanied reward and punishment outcomes arranged with different schedules. Then, the rats were allowed to freely choose from 4 different light signals. Once animals showed a stabilized pattern of preference (risk-averse or risk-seeking), they were injected with saline or cocaine (a single injection per day for 7 days) followed by 2 weeks of withdrawal. Then, their preference of choice was re-tested in the touch screen chamber while they were cocaine challenged. All rats significantly changed their preference toward more risky choices when they were exposed to and challenged with cocaine, except those in the risk-averse/isolated housing group. These results indicate that the pre-existing trait toward risk and the housing condition interact to affect the quality of decision-making, and cocaine may help to aggravate this process.

Reward Prediction Errors in Drug Addiction and Parkinson's Disease: from Neurophysiology to Neuroimaging

PURPOSE OF REVIEW

Surprises are important sources of learning. Cognitive scientists often refer to surprises as "reward prediction errors," a parameter that captures discrepancies between expectations and actual outcomes. Here, we integrate neurophysiological and functional magnetic resonance imaging (fMRI) results addressing the processing of reward prediction errors and how they might be altered in drug addiction and Parkinson's disease.

RECENT FINDINGS

By increasing phasic dopamine responses, drugs might accentuate prediction error signals, causing increases in fMRI activity in mesolimbic areas in response to drugs. Chronic substance dependence, by contrast, has been linked with compromised dopaminergic function, which might be associated with blunted fMRI responses to pleasant non-drug stimuli in mesocorticolimbic areas. In Parkinson's disease, dopamine replacement therapies seem to induce impairments in learning from negative outcomes. The present review provides a holistic overview of reward prediction errors across different pathologies and might inform future clinical strategies targeting impulsive/compulsive disorders.

Individual Differences in the Attribution of Incentive Salience to a Pavlovian Alcohol Cue

Individual differences exist in the attribution of incentive salience to conditioned stimuli associated with food. Here, we investigated whether individual differences also manifested with a Pavlovian alcohol conditioned stimulus (CS). We compiled data from five experiments that used a Pavlovian autoshaping paradigm and tests of conditioned reinforcement. In all experiments, male, Long-Evans rats with unrestricted access to food and water were acclimated to 15% ethanol. Next, rats received Pavlovian autoshaping training, in which a 10 s presentation of a retractable lever served as the CS and 0.2 mL of 15% ethanol served as the unconditioned stimulus (US). Finally, rats underwent conditioned reinforcement tests in which nose-pokes to an active aperture led to brief presentations of the lever-CS, but nose-pokes to an inactive aperture had no consequence. Rats were categorized as sign-trackers, goal-trackers and intermediates based on a response bias score that reflected their tendencies to sign-track or goal-track at different times during training. We found that distinct groups of rats either consistently interacted with the lever-CS (“sign-trackers”) or routinely approached the port during the lever-CS (“goal-trackers”) across a majority of the training sessions. However, some individuals (“shifted sign-trackers”) with an early tendency to goal-track later shifted to comparable asymptotic levels of sign-tracking as the group identified as sign-trackers. The lever-CS functioned as a conditioned reinforcer for sign-trackers and shifted sign-trackers, but not for goal-trackers. These results provide evidence of robust individual differences in the extent to which a Pavlovian alcohol cue gains incentive salience and functions as a conditioned reinforcer.

Terminal Dopamine Release Kinetics in the Accumbens Core and Shell Are Distinctly Altered after Withdrawal from Cocaine Self-Administration

Repeated self-administration of cocaine is associated with impairments in motivated behaviors as well as alterations in both dopamine (DA) release and neural signaling within the nucleus accumbens (NAc). These impairments are present even after several weeks of abstinence from drug taking, suggesting that the self-administration experience induces long-lasting neuroplastic alterations in the mesolimbic DA circuit. To understand these changes at the terminal level, rats were allowed to self-administer either cocaine intravenously (∼1 mg/kg per infusion) or water to a receptacle (control) in 2-h sessions over 14 days, followed by 30 days of enforced abstinence. Fast-scan cyclic voltammetry was used to record real-time DA release in either NAc core or shell after electrical stimulations of the ventral tegmental area (VTA) in freely-moving animals. In controls, the kinetics of DA release in the core and shell strikingly differed, with shell displaying slower release and reuptake rates than core. However, cocaine experience differentially altered these signaling patterns by NAc subregion. In the shell, cocaine rats showed less sensitivity to the dynamic range of applied stimulations than controls. In the core, by contrast, cocaine rats displayed robustly reduced peak DA release given the same stimulation, while also showing slower release and reuptake kinetics. The differential effects of cocaine self-administration on terminal function between core and shell is consistent with a region-specific functional reorganization of the mesolimbic DA system after repeated exposure and may provide an anatomical substrate for altered cognitive function after chronic drug-taking and addiction.

Premature responding is associated with approach to a food cue in male and female heterogeneous stock rats

RATIONALE

Disorders of behavioral regulation, including attention deficit hyperactivity disorder (ADHD) and drug addiction, are in part due to poor inhibitory control, attentional deficits, and hyper-responsivity to reward-associated cues.

OBJECTIVES

To determine whether these traits are related, we tested genetically variable male and female heterogeneous stock rats in the choice reaction time (CRT) task and Pavlovian conditioned approach (PavCA). Sex differences in the response to methylphenidate during the CRT were also assessed.

METHODS

In the CRT task, rats were required to withhold responding until one of two lights indicated whether responses into a left or right port would be reinforced with water. Reaction time on correct trials and premature responses were the operational definitions of attention and response inhibition, respectively. Rats were also pretreated with oral methylphenidate (0, 2, 4 mg/kg) during the CRT task to determine whether this drug would improve performance. Subsequently, during PavCA, presentation of an illuminated lever predicted the delivery of a food pellet into a food-cup. Lever-directed approach (sign-tracking) and food-cup approach (goal-tracking) were the primary measures, and rats were categorized as "sign-trackers" and "goal-trackers" using an index based on these measures.

RESULTS

Sign-trackers made more premature responses than goal-trackers but showed no differences in reaction time. There were sex differences in both tasks, with females having higher sign-tracking, completing more CRT trials, and making more premature responses after methylphenidate administration.

CONCLUSIONS

These results indicate that response inhibition is related to reward-cue responsivity, suggesting that these traits are influenced by common genetic factors.

Elucidating the role of D4 receptors in mediating attributions of salience to incentive stimuli on Pavlovian conditioned approach and conditioned reinforcement paradigms

The power of drug-associated cues to instigate drug 'wanting' and consequently promote drug seeking is a corner stone of contemporary theories of addiction. Gambling disorder has recently been added to the pantheon of addictive disorders due to the phenomenological similarities between the diseases. However, the neurobiological mechanism that may mediate increased sensitivity towards conditioned stimuli in addictive disorders is unclear. We have previously demonstrated using a rodent analogue of a simple slot machine that the dopamine D4 receptor is critically engaged in controlling animals' attribution of salience to stimuli associated with reward in this paradigm, and consequently may represent a target for the treatment of gambling disorder. Here, we investigated the role of acute administration of a D4 receptor agonist on animals' responsivity to conditioned stimuli on both a Pavlovian conditioned approach (autoshaping) and a conditioned reinforcement paradigm. Following training on one of the two tasks, separate cohorts of rats (male and female) were administered a dose of PD168077 shown to be maximally effective at precipitating errors in reward expectancy on the rat slot machine task (10mg/kg). However, augmenting the activity of the D4 receptors in this manner did not alter behaviour on either task. These data therefore provide novel evidence that the D4 receptor does not alter incentive motivation in response to cues on simple behavioural tasks.

Dorsolateral neostriatum contribution to incentive salience: opioid or dopamine stimulation makes one reward cue more motivationally attractive than another

Pavlovian cues for rewards can become attractive incentives: approached and 'wanted' as the rewards themselves. The motivational attractiveness of a previously learned cue is not fixed, but can be dynamically amplified during re-encounter by simultaneous activation of brain limbic circuitry. Here it was reported that opioid or dopamine microinjections in the dorsolateral quadrant of the neostriatum (DLS) of rats selectively amplify attraction toward a previously learned Pavlovian cue in an individualized fashion, at the expense of a competing cue. In an autoshaping (sign-tracking vs. goal-tracking) paradigm, microinjection of the mu opioid receptor agonist (DAMGO) or dopamine indirect agonist (amphetamine) in the DLS of sign-tracker individuals selectively enhanced their sign-tracking attraction toward the reward-predictive lever cue. By contrast, DAMGO or amphetamine in the DLS of goal-trackers selectively enhanced prepotent attraction toward the reward-proximal cue of sucrose dish. Amphetamine also enhanced goal-tracking in some sign-tracker individuals (if they ever defected to the dish even once). That DLS enhancement of cue attraction was due to stronger motivation, not stronger habits, was suggested by: (i) sign-trackers flexibly followed their cue to a new location when the lever was suddenly moved after DLS DAMGO microinjection; and (ii) DAMGO in the DLS also made sign-trackers work harder on a new instrumental nose-poke response required to earn presentations of their Pavlovian lever cue (instrumental conditioned reinforcement). Altogether, the current results suggest that DLS circuitry can enhance the incentive salience of a Pavlovian reward cue, selectively making that cue a stronger motivational magnet.

Cocaine Self-Administration Experience Induces Pathological Phasic Accumbens Dopamine Signals and Abnormal Incentive Behaviors in Drug-Abstinent Rats

Chronic exposure to drugs of abuse is linked to long-lasting alterations in the function of limbic system structures, including the nucleus accumbens (NAc). Although cocaine acts via dopaminergic mechanisms within the NAc, less is known about whether phasic dopamine (DA) signaling in the NAc is altered in animals with cocaine self-administration experience or if these animals learn and interact normally with stimuli in their environment. Here, separate groups of rats self-administered either intravenous cocaine or water to a receptacle (controls), followed by 30 d of enforced abstinence. Next, all rats learned an appetitive Pavlovian discrimination and voltammetric recordings of real-time DA release were taken in either the NAc core or shell of cocaine and control subjects. Cocaine experience differentially impaired DA signaling in the core and shell relative to controls. Although phasic DA signals in the shell were essentially abolished for all stimuli, in the core, DA did not distinguish between cues and was abnormally biased toward reward delivery. Further, cocaine rats were unable to learn higher-order associations and even altered simple conditioned approach behaviors, displaying enhanced preoccupation with cue-associated stimuli (sign-tracking; ST) but diminished time at the food cup awaiting reward delivery (goal-tracking). Critically, whereas control DA signaling correlated with ST behaviors, cocaine experience abolished this relationship. These findings show that cocaine has persistent, differential, and pathological effects on both DA signaling and DA-dependent behaviors and suggest that psychostimulant experience may remodel the very circuits that bias organisms toward repeated relapse.

SIGNIFICANCE STATEMENT

Relapsing to drug abuse despite periods of abstinence and sincere attempts to quit is one of the most pernicious facets of addiction. Unfortunately, little is known about how the dopamine (DA) system functions after periods of drug abstinence, particularly its role in behavior in nondrug situations. Here, rats learned about food-paired stimuli after prolonged abstinence from cocaine self-administration. Using voltammetry, we found that real-time DA signals in cocaine-experienced rats were strikingly altered relative to controls. Further, cocaine-experienced animals found reward-predictive stimuli abnormally salient and spent more time interacting with cues. Therefore, cocaine induces neuroplastic changes in the DA system that biases animals toward salient stimuli (including reward-associated cues), putting addicts at increasing risk to relapse as addiction increases in severity.

Roles of "Wanting" and "Liking" in Motivating Behavior: Gambling, Food, and Drug Addictions

The motivation to seek out and consume rewards has evolutionarily been driven by the urge to fulfill physiological needs. However in a modern society dominated more by plenty than scarcity, we tend to think of motivation as fueled by the search for pleasure. Here, we argue that two separate but interconnected subcortical and unconscious processes direct motivation: "wanting" and "liking." These two psychological and neuronal processes and their related brain structures typically work together, but can become dissociated, particularly in cases of addiction. In drug addiction, for example, repeated consumption of addictive drugs sensitizes the mesolimbic dopamine system, the primary component of the "wanting" system, resulting in excessive "wanting" for drugs and their cues. This sensitizing process is long-lasting and occurs independently of the "liking" system, which typically remains unchanged or may develop a blunted pleasure response to the drug. The result is excessive drug-taking despite minimal pleasure and intense cue-triggered craving that may promote relapse long after detoxification. Here, we describe the roles of "liking" and "wanting" in general motivation and review recent evidence for a dissociation of "liking" and "wanting" in drug addiction, known as the incentive sensitization theory (Robinson and Berridge 1993). We also make the case that sensitization of the "wanting" system and the resulting dissociation of "liking" and "wanting" occurs in both gambling disorder and food addiction.

Individual variability in behavioral flexibility predicts sign-tracking tendency

Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS-) and second-order (SOCS+, SOCS-) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.