Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists

Nucleus accumbens core dopamine signaling tracks the need-based motivational value of food-paired cues

Environmental reward-predictive stimuli provide a major source of motivation for instrumental reward-seeking activity and this has been linked to dopamine signaling in the nucleus accumbens (NAc) core. This cue-induced incentive motivation can be quite general, not restricted to instrumental actions that earn the same unique reward, and is also typically regulated by one's current need state, such that cues only motivate actions when this is adaptive. But it remains unknown whether cue-evoked dopamine signaling is similarly regulated by need state. Here, we used fast-scan cyclic voltammetry to monitor dopamine concentration changes in the NAc core of rats during a Pavlovian-to-instrumental transfer task in which the motivating influence of two cues, each signaling a distinct food reward (sucrose or food pellets), over an action earning a third unique food reward (polycose) was assessed in a state of hunger and of satiety. Both cues elicited a robust NAc dopamine response when hungry. The magnitude of the sucrose cue-evoked dopamine response correlated with the Pavlovian-to-instrumental transfer effect that was selectively induced by this stimulus. Satiety attenuated these cue-evoked dopamine responses and behavioral responding, even though rats had never experienced the specific food rewards in this state. These data demonstrate that cue-evoked NAc core responses are sensitive to current need state, one critical variable that determines the current adaptive utility of cue-motivated behavior. Food-predictive stimuli motivate food-seeking behavior. Here, we show that food cues evoke a robust nucleus accumbens core dopamine response when hungry that correlates with the cue's ability to invigorate general food seeking. This response is attenuated when sated, demonstrating that food cue-evoked accumbens dopamine responses are sensitive to the need state information that determines the current adaptive utility of cue-motivated action.

Dissociable hippocampal and amygdalar D1-like receptor contribution to discriminated Pavlovian conditioned approach learning

Pavlovian conditioning is an elementary form of reward-related behavioral adaptation. The mesolimbic dopamine system is widely considered to mediate critical aspects of reward-related learning. For example, initial acquisition of positively-reinforced operant behavior requires dopamine (DA) D1 receptor (D1R) activation in the basolateral amygdala (BLA), central nucleus of the amygdala (CeA), and the ventral subiculum (vSUB). However, the role of D1R activation in these areas on appetitive, non-drug-related, Pavlovian learning is not currently known. In separate experiments, microinfusions of the D1-like receptor antagonist SCH-23390 (3.0 nmol/0.5 μL per side) into the amygdala and subiculum preceded discriminated Pavlovian conditioned approach (dPCA) training sessions. D1-like antagonism in all three structures impaired the acquisition of discriminated approach, but had no effect on performance after conditioning was asymptotic. Moreover, dissociable effects of D1-like antagonism in the three structures on components of discriminated responding were obtained. Lastly, the lack of latent inhibition in drug-treated groups may elucidate the role of D1-like in reward-related Pavlovian conditioning. The present data suggest a role for the D1 receptors in the amygdala and hippocampus in learning the significance of conditional stimuli, but not in the expression of conditional responses.

Dynamic mesolimbic dopamine signaling during action sequence learning and expectation violation

Prolonged mesolimbic dopamine concentration changes have been detected during spatial navigation, but little is known about the conditions that engender this signaling profile or how it develops with learning. To address this, we monitored dopamine concentration changes in the nucleus accumbens core of rats throughout acquisition and performance of an instrumental action sequence task. Prolonged dopamine concentration changes were detected that ramped up as rats executed each action sequence and declined after earned reward collection. With learning, dopamine concentration began to rise increasingly earlier in the execution of the sequence and ultimately backpropagated away from stereotyped sequence actions, becoming only transiently elevated by the most distal and unexpected reward predictor. Action sequence-related dopamine signaling was reactivated in well-trained rats if they became disengaged in the task and in response to an unexpected change in the value, but not identity of the earned reward. Throughout training and test, dopamine signaling correlated with sequence performance. These results suggest that action sequences can engender a prolonged mode of dopamine signaling in the nucleus accumbens core and that such signaling relates to elements of the motivation underlying sequence execution and is dynamic with learning, overtraining and violations in reward expectation.

A Closer Look at the Effects of Repeated Cocaine Exposure on Adaptive Decision-Making under Conditions That Promote Goal-Directed Control

It has been proposed that compulsive drug seeking reflects an underlying dysregulation in adaptive behavior that favors habitual (automatic and inflexible) over goal-directed (deliberative and highly flexible) action selection. Rodent studies have established that repeated exposure to cocaine or amphetamine facilitates the development of habits, producing behavior that becomes unusually insensitive to a reduction in the value of its outcome. The current study more directly investigated the effects of cocaine pre-exposure on goal-directed learning and action selection using an approach that discourages habitual performance. After undergoing a 15-day series of cocaine (15 or 30 mg/kg, i.p.) or saline injections and a drug withdrawal period, rats were trained to perform two different lever-press actions for distinct reward options. During a subsequent outcome devaluation test, both cocaine- and saline-treated rats showed a robust bias in their choice between the two actions, preferring whichever action had been trained with the reward that retained its value. Thus, it appears that the tendency for repeated cocaine exposure to promote habit formation does not extend to a more complex behavioral scenario that encourages goal-directed control. To further explore this issue, we assessed how prior cocaine treatment would affect the rats' ability to learn about a selective reduction in the predictive relationship between one of the two actions and its outcome, which is another fundamental feature of goal-directed behavior. Interestingly, we found that cocaine-treated rats showed enhanced, rather than diminished, sensitivity to this action-outcome contingency degradation manipulation. Given their mutual dependence on striatal dopamine signaling, we suggest that cocaine's effects on habit formation and contingency learning may stem from a common adaptation in this neurochemical system.

N-methyl-D-aspartate receptors in the ventral tegmental area mediate the excitatory influence of Pavlovian stimuli on instrumental performance

Pavlovian stimuli predictive of food can markedly amplify instrumental responding for food. This effect is termed Pavlovian-instrumental transfer (PIT). The ventral tegmental area (VTA) plays a key role in mediating PIT, however, it is yet unknown whether N-methyl-D-aspartate (NMDA)-type glutamate receptors in the VTA are involved in PIT. Here, we examined the effects of an NMDA-receptor blockade in the VTA on PIT. Immediately prior to PIT testing, rats were subjected to intra-VTA infusions of vehicle or of the NMDA-receptor antagonist 2-amino-5-phosphonopentanoic acid (AP-5) (1, 5 µg/side). In rats that received AP-5 at the lower dose, the PIT effect was intact, i.e. presentation of the Pavlovian stimulus enhanced instrumental responding. By contrast, in rats that received AP-5 at the higher dose, the PIT effect was blocked. The data suggest that NMDA receptors in the VTA mediate the activating effects of Pavlovian stimuli on instrumental responding.

The Relationship Between Specific Pavlovian Instrumental Transfer and Instrumental Reward Probability

Goal-directed behavior is influenced by environmental cues: in particular, cues associated with a reward can bias action choice toward actions directed to that same reward. This effect is studied experimentally as specific Pavlovian-instrumental transfer (specific PIT). We have investigated the hypothesis that cues associated to an outcome elicit specific PIT by rising the estimates of reward probability of actions associated to that same outcome. In other words, cues reduce the uncertainty on the efficacy of instrumental actions. We used a human PIT experimental paradigm to test the effects of two different instrumental contingencies: one group of participants had a 33% chance of being rewarded for each button press, while another had a 100% chance. The group trained with 33% reward probability showed a stronger PIT effect than the 100% group, in line with the hypothesis that Pavlovian cues linked to an outcome work by reducing the uncertainty of receiving it. The 100% group also showed a significant specific PIT effect, highlighting additional factors that could contribute to specific PIT beyond the instrumental training contingency. We hypothesize that the uncertainty about reward delivery due to testing in extinction might be one of these factors. These results add knowledge on how goal-directed behavior is influenced by the presence of environmental cues associated with a reward: such influence depends on the probability that we have to reach a reward, namely when there is less chance of getting a reward we are more influenced by cues associated with it, and vice versa.

Dopamine, reward learning, and active inference

Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.

Pavlovian-to-instrumental transfer in alcohol dependence: a pilot study

BACKGROUND

Pavlovian processes are thought to play an important role in the development, maintenance and relapse of alcohol dependence, possibly by influencing and usurping ongoing thought and behavior. The influence of pavlovian stimuli on ongoing behavior is paradigmatically measured by pavlovian-to-instrumental transfer (PIT) tasks. These involve multiple stages and are complex. Whether increased PIT is involved in human alcohol dependence is uncertain. We therefore aimed to establish and validate a modified PIT paradigm that would be robust, consistent and tolerated by healthy controls as well as by patients suffering from alcohol dependence, and to explore whether alcohol dependence is associated with enhanced PIT.

METHODS

Thirty-two recently detoxified alcohol-dependent patients and 32 age- and gender-matched healthy controls performed a PIT task with instrumental go/no-go approach behaviors. The task involved both pavlovian stimuli associated with monetary rewards and losses, and images of drinks.

RESULTS

Both patients and healthy controls showed a robust and temporally stable PIT effect. Strengths of PIT effects to drug-related and monetary conditioned stimuli were highly correlated. Patients more frequently showed a PIT effect, and the effect was stronger in response to aversively conditioned CSs (conditioned suppression), but there was no group difference in response to appetitive CSs.

CONCLUSION

The implementation of PIT has favorably robust properties in chronic alcohol-dependent patients and in healthy controls. It shows internal consistency between monetary and drug-related cues. The findings support an association of alcohol dependence with an increased propensity towards PIT.

Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior

Pavlovian-to-instrumental transfer (PIT) refers to the process of a Pavlovian reward-paired cue acquiring incentive motivational proprieties that drive choices. It represents a crucial phenomenon for understanding cue-controlled behavior, and it has both adaptive and maladaptive implications (i.e., drug-taking). In animals, individual differences in the degree to which such cues bias performance have been identified in two types of individuals that exhibit distinct Conditioned Responses (CR) during Pavlovian conditioning: Sign-Trackers (ST) and Goal-Trackers (GT). Using an appetitive PIT procedure with a monetary reward, the present study investigated, for the first time, the extent to which such individual differences might affect the influence of reward-paired cues in humans. In a first task, participants learned an instrumental response leading to reward; then, in a second task, a visual Pavlovian cue was associated with the same reward; finally, in a third task, PIT was tested by measuring the preference for the reward-paired instrumental response when the task-irrelevant reward-paired cue was presented, in the absence of the reward itself. In ST individuals, but not in GT individuals, reward-related cues biased behavior, resulting in an increased likelihood to perform the instrumental response independently paired with the same reward when presented with the task-irrelevant reward-paired cue, even if the reward itself was no longer available (i.e., stronger PIT effect). This finding has important implications for developing individualized treatment for maladaptive behaviors, such as addiction.

The algorithmic anatomy of model-based evaluation

Despite many debates in the first half of the twentieth century, it is now largely a truism that humans and other animals build models of their environments and use them for prediction and control. However, model-based (MB) reasoning presents severe computational challenges. Alternative, computationally simpler, model-free (MF) schemes have been suggested in the reinforcement learning literature, and have afforded influential accounts of behavioural and neural data. Here, we study the realization of MB calculations, and the ways that this might be woven together with MF values and evaluation methods. There are as yet mostly only hints in the literature as to the resulting tapestry, so we offer more preview than review.

Toward sophisticated basal ganglia neuromodulation: Review on basal ganglia deep brain stimulation

This review presents state-of-the-art knowledge about the roles of the basal ganglia (BG) in action-selection, cognition, and motivation, and how this knowledge has been used to improve deep brain stimulation (DBS) treatment of neurological and psychiatric disorders. Such pathological conditions include Parkinson's disease, Huntington's disease, Tourette syndrome, depression, and obsessive-compulsive disorder. The first section presents evidence supporting current hypotheses of how the cortico-BG circuitry works to select motor and emotional actions, and how defects in this circuitry can cause symptoms of the BG diseases. Emphasis is given to the role of striatal dopamine on motor performance, motivated behaviors and learning of procedural memories. Next, the use of cutting-edge electrochemical techniques in animal and human studies of BG functioning under normal and disease conditions is discussed. Finally, functional neuroimaging studies are reviewed; these works have shown the relationship between cortico-BG structures activated during DBS and improvement of disease symptoms.

Model-based and model-free Pavlovian reward learning: revaluation, revision, and revelation

Evidence supports at least two methods for learning about reward and punishment and making predictions for guiding actions. One method, called model-free, progressively acquires cached estimates of the long-run values of circumstances and actions from retrospective experience. The other method, called model-based, uses representations of the environment, expectations, and prospective calculations to make cognitive predictions of future value. Extensive attention has been paid to both methods in computational analyses of instrumental learning. By contrast, although a full computational analysis has been lacking, Pavlovian learning and prediction has typically been presumed to be solely model-free. Here, we revise that presumption and review compelling evidence from Pavlovian revaluation experiments showing that Pavlovian predictions can involve their own form of model-based evaluation. In model-based Pavlovian evaluation, prevailing states of the body and brain influence value computations, and thereby produce powerful incentive motivations that can sometimes be quite new. We consider the consequences of this revised Pavlovian view for the computational landscape of prediction, response, and choice. We also revisit differences between Pavlovian and instrumental learning in the control of incentive motivation.

Skewed by Cues? The Motivational Role of Audiovisual Stimuli in Modelling Substance Use and Gambling Disorders

The similarity between gambling disorder (GD) and drug addiction has recently been recognized at the diagnostic level. Understanding the core cognitive processes involved in these addiction disorders, and in turn their neurobiological mechanisms, remains a research priority due to the enormous benefits such knowledge would have in enabling effective treatment design. Animal models can be highly informative in this regard. Although numerous rodent behavioural paradigms that capture different facets of gambling-like behaviour have recently been developed, the motivational power of cues in biasing individuals towards risky choice has so far received little attention despite the central role played by drug-paired cues in successful laboratory models of chemical dependency. Here, we review some of the comparatively simple paradigms in which reward-paired cues are known to modulate behaviour in rodents, such as sign-tracking, Pavlovian-to-instrumental transfer and conditioned reinforcement. Such processes are thought to play an important role in mediating responding for drug reward, and the need for future studies to address whether similar processes contribute to cue-driven risky choice is highlighted.

Learning and Motivational Processes Contributing to Pavlovian-Instrumental Transfer and Their Neural Bases: Dopamine and Beyond

Pavlovian stimuli exert a range of effects on behavior from simple conditioned reflexes, such as salivation, to altering the vigor and direction of instrumental actions. It is currently accepted that these distinct behavioral effects stem from two sources (i) the various associative connections between predictive stimuli and the component features of the events that these stimuli predict and (ii) the distinct motivational and cognitive functions served by cues, particularly their arousing and informational effects on the selection and performance of specific actions. Here, we describe studies that have assessed these latter phenomena using a paradigm that has come to be called Pavlovian-instrumental transfer. We focus first on behavioral experiments that have described distinct sources of stimulus control derived from the general affective and outcome-specific predictions of conditioned stimuli, referred to as general transfer and specific transfer, respectively. Subsequently, we describe research efforts attempting to establish the neural bases of these transfer effects, largely in the afferent and efferent connections of the nucleus accumbens (NAc) core and shell. Finally, we examine the role of predictive cues in examples of aberrant stimulus control associated with psychiatric disorders and addiction.

Effects of pramipexole on the acquisition of responding with opioid-conditioned reinforcement in the rat

RATIONALE

Dopamine D3 receptor-preferring ligands may be able to modify the conditioned reinforcing effects of drug-associated stimuli. In evaluating the effects of these compounds, it is important to clarify the extent to which responding depends on (1) conditioned reinforcement vs. other behavioral mechanisms and (2) dopamine D3 vs. D2 receptor activity.

OBJECTIVES

To use behaviorally stringent new-response acquisition procedures to characterize the effects of the D3-preferring agonist, pramipexole, on the conditioned reinforcing effects of a stimulus paired with the opioid agonist, remifentanil.

METHODS

First, in Pavlovian conditioning (PAV) sessions, rats received response-independent IV injections of remifentanil and presentations of a light-noise stimulus. In separate groups, injections and stimuli either always co-occurred ("paired PAV") or occurred with no consistent relationship ("random PAV" control). Next, in instrumental acquisition (ACQ) sessions, all animals could respond in two nose-poke manipulanda: an active nose-poke, which produced the stimulus alone, or an inactive nose-poke. Pramipexole was injected SC prior to ACQ sessions with or without pretreatments of the D3-preferring antagonist, SB-277011A, or the D2-preferring antagonist, L-741,626.

RESULTS

After paired PAV, but not random PAV, rats acquired nose-poke responding during ACQ (i.e., active > inactive). Pramipexole dose-dependently increased active responding without changing inactive responding. Pramipexole-induced increases in responding were blocked by pretreatment with L-741,626, but not SB-277011A.

CONCLUSIONS

Pramipexole specifically enhanced remifentanil-conditioned reinforcement: active responding was selectively increased only after the stimulus was paired with remifentanil. Although pramipexole is D3-preferring, the antagonist effects obtained presently suggest an important role for the D2 receptor in opioid-conditioned reinforcement.

Dopamine D1/D2 receptors do not mediate the expression of conditioned place preference induced by the aftereffect of wheel running

BACKGROUND

Rats lever-press for access to running wheels suggesting that wheel running by itself is reinforcing. Furthermore, pairings of an episode of wheel running and subsequent confinement in a specific environment can establish a conditioned place preference (CPP). This finding implies that the reinforcing effects of wheel running outlast the actual occurrence of physical activity, a phenomenon referred to as aftereffect of wheel running. Aftereffect-induced CPP involves Pavlovian conditioning, i.e. repeated pairings of the aftereffect of wheel running with a specific environment creates a learned association between aftereffect and environment and, in turn, a preference for that environment. Given the involvement of dopamine systems in mediating effects of Pavlovian stimuli on appetitive behavior, a role of dopamine in mediating aftereffect-induced CPP seems plausible. Here we assessed whether the mixed D1/D2 receptor antagonist flupenthixol (0.25 mg/kg, i.p.) can block the expression of an aftereffect-induced CPP.

RESULTS

In line with earlier studies, our results demonstrate that rats displayed a conditioned preference for environments paired with the aftereffect of wheel running and further show that the magnitude of CPP was not related to the wheel running rate. Furthermore, we found that flupenthixol (0.25 mg/kg, i.p.) reduced locomotor activity but did not attenuate the expression of an aftereffect-induced CPP.

CONCLUSION

The expression of a CPP produced by the aftereffect of wheel running seems not to depend on dopamine D1/D2 receptor activation.

Appetitive cue-evoked ERK signaling in the nucleus accumbens requires NMDA and D1 dopamine receptor activation and regulates CREB phosphorylation

Conditioned stimuli (CS) can modulate reward-seeking behavior. This modulatory effect can be maladaptive and has been implicated in excessive reward seeking and relapse to drug addiction. We previously demonstrated that exposure to an appetitive CS causes an increase in the activation of extracellular signal-regulated kinase (ERK) and cyclic-AMP response-element binding protein (CREB) in the nucleus accumbens (NAc) of rats, and that CS-evoked ERK activation is critical for CS control over reward seeking. To elucidate the mechanism that mediates CS-driven ERK activation in the NAc, we selectively blocked NMDA glutamate or D1 dopamine receptors in the NAc. To determine whether CS-driven ERK and CREB activation are linked, we selectively blocked ERK signaling in the NAc. We found that both NMDA and D1 receptors are critical for CS-driven ERK signaling in the NAc, and that this recruitment of the ERK cascade is responsible for increased CREB activation in the presence of the CS. Our findings suggest that activation of the NMDAR-D1R/ERK/CREB signal transduction pathway plays a critical role in the control of reward-seeking behavior by reward-predictive cues.

Role of cues and contexts on drug-seeking behaviour

Environmental stimuli are powerful mediators of craving and relapse in substance-abuse disorders. This review examined how animal models have been used to investigate the cognitive mechanisms through which cues are able to affect drug-seeking behaviour. We address how animal models can describe the way drug-associated cues come to facilitate the development and persistence of drug taking, as well as how these cues are critical to the tendency to relapse that characterizes substance-abuse disorders. Drug-associated cues acquire properties of conditioned reinforcement, incentive motivation and discriminative control, which allow them to influence drug-seeking behaviour. Using these models, researchers have been able to investigate the pharmacology subserving the behavioural impact of environmental stimuli, some of which we highlight. Subsequently, we examine whether the impact of drug-associated stimuli can be attenuated via a process of extinction, and how this question is addressed in the laboratory. We discuss how preclinical research has been translated into behavioural therapies targeting substance abuse, as well as highlight potential developments to therapies that might produce more enduring changes in behaviour.

Habitual responding for alcohol depends upon both AMPA and D2 receptor signaling in the dorsolateral striatum

Chronic alcohol self-administration leads to alcohol-seeking behavior that is habitual and insensitive to changes in the value of the earned alcohol. Such behavior has been shown to rely on the dorsolateral region of the striatum in rats but the specific pharmacological control of output from this region is not yet understood. In the following experiments rats were trained to self-administer unsweetened 10% (v/v) ethanol in daily sessions for 8 weeks prior to testing for sensitivity to outcome devaluation. We examined the role of glutamatergic AMPA-receptor activation by testing the effects of the antagonist NBQX (0.3 and 1.0 μg/μl) infused specifically into the dorsolateral striatum (DLS) before devaluation testing. In a separate group of rats we examined the role of dopaminergic D2-receptor activation using the D2-receptor antagonist raclopride (0.2 and 1.0 μg/μl) infused into the DLS before devaluation testing. Following control (saline) infusions rats’ lever-press performance was insensitive to devaluation of ethanol thus showing evidence of habitual responding. NBQX and racolpride each restored goal-directed control of responding at doses that did not impair overall lever-press rates. These data demonstrate that expression of habitual alcohol seeking relies on glutamatergic inputs to the DLS and D2 receptors within the DLS.

Phasic mesolimbic dopamine signaling encodes the facilitation of incentive motivation produced by repeated cocaine exposure

Drug addiction is marked by pathological drug seeking and intense drug craving, particularly in response to drug-related stimuli. Repeated psychostimulant administration is known to induce long-term alterations in mesolimbic dopamine (DA) signaling that are hypothesized to mediate this heightened sensitivity to environmental stimuli. However, there is little direct evidence that drug-induced alteration in mesolimbic DA function underlies this hypersensitivity to motivational cues. In the current study, we tested this hypothesis using fast-scan cyclic voltammetry to monitor phasic DA signaling in the nucleus accumbens core of cocaine-pretreated (6 once-daily injections of 15 mg/kg, i.p.) and drug-naive rats during a test of cue-evoked incentive motivation for food-the Pavlovian-to-instrumental transfer task. We found that prior cocaine exposure augmented both reward seeking and DA release triggered by the presentation of a reward-paired cue. Furthermore, cue-evoked DA signaling positively correlated with cue-evoked food seeking and was found to be a statistical mediator of this behavioral effect of cocaine. Taken together, these findings provide support for the hypothesis that repeated cocaine exposure enhances cue-evoked incentive motivation through augmented phasic mesolimbic DA signaling. This work sheds new light on a fundamental neurobiological mechanism underlying motivated behavior and its role in the expression of compulsive reward seeking.

The motivational drive to natural rewards is modulated by prenatal glucocorticoid exposure

Exposure to elevated levels of glucocorticoids (GCs) during neurodevelopment has been identified as a triggering factor for the development of reward-associated disorders in adulthood. Disturbances in the neural networks responsible for the complex processes that assign value to rewards and associated stimuli are critical for disorders such as depression, obsessive–compulsive disorders, obesity and addiction. Essential in the understanding on how cues influence behavior is the Pavlovian–instrumental transfer (PIT), a phenomenon that refers to the capacity of a Pavlovian stimulus that predicts a reward to elicit instrumental responses for that same reward. Here, we demonstrate that in utero exposure to GCs (iuGC) impairs both general and selective versions of the PIT paradigm, suggestive of deficits in motivational drive. The iuGC animals presented impaired neuronal activation pattern upon PIT performance in cortical and limbic regions, as well as morphometric changes and reduced levels of dopamine in prefrontal and orbitofrontal cortices, key regions involved in the integration of Pavlovian and instrumental stimuli. Normalization of dopamine levels rescued this behavior, a process that relied on D2/D3, but not D1, dopamine receptor activation. In summary, iuGC exposure programs the mesocorticolimbic dopaminergic circuitry, leading to a reduction in the attribution of the incentive salience to cues, in a dopamine-D2/D3-dependent manner. Ultimately, these results are important to understand how GCs bias incentive processes, a fact that is particularly relevant for disorders where differential attribution of incentive salience is critical.

Multiple memory systems as substrates for multiple decision systems

It has recently become widely appreciated that value-based decision making is supported by multiple computational strategies. In particular, animal and human behavior in learning tasks appears to include habitual responses described by prominent model-free reinforcement learning (RL) theories, but also more deliberative or goal-directed actions that can be characterized by a different class of theories, model-based RL. The latter theories evaluate actions by using a representation of the contingencies of the task (as with a learned map of a spatial maze), called an "internal model." Given the evidence of behavioral and neural dissociations between these approaches, they are often characterized as dissociable learning systems, though they likely interact and share common mechanisms. In many respects, this division parallels a longstanding dissociation in cognitive neuroscience between multiple memory systems, describing, at the broadest level, separate systems for declarative and procedural learning. Procedural learning has notable parallels with model-free RL: both involve learning of habits and both are known to depend on parts of the striatum. Declarative memory, by contrast, supports memory for single events or episodes and depends on the hippocampus. The hippocampus is thought to support declarative memory by encoding temporal and spatial relations among stimuli and thus is often referred to as a relational memory system. Such relational encoding is likely to play an important role in learning an internal model, the representation that is central to model-based RL. Thus, insofar as the memory systems represent more general-purpose cognitive mechanisms that might subserve performance on many sorts of tasks including decision making, these parallels raise the question whether the multiple decision systems are served by multiple memory systems, such that one dissociation is grounded in the other. Here we investigated the relationship between model-based RL and relational memory by comparing individual differences across behavioral tasks designed to measure either capacity. Human subjects performed two tasks, a learning and generalization task (acquired equivalence) which involves relational encoding and depends on the hippocampus; and a sequential RL task that could be solved by either a model-based or model-free strategy. We assessed the correlation between subjects' use of flexible, relational memory, as measured by generalization in the acquired equivalence task, and their differential reliance on either RL strategy in the decision task. We observed a significant positive relationship between generalization and model-based, but not model-free, choice strategies. These results are consistent with the hypothesis that model-based RL, like acquired equivalence, relies on a more general-purpose relational memory system.

Differential effects of systemic cholinergic receptor blockade on Pavlovian incentive motivation and goal-directed action selection

Reward-seeking actions can be guided by external cues that signal reward availability. For instance, when confronted with a stimulus that signals sugar, rats will prefer an action that produces sugar over a second action that produces grain pellets. Action selection is also sensitive to changes in the incentive value of potential rewards. Thus, rats that have been prefed a large meal of sucrose will prefer a grain-seeking action to a sucrose-seeking action. The current study investigated the dependence of these different aspects of action selection on cholinergic transmission. Hungry rats were given differential training with two unique stimulus-outcome (S1-O1 and S2-O2) and action-outcome (A1-O1 and A2-O2) contingencies during separate training phases. Rats were then given a series of Pavlovian-to-instrumental transfer tests, an assay of cue-triggered responding. Before each test, rats were injected with scopolamine (0, 0.03, or 0.1 mg/kg, intraperitoneally), a muscarinic receptor antagonist, or mecamylamine (0, 0.75, or 2.25 mg/kg, intraperitoneally), a nicotinic receptor antagonist. Although the reward-paired cues were capable of biasing action selection when rats were tested off-drug, both anticholinergic treatments were effective in disrupting this effect. During a subsequent round of outcome devaluation testing-used to assess the sensitivity of action selection to a change in reward value--we found no effect of either scopolamine or mecamylamine. These results reveal that cholinergic signaling at both muscarinic and nicotinic receptors mediates action selection based on Pavlovian reward expectations, but is not critical for flexibly selecting actions using current reward values.

Synergistic interaction between caloric restriction and amphetamine in food-unrelated approach behavior of rats

RATIONALE

Approach behavior is regulated by the brain integrating information about environment and body state. Psychoactive drugs interact with this process.

OBJECTIVES

We examined the extent to which caloric (i.e., food) restriction, amphetamine (AMPH) and lithium interact in potentiating locomotor activity and responding reinforced by visual stimulus (VS), a reward unrelated to energy homeostasis.

METHODS

Rats either had ad libitum access to food or received daily rations that maintained 85-90 % of their original body weights. Leverpressing turned on a cue light for 1 s and turned off house light for 5 s. AMPH and lithium were administered through intraperitoneal injections and diet, respectively.

RESULTS

Food restriction or AMPH (1 mg/kg) alone had little effect on VS-reinforced responding; however, the combination of the two conditions markedly potentiated VS-reinforced responding (fourfold). Food restriction lasting 7 days or longer was needed to augment AMPH's effect on VS-reinforced responding. AMPH (0.3-3 mg/kg) potentiated locomotor activity similarly between food-restricted and ad libitum groups. Repeated injections of AMPH-sensitized locomotor activity, but not VS-reinforced responding. In addition, while chronic lithium treatments (0.2 % lithium carbonate chow) reduced VS-reinforced responding, chronic lithium further augmented AMPH-potentiated VS-reinforced responding.

CONCLUSIONS

Food restriction interacts with psychoactive drugs to potentiate goal-directed responding unrelated to food seeking in a much more powerful manner than previously thought. The novel finding that lithium can augment a psychostimulant effect of AMPH suggests caution when combining lithium and psychostimulant drugs in clinical settings.

Dopamine dependency for acquisition and performance of Pavlovian conditioned response

During Pavlovian conditioning, pairing of a neutral conditioned stimulus (CS) with a reward leads to conditioned reward-approach responses (CRs) that are elicited by presentation of the CS. CR behaviors can be sign tracking, in which animals engage the CS, or goal tracking, in which animals go to the reward location. We investigated CR behaviors in mice with only ∼5% of normal dopamine in the striatum using a Pavlovian conditioning paradigm. These mice had severely impaired acquisition of the CR, which was ameliorated by pharmacological restoration of dopamine synthesis with l-dopa. Surprisingly, after they had learned the CR, its expression decayed only gradually in following sessions that were conducted without l-dopa treatment. To assess specific contributions of dopamine signaling in the dorsal or ventral striatum, we performed virus-mediated restoration of dopamine synthesis in completely dopamine-deficient (DD) mice. Mice with dopamine signaling only in the dorsal striatum did not acquire a CR, whereas mice with dopamine signaling only in in the ventral striatum acquired a CR. The CR in mice with dopamine signaling only in the dorsal striatum was restored by subjecting the mice to instrumental training in which they had to interact with the CS to obtain rewards. We conclude that dopamine is essential for learning and performance of CR behavior that is predominantly goal tracking. Furthermore, although dopamine signaling in the ventral striatum is sufficient to support a CR, dopamine signaling only in the dorsal striatum can also support a CR under certain circumstances.

Dorsal and ventral streams: the distinct role of striatal subregions in the acquisition and performance of goal-directed actions

Considerable evidence suggests that distinct neural processes mediate the acquisition and performance of goal-directed instrumental actions. Whereas a cortical-dorsomedial striatal circuit appears critical for the acquisition of goal-directed actions, a cortical-ventral striatal circuit appears to mediate instrumental performance, particularly the motivational control of performance. Here we review evidence that these distinct mechanisms of learning and performance constitute two distinct 'streams' controlling instrumental conditioning. From this perspective, the regulation of the interaction between these 'streams' becomes a matter of considerable importance. We describe evidence that the basolateral amygdala, which is heavily interconnected with both the dorsal and ventral subregions of the striatum, coordinates this interaction providing input to the final common path to action as a critical component of the limbic-motor interface.

Cue reactivity is associated with duration and severity of alcohol dependence: an FMRI study

INTRODUCTION

With the progression of substance dependence, drug cue-related brain activation is thought to shift from motivational towards habit pathways. However, a direct association between cue-induced brain activation and dependence duration has not yet been shown. We therefore examined the relationship between alcohol cue-reactivity in the brain, cue-induced subjective craving and alcohol dependence duration and severity. Since alcohol dependence is highly comorbid with depression/anxiety, which may modulate brain responses to alcohol cues, we also examined the relation between comorbid depression/anxiety and cue-reactivity.

METHODS

We compared 30 alcohol dependent patients with 15 healthy controls and 15 depression/anxiety patients during a visual alcohol cue-reactivity task using functional magnetic resonance imaging blood oxygenated level-dependent responses and subjective craving as outcomes. Within the alcohol dependent group we correlated cue-reactivity with alcohol dependence severity and duration, with cue-induced craving and with depression/anxiety levels.

RESULTS

Alcohol dependent patients showed greater cue-reactivity in motivational brain pathways and stronger subjective craving than depression/anxiety patients and healthy controls. Depression/anxiety was not associated with cue-reactivity, but depression severity in alcohol dependent patients was positively associated with craving. Within alcohol dependence, longer duration of alcohol dependence was associated with stronger cue-related activation of the posterior putamen, a structure involved in habits, whereas higher alcohol dependence severity was associated with lower cue-reactivity in the anterior putamen, an area implicated in goal-directed behavior preceding habit formation.

CONCLUSION

Cue-reactivity in alcohol dependence is not modulated by comorbid depression or anxiety. More importantly, the current data confirm the hypothesis of a ventral to dorsal striatal shift of learning processes with longer dependence duration, which could underlie increasingly habitual substance use with progressing substance dependence.

Individual variation in resisting temptation: implications for addiction

When exposed to the sights, sounds, smells and/or places that have been associated with rewards, such as food or drugs, some individuals have difficulty resisting the temptation to seek out and consume them. Others have less difficulty restraining themselves. Thus, Pavlovian reward cues may motivate maladaptive patterns of behavior to a greater extent in some individuals than in others. We are just beginning to understand the factors underlying individual differences in the extent to which reward cues acquire powerful motivational properties, and therefore, the ability to act as incentive stimuli. Here we review converging evidence from studies in both human and non-human animals suggesting that a subset of individuals are more "cue reactive", in that certain reward cues are more likely to attract these individuals to them and motivate actions to get them. We suggest that those individuals for whom Pavlovian reward cues become especially powerful incentives may be more vulnerable to impulse control disorders, such as binge eating and addiction.

The effects of amphetamine sensitization on conditioned inhibition during a Pavlovian-instrumental transfer task in rats

RATIONALE

Psychostimulant sensitization heightens behavioral and motivational responses to reward-associated stimuli; however, its effects on stimuli associated with reward absence are less understood.

OBJECTIVES

We examined whether amphetamine sensitization alters performance during Pavlovian-instrumental transfer (PIT) to conditioned excitors and inhibitors. We further sought to characterize the effects of amphetamine sensitization on learning versus performance by exposing rats to amphetamine prior to Pavlovian training or between training and test.

METHODS

Adult male Long-Evans rats were given conditioned inhibition (A+/AX-) and Pavlovian (B+) training, followed by variable-interval instrumental conditioning. Rats were sensitized to D-amphetamine (2 mg/kg daily injections for 7 days) or served as non-exposed controls. Rats were given a PIT test, in which they were presented with stimulus B alone or in compound with the conditioned inhibitor (BX).

RESULTS

During the PIT test, control rats significantly reduced instrumental responding on BX trials (to approximately 50 % of responding to B). Amphetamine sensitization prior to Pavlovian conditioning increased lever pressing on BX trials and reduced lever pressing on B trials compared to controls. Amphetamine sensitization between training and test increased lever pressing on B and BX trials compared to controls. No effects of sensitization were observed on conditioned food cup approach.

CONCLUSIONS

Amphetamine sensitization increases instrumental responding during PIT to a conditioned inhibitor by enhancing the excitation of conditioned stimuli and reducing the inhibition of conditioned inhibitors.

Cue-evoked cocaine "craving": role of dopamine in the accumbens core

Drug-associated cues can acquire powerful motivational control over the behavior of addicts, and can contribute to relapse via multiple, dissociable mechanisms. Most preclinical models of relapse focus on only one of these mechanisms: the ability of drug cues to reinforce drug-seeking actions following a period of extinction training. However, in addicts, drug cues typically do not follow seeking actions; they precede them. They often produce relapse by evoking a conditioned motivational state ("wanting" or "craving") that instigates and/or invigorates drug-seeking behavior. Here we used a conflict-based relapse model to ask whether individual variation in the propensity to attribute incentive salience to reward cues predicts variation in the ability of a cocaine cue to produce conditioned motivation (craving) for cocaine. Following self-administration training, responding was curtailed by requiring rats to cross an electrified floor to take cocaine. The subsequent response-independent presentation of a cocaine-associated cue was sufficient to reinstate drug-seeking behavior, despite the continued presence of the adverse consequence. Importantly, there were large individual differences in the motivational properties of the cocaine cue, which were predicted by variation in the propensity to attribute incentive salience to a food cue. Finally, a dopamine antagonist injected into the nucleus accumbens core attenuated, and amphetamine facilitated, cue-evoked cocaine seeking, implicating dopamine signaling in cocaine cue-evoked craving. These data provide a promising preclinical approach for studying sources of individual variation in susceptibility to relapse due to conditioned craving and implicate mesolimbic dopamine in this process.

Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis

Background

Depression is characterised partly by blunted reactions to reward. However, tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. We attempted to disentangle these factors with respect to anhedonia in the context of stress, Major Depressive Disorder (MDD), Bipolar Disorder (BPD) and a dopaminergic challenge.

Methods

Six behavioural datasets involving 392 experimental sessions were subjected to a model-based, Bayesian meta-analysis. Participants across all six studies performed a probabilistic reward task that used an asymmetric reinforcement schedule to assess reward learning. Healthy controls were tested under baseline conditions, stress or after receiving the dopamine D₂ agonist pramipexole. In addition, participants with current or past MDD or BPD were evaluated. Reinforcement learning models isolated the contributions of variation in reward sensitivity and learning rate.

Results

MDD and anhedonia reduced reward sensitivity more than they affected the learning rate, while a low dose of the dopamine D₂ agonist pramipexole showed the opposite pattern. Stress led to a pattern consistent with a mixed effect on reward sensitivity and learning rate.

Conclusion

Reward-related learning reflected at least two partially separable contributions. The first related to phasic prediction error signalling, and was preferentially modulated by a low dose of the dopamine agonist pramipexole. The second related directly to reward sensitivity, and was preferentially reduced in MDD and anhedonia. Stress altered both components. Collectively, these findings highlight the contribution of model-based reinforcement learning meta-analysis for dissecting anhedonic behavior.

On the motivational properties of reward cues: Individual differences

Cues associated with rewards, such as food or drugs of abuse, can themselves acquire motivational properties. Acting as incentive stimuli, such cues can exert powerful control over motivated behavior, and in the case of cues associated with drugs, they can goad continued drug-seeking behavior and relapse. However, recent studies reviewed here suggest that there are large individual differences in the extent to which food and drug cues are attributed with incentive salience. Rats prone to approach reward cues (sign-trackers) attribute greater motivational value to discrete localizable cues and interoceptive cues than do rats less prone to approach reward cues (goal-trackers). In contrast, contextual cues appear to exert greater control over motivated behavior in goal-trackers than sign-trackers. It is possible to predict, therefore, before any experience with drugs, in which animals specific classes of drug cues will most likely reinstate drug-seeking behavior. The finding that different individuals may be sensitive to different triggers capable of motivating behavior and producing relapse suggests there may be different pathways to addiction, and has implications for thinking about individualized treatment. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'.

Accelerated habit formation following amphetamine exposure is reversed by D1, but enhanced by D2, receptor antagonists

Repeated exposure to the psychostimulant amphetamine has been shown to disrupt goal-directed instrumental actions and promote the early and abnormal development of goal-insensitive habitual responding (Nelson and Killcross, 2006). To investigate the neuropharmacological specificity of this effect as well as restore goal-directed responding in animals with pre-training amphetamine exposure, animals were treated with the non-selective dopamine antagonist α-flupenthixol, the selective D₁ antagonist SCH 23390 or the selective D₂ antagonist eticlopride, prior to instrumental training (three sessions). Subsequently, the reinforcer was paired with LiCL-induced gastric-malaise and animals were given a test of goal-sensitivity both in extinction and reacquisition. The effect of these dopaminergic antagonists on the sensitivity of lever press performance to outcome devaluation was assessed in animals with pre-training exposure to amphetamine (Experiments 1A–C) or in non-sensitized animals (Experiment 2). Both α-flupenthixol and SCH23390 reversed accelerated habit formation following amphetamine sensitization. However, eticlopride appeared to enhance this effect and render instrumental performance compulsive as these animals were unable to inhibit responding both in extinction and reacquisition, even though a consumption test confirmed they had acquired an aversion to the reinforcer. These findings demonstrate that amphetamine induced-disruption of goal-directed behavior is mediated by activity at distinct dopamine receptor subtypes and may represent a putative model of the neurochemical processes involved in the loss of voluntary control over behavior.

Dopamine or opioid stimulation of nucleus accumbens similarly amplify cue-triggered 'wanting' for reward: entire core and medial shell mapped as substrates for PIT enhancement

Pavlovian cues [conditioned stimulus (CS+)] often trigger intense motivation to pursue and consume related reward [unconditioned stimulus (UCS)]. But cues do not always trigger the same intensity of motivation. Encountering a reward cue can be more tempting on some occasions than on others. What makes the same cue trigger more intense motivation to pursue reward on a particular encounter? The answer may be the level of incentive salience ('wanting') that is dynamically generated by mesocorticolimbic brain systems, influenced especially by dopamine and opioid neurotransmission in the nucleus accumbens (NAc) at that moment. We tested the ability of dopamine stimulation (by amphetamine microinjection) vs. mu opioid stimulation [by d-Ala, nMe-Phe, Glyol-enkephalin (DAMGO) microinjection] of either the core or shell of the NAc to amplify cue-triggered levels of motivation to pursue sucrose reward, measured with a Pavlovian-Instrumental Transfer (PIT) procedure, a relatively pure assay of incentive salience. Cue-triggered 'wanting' in PIT was enhanced by amphetamine or DAMGO microinjections equally, and also equally at nearly all sites throughout the entire core and medial shell (except for a small far-rostral strip of shell). NAc dopamine/opioid stimulations specifically enhanced CS+ ability to trigger phasic peaks of 'wanting' to obtain UCS, without altering baseline efforts when CS+ was absent. We conclude that dopamine/opioid stimulation throughout nearly the entire NAc can causally amplify the reactivity of mesocorticolimbic circuits, and so magnify incentive salience or phasic UCS 'wanting' peaks triggered by a CS+. Mesolimbic amplification of incentive salience may explain why a particular cue encounter can become irresistibly tempting, even when previous encounters were successfully resisted before.

Dopamine restores reward prediction errors in old age

Senescence affects the ability to utilize information about the likelihood of rewards for optimal decision-making. Using functional magnetic resonance imaging in humans, we found that healthy older adults had an abnormal signature of expected value, resulting in an incomplete reward prediction error (RPE) signal in the nucleus accumbens, a brain region that receives rich input projections from substantia nigra/ventral tegmental area (SN/VTA) dopaminergic neurons. Structural connectivity between SN/VTA and striatum, measured by diffusion tensor imaging, was tightly coupled to inter-individual differences in the expression of this expected reward value signal. The dopamine precursor levodopa (L-DOPA) increased the task-based learning rate and task performance in some older adults to the level of young adults. This drug effect was linked to restoration of a canonical neural RPE. Our results identify a neurochemical signature underlying abnormal reward processing in older adults and indicate that this can be modulated by L-DOPA.

Repeated cocaine exposure facilitates the expression of incentive motivation and induces habitual control in rats

There is growing evidence that mere exposure to drugs can induce long-term alterations in the neural systems that mediate reward processing, motivation, and behavioral control, potentially causing the pathological pursuit of drugs that characterizes the addicted state. The incentive sensitization theory proposes that drug exposure potentiates the influence of reward-paired cues on behavior. It has also been suggested that drug exposure biases action selection towards the automatic execution of habits and away from more deliberate goal-directed control. The current study investigated whether rats given repeated exposure to peripherally administered cocaine would show alterations in incentive motivation (assayed using the Pavlovian-to-instrumental transfer (PIT) paradigm) or habit formation (assayed using sensitivity to reward devaluation). After instrumental and Pavlovian training for food pellet rewards, rats were given 6 daily injections of cocaine (15 mg/kg, IP) or saline, followed by a 10-d period of rest. Consistent with the incentive sensitization theory, cocaine-treated rats showed stronger cue-evoked lever pressing than saline-treated rats during the PIT test. The same rats were then trained on a new instrumental action with a new food pellet reward before undergoing a reward devaluation testing. Although saline-treated rats exhibited sensitivity to reward devaluation, indicative of goal-directed performance, cocaine-treated rats were insensitive to this treatment, suggesting a reliance on habitual processes. These findings, when taken together, indicate that repeated exposure to cocaine can cause broad alterations in behavioral control, spanning both motivational and action selection processes, and could therefore help explain aberrations of decision-making that underlie drug addiction.

Dopamine encoding of Pavlovian incentive stimuli diminishes with extended training

Dopamine is highly implicated both as a teaching signal in reinforcement learning and in motivating actions to obtain rewards. However, theoretical disconnects remain between the temporal encoding properties of dopamine neurons and the behavioral consequences of its release. Here, we demonstrate in rats that dopamine evoked by Pavlovian cues increases during acquisition, but dissociates from stable conditioned appetitive behavior as this signal returns to preconditioning levels with extended training. Experimental manipulation of the statistical parameters of the behavioral paradigm revealed that this attenuation of cue-evoked dopamine release during the postasymptotic period was attributable to acquired knowledge of the temporal structure of the task. In parallel, conditioned behavior became less dopamine dependent after extended training. Thus, the current work demonstrates that as the presentation of reward-predictive stimuli becomes anticipated through the acquisition of task information, there is a shift in the neurobiological substrates that mediate the motivational properties of these incentive stimuli.

Phasic mesolimbic dopamine release tracks reward seeking during expression of Pavlovian-to-instrumental transfer

BACKGROUND

Recent theories addressing mesolimbic dopamine's role in reward processing emphasize two apparently distinct functions, one in reinforcement learning (i.e., prediction error) and another in incentive motivation (i.e., the invigoration of reward seeking elicited by reward-paired cues). Here, we evaluate the latter.

METHODS

Using fast-scan cyclic voltammetry, we monitored, in real time, dopamine release in the nucleus accumbens core of rats (n = 9) during a Pavlovian-to-instrumental transfer task in which the effects of a reward-predictive cue on an independently trained instrumental action were assessed. Voltammetric data were parsed into slow and phasic components to determine whether these forms of dopamine signaling were differentially related to task performance.

RESULTS

We found that a reward-paired cue, which increased reward-seeking actions, induced an increase in phasic mesolimbic dopamine release and produced slower elevations in extracellular dopamine. Interestingly, phasic dopamine release was temporally related to and positively correlated with lever-press activity generally, while slow dopamine changes were not significantly related to such activity. Importantly, the propensity of the reward-paired cue to increase lever pressing was predicted by the amplitude of phasic dopamine release events, indicating a possible mechanism through which cues initiate reward-seeking actions.

CONCLUSIONS

These data suggest that those phasic mesolimbic dopamine release events thought to signal reward prediction error may also be related to the incentive motivational impact of reward-paired cues on reward-seeking actions.

The relationship between impulsivity and craving in alcohol dependent patients

RATIONALE

Impulsivity and craving are both associated with higher relapse rates and a worse prognosis in patients with a substance use disorder, but the relationship between these two phenomena has been largely ignored in the field of alcohol use disorders.

OBJECTIVES

The primary aim of this study was to investigate the relationship between different dimensions of impulsivity and different forms of self-reported craving. Additionally, the influence of the severity of alcohol dependence on impulsivity, craving, and on their relationship was exploed.

METHODS

Impulsivity and craving levels were investigated in 87 abstinent alcohol-dependent (AD) patients using a broad range of self-report questionnaires and behavioral impulsivity measures. Alcohol use was measured by means of the timeline followback method.

RESULTS

Higher scores of emotional craving (Alcohol Urge Questionnaire-AUQ) were significantly related to higher self-reported impulsivity (Barratt Impulsiveness Scale, version 11) and to higher cognitive impulsivity (information sampling task). Additionally, exploratory analyses suggest that these relationships are more pronounced in severe AD patients compared to less severe AD patients. No significant relationships were found between emotional craving (AUQ) and motor impulsivity (stop signal task) or delay discounting and between obsessive-compulsive craving (Obsessive Compulsive Drinking Scale) and measures of impulsivity.

CONCLUSIONS

Emotional craving is related to self-reported impulsivity and to cognitive impulsivity. These relationships seem to be more pronounced in AD patients with severe alcohol dependence. Further research is needed to explore the effect of this relationship on treatment outcome and relapse.

Instrumental vigour in punishment and reward

Recent notions about the vigour of responding in operant conditioning suggest that the long-run average rate of reward should control the alacrity of action in cases in which the actual cost of speed is balanced against the opportunity cost of sloth. The average reward rate is suggested as being reported by tonic activity in the dopamine system and thereby influencing all actions, including ones that do not themselves lead directly to the rewards. This idea is syntactically problematical for the case of punishment. Here, we broaden the scope of the original suggestion, providing a two-factor analysis of obviated punishment in a variety of operant circumstances. We also consider the effects of stochastically successful actions, which turn out to differ rather markedly between appetitive and aversive cases. Finally, we study how to fit these ideas into nascent treatments that extend concepts of opponency between dopamine and serotonin from valence to invigoration.

From prediction error to incentive salience: mesolimbic computation of reward motivation

Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g. drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus, one consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To understand these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience.

Pawlowsch-Instrumentelle Transfereffekte bei Alkoholabhängigkeit

Hintergrund: Alkoholabhängigkeit ist eine Verkettung ungünstiger Entscheidungen in Bezug auf Alkoholkonsum. Dieses Entscheidungsmuster scheint sich u. a wegen pawlowsch-instrumentellen Transfereffekten (PIT-Effekten) immer wieder zu reproduzieren. Ziel dieser Literaturzusammenschau ist, wichtige Befunde zum Zusammenhang zwischen PIT-Effekten und Suchterkrankungen zusammenzutragen und offene Fragen im Hinblick auf PIT bei Alkoholabhängigkeit aufzuzeigen. Methoden: Die Literaturzusammenschau nutzte keine systematische Literaturrecherche, sondern basierte auf den Recherchen im Rahmen der Forschergruppe 1617 (Learning and Habitization in Alcohol Dependence, LeAD) der Deutschen Forschungsgemeinschaft (DFG). Ergebnisse: PIT-Effekte könnten im Zusammenhang mit Alkoholabhängigkeit möglicherweise zu einem Teufelskreis führen. Dieser besteht aus der Verstärkung von PIT-Effekten durch Alkoholkonsum und verstärktem Alkoholkonsum aufgrund von verstärkten PIT-Effekten. Diskussion: PIT-Effekte bei Alkoholabhängigkeit sind bisher vorwiegend aus Tierstudien bekannt. Das PIT-Paradigma kann uns allerdings auch in der humanen Suchtforschung Aufschluss darüber geben, wie bestimmte Reizmuster Alkoholabhängige zum wiederholten Alkoholkonsum motivieren. Demnach können PIT-Experimente womöglich auch helfen Alkoholrückfälle vorherzusagen.

Twenty-five lessons from computational neuromodulation

Neural processing faces three rather different, and perniciously tied, communication problems. First, computation is radically distributed, yet point-to-point interconnections are limited. Second, the bulk of these connections are semantically uniform, lacking differentiation at their targets that could tag particular sorts of information. Third, the brain's structure is relatively fixed, and yet different sorts of input, forms of processing, and rules for determining the output are appropriate under different, and possibly rapidly changing, conditions. Neuromodulators address these problems by their multifarious and broad distribution, by enjoying specialized receptor types in partially specific anatomical arrangements, and by their ability to mold the activity and sensitivity of neurons and the strength and plasticity of their synapses. Here, I offer a computationally focused review of algorithmic and implementational motifs associated with neuromodulators, using decision making in the face of uncertainty as a running example.

Establishing the dopamine dependency of human striatal signals during reward and punishment reversal learning

Drugs that alter dopamine transmission have opposite effects on reward and punishment learning. These opposite effects have been suggested to depend on dopamine in the striatum. Here, we establish for the first time the neurochemical specificity of such drug effects, during reward and punishment learning in humans, by adopting a coadministration design. Participants (N = 22) were scanned on 4 occasions using functional magnetic resonance imaging, following intake of placebo, bromocriptine (dopamine-receptor agonist), sulpiride (dopamine-receptor antagonist), or a combination of both drugs. A reversal-learning task was employed, in which both unexpected rewards and punishments signaled reversals. Drug effects were stratified with baseline working memory to take into account individual variations in drug response. Sulpiride induced parallel span-dependent changes on striatal blood oxygen level-dependent (BOLD) signal during unexpected rewards and punishments. These drug effects were found to be partially dopamine-dependent, as they were blocked by coadministration with bromocriptine. In contrast, sulpiride elicited opposite effects on behavioral measures of reward and punishment learning. Moreover, sulpiride-induced increases in striatal BOLD signal during both outcomes were associated with behavioral improvement in reward versus punishment learning. These results provide a strong support for current theories, suggesting that drug effects on reward and punishment learning are mediated via striatal dopamine.

Relative response cost determines the sensitivity of instrumental reward seeking to dopamine receptor blockade

Dopamine is a critical mediator of instrumental reward seeking behavior and appears to have a particularly important role in motivating actions that require considerable effort. As with rewards, response costs can be evaluated in both absolute and relative terms. The current study investigated whether the extent to which instrumental performance is dependent on dopamine transmission is influenced by relative or absolute response cost. Three groups of rats were rewarded for lever pressing on different fixed ratio (FR) schedules that required 1 (FR-1), 10 (FR-10), or 20 (FR-20) presses for each food reward. Rats were then injected systemically with flupentixol, a dopamine receptor antagonist, or vehicle before testing all groups on an intermediate-cost (FR-10) schedule, such that only the relative cost of responding differed across groups. Rats experiencing an upshift in cost (group FR-1/FR-10) showed greater response suppression following flupentixol administration than rats experiencing no shift in cost (group FR-10/FR-10), whereas flupentixol treatment had no effect on rats experiencing a downshift in cost (group FR-20/FR-10). A second round of flupentixol tests was conducted using the rats' maintenance schedules, such that only absolute response costs differed across groups. Here, the pattern was reversed among the groups, in line with previous reports. Specifically, flupentixol had a stronger suppressive effect in group FR-20/FR-20 than in group FR-10/FR-10, and had no detectable effect in group FR-1/FR-1. These findings suggest that response costs are evaluated in both absolute and relative terms and that dopamine has a role in overcoming both kinds of cost.

Cognition-emotion dysinteraction in schizophrenia

Evolving theories of schizophrenia emphasize a "disconnection" in distributed fronto-striatal-limbic neural systems, which may give rise to breakdowns in cognition and emotional function. We discuss these diverse domains of function from the perspective of disrupted neural circuits involved in "cold" cognitive vs. "hot" affective operations and the interplay between these processes. We focus on three research areas that highlight cognition-emotion dysinteractions in schizophrenia: First, we discuss the role of cognitive deficits in the "maintenance" of emotional information. We review recent evidence suggesting that motivational abnormalities in schizophrenia may in part arise due to a disrupted ability to "maintain" affective information over time. Here, dysfunction in a prototypical "cold" cognitive operation may result in "affective" deficits in schizophrenia. Second, we discuss abnormalities in the detection and ascription of salience, manifest as excessive processing of non-emotional stimuli and inappropriate distractibility. We review emerging evidence suggesting deficits in some, but not other, specific emotional processes in schizophrenia - namely an intact ability to perceive emotion "in-the-moment" but poor prospective valuation of stimuli and heightened reactivity to stimuli that ought to be filtered. Third, we discuss abnormalities in learning mechanisms that may give rise to delusions, the fixed, false, and often emotionally charged beliefs that accompany psychosis. We highlight the role of affect in aberrant belief formation, mostly ignored by current theoretical models. Together, we attempt to provide a consilient overview for how breakdowns in neural systems underlying affect and cognition in psychosis interact across symptom domains. We conclude with a brief treatment of the neurobiology of schizophrenia and the need to close our explanatory gap between cellular-level hypotheses and complex behavioral symptoms observed in this illness.

Avoidance disruptive effect of clozapine and olanzapine is potentiated by increasing the test trials: further test of the motivational salience hypothesis

Antipsychotic drugs suppress animals' ability to avoid an aversive stimulus in the conditioned avoidance response model (CAR). This behavioral effect is thought to reflect antipsychotic activity and is suggested to be mediated by a drug's action in attenuating the motivational salience of a conditioned stimulus (CS). In the present study, we tested whether atypical antipsychotic drugs clozapine and olanzapine act through this behavioral mechanism by manipulating the number of avoidance test trials. We reasoned that more CS trials in the presence of clozapine or olanzapine would afford the drug more opportunities to decrease the motivational salience of the CS, thus avoidance decline would be greater with the increase of CS trials in each test session. In two separate experiments, adult male Sprague-Dawley rats were tested under clozapine (5.0mg/kg, sc), olanzapine (0.5mg/kg, sc) or vehicle (sterile water) for 6 consecutive days in three CS trial conditions (i.e. 3, 10, and 40 CS trials per session). Two days later, all rats were tested under the same 40-trial session after receiving clozapine (5.0mg/kg, sc) or olanzapine (0.5mg/kg, sc). Results show that repeated clozapine and olanzapine treatment persistently decreased avoidance response, and this effect was potentiated by the increase of number of CS trials in the test sessions, as the clozapine-treated or olanzapine-treated rats tested under the 40-trial or 10-trial condition had significantly lower avoidance and faster decline across-sessions than those tested under the 3-trial condition. This potentiated effect was not only seen in the total avoidance percentage, but also observed in the within-session decline pattern in the last three drug test sessions and in the final 40-trial test session. These findings suggest that the clinical efficacy of a drug can be enhanced by increasing the exposure of symptoms in the presence of the drug.