Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in rats

Anticipation: An Essential Feature of Anhedonia

The following essay addresses the evolution of the term "anhedonia" as a key construct in biological psychiatry, especially as it pertains to positive emotional and motivational states central to mental health and well-being. In its strictest definition, anhedonia was intended to convey an inability to experience "pleasure" derived from ingestion of sweet tastes or the experience of pleasant odors and tactile sensations, among a host of positive sensations. However, this definition has proved to be too restrictive to capture the complexity of key psychological factors linked to major depression, schizophrenia, and substance use disorders it was originally intended to address. Despite the appeal of the elegant simplicity of the term anhedonia, its limitations soon became apparent when used to explain psychological constructs including aspects of learning, memory, and incentive motivation that are major determinants of success in securing the necessities of life. Accordingly, the definition of anhedonia has morphed into a much broader term that includes key roles in the disturbance of motivation in the form of anergia, impaired incentive motivation, along with deficits in associative learning and key aspects of memory, on which the ability to predict the consequences of one's actions are based. Here we argue that it is this latter capacity, namely predicting the likely consequences of motivated behavior, which can be termed "anticipation," that is especially important in the key deficits implied by the general term anhedonia in the context of neuropsychiatric conditions.

Prior chronic alcohol exposure enhances Pavlovian-to-instrumental transfer

Alcohol dependence is associated with aberrant decision-making processes, particularly in the presence of alcohol-related environmental cues. For instance, alcohol cues can trigger alcohol seeking, consumption, and even relapse behavior. Recently, works have suggested that alcohol dependence may induce more general alterations in cued processes that support adaptive behavior, including enhanced cue control of volitional behavior unrelated to alcohol use. Here we examine this hypothesis by combining prior exposure to chronic intermittent ethanol and repeated withdrawal (CIE) procedures with a Pavlovian-to-instrumental transfer (PIT) task in mice. The PIT task entails training a Pavlovian association, separately training an instrumental contingency, and a final test during which the Pavlovian cue and instrumental action are combined for the first time. We first tested two variants of the PIT procedure in ethanol-naïve mice, differing in part in the duration of Pavlovian conditioned cues (short or long). We found in the PIT test that the short cue procedure produced negative transfer, whereas the long cue procedure produced positive transfer. We then used the long cue variant to examine PIT behavior in mice previously exposed to either CIE or air vapor. We found that prior CIE exposure strengthened PIT behavior, with enhanced instrumental responding during presentation of the food-associated cue. We further found that this enhancement in CIE mice persisted even after devaluation of the food outcome. Our findings suggest that ethanol dependence can enhance the influence of reward-predictive cues on ongoing behavior. Greater non-alcohol cue control of behavior may reflect the effect of chronic ethanol exposure on neural circuitry critical for cue-guided behavior in general.

Interfacing behavioral and neural circuit models for habit formation

Habits are an important mechanism by which organisms can automate the control of behavior to alleviate cognitive demand. However, transitions to habitual control are risky because they lead to inflexible responding in the face of change. The question of how the brain controls transitions into habit is thus an intriguing one. How do we regulate when our repeated actions become automated? When is it advantageous or disadvantageous to release actions from cognitive control? Decades of research have identified a variety of methods for eliciting habitual responding in animal models. Progress has also been made to understand which brain areas and neural circuits control transitions into habit. Here, I discuss existing research on behavioral and neural circuit models for habit formation (with an emphasis on striatal circuits), and discuss strategies for combining information from different paradigms and levels of analysis to prompt further progress in the field.

Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico-basal ganglia systems

This article aims to provide a synthesis on the question how brain structures cooperate to accomplish hierarchically organized behaviors, characterized by low‐level, habitual routines nested in larger sequences of planned, goal‐directed behavior. The functioning of a connected set of brain structures—prefrontal cortex, hippocampus, striatum, and dopaminergic mesencephalon—is reviewed in relation to two important distinctions: (a) goal‐directed as opposed to habitual behavior and (b) model‐based and model‐free learning. Recent evidence indicates that the orbitomedial prefrontal cortices not only subserve goal‐directed behavior and model‐based learning, but also code the “landscape” (task space) of behaviorally relevant variables. While the hippocampus stands out for its role in coding and memorizing world state representations, it is argued to function in model‐based learning but is not required for coding of action–outcome contingencies, illustrating that goal‐directed behavior is not congruent with model‐based learning. While the dorsolateral and dorsomedial striatum largely conform to the dichotomy between habitual versus goal‐directed behavior, ventral striatal functions go beyond this distinction. Next, we contextualize findings on coding of reward‐prediction errors by ventral tegmental dopamine neurons to suggest a broader role of mesencephalic dopamine cells, viz. in behavioral reactivity and signaling unexpected sensory changes. We hypothesize that goal‐directed behavior is hierarchically organized in interconnected cortico‐basal ganglia loops, where a limbic‐affective prefrontal‐ventral striatal loop controls action selection in a dorsomedial prefrontal–striatal loop, which in turn regulates activity in sensorimotor‐dorsolateral striatal circuits. This structure for behavioral organization requires alignment with mechanisms for memory formation and consolidation. We propose that frontal corticothalamic circuits form a high‐level loop for memory processing that initiates and temporally organizes nested activities in lower‐level loops, including the hippocampus and the ripple‐associated replay it generates. The evidence on hierarchically organized behavior converges with that on consolidation mechanisms in suggesting a frontal‐to‐caudal directionality in processing control.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

Nucleus accumbens shell moderates preference bias during voluntary choice behavior

The nucleus accumbens (NAc) shell lies anatomically at a critical intersection within the brain’s reward system circuitry, however, its role in voluntary choice behavior remains unclear. Rats with electrolytic lesions in the NAc shell were tested in a novel foraging paradigm. Over a continuous two-week period they freely chose among four nutritionally identical but differently flavored food pellets by pressing corresponding levers. We examined the lesion’s effects on three behavioral dynamics components: motivation (when to eat), preference bias (what to choose) and persistence (how long to repeat the same choice). The lesion led to a marked increase in the preference bias: i.e., increased selection of the most-preferred choice option, and decreased selection of the others. We found no effects on any other behavioral measures, suggesting no effect on motivation or choice persistence. The results implicate the NAc shell in moderating the instrumental valuation process by inhibiting excessive bias toward preferred choice options.

Temporal specificity in Pavlovian-to-instrumental transfer

Presentation of a previously trained Pavlovian conditioned stimulus while an organism is engaged in operant responding can moderate the rate of responding, a phenomenon known as Pavlovian-to-instrumental transfer. Although it is well known that Pavlovian contingencies will generate conditioned behavior that is temporally organized with respect to the arrival of the predicted outcome, little work has examined the temporal dynamics of responding during Pavlovian-instrumental transfer. We trained rats using a fixed time 60-sec, fixed time 120-sec, or random time 60-sec schedule in an appetitive Pavlovian task, and found that presentation of the conditioned stimulus potentiated operant responding in a manner that reflected these previously established temporal expectancies. Further, this temporal specificity conformed to the scalar property as seen with other forms of interval timing behavior. Surprisingly, this effect was only seen when the conditioned stimulus was a visual cue, but not when it was an auditory cue. These data suggest that the motivational processes triggered by Pavlovian cues are not static, but fluctuate in strength as a function of temporally specific expectations of reward.

Reassessing wanting and liking in the study of mesolimbic influence on food intake

Humans and animals such as rats and mice tend to overconsume calorie-dense foods, a phenomenon that likely contributes to obesity. One often-advanced explanation for why we preferentially consume sweet and fatty foods is that they are more "rewarding" than low-calorie foods. "Reward" has been subdivided into three interdependent psychological processes: hedonia (liking a food), reinforcement (formation of associations among stimuli, actions, and/or the food), and motivation (wanting the food). Research into these processes has focused on the mesolimbic system, which comprises both dopamine neurons in the ventral tegmental area and neurons in their major projection target, the nucleus accumbens. The mesolimbic system and closely connected structures are commonly referred to as the brain's "reward circuit." Implicit in this title is the assumption that "rewarding" experiences are generally the result of activity in this circuit. In this review, I argue that food intake and the preference for calorie-dense foods can be explained without reference to subjective emotions. Furthermore, the contribution of mesolimbic dopamine to food intake and preference may not be a general one of promoting or coordinating behaviors that result in the most reward or caloric intake but may instead be limited to the facilitation of a specific form of neural computation that results in conditioned approach behavior. Studies on the neural mechanisms of caloric intake regulation must address how sensory information about calorie intake affects not just the mesolimbic system but also many other forms of computation that govern other types of food-seeking and food-oriented behaviors.

Goals and task difficulty expectations modulate striatal responses to feedback

The striatum plays a critical role in learning from reward, and it has been implicated in learning from performance-related feedback as well. Positive and negative performance-related feedback is known to engage the striatum during learning by eliciting a response similar to the reinforcement signal for extrinsic rewards and punishments. Feedback is an important tool used to teach new skills and promote healthful lifestyle changes, so it is important to understand how motivational contexts can modulate its effectiveness at promoting learning. While it is known that striatal responses scale with subjective factors influencing the desirability of rewards, it is less clear how expectations and goals might modulate the striatal responses to cognitive feedback during learning. We used functional magnetic resonance imaging to investigate the effects of task difficulty expectations and achievement goals on feedback processing during learning. We found that individuals who scored high in normative goals, which reflect a desire to outperform other students academically, showed the strongest effects of our manipulation. High levels of normative goals were associated with greater performance gains and exaggerated striatal sensitivity to positive versus negative feedback during blocks that were expected to be more difficult. Our findings suggest that normative goals may enhance performance when difficulty expectations are high, while at the same time modulating the subjective value of feedback as processed in the striatum.

Learning and Motivational Processes Contributing to Pavlovian-Instrumental Transfer and Their Neural Bases: Dopamine and Beyond

Pavlovian stimuli exert a range of effects on behavior from simple conditioned reflexes, such as salivation, to altering the vigor and direction of instrumental actions. It is currently accepted that these distinct behavioral effects stem from two sources (i) the various associative connections between predictive stimuli and the component features of the events that these stimuli predict and (ii) the distinct motivational and cognitive functions served by cues, particularly their arousing and informational effects on the selection and performance of specific actions. Here, we describe studies that have assessed these latter phenomena using a paradigm that has come to be called Pavlovian-instrumental transfer. We focus first on behavioral experiments that have described distinct sources of stimulus control derived from the general affective and outcome-specific predictions of conditioned stimuli, referred to as general transfer and specific transfer, respectively. Subsequently, we describe research efforts attempting to establish the neural bases of these transfer effects, largely in the afferent and efferent connections of the nucleus accumbens (NAc) core and shell. Finally, we examine the role of predictive cues in examples of aberrant stimulus control associated with psychiatric disorders and addiction.

Value and probability coding in a feedback-based learning task utilizing food rewards

For the consequences of our actions to guide behavior, the brain must represent different types of outcome-related information. For example, an outcome can be construed as negative because an expected reward was not delivered or because an outcome of low value was delivered. Thus behavioral consequences can differ in terms of the information they provide about outcome probability and value. We investigated the role of the striatum in processing probability-based and value-based negative feedback by training participants to associate cues with food rewards and then employing a selective satiety procedure to devalue one food outcome. Using functional magnetic resonance imaging, we examined brain activity related to receipt of expected rewards, receipt of devalued outcomes, omission of expected rewards, omission of devalued outcomes, and expected omissions of an outcome. Nucleus accumbens activation was greater for rewarding outcomes than devalued outcomes, but activity in this region did not correlate with the probability of reward receipt. Activation of the right caudate and putamen, however, was largest in response to rewarding outcomes relative to expected omissions of reward. The dorsal striatum (caudate and putamen) at the time of feedback also showed a parametric increase correlating with the trialwise probability of reward receipt. Our results suggest that the ventral striatum is sensitive to the motivational relevance, or subjective value, of the outcome, while the dorsal striatum codes for a more complex signal that incorporates reward probability. Value and probability information may be integrated in the dorsal striatum, to facilitate action planning and allocation of effort.

Internally generated sequences in learning and executing goal-directed behavior

A network of brain structures including hippocampus (HC), prefrontal cortex, and striatum controls goal-directed behavior and decision making. However, the neural mechanisms underlying these functions are unknown. Here, we review the role of 'internally generated sequences': structured, multi-neuron firing patterns in the network that are not confined to signaling the current state or location of an agent, but are generated on the basis of internal brain dynamics. Neurophysiological studies suggest that such sequences fulfill functions in memory consolidation, augmentation of representations, internal simulation, and recombination of acquired information. Using computational modeling, we propose that internally generated sequences may be productively considered a component of goal-directed decision systems, implementing a sampling-based inference engine that optimizes goal acquisition at multiple timescales of on-line choice, action control, and learning.

ADHD, altered dopamine neurotransmission, and disrupted reinforcement processes: implications for smoking and nicotine dependence

Attention deficit hyperactivity disorder (ADHD) is a common and impairing disorder affecting millions of children, adolescents, and adults. Individuals with ADHD smoke cigarettes at rates significantly higher than their non-diagnosed peers and the disorder also confers risk for a number of related adverse smoking outcomes including earlier age of initiation, faster progression to regular use, heavier smoking/greater dependence, and more difficulty quitting. Progress in our understanding of dopamine neurotransmission and basic behavioral reinforcement processes in ADHD may help increase our understanding of the ADHD-smoking comorbidity. This review will examine how these areas have been studied and how further work may aid in the development of better prevention and treatment for smoking in those with ADHD.

The motivational drive to natural rewards is modulated by prenatal glucocorticoid exposure

Exposure to elevated levels of glucocorticoids (GCs) during neurodevelopment has been identified as a triggering factor for the development of reward-associated disorders in adulthood. Disturbances in the neural networks responsible for the complex processes that assign value to rewards and associated stimuli are critical for disorders such as depression, obsessive–compulsive disorders, obesity and addiction. Essential in the understanding on how cues influence behavior is the Pavlovian–instrumental transfer (PIT), a phenomenon that refers to the capacity of a Pavlovian stimulus that predicts a reward to elicit instrumental responses for that same reward. Here, we demonstrate that in utero exposure to GCs (iuGC) impairs both general and selective versions of the PIT paradigm, suggestive of deficits in motivational drive. The iuGC animals presented impaired neuronal activation pattern upon PIT performance in cortical and limbic regions, as well as morphometric changes and reduced levels of dopamine in prefrontal and orbitofrontal cortices, key regions involved in the integration of Pavlovian and instrumental stimuli. Normalization of dopamine levels rescued this behavior, a process that relied on D2/D3, but not D1, dopamine receptor activation. In summary, iuGC exposure programs the mesocorticolimbic dopaminergic circuitry, leading to a reduction in the attribution of the incentive salience to cues, in a dopamine-D2/D3-dependent manner. Ultimately, these results are important to understand how GCs bias incentive processes, a fact that is particularly relevant for disorders where differential attribution of incentive salience is critical.

Acquisition of responding with a remifentanil-associated conditioned reinforcer in the rat

RATIONALE

Drug-associated environmental stimuli may serve as conditioned reinforcers to enhance drug self-administration behaviors in humans and laboratory animals. However, it can be difficult to distinguish experimentally the conditioned reinforcing effects of a stimulus from other behavioral processes that can change rates of responding.

OBJECTIVES

To characterize the conditioned reinforcing effects of a stimulus paired with the μ-opioid agonist, remifentanil, using a new-response acquisition procedure in the rat.

METHODS

First, in Pavlovian conditioning (PAV) sessions, rats received response-independent IV injections of remifentanil and presentations of a light-noise compound stimulus. In paired PAV groups, injections and stimulus presentations always co-occurred. In random PAV control groups, injections and stimulus presentations occurred with no consistent relationship. Second, in instrumental acquisition (ACQ) sessions, all animals could respond in an active nose-poke that produced the stimulus alone or in an inactive nose-poke that had no scheduled consequences.

RESULTS

During ACQ, rats made significantly more active nose-pokes than inactive nose-pokes after paired PAV, but not after random PAV. Between groups, rats also made more active nose-pokes after paired PAV than after random PAV. After paired PAV, increased active responding was obtained under different schedules of reinforcement, persisted across multiple ACQ sessions, and depended on the number of PAV sessions conducted.

CONCLUSIONS

The remifentanil-paired stimulus served as a conditioned reinforcer for nose-poking: responding depended on both the contingency between the stimulus and remifentanil and the contingency between the nose-poke and the stimulus. Generally, new-response acquisition procedures may provide valid, flexible models for studying opioid-based conditioned reinforcement.

Influence of palatability on motivation to operate for caloric and non-caloric food in non food-deprived and food-deprived rats

Palatability is the hedonic food component that is considered to override the homeostatic mechanisms that control food intake, and we compared how much effort non food-deprived and food-deprived rats were willing to spend in order to earn a palatable caloric (sucrose) or non-caloric (saccharin) snack. We first studied the dopaminergic response, in terms of dopamine levels and dopamine and cAMP-regulated phosphoprotein Mr 32,000 (DARPP-32) phosphorylation pattern, to two consecutive palatable caloric or non-caloric snacks in the nucleus accumbens shell (NAcS) of non food-deprived and fasted rats. We report that non food-deprived rats developed rapid habituation in the NAcS dopaminergic response to the second consumption of both caloric and non-caloric palatable food, while food-deprived rats developed rapid habituation only to saccharin. Next, we show that in self-administration experiments, non food-deprived rats spent a similar effort when operating for sucrose or saccharin. However, the same rats showed an increased response specifically for sucrose after 18-h fasting. After pre-feeding devaluation, rats reduced their response to sucrose but not for saccharin. These results strengthen the hypothesis that food intake is mainly controlled by palatability in non food-deprived rats and by caloric content in food-deprived rats. Moreover, they show that rapid habituation development was associated with a similar, basal working activity aimed at ingesting both caloric and non-caloric food, as observed in non food-deprived rats consuming sucrose or saccharin and in fasted rats consuming saccharin. Conversely, lack of habituation, as present in fasted rats consuming a caloric food, was associated with extra energy expenditure.

Context, emotion, and the strategic pursuit of goals: interactions among multiple brain systems controlling motivated behavior

Motivated behavior exhibits properties that change with experience and partially dissociate among a number of brain structures. Here, we review evidence from rodent experiments demonstrating that multiple brain systems acquire information in parallel and either cooperate or compete for behavioral control. We propose a conceptual model of systems interaction wherein a ventral emotional memory network involving ventral striatum (VS), amygdala, ventral hippocampus, and ventromedial prefrontal cortex triages behavioral responding to stimuli according to their associated affective outcomes. This system engages autonomic and postural responding (avoiding, ignoring, approaching) in accordance with associated stimulus valence (negative, neutral, positive), but does not engage particular operant responses. Rather, this emotional system suppresses or invigorates actions that are selected through competition between goal-directed control involving dorsomedial striatum (DMS) and habitual control involving dorsolateral striatum (DLS). The hippocampus provides contextual specificity to the emotional system, and provides an information rich input to the goal-directed system for navigation and discriminations involving ambiguous contexts, complex sensory configurations, or temporal ordering. The rapid acquisition and high capacity for episodic associations in the emotional system may unburden the more complex goal-directed system and reduce interference in the habit system from processing contingencies of neutral stimuli. Interactions among these systems likely involve inhibitory mechanisms and neuromodulation in the striatum to form a dominant response strategy. Innate traits, training methods, and task demands contribute to the nature of these interactions, which can include incidental learning in non-dominant systems. Addition of these features to reinforcement learning models of decision-making may better align theoretical predictions with behavioral and neural correlates in animals.

Stress Transiently Affects Pavlovian-to-Instrumental Transfer

Stress has a strong impact in the brain, impairing decision-making processes as a result of changes in circuits involving the prefrontal and orbitofrontal cortices and the striatum. Given that these same circuits are key for action control and outcome encoding, we hypothesized that adaptive responses to which these are essential functions, could also be targeted by stress. To test this hypothesis we herein assessed the impact of chronic stress in a Pavlovian-to-instrumental transfer (PIT) paradigm, a model of an adaptive response in which a previously conditioned cue biases an instrumental goal-directed action. Data reveals that rats submitted to chronic unpredictable stress did not display deficits in pavlovian conditioning nor on the learning of the instrumental task, but were impaired in PIT; importantly, after a stress-free period the PIT deficits were no longer observed. These results are relevant to understand how stress biases multiple incentive processes that contribute to instrumental performance.

Model-based learning and the contribution of the orbitofrontal cortex to the model-free world

Learning is proposed to occur when there is a discrepancy between reward prediction and reward receipt. At least two separate systems are thought to exist: one in which predictions are proposed to be based on model-free or cached values; and another in which predictions are model-based. A basic neural circuit for model-free reinforcement learning has already been described. In the model-free circuit the ventral striatum (VS) is thought to supply a common-currency reward prediction to midbrain dopamine neurons that compute prediction errors and drive learning. In a model-based system, predictions can include more information about an expected reward, such as its sensory attributes or current, unique value. This detailed prediction allows for both behavioral flexibility and learning driven by changes in sensory features of rewards alone. Recent evidence from animal learning and human imaging suggests that, in addition to model-free information, the VS also signals model-based information. Further, there is evidence that the orbitofrontal cortex (OFC) signals model-based information. Here we review these data and suggest that the OFC provides model-based information to this traditional model-free circuitry and offer possibilities as to how this interaction might occur.

Changes in nucleus accumbens and neostriatal c-Fos and DARPP-32 immunoreactivity during different stages of food-reinforced instrumental training

Nucleus accumbens is involved in several aspects of instrumental behavior, motivation and learning. Recent studies showed that dopamine (DA) release in the accumbens shell was significantly increased on the first day of training on a fixed ratio (FR) 5 schedule (i.e. the transition from FR1 to FR5) compared with those rats that continued FR1 training, even though the rats on their first day of FR5 training received less food reinforcement than rats continuing on the FR1 schedule. Additionally, the second day of FR5 responding was marked by a significant increase in DA release in accumbens core. The present studies employed immunohistochemical methods to characterize the changes in cellular markers of accumbens and neostriatal neural activity that occur during various stages of food-reinforced FR5 training. c-Fos and DARPP-32 immunoreactivity in accumbens shell was significantly increased on the first day of FR5 training, while core c-Fos and DARPP-32 expression showed large increases on the second day of FR5 training. Additional studies showed that c-Fos and DARPP-32 expression in neostriatum increased after more extensive training. Double-labeling studies with immunofluorescence methods indicated that increases in accumbens c-Fos and DARPP-32 expression were primarily seen in substance-P-positive neurons. These increases in accumbens c-Fos and DARPP-32 immunoreactivity seen during the initial phases of FR training may reflect several factors, including novelty, learning, stress or the presentation of a work-related challenge to the organism. Moreover, it appears that the separate subregions of the striatal complex are differentially activated at distinct phases of instrumental training.

Roles of nucleus accumbens and basolateral amygdala in autoshaped lever pressing

Initially-neutral cues paired with rewards are thought to acquire motivational significance, as if the incentive motivational value of the reward is transferred to the cue. Such cues may serve as secondary reinforcers to establish new learning, modulate the performance of instrumental action (Pavlovian-instrumental transfer, PIT), and be the targets of approach and other cue-directed behaviors. Here we examined the effects of lesions of the ventral striatal nucleus accumbens (ACb) and the basolateral amygdala (BLA) on the acquisition of discriminative autoshaped lever-pressing in rats. Insertion of one lever into the experimental chamber was reinforced by sucrose delivery, but insertion of another lever was not reinforced. Although sucrose was delivered independently of the rats' behavior, sham-lesioned rats rapidly came to press the reinforced but not the nonreinforced lever. Bilateral ACb lesions impaired the initial acquisition of sign-tracking but not its terminal levels. In contrast, BLA lesions produced substantial deficits in terminal levels of sign-tracking. Furthermore, whereas ACb lesions primarily affected the probability of lever press responses, BLA lesions mostly affected the rate of responding once it occurred. Finally, disconnection lesions that disrupted communication between ACb and BLA produced both sets of deficits. We suggest that ACb is important for initial acquisition of consummatory-like responses that incorporate hedonic aspects of the reward, while BLA serves to enhance such incentive salience once it is acquired.

The neural basis of drug stimulus processing and craving: an activation likelihood estimation meta-analysis

BACKGROUND

The capacity of drug cues to elicit drug-seeking behavior is believed to play a fundamental role in drug dependence; yet the neurofunctional basis of human drug cue-reactivity is not fully understood. We performed a meta-analysis to identify brain regions that are consistently activated by presentation of drug cues. Studies involving treatment-seeking and nontreatment-seeking substance users were contrasted to determine whether there were consistent differences in the neural response to drug cues between these populations. Finally, to assess the neural basis of craving, consistency across studies in brain regions that show correlated activation with craving was assessed.

METHODS

Appropriate studies, assessing the effect of drug-related cues or manipulations of drug craving in drug-user populations across the whole brain, were obtained via the PubMed database and literature search. Activation likelihood estimation, a method of quantitative meta-analysis that estimates convergence across experiments by modeling the spatial uncertainty of neuroimaging data, was used to identify consistent regions of activation.

RESULTS

Cue-related activation was observed in the ventral striatum (across both subgroups), amygdala (in the treatment-seeking subgroup and overall), and orbitofrontal cortex (in the nontreatment-seeking subgroup and overall) but not insula cortex. Although a different pattern of frontal and temporal lobe activation between the subgroups was observed, these differences were not significant. Finally, right amygdala and left middle frontal gyrus activity were positively associated with craving.

CONCLUSIONS

These results substantiate the key neural substrates underlying reactivity to drug cues and drug craving.

Dorsal and ventral striatal protein synthesis inhibition affect reinforcer valuation but not the consolidation of instrumental learning

The evidence for a role of the striatum in the acquisition of uncued instrumental responding is ambiguous. It has been shown that post-session infusions of anisomycin into the core of the nucleus accumbens (NAcc) impaired instrumental acquisition, but pre-training lesions of the NAcc suggest that it is not necessary. Recently, we demonstrated that the infusion of anisomycin into the anterior cingulate cortex impaired instrumental acquisition indirectly through a taste aversion. Thus, we hypothesized that post-session anisomycin infusions into the NAcc affected instrumental acquisition through an effect on reinforcer valuation. For the dorsal striatum, both post-session infusions of anisomycin and pre-training lesion studies suggest that neither the dorsolateral nor the dorsomedial striatum is necessary for the acquisition of instrumental responding. However, it has not been attempted to block plasticity in both regions concurrently, and we hypothesized that both regions independently contribute to acquisition through goal-directed and habitual learning. In the current experiments, we first replicated the effect of unprotected post-session anisomycin infusions into the NAcc on instrumental acquisition. Subsequently, we investigated the effect of protein synthesis inhibition in the NAcc and dorsomedial and dorsolateral striatum concurrently on instrumental acquisition, critically controlling for effects on reinforcer valuation. The anisomycin infusions induced an aversive state, but did not affect instrumental acquisition.

Neural correlates of Pavlovian-to-instrumental transfer in the nucleus accumbens shell are selectively potentiated following cocaine self-administration

During Pavlovian-to-instrumental transfer (PIT), learned Pavlovian cues significantly modulate ongoing instrumental actions. This phenomenon is suggested as a mechanism under which conditioned stimuli may lead to relapse in addicted populations. Following discriminative Pavlovian learning and instrumental conditioning with sucrose, one group of rats (naive) underwent electrophysiological recordings in the nucleus accumbens core and shell during a single PIT session. Other groups, following Pavlovian and instrumental conditioning, were subsequently trained to self-administer cocaine with nosepoke responses, or received yoked saline infusions and nosepoked for water rewards, and then performed PIT while electrophysiological recordings were taken in the nucleus accumbens. Behaviorally, although both naive and saline-treated groups showed increases in lever pressing during the conditioned stimulus cue, this effect was significantly enhanced in the cocaine-treated group. Neurons in the core and shell tracked these behavioral changes. In control animals, core neurons were significantly more likely to encode general information about cues, rewards and responses than those in the shell, and positively correlated with behavioral PIT performance, whereas PIT-specific encoding in the shell, but not core, tracked PIT performance. In contrast, following cocaine exposure, there was a significant increase in neural encoding of all task-relevant events that was selective to the shell. Given that cocaine exposure enhanced both behavior and shell-specific task encoding, these findings suggest that, whereas the core is important for acquiring the information about cues and response contingencies, the shell is important for using this information to guide and modulate behavior and is specifically affected following a history of cocaine self-administration.

Ventral striatum: a critical look at models of learning and evaluation

Extensive evidence implicates the ventral striatum in multiple distinct facets of action selection. Early work established a role in modulating ongoing behavior, as engaged by the energizing and directing influences of motivationally relevant cues and the willingness to expend effort in order to obtain reward. More recently, reinforcement learning models have suggested the notion of ventral striatum primarily as an evaluation step during learning, which serves as a critic to update a separate actor. Recent computational and experimental work may provide a resolution to the differences between these two theories through a careful parsing of behavior and the instrinsic heterogeneity that characterizes this complex structure.

Measuring Pavlovian fear with conditioned freezing and conditioned suppression reveals different roles for the basolateral amygdala

In Pavlovian fear conditioning, pairing a neutral cue with aversive foot shock endows a cue with fear-eliciting properties. Studies of Pavlovian fear conditioning measuring freezing have demonstrated the basolateral amygdala (BLA) to be critical to both fear learning and memory. The nucleus accumbens core (NAc), while not important to freezing, is important to the enhancement of instrumental responding by cues paired with food reward. In the present study we investigated the role of the BLA and the NAc in another property of fear cues, the ability to suppress instrumental responding for food rewards (conditioned suppression). Sham, BLA and NAc-lesioned rats received a fear discrimination procedure in which one visual cue (CS+) predicted foot shock while a second cue (CS-) did not. Conditioning took place over a baseline of instrumental responding, allowing for concurrent measure of freezing and instrumental suppression. NAc lesions left fear conditioning fully intact. BLA lesions impaired acquisition and discrimination of fear when assessed with conditioned freezing. However, BLA lesions only altered fear acquisition and left discrimination completely intact when assessed with conditioned suppression. These findings suggest a critical role for the BLA in fear when assessed with conditioned freezing but a diminished role when assessed with conditioned suppression.

Nucleus Accumbens Core and Shell are Necessary for Reinforcer Devaluation Effects on Pavlovian Conditioned Responding

The nucleus accumbens (NA) has been hypothesized to be part of a circuit in which cue-evoked information about expected outcomes is mobilized to guide behavior. Here we tested this hypothesis using a Pavlovian reinforcer devaluation task, previously applied to assess outcome-guided behavior after damage to regions such as the orbitofrontal cortex and amygdala that send projections to NA. Rats with sham lesions or neurotoxic lesions of either the core or shell subdivision of NA were trained to associate a 10-s CS+ with delivery of three food pellets. After training, half of the rats in each lesion group received food paired with illness induced by LiCl injections; the remaining rats received food and illness unpaired. Subsequently, responding to the CS+ was assessed in an extinction probe test. Both sham and lesioned rats conditioned to the CS+ and formed a conditioned taste aversion. However only sham rats reduced their conditioned responding as a result of reinforcer devaluation; devalued rats with lesions of either core or shell showed levels of responding that were similar to lesioned, non-devalued rats. This impairment was not due to the loss of motivational salience conferred to the CS+ in lesioned rats as both groups responded similarly for the cue in conditioned reinforcement testing. These data suggest that NA core and shell are part of a circuit necessary for the use of cue-evoked information about expected outcomes to guide behavior.

“Virus and Epidemic”: Causal Knowledge Activates Prediction Error Circuitry

Knowledge about cause and effect relationships (e.g., virus–epidemic) is essential for predicting changes in the environment and for anticipating the consequences of events and one's own actions. Although there is evidence that predictions and learning from prediction errors are instrumental in acquiring causal knowledge, it is unclear whether prediction error circuitry remains involved in the mental representation and evaluation of causal knowledge already stored in semantic memory. In an fMRI study, participants assessed whether pairs of words were causally related (e.g., virus–epidemic) or noncausally associated (e.g., emerald–ring). In a second fMRI study, a task cue prompted the participants to evaluate either the causal or the noncausal associative relationship between pairs of words. Causally related pairs elicited higher activity in OFC, amygdala, striatum, and substantia nigra/ventral tegmental area than noncausally associated pairs. These regions were also more activated by the causal than by the associative task cue. This network overlaps with the mesolimbic and mesocortical dopaminergic network known to code prediction errors, suggesting that prediction error processing might participate in assessments of causality even under conditions when it is not explicitly required to make predictions.

External incentives and internal states guide goal-directed behavior via the differential recruitment of the nucleus accumbens and the medial prefrontal cortex

Goal-directed behavior is governed by internal physiological states and external incentives present in the environment (e.g. hunger and food). While the role of the mesocorticolimbic dopamine (DA) system in behavior guided by environmental incentives has been well studied, the effect of relevant physiological states on the function of this system is less understood. The current study examined the role of the medial prefrontal cortex (mPFC) and the nucleus accumbens (NAcc) in the kind of food-reinforced behaviors known to be sensitive to the internal state produced by food deprivation conditions. Operant lever-press reinforced on fixed ratio 1 (FR1) and progressive ratio (PR) schedules was tested after temporary inactivation of, or DA receptor blockade in, the prelimbic mPFC or NAcc core of rats with differing levels of food deprivation (0, 12 and 36-h). Food deprivation increased PR breakpoints, as well as the number of lever-presses emitted on the FR1 schedule. Both temporary inactivation and DA blockade of NAcc reduced breakpoints across deprivation conditions, while temporary inactivation and DA blockade of mPFC reduced breakpoints only in food-deprived rats. Neither manipulation of mPFC and NAcc had any effect on behavior reinforced on the FR1 schedule. Thus, mPFC and NAcc were differentially relevant to the behaviors tested-NAcc was recruited when the behavioral cost per reinforcer was rising or high regardless of food deprivation conditions, while mPFC was recruited when food-deprived animals behaved through periods of sparse reinforcement density in order to maximize available gain.

All that glitters ... dissociating attention and outcome expectancy from prediction errors signals

Initially reported in dopamine neurons, neural correlates of prediction errors have now been shown in a variety of areas, including orbitofrontal cortex, ventral striatum, and amygdala. Yet changes in neural activity to an outcome or cues that precede it can reflect other processes. We review the recent literature and show that although activity in dopamine neurons appears to signal prediction errors, similar activity in orbitofrontal cortex, basolateral amygdala, and ventral striatum does not. Instead, increased firing in basolateral amygdala to unexpected outcomes likely reflects attention, whereas activity in orbitofrontal cortex and ventral striatum is unaffected by prior expectations and may provide information on outcome expectancy. These results have important implications for how these areas interact to facilitate learning and guide behavior.

The psychological and neurochemical mechanisms of drug memory reconsolidation: implications for the treatment of addiction

Memory reconsolidation is the process by which memories, destabilised at retrieval, require restabilisation to persist in the brain. It has been demonstrated that even old, well-established memories require reconsolidation following retrieval; therefore, memory reconsolidation could potentially be exploited to disrupt, or even erase, aberrant memories that underlie psychiatric disorders, thereby providing a novel therapeutic target. Drug addiction is one such disorder; it is both chronic and relapsing, and one prominent risk factor for a relapse episode is the presentation of environmental cues that have previously been associated with drugs of abuse. This 'cue-induced relapse' can be accounted for in psychological terms by reinforcing memories of the pavlovian association between the cue and the drug, which can thus influence behaviour through at least three psychologically and neurobiologically dissociable mechanisms: conditioned reinforcement, conditioned approach and conditioned motivation. As each of these psychological processes could contribute to the resumption of drug-seeking following abstinence, it is important to develop treatments that can reduce drug-seeking re-established via influences on each or all of these pavlovian processes, in order to minimise the risk of a subsequent relapse. Investigation of the memory reconsolidation mechanisms of the memories underlying conditioned reinforcement, conditioned approach and conditioned motivation indicate that they depend upon different neurochemical systems, including the glutamatergic and adrenergic systems within limbic corticostriatal circuitry. We also discuss here the subsequent translation to the clinic of this preclinical work.

Expectancies in decision making, reinforcement learning, and ventral striatum

Decisions can arise in different ways, such as from a gut feeling, doing what worked last time, or planful deliberation. Different decision-making systems are dissociable behaviorally, map onto distinct brain systems, and have different computational demands. For instance, “model-free” decision strategies use prediction errors to estimate scalar action values from previous experience, while “model-based” strategies leverage internal forward models to generate and evaluate potentially rich outcome expectancies. Animal learning studies indicate that expectancies may arise from different sources, including not only forward models but also Pavlovian associations, and the flexibility with which such representations impact behavior may depend on how they are generated. In the light of these considerations, we review the results of van der Meer and Redish (2009a), who found that ventral striatal neurons that respond to reward delivery can also be activated at other points, notably at a decision point where hippocampal forward representations were also observed. These data suggest the possibility that ventral striatal reward representations contribute to model-based expectancies used in deliberative decision making.

Pavlovian to instrumental transfer: a neurobehavioural perspective

Pavlovian-to-instrumental transfer (PIT) is a key concept in developing our understanding of cue-controlled behaviours. Here we have reviewed the literature on behavioural and neurobiological factors that influence PIT. Meta-analyses of the data for individual groups in PIT studies revealed that PIT is related to both the order and amounts of instrumental and Pavlovian training, and that it is critically determined by competition between instrumental and Pavlovian responses. We directly addressed the role of response competition in PIT in two experiments which showed that extensive Pavlovian conditioning produced more Pavlovian magazine visits and weaker PIT than moderate Pavlovian conditioning (Experiment 1); and that PIT lost after extensive Pavlovian conditioning was restored by Pavlovian extinction training (Experiment 2). These findings confirm that response competition is indeed an important determinant of PIT. This has significant implications for lesion and inactivation studies that assess the neurobiological substrates of PIT, as well as attempts to demonstrate PIT in the drug self-administration paradigm where the effect is yet to be reliably shown.

How do you (estimate you will) like them apples? Integration as a defining trait of orbitofrontal function

The past 15 years have seen a rapid increase in our understanding of orbitofrontal function. Today this region is the focus of an enormous amount of research, including work on such complex phenomena as regret, ambiguity, and willingness to pay. The orbitofrontal cortex is also credited as a major player in a host of neuropsychiatric diseases. This transformation arguably began with the application of concepts derived from animal learning theory. We will review data from studies emphasizing these approaches to argue that the orbitofrontal cortex forms a crucial part of a network of structures that signals information about expected outcomes. Further we will suggest that, within this network, the orbitofrontal cortex provides the critical ability to integrate information in real-time to make what amounts to actionable predictions or estimates about future outcomes. As we will show, the influence of these estimates can be demonstrated experimentally in appropriate behavioral settings, and their operation can also readily explain the role of orbitofrontal cortex in much more complex phenomena such as those cited above.

Functional interaction of medial mediodorsal thalamic nucleus but not nucleus accumbens with amygdala and orbital prefrontal cortex is essential for adaptive response selection after reinforcer devaluation

In nonhuman primates, reward-based decision making may be assessed through choices of objects overlying two different foods, one of which has been devalued by selective satiation. The most adaptive object choices yield the food of higher value. A large body of data identifies the amygdala and orbital prefrontal cortex (PFo) as neural mediators of adaptive responses to reinforcer devaluation. More recent work in nonhuman primates reveals the critical role of the medial, magnocellular portion of the mediodorsal nucleus of the thalamus (MDm) as well. Because both the nucleus accumbens (NA) and the MDm are anatomically related to the amygdala and PFo, and because both regions are implicated in reward processing, we tested whether either region necessarily interacts with the amygdala and PFo to mediate reinforcer devaluation effects. We used a crossed-disconnection design in which monkeys received amygdala and PFo lesions in one hemisphere combined with either NA or MDm lesions in the contralateral hemisphere. Monkeys that sustained NA disconnection, like controls, showed robust shifts in object choices in response to reinforcer devaluation. In contrast, monkeys that sustained MDm disconnection failed to adjust their object choices. Thus, MDm, but not NA, works together with the amygdala and PFo to support reward-based decision making.

Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action

Recent behavioral studies in both humans and rodents have found evidence that performance in decision-making tasks depends on two different learning processes; one encoding the relationship between actions and their consequences and a second involving the formation of stimulus-response associations. These learning processes are thought to govern goal-directed and habitual actions, respectively, and have been found to depend on homologous corticostriatal networks in these species. Thus, recent research using comparable behavioral tasks in both humans and rats has implicated homologous regions of cortex (medial prefrontal cortex/medial orbital cortex in humans and prelimbic cortex in rats) and of dorsal striatum (anterior caudate in humans and dorsomedial striatum in rats) in goal-directed action and in the control of habitual actions (posterior lateral putamen in humans and dorsolateral striatum in rats). These learning processes have been argued to be antagonistic or competing because their control over performance appears to be all or none. Nevertheless, evidence has started to accumulate suggesting that they may at times compete and at others cooperate in the selection and subsequent evaluation of actions necessary for normal choice performance. It appears likely that cooperation or competition between these sources of action control depends not only on local interactions in dorsal striatum but also on the cortico-basal ganglia network within which the striatum is embedded and that mediates the integration of learning with basic motivational and emotional processes. The neural basis of the integration of learning and motivation in choice and decision-making is still controversial and we review some recent hypotheses relating to this issue.

The role of nucleus accumbens dopamine in outcome encoding in instrumental and Pavlovian conditioning

Considerable evidence suggests that dopamine in the core subregion of the nucleus accumbens is not only involved in Pavlovian conditioning but also supports instrumental performance. However, it is largely unknown whether NAc dopamine is required for outcome encoding which plays an important role both in Pavlovian stimulus-outcome learning and instrumental action-outcome learning. Therefore, we tested rats with 6-hydroxydopamine (6-OHDA) induced dopamine depletion of the NAc core for their sensitivity to outcome devaluation in a Pavlovian and an instrumental task. Results indicate that 6-OHDA-lesioned animals were sensitive to outcome devaluation in an instrumental task. This finding provides support to the notion that NAc core dopamine may not be crucial in encoding action-outcome associations. However, during instrumental conditioning lever pressing rates in 6-OHDA-lesioned animals were markedly lower which could reflect an impaired behavioral activation. By contrast, after outcome-specific devaluation in a Pavlovian task, performance in 6-OHDA-lesioned animals was impaired, i.e. their magazine-directed responding was non-selectively reduced. One possibility to explain non-selective responding is that NAc core DA depletion impaired the ability of conditioned stimuli to activate the memory of the current value of the reinforcer.

Regional specificity in the real-time development of phasic dopamine transmission patterns during acquisition of a cue-cocaine association in rats

Drug seeking is significantly regulated by drug-associated cues and associative learning between environmental cues and cocaine reward is mediated by dopamine transmission within the nucleus accumbens (NAc). However, dopamine transmission during early acquisition of a cue-cocaine association has never been assessed because of the technical difficulties associated with resolving cue-evoked and cocaine-evoked dopamine release within the same conditioning trial. Here, we used fast-scan cyclic voltammetry to measure sub-second fluctuations in dopamine concentration within the NAc core and shell during the initial acquisition of a cue-cocaine Pavlovian association. Within the NAc core, cue-evoked dopamine release developed during conditioning. However, within the NAc shell, the predictive cue appeared to cause an unconditioned decrease in dopamine concentration. The pharmacological effects of cocaine also differed between sub-regions, as cocaine increased phasic dopamine release events within the NAc shell but not the core. Thus, real-time measurements not only revealed the initial development of a conditioned neurochemical response but also demonstrated differential phasic dopamine transmission patterns across NAc sub-regions during the acquisition of a cue-cocaine association.

Cognitive control of drug craving inhibits brain reward regions in cocaine abusers

Loss of control over drug taking is considered a hallmark of addiction and is critical in relapse. Dysfunction of frontal brain regions involved with inhibitory control may underlie this behavior. We evaluated whether addicted subjects when instructed to purposefully control their craving responses to drug-conditioned stimuli can inhibit limbic brain regions implicated in drug craving. We used PET and 2-deoxy-2[18F]fluoro-d-glucose to measure brain glucose metabolism (marker of brain function) in 24 cocaine abusers who watched a cocaine-cue video and compared brain activation with and without instructions to cognitively inhibit craving. A third scan was obtained at baseline (without video). Statistical parametric mapping was used for analysis and corroborated with regions of interest. The cocaine-cue video increased craving during the no-inhibition condition (pre 3+/-3, post 6+/-3; p<0.001) but not when subjects were instructed to inhibit craving (pre 3+/-2, post 3+/-3). Comparisons with baseline showed visual activation for both cocaine-cue conditions and limbic inhibition (accumbens, orbitofrontal, insula, cingulate) when subjects purposefully inhibited craving (p<0.001). Comparison between cocaine-cue conditions showed lower metabolism with cognitive inhibition in right orbitofrontal cortex and right accumbens (p<0.005), which was associated with right inferior frontal activation (r=-0.62, p<0.005). Decreases in metabolism in brain regions that process the predictive (nucleus accumbens) and motivational value (orbitofrontal cortex) of drug-conditioned stimuli were elicited by instruction to inhibit cue-induced craving. This suggests that cocaine abusers may retain some ability to inhibit craving and that strengthening fronto-accumbal regulation may be therapeutically beneficial in addiction.

Associative theories of goal-directed behaviour: a case for animal-human translational models

Associative accounts of goal-directed action, developed in the fields of human ideomotor action and that of animal learning, can capture cognitive belief-desire psychology of human decision-making. Whereas outcome-response accounts can account for the fact that the thought of a goal can call to mind the action that has previously procured this goal, response-outcome accounts capture decision-making processes that start out with the consideration of possible response alternatives followed only in the second instance by evaluation of their consequences. We argue that while the outcome-response mechanism plays a crucial role in response priming effects, the response-outcome mechanism is particularly important for action selection on the basis of current needs and desires. We therefore develop an integrative account that encapsulates these two routes of action selection within the framework of the associative-cybernetic model. This model has the additional benefit of providing mechanisms for the incentive modulation of goal-directed action and for the development of behavioural autonomy, and therefore provides a promising account of the multi-faceted process of animal as well as human instrumental decision-making.

Differential representation of Pavlovian-instrumental transfer by prefrontal cortex subregions and striatum

Environmental cues that once predicted reward can restore extinguished behavior directed toward that reward. This process may be modeled by the Pavlovian-instrumental transfer (PIT) paradigm where a previously learned Pavlovian conditioned stimulus (CS) elicits a representation of the reward associated with that CS, prompts motivation toward the absent reward, and triggers an instrumental action. We recorded in the medial and orbital prefrontal cortex (mPFC and OFC) and dorsal striatum (DS) of freely moving rats during PIT and found that a Pavlovian CS, as compared with neutral or no stimuli, amplified the phasic neuronal responses to instrumental nosepokes ('transfer' event). In mPFC and OFC, but not the DS, representation of the transfer event correlated with the strength of PIT behavior. Neurons in all three regions showed CS-selective amplification of Pavlovian approaches toward the reward delivery site. Whereas striatal neurons represented transfer and approach behavior through mostly segregated neuronal subsets, overlapping subsets represented these events in the mPFC and OFC. These findings suggest that parallel phasic activation of mPFC and OFC neuronal subsets participates in the transfer from Pavlovian incentives to instrumental actions.

Covert Expectation-of-Reward in Rat Ventral Striatum at Decision Points

Flexible decision-making strategies (such as planning) are a key component of adaptive behavior, yet their neural mechanisms have remained resistant to experimental analysis. Theories of planning require prediction and evaluation of potential future rewards, suggesting that reward signals may covertly appear at decision points. To test this idea, we recorded ensembles of ventral striatal neurons on a spatial decision task, in which hippocampal ensembles are known to represent future possibilities at decision points. We found representations of reward which were not only activated at actual reward delivery sites, but also at a high-cost choice point and before error correction. This expectation-of-reward signal at decision points was apparent at both the single cell and the ensemble level, and vanished with behavioral automation. We conclude that ventral striatal representations of reward are more dynamic than suggested by previous reports of reward- and cue-responsive cells, and may provide the necessary signal for evaluation of internally generated possibilities considered during flexible decision-making.

Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks

Here we challenge the view that reward-guided learning is solely controlled by the mesoaccumbens pathway arising from dopaminergic neurons in the ventral tegmental area and projecting to the nucleus accumbens. This widely accepted view assumes that reward is a monolithic concept, but recent work has suggested otherwise. It now appears that, in reward-guided learning, the functions of ventral and dorsal striata, and the cortico-basal ganglia circuitry associated with them, can be dissociated. Whereas the nucleus accumbens is necessary for the acquisition and expression of certain appetitive Pavlovian responses and contributes to the motivational control of instrumental performance, the dorsal striatum is necessary for the acquisition and expression of instrumental actions. Such findings suggest the existence of multiple independent yet interacting functional systems that are implemented in iterating and hierarchically organized cortico-basal ganglia networks engaged in appetitive behaviors ranging from Pavlovian approach responses to goal-directed instrumental actions controlled by action-outcome contingencies.

Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer

Pavlovian stimuli previously paired with food can markedly elevate the rate of food-reinforced instrumental responding. This effect, termed Pavlovian-instrumental transfer (PIT), depends both on general activating and specific cueing properties of Pavlovian stimuli. Recent evidence suggests that the general activating properties of Pavlovian stimuli are mediated by mesoaccumbens dopamine systems; however, the role of NAC dopamine D1 and D2 receptors is still unknown. Here we examined the effects of a selective dopamine D1 and D2 receptor blockade in the shell and core subregion of the NAC on general PIT. Rats were trained to press a single lever for food, and the effect of a single Pavlovian stimulus previously associated with the same food on performance of that lever was measured in extinction. Results reveal that PIT, that is, the increase in instrumental responding during presentation of the Pavlovian stimulus, was reduced by microinjections of the dopamine D1 receptor antagonist SCH-23390 and, less pronounced, by microinjections of the dopamine D2 receptor antagonist raclopride into the NAC core or shell, respectively. Our data suggest that dopamine D1 and D2 receptors in the NAC core and shell mediate the general activating effects of Pavlovian stimuli on instrumental behavior.

Cue-elicited reward-seeking requires extracellular signal-regulated kinase activation in the nucleus accumbens

The motivation to seek out rewards can come under the control of stimuli associated with reward delivery. The ability of cues to motivate reward-seeking behavior depends on the nucleus accumbens (NAcc). The molecular mechanisms in the NAcc that underlie the ability of a cue to motivate reward-seeking are not well understood. We examined whether extracellular signal-regulated kinase (ERK), an important intracellular signaling pathway in learning and memory, has a role in these motivational processes. We first examined p42 ERK (ERK2) activation in the NAcc after rats were trained to associate an auditory stimulus with food delivery and found that, as a consequence of training, presentation of the auditory cue itself was sufficient to increase ERK2 activation in the NAcc. To examine whether inhibition of ERK in the NAcc prevents cue-induced reward-seeking, we infused an inhibitor of ERK, U0126, into the NAcc before assessing rats' instrumental responding in the presence versus absence of the conditioned cue. We found that, whereas vehicle-infused rats showed increased instrumental responding during cue presentation, rats infused with U0126 showed a profound impairment in cue-induced instrumental responding. In contrast, intra-NAcc U0126 infusion had no effect on rats' food-reinforced instrumental responding or their ability to execute conditioned approach behavior. Our results demonstrate learning-related changes in ERK signaling in the NAcc, and that disruption of ERK activation in this structure interferes with the incentive-motivational effects of conditioned stimuli. The molecular mechanisms described here may have implications for cue-elicited drug craving after repeated exposure to drugs of abuse.