The influence of Pavlovian cues on instrumental performance is mediated by CaMKII activity in the striatum

What Role Does Striatal Dopamine Play in Goal-directed Action?

Evidence suggests that dopamine activity provides a US-related prediction error for Pavlovian conditioning and the reinforcement signal supporting the acquisition of habits. However, its role in goal-directed action is less clear. There are currently few studies that have assessed dopamine release as animals acquire and perform self-paced instrumental actions. Here we briefly review the literature documenting the psychological, behavioral and neural bases of goal-directed actions in rats and mice, before turning to describe recent studies investigating the role of dopamine in instrumental learning and performance. Plasticity in dorsomedial striatum, a central node in the network supporting goal-directed action, clearly requires dopamine release, the timing of which, relative to cortical and thalamic inputs, determines the degree and form of that plasticity. Beyond this, bilateral release appears to reflect reward prediction errors as animals experience the consequences of an action. Such signals feedforward to update the value of the specific action associated with that outcome during subsequent performance, with dopamine release at the time of action reflecting the updated predicted action value. More recently, evidence has also emerged for a hemispherically lateralised signal associated with the action; dopamine release is greater in the hemisphere contralateral to the spatial target of the action. This effect emerges over the course of acquisition and appears to reflect the strength of the action-outcome association. Thus, during goal-directed action, dopamine release signals the action, the outcome and their association to shape the learning and performance processes necessary to support this form of behavioral control.

Task-Dependent Effects of SKF83959 on Operant Behaviors Associated With Distinct Changes of CaMKII Signaling in Striatal Subareas

BACKGROUND

SKF83959, an atypical dopamine (DA) D1 receptor agonist, has been used to test the functions of DA-related receptor complexes in vitro, but little is known about its impact on conditioned behavior. The present study examined the effects of SKF83959 on operant behaviors and assayed the neurochemical mechanisms involved.

METHODS

Male rats were trained and maintained on either a fixed-interval 30-second (FI30) schedule or a differential reinforcement of low-rate response 10-second (DRL10) schedule of reinforcement. After drug treatment tests, western blotting assayed the protein expressions of the calcium-/calmodulin-dependent protein kinase II (CaMKII) and the transcription factor cyclic AMP response element binding protein (CREB) in tissues collected from 4 selected DA-related areas.

RESULTS

SKF83959 disrupted the performance of FI30 and DRL10 behaviors in a dose-dependent manner by reducing the total number of responses in varying magnitudes. Moreover, the distinct profiles of the behavior altered by the drug were manifested by analyzing qualitative and quantitative measures on both tasks. Western-blot results showed that phospho-CaMKII levels decreased in the nucleus accumbens and the dorsal striatum of the drug-treated FI30 and DRL10 subjects, respectively, compared with their vehicle controls. The phospho-CREB levels decreased in the nucleus accumbens and the hippocampus of drug-treated FI30 subjects but increased in the nucleus accumbens of drug-treated DRL10 subjects.

CONCLUSIONS

Our results provide important insight into the neuropsychopharmacology of SKF83959, indicating that the drug-altered operant behavior is task dependent and related to regional-dependent changes of CaMKII-CREB signaling in the mesocorticolimbic DA systems.

K369I Tau Mice Demonstrate a Shift Towards Striatal Neuron Burst Firing and Goal-directed Behaviour

Pathological forms of the microtubule-associated protein tau are involved in a large group of neurodegenerative diseases named tauopathies, including frontotemporal lobar degeneration (FTLD-tau). K369I mutant tau transgenic mice (K3 mice) recapitulate neural and behavioural symptoms of FTLD, including tau aggregates in the cortex, alterations to nigrostriatum, memory deficits and parkinsonism. The aim of this study was to further characterise the K3 mouse model by examining functional alterations to the striatum. Whole-cell patch-clamp electrophysiology was used to investigate the properties of striatal neurons in K3 mice and wildtype controls. Additionally, striatal-based instrumental learning tasks were conducted to assess goal-directed versus habitual behaviours (i.e., by examining sensitivity to outcome devaluation and progressive ratios). The K3 model demonstrated significant alterations in the discharge properties of striatal neurons relative to wildtype mice, which manifested as a shift in neuronal output towards a burst firing state. K3 mice acquired goal-directed responding faster than control mice and were goal-directed at test unlike wildtype mice, which is likely to indicate reduced capacity to develop habitual behaviour. The observed pattern of behaviour in K3 mice is suggestive of deficits in dorsal lateral striatal function and this was supported by our electrophysiological findings. Thus, both the electrophysiological and behavioural alterations indicate that K3 mice have early deficits in striatal function. This finding adds to the growing literature which indicate that the striatum is impacted in tau-related neuropathies such as FTLD, and further suggests that the K3 model is a unique mouse model for investigating FTLD especially with striatal involvement.

CaMKII antagonism in the ventral tegmental area impairs acquisition of conditioned approach learning in rats

This study investigated the role of calcium2+/calmodulin-dependent protein kinase II (CaMKII), a protein in the second messenger pathway of NMDA receptors, in the ventral tegmental area (VTA) in the acquisition and performance of conditioned approach learning. Male Long-Evans rats (N = 79) were exposed to 3 (to test acquisition) or 7 (to test performance) conditioning sessions in which they received 30 paired presentations of a light stimulus (CS) and a food pellet (US) on a random time schedule. These conditioning sessions were then followed by one 30-min session without the CS or US and lastly by a CS-only test session, where only the light stimulus was presented (without food) according to the same schedule as the conditioning sessions. Bilateral intra-VTA injections of KN93 (vehicle, 3.0, 4.5 or 6.0 μg/0.5 μL), a CaMKII inhibitor, were administered prior to each conditioning session to test effects on the acquisition of conditioned approach or prior to the CS-only test session to test effects on the performance of conditioned approach. KN93, when given prior to conditioning sessions, significantly reduced the number of conditioned approach responses emitted during CS presentations in the CS-only test. When KN93 was given prior to the CS-only test it had no effect. These results suggest that CaMKII activation in the VTA is necessary for the acquisition, but not the performance, of reward-related learning.

Mechanistic Resolution Required to Mediate Operant Learned Behaviors: Insights from Neuronal Ensemble-Specific Inactivation

Many learned behaviors are directed by complex sets of highly specific stimuli or cues. The neural mechanisms mediating learned associations in these behaviors must be capable of storing complex cue information and distinguishing among different learned associations—we call this general concept “mechanistic resolution”. For many years, our understanding of the circuitry of these learned behaviors has been based primarily on inactivation of specific cell types or whole brain areas regardless of which neurons were activated during the cue-specific behaviors. However, activation of all cells or specific cell types in a brain area do not have enough mechanistic resolution to encode or distinguish high-resolution learned associations in these behaviors. Instead, these learned associations are likely encoded within specific patterns of sparsely distributed neurons called neuronal ensembles that are selectively activated by the cues. This review article focuses on studies of neuronal ensembles in operant learned responding to obtain food or drug rewards. These studies suggest that the circuitry of operant learned behaviors may need to be re-examined using ensemble-specific manipulations that have the requisite level of mechanistic resolution.

Strain commonalities and differences in response-outcome decision making in mice

The ability to select between actions that are more vs. less likely to be reinforced is necessary for survival and navigation of a changing environment. A task termed "response-outcome contingency degradation" can be used in the laboratory to determine whether rodents behave according to such goal-directed response strategies. In one iteration of this task, rodents are trained to perform two food-reinforced behaviors, then the predictive relationship between one instrumental response and the associated outcome is modified by providing the reinforcer associated with that response non-contingently. During a subsequent probe test, animals can select between the two trained responses. Preferential engagement of the behavior most likely to be reinforced is considered goal-directed, while non-selective responding is considered a failure in response-outcome conditioning, or "habitual." This test has largely been used with rats, and less so with mice. Here we compiled data collected from several cohorts of mice tested in our lab between 2012 and 2015. Mice were bred on either a C57BL/6 or predominantly BALB/c strain background. We report that both strains of mice can use information acquired as a result of instrumental contingency degradation training to select amongst multiple response options the response most likely to be reinforced. Mice differ, however, during the training sessions when the familiar response-outcome contingency is being violated. BALB/c mice readily generate perseverative or habit-like response strategies when the only available response is unlikely to be reinforced, while C57BL/6 mice more readily inhibit responding. These findings provide evidence of strain differences in response strategies when an anticipated reinforcer is unlikely to be delivered.

Optogenetic activation of the excitatory neurons expressing CaMKIIα in the ventral tegmental area upregulates the locomotor activity of free behaving rats

The ventral tegmental area (VTA) plays an important role in motivation and motor activity of mammals. Previous studies have reported that electrical stimulations of the VTA's neuronal projections were able to upregulate the locomotor activity of behaving rats. However, which types of neurons in the VTA that take part in the activation remain elusive. In this paper we employed optogenetic technique to selectively activate the excitatory neurons expressing CaMKIIα in the VTA region and induced a higher locomotor activity for free behaving rats. Further behavioral studies indicated that reward learning mediated in the enhancement of the rat locomotor activity. Finally the immunohistochemistry studies explored that the excitatory neurons under the optogenetic activation in VTA were partly dopaminergic that may participate as a vital role in the optogenetic activation of the locomotor activity. In total, our study provided an optogenetic approach to selectively upregulate the locomotor activity of free behaving rats, thus facilitating both neuroscience researches and neural engineering such as animal robotics in the future.

Molecular and cellular mechanisms of dopamine-mediated behavioral plasticity in the striatum

The striatum is the input structure of the basal ganglia system. By integrating glutamatergic signals from cortical and subcortical regions and dopaminergic signals from mesolimbic nuclei the striatum functions as an important neural substrate for procedural and motor learning as well as for reward-guided behaviors. In addition, striatal activity is significantly altered in pathological conditions in which either a loss of dopamine innervation (Parkinson's disease) or aberrant dopamine-mediated signaling (drug addiction and L-DOPA induced dyskinesia) occurs. Here we discuss cellular mechanisms of striatal synaptic plasticity and aspects of cell signaling underlying striatum-dependent behavior, with a major focus on the neuromodulatory action of the endocannabinoid system and on the role of the Ras-ERK cascade.

Repeated cocaine exposure facilitates the expression of incentive motivation and induces habitual control in rats

There is growing evidence that mere exposure to drugs can induce long-term alterations in the neural systems that mediate reward processing, motivation, and behavioral control, potentially causing the pathological pursuit of drugs that characterizes the addicted state. The incentive sensitization theory proposes that drug exposure potentiates the influence of reward-paired cues on behavior. It has also been suggested that drug exposure biases action selection towards the automatic execution of habits and away from more deliberate goal-directed control. The current study investigated whether rats given repeated exposure to peripherally administered cocaine would show alterations in incentive motivation (assayed using the Pavlovian-to-instrumental transfer (PIT) paradigm) or habit formation (assayed using sensitivity to reward devaluation). After instrumental and Pavlovian training for food pellet rewards, rats were given 6 daily injections of cocaine (15 mg/kg, IP) or saline, followed by a 10-d period of rest. Consistent with the incentive sensitization theory, cocaine-treated rats showed stronger cue-evoked lever pressing than saline-treated rats during the PIT test. The same rats were then trained on a new instrumental action with a new food pellet reward before undergoing a reward devaluation testing. Although saline-treated rats exhibited sensitivity to reward devaluation, indicative of goal-directed performance, cocaine-treated rats were insensitive to this treatment, suggesting a reliance on habitual processes. These findings, when taken together, indicate that repeated exposure to cocaine can cause broad alterations in behavioral control, spanning both motivational and action selection processes, and could therefore help explain aberrations of decision-making that underlie drug addiction.

Acute cocaine increases phosphorylation of CaMKII and GluA1 in the dorsolateral striatum of drug naïve rats, but not cocaine-experienced rats

Transport of GluA1-containing AMPA glutamate receptors to synapses in the nucleus accumbens, a process that involves phosphorylation of key serine residues by CaMKII, is associated with the reinstatement of cocaine-seeking behavior. A growing body of evidence indicates that the dorsal striatum contributes to aspects of cocaine addiction. However, the potential role of CaMKII-mediated phosphorylation of GluA1 subunits in the dorsolateral (DL) striatum during cocaine reinstatement has not been examined. In this study, rats were trained to self-administer cocaine and were partnered with saline-yoked rats that received injections of saline. Following extinction, each pair of rats received either a systemic priming injection of cocaine (10mg/kg, i.p.) or saline. As expected, cocaine-experienced rats displayed robust reinstatement of cocaine seeking in response to a challenge injection, whereas yoked saline controls did not. The DL striatum was dissected immediately following the reinstatement test session. Results from Western blotting experiments showed increased pGluA1-ser831 and pCaMKII-thr286 in the DL striatum of saline-yoked rats given an acute injection of cocaine. This effect was absent in cocaine-experienced rats that received a saline injection, and no changes were observed following a priming injection of cocaine in cocaine-experienced rats. These results indicate that acute exposure to cocaine in drug naïve rats increased CaMKII-mediated phosphorylation of GluA1-containing AMPA receptors in the DL striatum, an effect that was not observed during cocaine priming-induced reinstatement of drug seeking. It is possible; therefore, that increased phosphorylation of CaMKII and GluA1 following acute cocaine is a compensatory mechanism in the DL striatum.

The effect of ratio and interval training on Pavlovian-instrumental transfer in mice

Conditional stimuli (CS) that are paired with reward can be used to motivate instrumental responses. This process is called Pavlovian-instrumental transfer (PIT). A recent study in rats suggested that habitual responses are particularly sensitive to the motivational effects of reward cues. The current experiments examined this idea using ratio and interval training in mice. Two groups of animals were trained to lever press for food pellets that were delivered on random ratio or random interval schedules. Devaluation tests revealed that interval training led to habitual responding while ratio training produced goal-directed actions. The presentation of CSs paired with reward led to positive transfer in both groups, however, the size of this effect was much larger in mice that were trained on interval schedules. This result suggests that habitual responses are more sensitive to the motivational influence of reward cues than goal-directed actions. The implications for neurobiological models of motivation and drug seeking behaviors are discussed.

Genetic inhibition of CaMKII in dorsal striatal medium spiny neurons reduces functional excitatory synapses and enhances intrinsic excitability

Ca²⁺/calmodulin-dependent protein kinase II (CaMKII) is abundant in striatal medium spiny neurons (MSNs). CaMKII is dynamically regulated by changes in dopamine signaling, as occurs in Parkinson's disease as well as addiction. Although CaMKII has been extensively studied in the hippocampus where it regulates excitatory synaptic transmission, relatively little is known about how it modulates neuronal function in the striatum. Therefore, we examined the impact of selectively overexpressing an EGFP-fused CaMKII inhibitory peptide (EAC3I) in striatal medium spiny neurons (MSNs) using a novel transgenic mouse model. EAC3I-expressing cells exhibited markedly decreased excitatory transmission, indicated by a decrease in the frequency of spontaneous excitatory postsynaptic currents (sEPSCs). This decrease was not accompanied by changes in the probability of release, levels of glutamate at the synapse, or changes in dendritic spine density. CaMKII regulation of the AMPA receptor subunit GluA1 is a major means by which the kinase regulates neuronal function in the hippocampus. We found that the decrease in striatal excitatory transmission seen in the EAC3I mice is mimicked by deletion of GluA1. Further, while CaMKII inhibition decreased excitatory transmission onto MSNs, it increased their intrinsic excitability. These data suggest that CaMKII plays a critical role in setting the excitability rheostat of striatal MSNs by coordinating excitatory synaptic drive and the resulting depolarization response.

AMPAR-independent effect of striatal αCaMKII promotes the sensitization of cocaine reward

Changes in CaMKII-regulated synaptic excitability are a means through which experience may modify neuronal function and shape behavior. While behavior in rodent addiction models is linked with CaMKII activity in the nucleus accumbens (NAc) shell, the key cellular adaptations that forge this link are unclear. Using a mouse strain with striatal-specific expression of autonomously active CaMKII (T286D), we demonstrate that while persistent CaMKII activity induces behaviors comparable to those in mice repeatedly exposed to psychostimulants, it is insufficient to increase AMPAR-mediated synaptic strength in NAc shell. However, autonomous CaMKII upregulates A-type K(+) current (IA) and decreases firing in shell neurons. Importantly, inactivating the transgene with doxycycline eliminates both the IA-mediated firing decrease and the elevated behavioral response to cocaine. This study identifies CaMKII regulation of IA in NAc shell neurons as a novel cellular contributor to the sensitization of cocaine reward.

Molecular substrates of action control in cortico-striatal circuits

The purpose of this review is to describe the molecular mechanisms in the striatum that mediate reward-based learning and action control during instrumental conditioning. Experiments assessing the neural bases of instrumental conditioning have uncovered functional circuits in the striatum, including dorsal and ventral striatal sub-regions, involved in action-outcome learning, stimulus-response learning, and the motivational control of action by reward-associated cues. Integration of dopamine (DA) and glutamate neurotransmission within these striatal sub-regions is hypothesized to enable learning and action control through its role in shaping synaptic plasticity and cellular excitability. The extracellular signal regulated kinase (ERK) appears to be particularly important for reward-based learning and action control due to its sensitivity to combined DA and glutamate receptor activation and its involvement in a range of cellular functions. ERK activation in striatal neurons is proposed to have a dual role in both the learning and performance factors that contribute to instrumental conditioning through its regulation of plasticity-related transcription factors and its modulation of intrinsic cellular excitability. Furthermore, perturbation of ERK activation by drugs of abuse may give rise to behavioral disorders such as addiction.

Reward-related behavioral paradigms for addiction research in the mouse: performance of common inbred strains

The mouse has emerged as a uniquely valuable species for studying the molecular and genetic basis of complex behaviors and modeling neuropsychiatric disease states. While valid and reliable preclinical assays for reward-related behaviors are critical to understanding addiction-related processes, and various behavioral procedures have been developed and characterized in rats and primates, there have been relatively few studies using operant-based addiction-relevant behavioral paradigms in the mouse. Here we describe the performance of the C57BL/6J inbred mouse strain on three major reward-related paradigms, and replicate the same procedures in two other commonly used inbred strains (DBA/2J, BALB/cJ). We examined Pavlovian-instrumental transfer (PIT) by measuring the ability of an auditory cue associated with food reward to promote an instrumental (lever press) response. In a separate experiment, we assessed the acquisition and extinction of a simple stimulus-reward instrumental behavior on a touchscreen-based task. Reinstatement of this behavior was then examined following either continuous exposure to cues (conditioned reinforcers, CRs) associated with reward, brief reward and CR exposure, or brief reward exposure followed by continuous CR exposure. The third paradigm examined sensitivity of an instrumental (lever press) response to devaluation of food reward (a probe for outcome insensitive, habitual behavior) by repeated pairing with malaise. Results showed that C57BL/6J mice displayed robust PIT, as well as clear extinction and reinstatement, but were insensitive to reinforcer devaluation. DBA/2J mice showed good PIT and (rewarded) reinstatement, but were slow to extinguish and did not show reinforcer devaluation or significant CR-reinstatement. BALB/cJ mice also displayed good PIT, extinction and reinstatement, and retained instrumental responding following devaluation, but, unlike the other strains, demonstrated reduced Pavlovian approach behavior (food magazine head entries). Overall, these assays provide robust paradigms for future studies using the mouse to elucidate the neural, molecular and genetic factors underpinning reward-related behaviors relevant to addiction research.

Acquisition and performance of goal-directed instrumental actions depends on ERK signaling in distinct regions of dorsal striatum in rats

The performance of goal-directed actions relies on an animal's previous knowledge of the outcomes or consequences that result from its actions. Additionally, a sensorimotor learning process linking environmental stimuli with actions influences instrumental performance by selecting actions for additional evaluation. These distinct decision-making processes in rodents depend on separate subregions of the dorsal striatum. Whereas the posterior dorsomedial striatum (pDMS) is required for the encoding of actions with their outcomes or consequences, the dorsolateral striatum (DLS) mediates action selection based on sensorimotor learning. However, the molecular mechanisms within these brain regions that support learning and performance of goal-directed behavior are not known. Here we show that activation of extracellular signal-regulated kinase (ERK) in the dorsal striatum has a critical role in learning and performance of instrumental goal-directed behavior in rodents. We observed an increase in p42 ERK (ERK2) activation in both the pDMS and DLS during both the acquisition and performance of recently acquired instrumental goal-directed actions. Furthermore, disruption of ERK activation in the pDMS prevented both the acquisition of action-outcome associations, as well as the performance of goal-directed actions guided by previously acquired associations, whereas disruption of ERK activation in the DLS disrupted instrumental performance but left instrumental action-outcome learning intact. These results provide evidence of a critical, region-specific role for ERK signaling in the dorsal striatum during the acquisition of instrumental learning and suggest that processes sensitive to ERK signaling within these striatal subregions interact to control instrumental performance after initial acquisition.

High on habits

The neural circuits involved in learning and executing goal-directed actions, which are governed by action-outcome contingencies and sensitive to changes in the expected value of the outcome, have been shown to be different from those mediating habits, which are less dependent on action-outcome relations and changes in outcome value. Extended training, different reinforcement schedules, and substances of abuse have been shown to induce a shift from goal-directed performance to habitual performance. This shift can be beneficial in everyday life, but can also lead to loss of voluntary control and compulsive behavior, namely during drug seeking in addiction. Although the brain circuits underlying habit formation are becoming clearer, the molecular mechanisms underlying habit formation are still not understood. Here, we review a recent study where Hilario et al. (2007) established behavioral procedures to investigate habit formation in mice in order to investigate the molecular mechanisms underlying habit formation. Using those procedures, and a combination of genetic and pharmacological tools, the authors showed that endocannabinoid signaling is critical for habit formation.

A role for alpha-amino-3-hydroxy-5-methylisoxazole-4-propionic acid GluR1 phosphorylation in the modulatory effects of appetitive reward cues on goal-directed behavior

Alpha-amino-3-hydroxy-5-methylisoxazole-4-propionic acid (AMPA) receptor regulation has been shown to be critically involved in synaptic plasticity underlying learning and memory. This regulation occurs through trafficking of the receptor and modulation of the receptor's channel properties, both of which depend on protein phosphorylation. Using homologous recombination (knock-in) techniques we targeted two phosphorylation sites on the AMPA-GluR1 receptor: the Ser831 site, phosphorylated by calcium calmodulin-dependent protein kinase II/protein kinase C, and the Ser845 site, phosphorylated by protein kinase A. Mice with mutations that prevented phosphorylation at one or both of these sites were tested on a single-outcome Pavlovian-instrumental transfer task often used to assess the acquisition of incentive motivation by cues for food reinforcement. Mice were separately trained to associate a Pavlovian cue with food and to perform an instrumental lever-press response to earn that same reward. During a transfer test, the cue was presented while the mice were lever-pressing under extinction conditions. Whereas wild-type control mice showed substantial enhancement of lever-pressing when the cue was presented (i.e. showed Pavlovian-instrumental transfer), mice with mutations at both of these phosphorylation sites showed no evidence of such transfer. By contrast, mice with either serine site mutated alone showed normal transfer. These results suggest critical roles for GluR1 phosphorylation pathways in a form of incentive learning that can play an important part in regulating normal motivated behavior as well as maladaptive behaviors such as addiction and eating disorders.

Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks

Here we challenge the view that reward-guided learning is solely controlled by the mesoaccumbens pathway arising from dopaminergic neurons in the ventral tegmental area and projecting to the nucleus accumbens. This widely accepted view assumes that reward is a monolithic concept, but recent work has suggested otherwise. It now appears that, in reward-guided learning, the functions of ventral and dorsal striata, and the cortico-basal ganglia circuitry associated with them, can be dissociated. Whereas the nucleus accumbens is necessary for the acquisition and expression of certain appetitive Pavlovian responses and contributes to the motivational control of instrumental performance, the dorsal striatum is necessary for the acquisition and expression of instrumental actions. Such findings suggest the existence of multiple independent yet interacting functional systems that are implemented in iterating and hierarchically organized cortico-basal ganglia networks engaged in appetitive behaviors ranging from Pavlovian approach responses to goal-directed instrumental actions controlled by action-outcome contingencies.

A necessary role for GluR1 serine 831 phosphorylation in appetitive incentive learning

It is widely thought that regulation of post-synaptic AMPA receptors is a critical component in changes in synaptic efficacy underlying learning and memory. The regulation of AMPA receptors occurs through trafficking of the receptor and/or modulation of the receptor's channel properties and both of these processes depend on phosphorylation of the receptor. Using homologous recombination (knock-in) techniques we targeted two phosphorylation sites on the AMPA-GluR1 receptor: the CaMKII/PKC Ser 831 site and the PKA Ser 845 site. Mice with either or both of these sites mutated were then tested on an incentive learning task that assessed their ability to acquire a simple association between a cue and reward and to then use this cue as a reinforcer to guide their behavior (conditioned reinforcement). We report that, whereas WT mice showed enhanced responding for the reward-associated cue, mice with mutations of both phosphorylation sites or the Ser 831 site alone, failed to show such a conditioned reinforcement effect. By contrast, mice with only the Ser 845 site deficient showed normal CS+ reinforced responding. Thus, action at the Ser 831 phosphorylation site was necessary for normal conditioned reinforcement. Finally, the behavioral deficit was highly specific: performance on a number of other measures of motivated performance, including responding reinforced by the food itself, was unaffected by the mutations. Our findings provide novel evidence for a molecular mechanism in a form of appetitive incentive learning critical in regulating normal motivated behavior, as well as maladaptive forms such as addiction and eating disorders.

Cue-elicited reward-seeking requires extracellular signal-regulated kinase activation in the nucleus accumbens

The motivation to seek out rewards can come under the control of stimuli associated with reward delivery. The ability of cues to motivate reward-seeking behavior depends on the nucleus accumbens (NAcc). The molecular mechanisms in the NAcc that underlie the ability of a cue to motivate reward-seeking are not well understood. We examined whether extracellular signal-regulated kinase (ERK), an important intracellular signaling pathway in learning and memory, has a role in these motivational processes. We first examined p42 ERK (ERK2) activation in the NAcc after rats were trained to associate an auditory stimulus with food delivery and found that, as a consequence of training, presentation of the auditory cue itself was sufficient to increase ERK2 activation in the NAcc. To examine whether inhibition of ERK in the NAcc prevents cue-induced reward-seeking, we infused an inhibitor of ERK, U0126, into the NAcc before assessing rats' instrumental responding in the presence versus absence of the conditioned cue. We found that, whereas vehicle-infused rats showed increased instrumental responding during cue presentation, rats infused with U0126 showed a profound impairment in cue-induced instrumental responding. In contrast, intra-NAcc U0126 infusion had no effect on rats' food-reinforced instrumental responding or their ability to execute conditioned approach behavior. Our results demonstrate learning-related changes in ERK signaling in the NAcc, and that disruption of ERK activation in this structure interferes with the incentive-motivational effects of conditioned stimuli. The molecular mechanisms described here may have implications for cue-elicited drug craving after repeated exposure to drugs of abuse.

General and outcome-specific forms of Pavlovian-instrumental transfer: the effect of shifts in motivational state and inactivation of the ventral tegmental area

This study compared the contribution of the general activating and specific cueing properties of Pavlovian stimuli to Pavlovian-instrumental transfer (PIT) and the role of the ventral tegmental area (VTA) in mediating these effects. In Experiment 1, hungry rats initially received Pavlovian training, in which three distinct auditory stimuli predicted the delivery of three different food outcomes. Next, the rats were trained to perform two instrumental actions, each earning a unique outcome selected from the three used in Pavlovian conditioning. Finally, the effects of the three stimuli on performance of the two actions were assessed in extinction. Presentation of a stimulus that had been paired with the same outcome as an action increased its performance relative to the other action, demonstrating that PIT effects can be outcome selective. In contrast, presentation of the stimulus that predicted the outcome that was not earned during instrumental training facilitated the performance of both actions indiscriminately. This effect, but not the outcome-selective effect, was abolished by a shift from a hungry to a relatively sated state. Experiment 2 examined the effects of inactivation of the VTA on these two forms of PIT. VTA inactivation was found to attenuate PIT but, unlike satiety, did not appear to differentially affect the general or the outcome-selective forms of PIT. The VTA appears therefore to play an important but general role in the initiation of instrumental actions, enabling cues to influence performance whether they enhance responding by changes in arousal or by retrieving particular actions based on their consequences.

Endocannabinoid signaling is critical for habit formation

Extended training can induce a shift in behavioral control from goal-directed actions, which are governed by action-outcome contingencies and sensitive to changes in the expected value of the outcome, to habits which are less dependent on action-outcome relations and insensitive to changes in outcome value. Previous studies in rats have shown that interval schedules of reinforcement favor habit formation while ratio schedules favor goal-directed behavior. However, the molecular mechanisms underlying habit formation are not well understood. Endocannabinoids, which can function as retrograde messengers acting through presynaptic CB1 receptors, are highly expressed in the dorsolateral striatum, a key region involved in habit formation. Using a reversible devaluation paradigm, we confirmed that in mice random interval schedules also favor habit formation compared with random ratio schedules. We also found that training with interval schedules resulted in a preference for exploration of a novel lever, whereas training with ratio schedules resulted in less generalization and more exploitation of the reinforced lever. Furthermore, mice carrying either a heterozygous or a homozygous null mutation of the cannabinoid receptor type I (CB1) showed reduced habit formation and enhanced exploitation. The impaired habit formation in CB1 mutant mice cannot be attributed to chronic developmental or behavioral abnormalities because pharmacological blockade of CB1 receptors specifically during training also impairs habit formation. Taken together our data suggest that endocannabinoid signaling is critical for habit formation.