The contribution of NMDA receptor signaling in the corticobasal ganglia reward network to appetitive Pavlovian learning

Analysis of individual differences in neurofeedback training illuminates successful self-regulation of the dopaminergic midbrain

The dopaminergic midbrain is associated with reinforcement learning, motivation and decision-making – functions often disturbed in neuropsychiatric disorders. Previous research has shown that dopaminergic midbrain activity can be endogenously modulated via neurofeedback. However, the robustness of endogenous modulation, a requirement for clinical translation, is unclear. Here, we examine whether the activation of particular brain regions associates with successful regulation transfer when feedback is no longer available. Moreover, to elucidate mechanisms underlying effective self-regulation, we study the relation of successful transfer with learning (temporal difference coding) outside the midbrain during neurofeedback training and with individual reward sensitivity in a monetary incentive delay (MID) task. Fifty-nine participants underwent neurofeedback training either in standard (Study 1 N = 15, Study 2 N = 28) or control feedback group (Study 1, N = 16). We find that successful self-regulation is associated with prefrontal reward sensitivity in the MID task (N = 25), with a decreasing relation between prefrontal activity and midbrain learning signals during neurofeedback training and with increased activity within cognitive control areas during transfer. The association between midbrain self-regulation and prefrontal temporal difference and reward sensitivity suggests that reinforcement learning contributes to successful self-regulation. Our findings provide insights in the control of midbrain activity and may facilitate individually tailoring neurofeedback training.

Analysis of real-time fMRI data from 59 participants undergoing neurofeedback training suggests that reinforcement learning contributes to successful self-regulation in the dopaminergic midbrain.

Medial prefrontal cortical control of reward- and aversion-based behavioral output: Bottom-up modulation

How does the brain guide our actions? This is a complex issue, where the medial prefrontal cortex (mPFC) plays a crucial role. The mPFC is essential for cognitive flexibility and decision making. These functions are related to reward- and aversion-based learning, which ultimately drive behavior. Though, cortical projections and modulatory systems that may regulate those processes in the mPFC are less understood. How does the mPFC regulate approach-avoidance behavior in the case of conflicting aversive and appetitive stimuli? This is likely dependent on the bottom-up neuromodulation of the mPFC projection neurons. In this review, we integrate behavioral-, pharmacological-, and viral-based circuit manipulation data showing the involvement of mPFC dopaminergic, noradrenergic, cholinergic, and serotoninergic inputs in reward and aversion processing. Given that an incorrect balance of reward and aversion value could be a key problem in mental diseases such as substance use disorders, we discuss outstanding questions for future research on the role of mPFC modulation in reward and aversion.

Tonic GABAergic inhibition, via GABAA receptors containing αβƐ subunits, regulates excitability of ventral tegmental area dopamine neurons

The activity of midbrain dopamine neurons is strongly regulated by fast synaptic inhibitory γ‐Aminobutyric acid (GABA)ergic inputs. There is growing evidence in other brain regions that low concentrations of ambient GABA can persistently activate certain subtypes of GABA_A receptor to generate a tonic current. However, evidence for a tonic GABAergic current in midbrain dopamine neurons is limited. To address this, we conducted whole‐cell recordings from ventral tegmental area (VTA) dopamine neurons in brain slices from mice. We found that application of GABA_A receptor antagonists decreased the holding current, indicating the presence of a tonic GABAergic input. Global increases in GABA release, induced by either a nitric oxide donor or inhibition of GABA uptake, further increased this tonic current. Importantly, prolonged inhibition of the firing activity of local GABAergic neurons abolished the tonic current. A combination of pharmacology and immunohistochemistry experiments suggested that, unlike common examples of tonic inhibition, this current may be mediated by a relatively unusual combination of α4βƐ subunits. Lastly, we found that the tonic current reduced excitability in dopamine neurons suggesting a subtractive effect on firing activity.

Neurobiology of reward-related learning

A major goal in psychology is to understand how environmental stimuli associated with primary rewards come to function as conditioned stimuli, acquiring the capacity to elicit similar responses to those elicited by primary rewards. Our neurobiological model is predicated on the Hebbian idea that concurrent synaptic activity on the primary reward neural substrate-proposed to be ventral tegmental area (VTA) dopamine (DA) neurons-strengthens the synapses involved. We propose that VTA DA neurons receive both a strong unconditioned stimulus signal (acetylcholine stimulation of DA cells) from the primary reward capable of unconditionally activating DA cells and a weak stimulus signal (glutamate stimulation of DA cells) from the neutral stimulus. Through joint stimulation the weak signal is potentiated and capable of activating the VTA DA cells, eliciting a conditioned response. The learning occurs when this joint stimulation initiates intracellular second-messenger cascades resulting in enhanced glutamate-DA synapses. In this review we present evidence that led us to propose this model and the most recent evidence supporting it.

Anatomic resolution of neurotransmitter-specific projections to the VTA reveals diversity of GABAergic inputs

The ventral tegmental area (VTA) is important for reward processing and motivation. The anatomic organization of neurotransmitter-specific inputs to the VTA remains poorly resolved. In the present study, we mapped the major neurotransmitter projections to the VTA through cell-type-specific retrograde and anterograde tracing. We found that glutamatergic inputs arose from a variety of sources and displayed some connectivity biases toward specific VTA cell types. The sources of GABAergic projections were more widespread, displayed a high degree of differential innervation of subregions in the VTA and were largely biased toward synaptic contact with local GABA neurons. Inactivation of GABA release from the two major sources, locally derived versus distally derived, revealed distinct roles for these projections in behavioral regulation. Optogenetic manipulation of individual distal GABAergic inputs also revealed differential behavioral effects. These results demonstrate that GABAergic projections to the VTA are a major contributor to the regulation and diversification of the structure.

Selective Effects of the Loss of NMDA or mGluR5 Receptors in the Reward System on Adaptive Decision-Making

Selecting the most advantageous actions in a changing environment is a central feature of adaptive behavior. The midbrain dopamine (DA) neurons along with the major targets of their projections, including dopaminoceptive neurons in the frontal cortex and basal ganglia, play a key role in this process. Here, we investigate the consequences of a selective genetic disruption of NMDA receptor and metabotropic glutamate receptor 5 (mGluR5) in the DA system on adaptive choice behavior in mice. We tested the effects of the mutation on performance in the probabilistic reinforcement learning and probability-discounting tasks. In case of the probabilistic choice, both the loss of NMDA receptors in dopaminergic neurons or the loss mGluR5 receptors in D1 receptor-expressing dopaminoceptive neurons reduced the probability of selecting the more rewarded alternative and lowered the likelihood of returning to the previously rewarded alternative (win-stay). When observed behavior was fitted to reinforcement learning models, we found that these two mutations were associated with a reduced effect of the expected outcome on choice (i.e., more random choices). None of the mutations affected probability discounting, which indicates that all animals had a normal ability to assess probability. However, in both behavioral tasks animals with targeted loss of NMDA receptors in dopaminergic neurons or mGluR5 receptors in D1 neurons were significantly slower to perform choices. In conclusion, these results show that glutamate receptor-dependent signaling in the DA system is essential for the speed and accuracy of choices, but at the same time probably is not critical for correct estimation of probable outcomes.

Hippocampus and Prefrontal Cortex Predict Distinct Timescales of Activation in the Human Ventral Tegmental Area

The mesolimbic dopamine system contributes to a remarkable variety of behaviors at multiple timescales. Midbrain neurons have fast and slow signaling components, and specific afferent systems, such as the hippocampus (HPC) and prefrontal cortex (PFC), have been demonstrated to drive these components in anesthetized animals. Whether these interactions exist during behavior, however, is unknown. To address this question, we developed a novel analysis of human functional magnetic resonance imaging data that fits models of network excitation and inhibition on ventral tegmental area (VTA) activation. We show that specific afferent systems predict distinct temporal components of midbrain VTA signal. We found that PFC, but not HPC, positively predicted transient, event-evoked VTA activation. In contrast, HPC, but not PFC, positively predicted slow shifts in VTA baseline variability. Thus, unique functional contributions of afferent systems to VTA physiology are detectable at the network level in behaving humans. The findings support models of dopamine function in which dissociable neural circuits support different aspects of motivated behavior via active regulation of tonic and phasic signals.

NMDA receptor blockade specifically impedes the acquisition of incentive salience attribution

Glutamatergic signaling plays an important role in learning and memory. Using Pavlovian conditioned approach procedures, the mechanisms that drive stimulus-reward learning and memory have been investigated. However, there are instances where reward-predictive stimuli can function beyond being solely predictive and can be attributed with "motivational value" or incentive salience. Using a Pavlovian conditioned approach procedure consisting of two different but equally predictive stimuli (lever vs. tone) we investigated the role NMDA receptor function has in the attribution of incentive salience. The results revealed that the administration of MK-801, an NMDA receptor antagonist, during acquisition of Pavlovian conditioned approach promoted goal-tracking to a lever stimulus, while control animals learned to sign-track. Moreover, within the same animals, the use of a tone stimulus elicited goal-tracking responses that were unaffected by MK-801 pretreatments. Furthermore, a lever CS that elicited sign-tracking served as a more robust conditioned reinforcer than a tone CS that elicited goal-tracking or a lever CS that elicited goal-tracking via MK-801 pretreatments. Collectively, these results demonstrate that NMDA receptor antagonism can alter the stimulus-reward relationship learned and prevent the attribution of incentive salience, rather than impede general learning.

Striatopallidal Neuron NMDA Receptors Control Synaptic Connectivity, Locomotor, and Goal-Directed Behaviors

The basal ganglia (BG) control action selection, motor programs, habits, and goal-directed learning. The striatum, the principal input structure of BG, is predominantly composed of medium-sized spiny neurons (MSNs). Arising from these spatially intermixed MSNs, two inhibitory outputs form two main efferent pathways, the direct and indirect pathways. Striatonigral MSNs give rise to the activating, direct pathway MSNs and striatopallidal MSNs to the inhibitory, indirect pathway (iMSNs). BG output nuclei integrate information from both pathways to fine-tune motor procedures and to acquire complex habits and skills. Therefore, balanced activity between both pathways is crucial for harmonious functions of the BG. Despite the increase in knowledge concerning the role of glutamate NMDA receptors (NMDA-Rs) in the striatum, understanding of the specific functions of NMDA-R iMSNs is still lacking. For this purpose, we generated a conditional knock-out mouse to address the functions of the NMDA-R in the indirect pathway. At the cellular level, deletion of GluN1 in iMSNs leads to a reduction in the number and strength of the excitatory corticostriatopallidal synapses. The subsequent scaling down in input integration leads to dysfunctional changes in BG output, which is seen as reduced habituation, delay in goal-directed learning, lack of associative behavior, and impairment in action selection or skill learning. The NMDA-R deletion in iMSNs causes a decrease in the synaptic strength of striatopallidal neurons, which in turn might lead to a imbalanced integration between direct and indirect MSN pathways, making mice less sensitive to environmental change. Therefore, their ability to learn and adapt to the environment-based experience was significantly affected.

SIGNIFICANCE STATEMENT

The striatum controls habits, locomotion, and goal-directed behaviors by coordinated activation of two antagonistic pathways. Insofar as NMDA receptors (NMDA-Rs) play a key role in synaptic plasticity essential for sustaining these behaviors, we generated a mouse model lacking NMDA-Rs specifically in striatopallidal neurons. To our knowledge, this is the first time that a specific deletion of inhibitory, indirect pathway medium-sized spiny neuron (iMSN) NMDA-Rs has been used to address the role of these receptors in the inhibitory pathway. Importantly, we found that this specific deletion led to a significant reduction in the number and strength of the cortico-iMSN synapses, which resulted in the significant impairments of behaviors orchestrated by the basal ganglia. Our findings indicate that the NMDA-Rs of the indirect pathway are essential for habituation, action selection, and goal-directed learning.

Emergence of visually-evoked reward expectation signals in dopamine neurons via the superior colliculus in V1 lesioned monkeys

Responses of midbrain dopamine (DA) neurons reflecting expected reward from sensory cues are critical for reward-based associative learning. However, critical pathways by which reward-related visual information is relayed to DA neurons remain unclear. To address this question, we investigated Pavlovian conditioning in macaque monkeys with unilateral primary visual cortex (V1) lesions (an animal model of ‘blindsight’). Anticipatory licking responses to obtain juice drops were elicited in response to visual conditioned stimuli (CS) in the affected visual field. Subsequent pharmacological inactivation of the superior colliculus (SC) suppressed the anticipatory licking. Concurrent single unit recordings indicated that DA responses reflecting the reward expectation could be recorded in the absence of V1, and that these responses were also suppressed by SC inactivation. These results indicate that the subcortical visual circuit can relay reward-predicting visual information to DA neurons and integrity of the SC is necessary for visually-elicited classically conditioned responses after V1 lesion.

DOI:http://dx.doi.org/10.7554/eLife.24459.001

To survive and thrive, animals must learn to approach cues in their environment that are likely to lead to a desirable outcome and avoid those that might lead them to harm. A group of brain regions known as the midbrain dopamine system helps many animals to achieve this. Dopamine is the brain’s reward signal. Cues that predict rewards, such as the sight or smell of food, activate midbrain dopamine neurons. However, the details of this process remained unclear.

Takakuwa et al. have now examined how visual information that signals reward reaches the midbrain dopamine neurons. The anatomy of the visual system suggests two main possibilities. Information may travel directly from the eyes to an area of the midbrain called the superior colliculus, and then onto the dopamine neurons. Alternatively, information may travel to the midbrain indirectly via a pathway that includes additional processing in the brain’s outer layer, the visual cortex.

To distinguish between these routes, Takakuwa et al. studied monkeys in which the indirect pathway via the visual cortex had been damaged. Some people with damage to this pathway have a disorder called blindsight. They are able to detect the movement or location of stimuli, but they cannot consciously see those stimuli. The monkeys with damage to visual cortex were able to learn that an image on a screen predicted the delivery of fruit juice. After repeated trials, the monkeys began to lick the spout dispensing the juice whenever the image appeared, even if no juice was delivered. The monkeys’ midbrain dopamine neurons also sent more signals in response to the images, and showed greater activity when the images predicted large rewards than small ones. Takakuwa et al. next inactivated the superior colliculus with a drug and showed that this prevented both the licking behavior and the increased signaling.

Together the findings show that visual information about potential rewards can reach midbrain dopamine neurons via a direct route through the superior colliculus, without needing to pass via the visual cortex. The next step is to determine how and when the visual cortex may get involved in this process to help animals maximize rewards.

DOI:http://dx.doi.org/10.7554/eLife.24459.002

Viral strategies for targeting cortical circuits that control cocaine-taking and cocaine-seeking in rodents

Addiction to cocaine is a chronic disease characterized by persistent drug-taking and drug-seeking behaviors, and a high likelihood of relapse. The prefrontal cortex (PFC) has long been implicated in the development of cocaine addiction, and relapse. However, the PFC is a heterogeneous structure, and understanding the role of PFC subdivisions, cell types and afferent/efferent connections is critical for gaining a comprehensive picture of the contribution of the PFC in addiction-related behaviors. Here we provide an update on the role of the PFC in cocaine addiction from recent work that used viral-mediated optogenetic and chemogenetic tools to study the role of the PFC in drug-taking and drug-seeking behavior in rodents. Following overviews of rodent PFC neuroanatomy and of viral-mediated optogenetic and chemogenetic techniques, we review studies of manipulations within the PFC, followed by a review of work that utilized targeted manipulations to PFC inputs and outputs.

Schizophrenia-like phenotypes in mice with NMDA receptor ablation in intralaminar thalamic nucleus cells and gene therapy-based reversal in adults

In understanding the mechanism of schizophrenia pathogenesis, a significant finding is that drug abuse of phencyclidine or its analog ketamine causes symptoms similar to schizophrenia. Such drug effects are triggered even by administration at post-adolescent stages. Both drugs are N-methyl-d-aspartate receptor (NMDAR) antagonists, leading to a major hypothesis that glutamate hypofunction underlies schizophrenia pathogenesis. The precise region that depends on NMDAR function, however, is unclear. Here, we developed a mouse strain in which NMDARs in the intralaminar thalamic nuclei (ILN) were selectively disrupted. The mutant mice exhibited various schizophrenia-like phenotypes, including deficits in working memory, long-term spatial memory, and attention, as well as impulsivity, impaired prepulse inhibition, hyperlocomotion and hyperarousal. The electroencephalography analysis revealed that the mutant mice had a significantly reduced power in a wide range of frequencies including the alpha, beta and gamma bands, both during wake and rapid eye movement (REM) sleep, and a modest decrease of gamma power during non-REM sleep. Notably, restoring NMDARs in the adult ILN rescued some of the behavioral abnormalities. These findings suggest that NMDAR dysfunction in the ILN contributes to the pathophysiology of schizophrenia-related disorders. Furthermore, the reversal of inherent schizophrenia-like phenotypes in the adult mutant mice supports that ILN is a potential target site for a therapeutic strategy.

Altered fronto-striatal functions in the Gdi1-null mouse model of X-linked Intellectual Disability

RAB-GDP dissociation inhibitor 1 (GDI1) loss-of-function mutations are responsible for a form of non-specific X-linked Intellectual Disability (XLID) where the only clinical feature is cognitive impairment. GDI1 patients are impaired in specific aspects of executive functions and conditioned response, which are controlled by fronto-striatal circuitries. Previous molecular and behavioral characterization of the Gdi1-null mouse revealed alterations in the total number/distribution of hippocampal and cortical synaptic vesicles as well as hippocampal short-term synaptic plasticity, and memory deficits. In this study, we employed cognitive protocols with high translational validity to human condition that target the functionality of cortico-striatal circuitry such as attention and stimulus selection ability with progressive degree of complexity. We previously showed that Gdi1-null mice are impaired in some hippocampus-dependent forms of associative learning assessed by aversive procedures. Here, using appetitive-conditioning procedures we further investigated associative learning deficits sustained by the fronto-striatal system. We report that Gdi1-null mice are impaired in attention and associative learning processes, which are a key part of the cognitive impairment observed in XLID patients.

Manipulating gene expression in projection-specific neuronal populations using combinatorial viral approaches

The mammalian brain contains tremendous structural and genetic complexity that is vital for its function. The elucidation of gene expression profiles in the brain, coupled with the development of large-scale connectivity maps and emerging viral vector-based approaches for target-selective gene manipulation, now allow for detailed dissection of gene-circuit interfaces. This protocol details how to perform combinatorial viral injections to manipulate gene expression in subsets of neurons interconnecting two brain regions. This method utilizes stereotaxic injection of a retrograde transducing CAV2-Cre virus into one brain region, combined with injection of a locally transducing Cre-dependent AAV virus into another brain region. This technique is widely applicable to the genetic dissection of neural circuitry, as it enables selective expression of candidate genes, dominant-negatives, fluorescent reporters, or genetic tools within heterogeneous populations of neurons based upon their projection targets.

Pontomesencephalic Tegmental Afferents to VTA Non-dopamine Neurons Are Necessary for Appetitive Pavlovian Learning

The ventral tegmental area (VTA) receives phenotypically distinct innervations from the pedunculopontine tegmental nucleus (PPTg). While PPTg-to-VTA inputs are thought to play a critical role in stimulus-reward learning, direct evidence linking PPTg-to-VTA phenotypically distinct inputs in the learning process remains lacking. Here, we used optogenetic approaches to investigate the functional contribution of PPTg excitatory and inhibitory inputs to the VTA in appetitive Pavlovian conditioning. We show that photoinhibition of PPTg-to-VTA cholinergic or glutamatergic inputs during cue presentation dampens the development of anticipatory approach responding to the food receptacle during the cue. Furthermore, we employed in vivo optetrode recordings to show that photoinhibition of PPTg cholinergic or glutamatergic inputs significantly decreases VTA non-dopamine (non-DA) neural activity. Consistently, photoinhibition of VTA non-DA neurons disrupts the development of cue-elicited anticipatory approach responding. Taken together, our study reveals a crucial regulatory mechanism by PPTg excitatory inputs onto VTA non-DA neurons during appetitive Pavlovian conditioning.

NMDA Receptors on Dopaminoceptive Neurons Are Essential for Drug-Induced Conditioned Place Preference

Plasticity of the brain's dopamine system plays a crucial role in adaptive behavior by regulating appetitive motivation and the control of reinforcement learning. In this study, we investigated drug- and natural-reward conditioned behaviors in a mouse model in which the NMDA receptor-dependent plasticity of dopaminoceptive neurons was disrupted. We generated a transgenic mouse line with inducible selective inactivation of the NR1 subunit in neurons expressing dopamine D1 receptors (the NR1(D1CreERT2) mice). Whole-cell recordings of spontaneous EPSCs on neurons in the nucleus accumbens confirmed that a population of neurons lacked the NMDA receptor-dependent component of the current. This effect was accompanied by impaired long-term potentiation in the nucleus accumbens and in the CA1 area of the ventral, but not the dorsal, hippocampus. Mutant mice did not differ from control animals when tested for pavlovian or instrumental conditioning. However, NR1(D1CreERT2) mice acquired no preference for a context associated with administration of drugs of abuse. In the conditioned place preference paradigm, mutant mice did not spend more time in the context paired with cocaine, morphine, or ethanol, although these mice acquired a preference for sucrose jelly and an aversion to naloxone injections, as normal. Thus, we observed that the selective inducible ablation of the NMDA receptors specifically blocks drug-associated context memory with no effect on positive reinforcement in general.

N-methyl-D-aspartate receptors in the ventral tegmental area mediate the excitatory influence of Pavlovian stimuli on instrumental performance

Pavlovian stimuli predictive of food can markedly amplify instrumental responding for food. This effect is termed Pavlovian-instrumental transfer (PIT). The ventral tegmental area (VTA) plays a key role in mediating PIT, however, it is yet unknown whether N-methyl-D-aspartate (NMDA)-type glutamate receptors in the VTA are involved in PIT. Here, we examined the effects of an NMDA-receptor blockade in the VTA on PIT. Immediately prior to PIT testing, rats were subjected to intra-VTA infusions of vehicle or of the NMDA-receptor antagonist 2-amino-5-phosphonopentanoic acid (AP-5) (1, 5 µg/side). In rats that received AP-5 at the lower dose, the PIT effect was intact, i.e. presentation of the Pavlovian stimulus enhanced instrumental responding. By contrast, in rats that received AP-5 at the higher dose, the PIT effect was blocked. The data suggest that NMDA receptors in the VTA mediate the activating effects of Pavlovian stimuli on instrumental responding.

Neuronal Reward and Decision Signals: From Theories to Data

Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neuronal populations. The approach and choice functions involve subjective value, which is objectively assessed by behavioral choices eliciting internal, subjective reward preferences. Utility is the formal mathematical characterization of subjective value and a prime decision variable in economic choice theory. It is coded as utility prediction error by phasic dopamine responses. Utility can incorporate various influences, including risk, delay, effort, and social interaction. Appropriate for formal decision mechanisms, rewards are coded as object value, action value, difference value, and chosen value by specific neurons. Although all reward, reinforcement, and decision variables are theoretical constructs, their neuronal signals constitute measurable physical implementations and as such confirm the validity of these concepts. The neuronal reward signals provide guidance for behavior while constraining the free will to act.

Compromised NMDA/Glutamate Receptor Expression in Dopaminergic Neurons Impairs Instrumental Learning, But Not Pavlovian Goal Tracking or Sign Tracking

Behavior is shaped to a dramatic degree by the occurrence of rewards, through both pavlovian and instrumental conditioning processes; these mechanisms give rise to both normal and abnormal behavior. It is crucial to understand the neural mechanisms that give rise to normal actions and how they lead to pathological behaviors, such as overeating and drug addictions.

Two theories regarding the role for dopamine neurons in learning include the concepts that their activity serves as a (1) mechanism that confers incentive salience onto rewards and associated cues and/or (2) contingency teaching signal reflecting reward prediction error. While both theories are provocative, the causal role for dopamine cell activity in either mechanism remains controversial. In this study mice that either fully or partially lacked NMDARs in dopamine neurons exclusively, as well as appropriate controls, were evaluated for reward-related learning; this experimental design allowed for a test of the premise that NMDA/glutamate receptor (NMDAR)-mediated mechanisms in dopamine neurons, including NMDA-dependent regulation of phasic discharge activity of these cells, modulate either the instrumental learning processes or the likelihood of pavlovian cues to become highly motivating incentive stimuli that directly attract behavior. Loss of NMDARs in dopamine neurons did not significantly affect baseline dopamine utilization in the striatum, novelty evoked locomotor behavior, or consumption of a freely available, palatable food solution. On the other hand, animals lacking NMDARs in dopamine cells exhibited a selective reduction in reinforced lever responses that emerged over the course of instrumental learning. Loss of receptor expression did not, however, influence the likelihood of an animal acquiring a pavlovian conditional response associated with attribution of incentive salience to reward-paired cues (sign tracking). These data support the view that reductions in NMDAR signaling in dopamine neurons affect instrumental reward-related learning but do not lend support to hypotheses that suggest that the behavioral significance of this signaling includes incentive salience attribution.

Spine pruning drives antipsychotic-sensitive locomotion via circuit control of striatal dopamine

Psychiatric and neurodevelopmental disorders may arise from anomalies in long-range neuronal connectivity downstream of pathologies in dendritic spines. However, the mechanisms that may link spine pathology to circuit abnormalities relevant to atypical behavior remain unknown. Using a mouse model to conditionally disrupt a critical regulator of the dendritic spine cytoskeleton, Arp2/3, we report here a molecular mechanism that unexpectedly reveals the interrelationship of progressive spine pruning, elevated frontal cortical excitation of pyramidal neurons, and striatal hyperdopaminergia within a cortical-to-midbrain circuit abnormality. The main symptomatic manifestations of this circuit abnormality are psychomotor agitation and stereotypical behaviors, which are relieved by antipsychotics. Moreover, antipsychotic-responsive locomotion can be directly mimicked in wildtype mice by optogenetic activation of this circuit. Collectively these results reveal molecular and neural-circuit mechanisms, illustrating how diverse pathologies may converge to drive behaviors relevant to psychiatric disorders.

Specific contributions of N-methyl-D-aspartate receptors in the dorsal striatum to cognitive flexibility

Behavioral flexibility is known to be mediated by corticostriatal systems and to involve several major neurotransmitter signaling pathways. The current study investigated the effects of inactivation of glutamatergic N-methyl-D-aspartate-(NMDA) receptor signaling in the dorsal striatum on behavioral flexibility in mice. NMDA-receptor inactivation was achieved by virus-mediated inactivation of the Grin1 gene, which encodes the essential NR1 subunit of NMDA receptors. To assess behavioral flexibility, we used a water U-maze paradigm in which mice had to shift from an initially acquired rule to a new rule (strategy shifting) or had to reverse an initially learned rule (reversal learning). Inactivation of NMDA-receptors in all neurons of the dorsal striatum did not affect learning of the initial rule or reversal learning, but impaired shifting from one strategy to another. Strategy shifting was also compromised when NMDA-receptors were inactivated only in dynorphin-expressing neurons in the dorsal striatum, which represent the direct pathway. These data suggest that NMDA-receptor-mediated synaptic plasticity in the dorsal striatum contributes to strategy shifting and that striatal projection neurons of the direct pathway are particularly relevant for this process.

Dopamine dependency for acquisition and performance of Pavlovian conditioned response

During Pavlovian conditioning, pairing of a neutral conditioned stimulus (CS) with a reward leads to conditioned reward-approach responses (CRs) that are elicited by presentation of the CS. CR behaviors can be sign tracking, in which animals engage the CS, or goal tracking, in which animals go to the reward location. We investigated CR behaviors in mice with only ∼5% of normal dopamine in the striatum using a Pavlovian conditioning paradigm. These mice had severely impaired acquisition of the CR, which was ameliorated by pharmacological restoration of dopamine synthesis with l-dopa. Surprisingly, after they had learned the CR, its expression decayed only gradually in following sessions that were conducted without l-dopa treatment. To assess specific contributions of dopamine signaling in the dorsal or ventral striatum, we performed virus-mediated restoration of dopamine synthesis in completely dopamine-deficient (DD) mice. Mice with dopamine signaling only in the dorsal striatum did not acquire a CR, whereas mice with dopamine signaling only in in the ventral striatum acquired a CR. The CR in mice with dopamine signaling only in the dorsal striatum was restored by subjecting the mice to instrumental training in which they had to interact with the CS to obtain rewards. We conclude that dopamine is essential for learning and performance of CR behavior that is predominantly goal tracking. Furthermore, although dopamine signaling in the ventral striatum is sufficient to support a CR, dopamine signaling only in the dorsal striatum can also support a CR under certain circumstances.

Defining functional gene-circuit interfaces in the mouse nervous system

Complexity in the nervous system is established by developmental genetic programs, maintained by differential genetic profiles and sculpted by experiential and environmental influence over gene expression. Determining how specific genes define neuronal phenotypes, shape circuit connectivity and regulate circuit function is essential for understanding how the brain processes information, directs behavior and adapts to changing environments. Mouse genetics has contributed greatly to current percepts of gene-circuit interfaces in behavior, but considerable work remains. Large-scale initiatives to map gene expression and connectivity in the brain, together with advanced techniques in molecular genetics, now allow detailed exploration of the genetic basis of nervous system function at the level of specific circuit connections. In this review, we highlight several key advances for defining the function of specific genes within a neural network.

GluN2B in corticostriatal circuits governs choice learning and choice shifting

A choice that reliably produces a preferred outcome can be automated to liberate cognitive resources for other tasks. Should an outcome become less desirable, behavior must adapt in parallel or it becomes perseverative. Corticostriatal systems are known to mediate choice learning and flexibility, but the molecular mechanisms of these processes are not well understood. We integrated mouse behavioral, immunocytochemical, in vivo electrophysiological, genetic and pharmacological approaches to study choice. We found that the dorsal striatum (DS) was increasingly activated with choice learning, whereas reversal of learned choice engaged prefrontal regions. In vivo, DS neurons showed activity associated with reward anticipation and receipt that emerged with learning and relearning. Corticostriatal or striatal deletion of Grin2b (encoding the NMDA-type glutamate receptor subunit GluN2B) or DS-restricted GluN2B antagonism impaired choice learning, whereas cortical Grin2b deletion or OFC GluN2B antagonism impaired shifting. Our convergent data demonstrate how corticostriatal GluN2B circuits govern the ability to learn and shift choice behavior.

Amphetamine-induced sensitization has little effect on multiple learning paradigms and fails to rescue mice with a striatal learning defect

Behavioral sensitization to psychostimulants such as amphetamine (AMPH) is associated with synaptic modifications that are thought to underlie learning and memory. Because AMPH enhances extracellular dopamine in the striatum where dopamine and glutamate signaling are essential for learning, one might expect that the molecular and morphological changes that occur in the striatum in response to AMPH, including changes in synaptic plasticity, would affect learning. To ascertain whether AMPH sensitization affects learning, we tested wild-type mice and mice lacking NMDA receptor signaling in striatal medium spiny neurons in several different learning tests (motor learning, Pavlovian association, U-maze escape test with strategy shifting) with or without prior sensitization to AMPH. Prior sensitization had minimal effect on learning in any of these paradigms in wild-type mice and failed to restore learning in mutant mice, despite the fact that the mutant mice became sensitized by the AMPH treatment. We conclude that the changes in synaptic plasticity and many other signaling events that occur in response to AMPH sensitization are dissociable from those involved in learning the tasks used in our experiments.

Updating dopamine reward signals

Recent work has advanced our knowledge of phasic dopamine reward prediction error signals. The error signal is bidirectional, reflects well the higher order prediction error described by temporal difference learning models, is compatible with model-free and model-based reinforcement learning, reports the subjective rather than physical reward value during temporal discounting and reflects subjective stimulus perception rather than physical stimulus aspects. Dopamine activations are primarily driven by reward, and to some extent risk, whereas punishment and salience have only limited activating effects when appropriate controls are respected. The signal is homogeneous in terms of time course but heterogeneous in many other aspects. It is essential for synaptic plasticity and a range of behavioural learning situations.

K-ATP channels in dopamine substantia nigra neurons control bursting and novelty-induced exploration

Phasic activation of the dopamine (DA) midbrain system in response to unexpected reward or novelty is critical for adaptive behavioral strategies. This activation of DA midbrain neurons occurs via a synaptically triggered switch from low-frequency background spiking to transient high-frequency burst firing. We found that, in medial DA neurons of the substantia nigra (SN), activity of ATP-sensitive potassium (K-ATP) channels enabled NMDA-mediated bursting in vitro as well as spontaneous in vivo burst firing in anesthetized mice. Cell-selective silencing of K-ATP channel activity in medial SN DA neurons revealed that their K-ATP channel-gated burst firing was crucial for novelty-dependent exploratory behavior. We also detected a transcriptional upregulation of K-ATP channel and NMDA receptor subunits, as well as high in vivo burst firing, in surviving SN DA neurons from Parkinson's disease patients, suggesting that burst-gating K-ATP channel function in DA neurons affects phenotypes in both disease and health.

Utility of genetically modified mice for understanding the neurobiology of substance use disorders

Advances in our ability to modify the mouse genome have enhanced our understanding of the genetic and neurobiological mechanisms contributing to addiction-related behaviors underlying substance use and abuse. These experimentally induced manipulations permit greater spatial and temporal specificity for modification of gene expression within specific cellular populations and during select developmental time periods. In this review, we consider the current mouse genetic model systems that have been employed to understand aspects of addiction and highlight significant conceptual advances achieved related to substance use and abuse. The mouse models reviewed herein include conventional knock-out and knock-in, conditional knockout, transgenic, inducible transgenic, mice suitable for optogenetic control of discrete neuronal populations, and phenotype-selected mice. By establishing a reciprocal investigatory relationship between genetic findings in humans and genomic manipulations in mice, a far better understanding of the discrete neuromechanisms underlying addiction can be achieved, which is likely to provide a strong foundation for developing and validating novel therapeutics for the treatment of substance abuse disorders.

Subcellular location of PKA controls striatal plasticity: stochastic simulations in spiny dendrites

Dopamine release in the striatum has been implicated in various forms of reward dependent learning. Dopamine leads to production of cAMP and activation of protein kinase A (PKA), which are involved in striatal synaptic plasticity and learning. PKA and its protein targets are not diffusely located throughout the neuron, but are confined to various subcellular compartments by anchoring molecules such as A-Kinase Anchoring Proteins (AKAPs). Experiments have shown that blocking the interaction of PKA with AKAPs disrupts its subcellular location and prevents LTP in the hippocampus and striatum; however, these experiments have not revealed whether the critical function of anchoring is to locate PKA near the cAMP that activates it or near its targets, such as AMPA receptors located in the post-synaptic density. We have developed a large scale stochastic reaction-diffusion model of signaling pathways in a medium spiny projection neuron dendrite with spines, based on published biochemical measurements, to investigate this question and to evaluate whether dopamine signaling exhibits spatial specificity post-synaptically. The model was stimulated with dopamine pulses mimicking those recorded in response to reward. Simulations show that PKA colocalization with adenylate cyclase, either in the spine head or in the dendrite, leads to greater phosphorylation of DARPP-32 Thr34 and AMPA receptor GluA1 Ser845 than when PKA is anchored away from adenylate cyclase. Simulations further demonstrate that though cAMP exhibits a strong spatial gradient, diffusible DARPP-32 facilitates the spread of PKA activity, suggesting that additional inactivation mechanisms are required to produce spatial specificity of PKA activity.

The striatum is a part of the basal ganglia which plays a role in addiction and reward learning. Its importance is underscored by pathologies such as Parkinson's disease and Huntington's disease in which degeneration of the dopamine inputs to the striatum or degeneration of neurons in the striatum, respectively, produces motor dysfunction. Dopamine in the striatum activates cascades of signaling molecules, ultimately producing an activity dependent change in the strength of connections between neurons. However, the dispersive movement of signaling molecules seems incompatible with the strengthening of specific subsets of connections, which is required for formation of distinct memories. Anchoring proteins, which restrict molecules to particular compartments within the neuron, are proposed to achieve specificity. We develop a reaction-diffusion model of dopamine activated signaling pathways to explore mechanisms whereby anchoring proteins can produce specificity. We use an efficient Monte-Carlo simulator to implement the cascades of signaling molecules in a neuronal dendrite with multiple dendritic spines. Simulations demonstrate that spatial specificity requires both anchoring proteins and inactivation mechanisms that limit the diffusion of signaling molecules.

Severely impaired learning and altered neuronal morphology in mice lacking NMDA receptors in medium spiny neurons

The striatum is composed predominantly of medium spiny neurons (MSNs) that integrate excitatory, glutamatergic inputs from the cortex and thalamus, and modulatory dopaminergic inputs from the ventral midbrain to influence behavior. Glutamatergic activation of AMPA, NMDA, and metabotropic receptors on MSNs is important for striatal development and function, but the roles of each of these receptor classes remain incompletely understood. Signaling through NMDA-type glutamate receptors (NMDARs) in the striatum has been implicated in various motor and appetitive learning paradigms. In addition, signaling through NMDARs influences neuronal morphology, which could underlie their role in mediating learned behaviors. To study the role of NMDARs on MSNs in learning and in morphological development, we generated mice lacking the essential NR1 subunit, encoded by the Grin1 gene, selectively in MSNs. Although these knockout mice appear normal and display normal 24-hour locomotion, they have severe deficits in motor learning, operant conditioning and active avoidance. In addition, the MSNs from these knockout mice have smaller cell bodies and decreased dendritic length compared to littermate controls. We conclude that NMDAR signaling in MSNs is critical for normal MSN morphology and many forms of learning.