Stimulus-response and response-outcome learning mechanisms in the striatum

Ideomotor learning: Time to generalize a longstanding principle

The ideomotor principle holds that anticipating the sensory consequences of a movement triggers an associated motor response. Even though this framework dates back to the 19th century, it continues to lie at the heart of many contemporary approaches to human action control. Here we specifically focus on the ideomotor learning mechanism that has to precede action initiation via effect anticipation. Traditional approaches to this learning mechanism focused on establishing novel action-effect (or response-effect) associations. Here we apply the theoretical concept of common coding for action and perception to argue that the same learning principle should result in response-response and stimulus-stimulus associations just as well. Generalizing ideomotor learning in such a way results in a powerful and general framework of ideomotor action control, and it allows for integrating the two seemingly separate fields of ideomotor approaches and hierarchical learning.

The acute exercise-cognition interaction: From the catecholamines hypothesis to an interoception model

An interoception model for the acute exercise-cognition interaction is presented. During exercise following the norepinephrine threshold, interoceptive feedback induces increased tonic release of extracellular catecholamines, facilitating phasic release hence better cognitive performance of executive functions. When exercise intensity increases to maximum, the nature of task-induced norepinephrine release from the locus coeruleus is dependent on interaction between motivation, perceived effort costs and perceived availability of resources. This is controlled by interaction between the rostral and dorsolateral prefrontal cortices, orbitofrontal cortex, anterior cingulate cortex and anterior insula cortex. If perceived available resources are sufficient to meet predicted effort costs and reward value is high, tonic release from the locus coeruleus is attenuated thus facilitating phasic release, therefore cognition is not inhibited. However, if perceived available resources are insufficient to meet predicted effort costs or reward value is low, tonic release from the locus coeruleus is induced, attenuating phasic release. As a result, cognition is inhibited, although long-term memory and tasks that require switching to new stimuli-response couplings are probably facilitated.

Retrograde Labeling Illuminates Distinct Topographical Organization of D1 and D2 Receptor-Positive Pyramidal Neurons in the Prefrontal Cortex of Mice

The cortex plays an important role in regulating motivation and cognition, and does so by regulating multiple subcortical brain circuits. Glutamatergic pyramidal neurons in the prefrontal cortex (PFC) are topographically organized in different subregions such as the prelimbic, infralimbic (IL), and orbitofrontal and project to topographically-organized subcortical target regions. Dopamine D1 and D2 receptors are expressed on glutamatergic pyramidal neurons in the PFC. However, it is unclear whether D1 and D2 receptor-expressing pyramidal neurons in the PFC are also topographically organized. We used a retrograde adeno-associated virus (AAVRG)-based approach to illuminate the topographical organization of D1 and D2 receptor-expressing neurons, projecting to distinct striatal and midbrain subregions. Our experiments reveal that AAVRG injection in the nucleus accumbens (NAcc) or dorsal striatum (dSTR) of D1Cre mice labeled distinct neuronal subpopulations in medial orbitofrontal or prelimbic PFC, respectively. However, AAVRG injection in NAcc or dSTR of D2Cre mice labeled medial orbitofrontal, but not medial prelimbic PFC, respectively. Additionally, D2R+ but not D1R+ PFC neurons were labeled on injection of AAVRG in substantia nigra pars compacta (SNpc). Thus, our data are the first to highlight a unique dopamine receptor-specific topographical pattern in the PFC, which could have profound implications for corticostriatal signaling in the basal ganglia.

A role for the medial temporal lobes in category learning

Despite much research, the role of the medial temporal lobes (MTL) in category learning is unclear. Two unstructured categorization experiments explored conditions that might recruit MTL category learning and memory systems—namely, whether the stimulus display includes one or two stimuli, and whether category membership depends on configural properties of the stimulus features. The results supported three conclusions. First, in agreement with prior research, learning with single stimulus displays depended on striatal-mediated procedural learning. Second, and most important, learning with pair displays was mediated by MTL declarative memory systems. Third, the use of stimuli in which category membership depends on configural properties of the stimulus features made MTL learning slightly more likely. Overall, the results suggested that the MTL are most likely to mediate learning when the participant must decide which of two configural stimuli belongs to a selected category.

Synergy of Distinct Dopamine Projection Populations in Behavioral Reinforcement

Dopamine neurons of the ventral tegmental area (VTA) regulate reward association and motivation. It remains unclear whether there are distinct dopamine populations to mediate these functions. Using mouse genetics, we isolated two populations of dopamine-producing VTA neurons with divergent projections to the nucleus accumbens (NAc) core and shell. Inhibition of VTA-core-projecting neurons disrupted Pavlovian reward learning, and activation of these cells promoted the acquisition of an instrumental response. VTA-shell-projecting neurons did not regulate Pavlovian reward learning and could not facilitate acquisition of an instrumental response, but their activation could drive robust responding in a previously learned instrumental task. Both populations are activated simultaneously by cues, actions, and rewards, and this co-activation is required for robust reinforcement of behavior. Thus, there are functionally distinct dopamine populations in the VTA for promoting motivation and reward association, which operate on the same timescale to optimize behavioral reinforcement.

The role of the nucleus accumbens in learned approach behavior diminishes with training

Nucleus accumbens dopamine plays a key role in reward-directed approach. Past findings suggest that dopamine's role in the expression of learned behavior diminishes with extended training. However, little is known about the central substrates that mediate the shift to dopamine-independent reward approach. In the present study, rats approached and inserted the head into a reward compartment in response to a cue signaling food delivery. On days 4 and 5 of 28-trial-per-day sessions, D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390) infused to the NAc core reduced the probability and speed of cued approach. The disruptive effect of D1 receptor blockade was specific to the nucleus accumbens core and not seen with drug infusions to nearby dopamine target regions. In rats that received drug infusions after extended training (days 10 or 11), accumbens core D1 receptor blockade produced little effect on the expression of the same behavior. These results could have been due to a continued accumbens mediation of cued approach even after the behavior had become independent of accumbens D1 receptors. However, accumbens core ionotropic glutamate receptor blockade disrupted cued approach during early but not late stages of training, similar to the effects of D1 antagonist infusions. The results suggest that with extended training, a nucleus accumbens D1-dependent behavior becomes less dependent not only on nucleus accumbens D1 transmission but also on excitatory transmission in the nucleus accumbens. These findings fill an important gap in a growing literature on reorganization of striatal function over the course of training.

Contributions of the basal ganglia to action sequence learning and performance

Animals engage in intricately woven and choreographed action sequences that are constructed from trial-and-error learning. The mechanisms by which the brain links together individual actions which are later recalled as fluid chains of behavior are not fully understood, but there is broad consensus that the basal ganglia play a crucial role in this process. This paper presents a comprehensive review of the role of the basal ganglia in action sequencing, with a focus on whether the computational framework of reinforcement learning can capture key behavioral features of sequencing and the neural mechanisms that underlie them. While a simple neurocomputational model of reinforcement learning can capture key features of action sequence learning, this model is not sufficient to capture goal-directed control of sequences or their hierarchical representation. The hierarchical structure of action sequences, in particular, poses a challenge for building better models of action sequencing, and it is in this regard that further investigations into basal ganglia information processing may be informative.

Coding of self-motion-induced and self-independent visual motion in the rat dorsomedial striatum

Evolutionary development of vision has provided us with the capacity to detect moving objects. Concordant shifts of visual features suggest movements of the observer, whereas discordant changes are more likely to be indicating independently moving objects, such as predators or prey. Such distinction helps us to focus attention, adapt our behavior, and adjust our motor patterns to meet behavioral challenges. However, the neural basis of distinguishing self-induced and self-independent visual motions is not clarified in unrestrained animals yet. In this study, we investigated the presence and origin of motion-related visual information in the striatum of rats, a hub of action selection and procedural memory. We found that while almost half of the neurons in the dorsomedial striatum are sensitive to visual motion congruent with locomotion (and that many of them also code for spatial location), only a small subset of them are composed of fast-firing interneurons that could also perceive self-independent visual stimuli. These latter cells receive their visual input at least partially from the secondary visual cortex (V2). This differential visual sensitivity may be an important support in adjusting behavior to salient environmental events. It emphasizes the importance of investigating visual motion perception in unrestrained animals.

Behavioural and neural sequelae of stressor exposure are not modulated by controllability in females

The degree of behavioural control that an organism has over a stressor is a potent modulator of the stressor's impact; controllable stressors produce none of the neurochemical and behavioural sequelae that occur if the stressor is uncontrollable. Research demonstrating the importance of control and the neural mechanisms responsible has been conducted almost entirely with male rats. It is unknown if behavioural control is stress blunting in females, and whether or not a similar resilience circuitry is engaged. Female rats were exposed to controllable, yoked uncontrollable or no tailshock. In separate experiments, behavioural (juvenile social exploration, fear and shuttle box escape) and neurochemical (activation of dorsal raphe serotonin and dorsal raphe-projecting prelimbic neurons) outcomes, which are sensitive to the dimension of control in males, were assessed. Despite successful acquisition of the controlling response, behavioural control did not mitigate dorsal raphe serotonergic activation and behavioural outcomes induced by tailshock, as it does in males. Moreover, behavioural control failed to selectively engage prelimbic cells that project to the dorsal raphe as in males. Pharmacological activation of the prelimbic cortex restored the stress-buffering effects of control. Collectively, the data demonstrate stressor controllability phenomena are absent in females and that the protective prelimbic circuitry is present but not engaged. Reduced benefit from coping responses may represent a novel approach for understanding differential sex prevalence in stress-related psychiatric disorders.

Striatopallidal Neuron NMDA Receptors Control Synaptic Connectivity, Locomotor, and Goal-Directed Behaviors

The basal ganglia (BG) control action selection, motor programs, habits, and goal-directed learning. The striatum, the principal input structure of BG, is predominantly composed of medium-sized spiny neurons (MSNs). Arising from these spatially intermixed MSNs, two inhibitory outputs form two main efferent pathways, the direct and indirect pathways. Striatonigral MSNs give rise to the activating, direct pathway MSNs and striatopallidal MSNs to the inhibitory, indirect pathway (iMSNs). BG output nuclei integrate information from both pathways to fine-tune motor procedures and to acquire complex habits and skills. Therefore, balanced activity between both pathways is crucial for harmonious functions of the BG. Despite the increase in knowledge concerning the role of glutamate NMDA receptors (NMDA-Rs) in the striatum, understanding of the specific functions of NMDA-R iMSNs is still lacking. For this purpose, we generated a conditional knock-out mouse to address the functions of the NMDA-R in the indirect pathway. At the cellular level, deletion of GluN1 in iMSNs leads to a reduction in the number and strength of the excitatory corticostriatopallidal synapses. The subsequent scaling down in input integration leads to dysfunctional changes in BG output, which is seen as reduced habituation, delay in goal-directed learning, lack of associative behavior, and impairment in action selection or skill learning. The NMDA-R deletion in iMSNs causes a decrease in the synaptic strength of striatopallidal neurons, which in turn might lead to a imbalanced integration between direct and indirect MSN pathways, making mice less sensitive to environmental change. Therefore, their ability to learn and adapt to the environment-based experience was significantly affected.

SIGNIFICANCE STATEMENT

The striatum controls habits, locomotion, and goal-directed behaviors by coordinated activation of two antagonistic pathways. Insofar as NMDA receptors (NMDA-Rs) play a key role in synaptic plasticity essential for sustaining these behaviors, we generated a mouse model lacking NMDA-Rs specifically in striatopallidal neurons. To our knowledge, this is the first time that a specific deletion of inhibitory, indirect pathway medium-sized spiny neuron (iMSN) NMDA-Rs has been used to address the role of these receptors in the inhibitory pathway. Importantly, we found that this specific deletion led to a significant reduction in the number and strength of the cortico-iMSN synapses, which resulted in the significant impairments of behaviors orchestrated by the basal ganglia. Our findings indicate that the NMDA-Rs of the indirect pathway are essential for habituation, action selection, and goal-directed learning.

A Dynamic Circuit Hypothesis for the Pathogenesis of Blepharospasm

Blepharospasm (sometimes called “benign essential blepharospasm,” BEB) is one of the most common focal dystonias. It involves involuntary eyelid spasms, eye closure, and increased blinking. Despite the success of botulinum toxin injections and, in some cases, pharmacologic or surgical interventions, BEB treatments are not completely efficacious and only symptomatic. We could develop principled strategies for preventing and reversing the disease if we knew the pathogenesis of primary BEB. The objective of this study was to develop a conceptual framework and dynamic circuit hypothesis for the pathogenesis of BEB. The framework extends our overarching theory for the multifactorial pathogenesis of focal dystonias (Peterson et al., 2010) to incorporate a two-hit rodent model specifically of BEB (Schicatano et al., 1997). We incorporate in the framework three features critical to cranial motor control: (1) the joint influence of motor cortical regions and direct descending projections from one of the basal ganglia output nuclei, the substantia nigra pars reticulata, on brainstem motor nuclei, (2) nested loops composed of the trigeminal blink reflex arc and the long sensorimotor loop from trigeminal nucleus through thalamus to somatosensory cortex back through basal ganglia to the same brainstem nuclei modulating the reflex arc, and (3) abnormalities in the basal ganglia dopamine system that provide a sensorimotor learning substrate which, when combined with patterns of increased blinking, leads to abnormal sensorimotor mappings manifest as BEB. The framework explains experimental data on the trigeminal reflex blink excitability (TRBE) from Schicatano et al. and makes predictions that can be tested in new experimental animal models based on emerging genetics in dystonia, including the recently characterized striatal-specific D1R dopamine transduction alterations caused by the GNAL mutation. More broadly, the model will provide a guide for future efforts to mechanistically link multiple factors in the pathogenesis of BEB and facilitate simulations of how exogenous manipulations of the pathogenic factors could ultimately be used to prevent and reverse the disorder.

Determinants of conditioned reinforcing effectiveness: Dopamine D2-like receptor agonist-stimulated responding for cocaine-associated stimuli

Environmental stimuli associated with drug use can take on conditioned properties capable of promoting drug-seeking behaviors during abstinence. This study investigated the relative importance of the amount of reinforced responding, number of cocaine-stimulus pairings, total cocaine intake, and reinforcing effectiveness of the self-administered dose of cocaine to the conditioned reinforcing effectiveness of cocaine-associated stimuli (CS). Male rats were trained to self-administer cocaine (0.1 [small] or 1.0mg/kg/inf [large]) under a fixed ratio schedule of reinforcement. A progressive ratio (PR) schedule was used to quantify the reinforcing effectiveness of each dose of cocaine, as well as the conditioned reinforcing effectiveness of the CS following treatment with saline or the dopamine D2-like receptor agonist pramipexole (0.1-3.2mg/kg). The large dose of cocaine maintained larger final ratios and greater levels of cocaine intake, whereas the small dose resulted in more cocaine-CS pairings. The total amount of responding was comparable between groups. During PR tests of conditioned reinforcement, pramipexole increased responding for CS presentations in both groups; however, the final ratio completed was significantly greater in large- as compared to small-dose group. In addition to highlighting a central role for dopamine D2-like receptors in modulating the effectiveness of cocaine-paired stimuli to reinforce behavior, these results suggest that conditioned reinforcing effectiveness is primarily determined by the reinforcing effectiveness of the self-administered dose of cocaine and/or total cocaine intake, and not the total amount of responding or number cocaine-stimulus pairings. These findings have implications for understanding how different patterns of drug-taking might impact vulnerability to relapse.

Dissociable contribution of nucleus accumbens and dorsolateral striatum to the acquisition of risk choice behavior in the rat

While a growing body of research has suggested that the mesocorticolimbic dopamine systems play a key role in decision making under risk, how the nucleus accumbens (NAC) is involved in the acquisition of risk choice behavior remains unclear. This study used a T-maze task to assess risk-based decision making in which the rat was required to assess the risk by choosing to enter either a small and certain reward arm or a large but uncertain reward arm of the maze. The latter option, when chosen, resulted in provision of 2, 4, or 8 sweeten pellets with a probability (p) of 0.5, 0.25, or 0.125, respectively. Thus the latter arm provided three different conditions of reward ratio, compared to the choice of former arm, which always provided 1 pellet with p=1. This risk choice task was then run with the expected value being equality between the binary choice options. The experimental rats first received an excitoneurotoxic lesion affecting either the NAC or the dorsolateral striatum (DLS) and this was followed by post-lesion behavioral examination. The sham lesion control rats acquired a stable risk choice with regard to each reward ratio over a 10-day test. The pattern of choice behavior appeared in risk-seeking when p=0.5 to obtain 2 pellets, and was risk-averse when larger reward resulted in lower p. The NAC lesion significantly disrupted the acquisition of the aforementioned risk choice behavior and apparently shifted the choice into a risk-averse style for all three reward ratios. No such effect was observed in the rats with DLS lesions. Neither the gross motor action nor the discrimination of different reward magnitudes was impaired by the lesions affecting either the NAC or DLS as assessed by an additional experiment. These findings suggest that firstly there is heterogeneity between NAC and DLS with respect to risk-based decision making, and that secondly the NAC is involved and critical to the acquisition of behavioral choice under risk, specially when the expected value of the reward under the two choice options is equal.

Differential contributions of the globus pallidus and ventral thalamus to stimulus-response learning in humans

The ability to learn associations between stimuli, responses and rewards is a prerequisite for survival. Models of reinforcement learning suggest that the striatum, a basal ganglia input nucleus, vitally contributes to these learning processes. Our recently presented computational model predicts, first, that not only the striatum, but also the globus pallidus contributes to the learning (i.e., exploration) of stimulus-response associations based on rewards. Secondly, it predicts that the stable execution (i.e., exploitation) of well-learned associations involves further learning in the thalamus. To test these predictions, we postoperatively recorded local field potentials (LFPs) from patients that had undergone surgery for deep brain stimulation to treat severe movement disorders. Macroelectrodes were placed either in the globus pallidus or in the ventral thalamus. During recordings, patients performed a reward-based stimulus-response learning task that comprised periods of exploration and exploitation. We analyzed correlations between patients' LFP amplitudes and model-based estimates of their reward expectations and reward prediction errors. In line with our first prediction, pallidal LFP amplitudes during the presentation of rewards and reward omissions correlated with patients' reward prediction errors, suggesting pallidal access to reward-based teaching signals. Unexpectedly, the same was true for the thalamus. In further support of this prediction, pallidal LFP amplitudes during stimulus presentation correlated with patients' reward expectations during phases of low reward certainty - suggesting pallidal participation in the learning of stimulus-response associations. In line with our second prediction, correlations between thalamic stimulus-related LFP amplitudes and patients' reward expectations were significant within phases of already high reward certainty, suggesting thalamic participation in exploitation.

Molecular and cellular mechanisms of dopamine-mediated behavioral plasticity in the striatum

The striatum is the input structure of the basal ganglia system. By integrating glutamatergic signals from cortical and subcortical regions and dopaminergic signals from mesolimbic nuclei the striatum functions as an important neural substrate for procedural and motor learning as well as for reward-guided behaviors. In addition, striatal activity is significantly altered in pathological conditions in which either a loss of dopamine innervation (Parkinson's disease) or aberrant dopamine-mediated signaling (drug addiction and L-DOPA induced dyskinesia) occurs. Here we discuss cellular mechanisms of striatal synaptic plasticity and aspects of cell signaling underlying striatum-dependent behavior, with a major focus on the neuromodulatory action of the endocannabinoid system and on the role of the Ras-ERK cascade.

Dopaminergic Control of the Exploration-Exploitation Trade-Off via the Basal Ganglia

We continuously face the dilemma of choosing between actions that gather new information or actions that exploit existing knowledge. This "exploration-exploitation" trade-off depends on the environment: stability favors exploiting knowledge to maximize gains; volatility favors exploring new options and discovering new outcomes. Here we set out to reconcile recent evidence for dopamine's involvement in the exploration-exploitation trade-off with the existing evidence for basal ganglia control of action selection, by testing the hypothesis that tonic dopamine in the striatum, the basal ganglia's input nucleus, sets the current exploration-exploitation trade-off. We first advance the idea of interpreting the basal ganglia output as a probability distribution function for action selection. Using computational models of the full basal ganglia circuit, we showed that, under this interpretation, the actions of dopamine within the striatum change the basal ganglia's output to favor the level of exploration or exploitation encoded in the probability distribution. We also found that our models predict striatal dopamine controls the exploration-exploitation trade-off if we instead read-out the probability distribution from the target nuclei of the basal ganglia, where their inhibitory input shapes the cortical input to these nuclei. Finally, by integrating the basal ganglia within a reinforcement learning model, we showed how dopamine's effect on the exploration-exploitation trade-off could be measurable in a forced two-choice task. These simulations also showed how tonic dopamine can appear to affect learning while only directly altering the trade-off. Thus, our models support the hypothesis that changes in tonic dopamine within the striatum can alter the exploration-exploitation trade-off by modulating the output of the basal ganglia.

Molecular substrates of action control in cortico-striatal circuits

The purpose of this review is to describe the molecular mechanisms in the striatum that mediate reward-based learning and action control during instrumental conditioning. Experiments assessing the neural bases of instrumental conditioning have uncovered functional circuits in the striatum, including dorsal and ventral striatal sub-regions, involved in action-outcome learning, stimulus-response learning, and the motivational control of action by reward-associated cues. Integration of dopamine (DA) and glutamate neurotransmission within these striatal sub-regions is hypothesized to enable learning and action control through its role in shaping synaptic plasticity and cellular excitability. The extracellular signal regulated kinase (ERK) appears to be particularly important for reward-based learning and action control due to its sensitivity to combined DA and glutamate receptor activation and its involvement in a range of cellular functions. ERK activation in striatal neurons is proposed to have a dual role in both the learning and performance factors that contribute to instrumental conditioning through its regulation of plasticity-related transcription factors and its modulation of intrinsic cellular excitability. Furthermore, perturbation of ERK activation by drugs of abuse may give rise to behavioral disorders such as addiction.

High-frequency, short-latency disinhibition bursting of midbrain dopaminergic neurons

During reinforcement and sequence learning, dopaminergic neurons fire bursts of action potentials. Dopaminergic neurons in vivo receive strong background excitatory and inhibitory inputs, suggesting that one mechanism by which bursts may be produced is disinhibition. Unfortunately, these inputs are lost during slice preparation and are not precisely controlled during in vivo experiments. In the present study we show that dopaminergic neurons can be shifted into a balanced state in which constant synaptic N-methyl-d-aspartate (NMDA) and GABA(A) conductances are mimicked either pharmacologically or using dynamic clamp. From this state, a disinhibition burst can be evoked by removing the background inhibitory conductance. We demonstrate three functional characteristics of network-based disinhibition that promote high-frequency, short-latency bursting in dopaminergic neurons. First, we found that increasing the total background NMDA and GABA(A) synaptic conductances increased the intraburst firing frequency and reduced its latency. Second, we found that the disinhibition burst is sensitive to the proportion of background inhibitory input that is removed. In particular, we found that high-frequency, short-latency bursts were enhanced by increasing the degree of disinhibition. Third, the time course over which inhibition is removed had a large effect on the burst, namely, that synchronous removal of weak inhibitory inputs produces bursts of high intraburst frequency and shorter latency. Our results suggest that fast, more precisely timed bursts can be evoked by complete and synchronous disinhibition of dopaminergic neurons in a high-conductance state.

Contributions of ERK signaling in the striatum to instrumental learning and performance

The striatum is critical for learning and decision making; however, the molecular mechanisms that govern striatum function are not fully understood. The extracellular signal regulated kinase (ERK) cascade is an important signaling pathway that underlies synaptic plasticity, cellular excitability, learning and arousal. This review focuses on the role of ERK signaling in striatum function. ERK is activated in the striatum by coordinated dopamine and glutamate receptor signaling, where it underlies corticostriatal synaptic plasticity and influences striatal cell excitability. ERK activation in the dorsal striatum is necessary for action-outcome learning and performance of goal-directed actions. In the ventral striatum, ERK is necessary for the motivating effects of reward-associated stimuli on instrumental performance. Dysregulation of ERK signaling in the striatum by repeated drug exposure contributes to the development of addictive behavior. These results highlight the importance of ERK signaling in the striatum as a critical substrate for learning and as a regulator of ongoing behavior. Furthermore, they suggest that ERK may be a suitable target for therapeutics to treat disorders of learning and decision making that arise from compromised striatum function.

Impaired appetitively as well as aversively motivated behaviors and learning in PDE10A-deficient mice suggest a role for striatal signaling in evaluative salience attribution

Phosphodiesterase 10A (PDE10A) hydrolyzes both cAMP and cGMP, and is a key element in the regulation of medium spiny neuron (MSN) activity in the striatum. In the present report, we investigated the effects of targeted disruption of PDE10A on spatial learning and memory as well as aversive and appetitive conditioning in C57BL/6J mice. Because of its putative role in motivational processes and reward learning, we also determined the expression of the immediate early gene zif268 in striatum and anterior cingulate cortex. Animals showed decreased response rates in scheduled appetitive operant conditioning, as well as impaired aversive conditioning in a passive avoidance task. Morris water maze performance revealed not-motor related spatial learning and memory deficits. Anxiety and social explorative behavior was not affected in PDE10A-deficient mice. Expression of zif268 was increased in striatum and anterior cingulate cortex, which suggests alterations in the neural connections between striatum and anterior cingulate cortex in PDE10A-deficient mice. The changes in behavior and plasticity in these PDE10A-deficient mice were in accordance with the proposed role of striatal MSNs and corticostriatal connections in evaluative salience attribution.

Goal-directed and habitual control in the basal ganglia: implications for Parkinson's disease

Progressive loss of the ascending dopaminergic projection in the basal ganglia is a fundamental pathological feature of Parkinson's disease. Studies in animals and humans have identified spatially segregated functional territories in the basal ganglia for the control of goal-directed and habitual actions. In patients with Parkinson's disease the loss of dopamine is predominantly in the posterior putamen, a region of the basal ganglia associated with the control of habitual behaviour. These patients may therefore be forced into a progressive reliance on the goal-directed mode of action control that is mediated by comparatively preserved processing in the rostromedial striatum. Thus, many of their behavioural difficulties may reflect a loss of normal automatic control owing to distorting output signals from habitual control circuits, which impede the expression of goal-directed action.

Decreased striatal dopamine receptor binding in primary focal dystonia: a D2 or D3 defect?

Dystonia is an involuntary movement disorder characterized by repetitive patterned or sustained muscle contractions causing twisting or abnormal postures. Several lines of evidence suggest that abnormalities of dopaminergic pathways contribute to the pathophysiology of dystonia. In particular, dysfunction of D2-like receptors that mediate function of the indirect pathway in the basal ganglia may play a key role. We have demonstrated with positron emission tomography that patients with primary focal cranial or hand dystonia have reduced putamenal specific binding of [(18)F]spiperone, a nonselective D2-like radioligand with nearly equal affinity for serotonergic 5-HT(2A) sites. We then repeated the study with [(18)F]N-methyl-benperidol (NMB), a more selective D2-like receptor radioligand with minimal affinity for 5-HT(2A). Surprisingly, there was no decrease in NMB binding in the putamen of subjects with dystonia. Our findings excluded reductions of putamenal uptake greater than 20% with 95% confidence intervals. The analysis of the in vitro selectivity of NMB and spiperone demonstrated that NMB was highly selective for D2 receptors relative to D3 receptors (200-fold difference in affinity), whereas spiperone has similar affinity for all three of the D2-like receptor subtypes. These findings when coupled with other literature suggest that a defect in D3, rather than D2, receptor expression may be associated with primary focal dystonia.

Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning

The basal ganglia are implicated in a remarkable range of functions influencing emotion and cognition as well as motor behavior. Current models of basal ganglia function hypothesize that parallel limbic, associative, and motor cortico-basal ganglia loops contribute to this diverse set of functions, but little is yet known about how these loops operate and how their activities evolve during learning. To address these issues, we recorded simultaneously in sensorimotor and associative regions of the striatum as rats learned different versions of a conditional T-maze task. We found highly contrasting patterns of activity in these regions during task performance and found that these different patterns of structured activity developed concurrently, but with sharply different dynamics. Based on the region-specific dynamics of these patterns across learning, we suggest a working model whereby dorsomedial associative loops can modulate the access of dorsolateral sensorimotor loops to the control of action.

Functional disconnection of the substantia nigra pars compacta from the pedunculopontine nucleus impairs learning of a conditioned avoidance task

The pedunculopontine tegmental nucleus (PPTg) targets nuclei in the basal ganglia, including the substantia nigra pars compacta (SNc), in which neuronal loss occurs in Parkinson's disease, a condition in which patients show cognitive as well as motor disturbances. Partial loss and functional abnormalities of neurons in the PPTg are also associated with Parkinson's disease. We hypothesized that the interaction of PPTg and SNc might be important for cognitive impairments and so investigated whether disrupting the connections between the PPTg and SNc impaired learning of a conditioned avoidance response (CAR) by male Wistar rats. The following groups were tested: PPTg unilateral; SNc unilateral; PPTg-SNc ipsilateral (ipsilateral lesions in PPTg and SNc); PPTg-SNc contralateral (contralateral lesions in PPTg and SNc); sham lesions (of each type). SNc lesions were made with 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine HCl (MPTP, 0.6micromol); PPTg lesions with ibotenate (24nmol). After recovery, all rats underwent 50-trial sessions of 2-way active avoidance conditioning for 3 consecutive days. Rats with unilateral lesions in PPTg or SNc learnt this, however rats with contralateral (but not ipsilateral) combined lesions in both structures presented no sign of learning. This effect was not likely to be due to sensorimotor impairment because lesions did not affect reaction time to the tone or footshock during conditioning. However, an increased number of non-responses were observed in the rats with contralateral lesions. The results support the hypothesis that a functional interaction between PPTg and SNc is needed for CAR learning and performance.

Cortical and basal ganglia contributions to habit learning and automaticity

In the 20th century it was thought that novel behaviors are mediated primarily in cortex and that the development of automaticity is a process of transferring control to subcortical structures. However, evidence supports the view that subcortical structures, such as the striatum, make significant contributions to initial learning. More recently, there has been increasing evidence that neurons in the associative striatum are selectively activated during early learning, whereas those in the sensorimotor striatum are more active after automaticity has developed. At the same time, other recent reports indicate that automatic behaviors are striatum- and dopamine-independent, and might be mediated entirely within cortex. Resolving this apparent conflict should be a major goal of future research.

Convergent evidence for abnormal striatal synaptic plasticity in dystonia

Dystonia is a functionally disabling movement disorder characterized by abnormal movements and postures. Although substantial recent progress has been made in identifying genetic factors, the pathophysiology of the disease remains a mystery. A provocative suggestion gaining broader acceptance is that some aspect of neural plasticity may be abnormal. There is also evidence that, at least in some forms of dystonia, sensorimotor "use" may be a contributing factor. Most empirical evidence of abnormal plasticity in dystonia comes from measures of sensorimotor cortical organization and physiology. However, the basal ganglia also play a critical role in sensorimotor function. Furthermore, the basal ganglia are prominently implicated in traditional models of dystonia, are the primary targets of stereotactic neurosurgical interventions, and provide a neural substrate for sensorimotor learning influenced by neuromodulators. Our working hypothesis is that abnormal plasticity in the basal ganglia is a critical link between the etiology and pathophysiology of dystonia. In this review we set up the background for this hypothesis by integrating a large body of disparate indirect evidence that dystonia may involve abnormalities in synaptic plasticity in the striatum. After reviewing evidence implicating the striatum in dystonia, we focus on the influence of two neuromodulatory systems: dopamine and acetylcholine. For both of these neuromodulators, we first describe the evidence for abnormalities in dystonia and then the means by which it may influence striatal synaptic plasticity. Collectively, the evidence suggests that many different forms of dystonia may involve abnormal plasticity in the striatum. An improved understanding of these altered plastic processes would help inform our understanding of the pathophysiology of dystonia, and, given the role of the striatum in sensorimotor learning, provide a principled basis for designing therapies aimed at the dynamic processes linking etiology to pathophysiology of the disease.

The dorsomedial striatum reflects response bias during learning

Previous studies have established that neurons in the dorsomedial striatum track the behavioral significance of external stimuli, are sensitive to contingencies between actions and outcomes, and show rapid flexibility in representing task-related information. Here, we describe how neural activity in the dorsomedial striatum changes during the initial acquisition of a Go/NoGo task and during an initial reversal of stimulus-response contingencies. Rats made nosepoke responses over delay periods and then received one of two acoustic stimuli. Liquid rewards were delivered after one stimulus (S+) if the rats made a Go response (entering a reward port on the opposite wall of the chamber). If a Go response was made to other stimulus (S-), rats experienced a timeout. On 10% of trials, no stimulus was presented. These trials were used to assess response bias, the animals' tendency to collect reward independent of the stimulus. Response bias increased during the reversal, corresponding to the animals' uncertainty about the stimulus-response contingencies. Most task-modulated neurons fired during the response at the end of the delay period. The fraction of response-modulated neurons was correlated with response bias and neural activity was sensitive to the behavioral response made on the previous trial. During initial task acquisition and initial reversal learning, there was a remarkable change in the percentages of neurons that fired in relation to the task events, especially during withdrawal from the nosepoke aperture. These results suggest that changes in task-related activity in the dorsomedial striatum during learning are driven by the animal's bias to collect rewards.

Dopamine D1 and D2 antagonist effects on response likelihood and duration

Experimentally induced and parkinsonian disruptions in dopamine (DA) transmission are associated with motor abnormalities that include a reduced likelihood of behavioral response initiation and an increased duration of executed responses. Here we investigated the dopamine receptor subtypes involved in regulating these two aspects of behavior. We examined the effects of D1 family (D1/D5) antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390; 0, 0.04, 0.08, or 0.16 mg/kg) and D2/D3 antagonist 3,5-dichloro-N-(1-ethylpyrrolidin-2-ylmethyl)-2-hydroxy-6-methoxybenzamide (+)-tartrate salt (raclopride; 0, 0.2, or 0.4 mg/kg) on the likelihood and duration of a cued Pavlovian approach and a cued operant lever-press response. While the high doses of the D1 and D2 antagonists produced similar levels of overall locomotor suppression, only the D2 antagonist increased the duration of time that animals' heads remained in the food compartment during both Pavlovian and operant task performance. In contrast, D1 antagonist SCH23390 decreased the proportion of trials in which animals executed both the Pavlovian approach and operant lever-press, while raclopride did not. The results suggest that D2 receptor blockade preferentially increases response duration, and, under the simple discrete-trial procedures employed here, D1 receptor blockade preferential reduces Pavlovian and operant response likelihood.

The uncertainty processing theory of motivation

Most theories describe motivation using basic terminology (drive, 'wanting', goal, pleasure, etc.) that fails to inform well about the psychological mechanisms controlling its expression. This leads to a conception of motivation as a mere psychological state 'emerging' from neurophysiological substrates. However, the involvement of motivation in a large number of behavioural parameters (triggering, intensity, duration, and directedness) and cognitive abilities (learning, memory, decision, etc.) suggest that it should be viewed as an information processing system. The uncertainty processing theory (UPT) presented here suggests that motivation is the set of cognitive processes allowing organisms to extract information from the environment by reducing uncertainty about the occurrence of psychologically significant events. This processing of information is shown to naturally result in the highlighting of specific stimuli. The UPT attempts to solve three major problems: (i) how motivations can affect behaviour and cognition so widely, (ii) how motivational specificity for objects and events can result from nonspecific neuropharmacological causal factors (such as mesolimbic dopamine), and (iii) how motivational interactions can be conceived in psychological terms, irrespective of their biological correlates. The UPT is in keeping with the conceptual tradition of the incentive salience hypothesis while trying to overcome the shortcomings inherent to this view.

Learning processing in the basal ganglia: a mosaic of broken mirrors

In the present review we propose a model to explain the role of the basal ganglia in sensorimotor and cognitive functions based on a growing body of behavioural, anatomical, physiological, and neurochemical evidence accumulated over the last decades. This model proposes that the body and its surrounding environment are represented in the striatum in a fragmented and repeated way, like a mosaic consisting of the fragmented images of broken mirrors. Each fragment forms a functional unit representing articulated parts of the body with motion properties, objects of the environment which the subject can approach or manipulate, and locations the subject can move to. These units integrate the sensory properties and movements related to them. The repeated and widespread distribution of such units amplifies the combinatorial power of the associations among them. These associations depend on the phasic release of dopamine in the striatum triggered by the saliency of stimuli and will be reinforced by the rewarding consequences of the actions related to them. Dopamine permits synaptic plasticity in the corticostriatal synapses. The striatal units encoding the same stimulus/action send convergent projections to the internal segment of the globus pallidus (GPi) and to the substantia nigra pars reticulata (SNr) that stimulate or hold the action through a thalamus-frontal cortex pathway. According to this model, this is how the basal ganglia select actions based on environmental stimuli and store adaptive associations as nondeclarative memories such as motor skills, habits, and memories formed by Pavlovian and instrumental conditioning.