Dopaminergic Modulation of Striatal Inhibitory Transmission and Long-Term Plasticity

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.

Motor learning and metaplasticity in striatal neurons: relevance for Parkinson's disease

Nigro-striatal dopamine transmission is central to a wide range of neuronal functions, including skill learning, which is disrupted in several pathologies such as Parkinson's disease. The synaptic plasticity mechanisms, by which initial motor learning is stored for long time periods in striatal neurons, to then be gradually optimized upon subsequent training, remain unexplored. Addressing this issue is crucial to identify the synaptic and molecular mechanisms involved in striatal-dependent learning impairment in Parkinson's disease. In this study, we took advantage of interindividual differences between outbred rodents in reaching plateau performance in the rotarod incremental motor learning protocol, to study striatal synaptic plasticity ex vivo. We then assessed how this process is modulated by dopamine receptors and the dopamine active transporter, and whether it is impaired by overexpression of human α-synuclein in the mesencephalon; the latter is a progressive animal model of Parkinson's disease. We found that the initial acquisition of motor learning induced a dopamine active transporter and D1 receptors mediated long-term potentiation, under a protocol of long-term depression in striatal medium spiny neurons. This effect disappeared in animals reaching performance plateau. Overexpression of human α-synuclein reduced striatal dopamine active transporter levels, impaired motor learning, and prevented the learning-induced long-term potentiation, before the appearance of dopamine neuronal loss. Our findings provide evidence of a reorganization of cellular plasticity within the dorsolateral striatum that is mediated by dopamine receptors and dopamine active transporter during the acquisition of a skill. This newly identified mechanism of cellular memory is a form of metaplasticity that is disrupted in the early stage of synucleinopathies, such as Parkinson's disease, and that might be relevant for other striatal pathologies, such as drug abuse.

Striatal Neurodegeneration that Mimics Huntington's Disease Modifies GABA-induced Currents

Huntington's Disease (HD) is a degenerative disease which produces cognitive and motor disturbances. Treatment with GABAergic agonists improves the behavior and activity of mitochondrial complexes in rodents treated with 3-nitropropionic acid to mimic HD symptomatology. Apparently, GABA receptors activity may protect striatal medium spiny neurons (MSNs) from excitotoxic damage. This study evaluates whether mitochondrial inhibition with 3-NP that mimics the early stages of HD, modifies the kinetics and pharmacology of GABA receptors in patch clamp recorded dissociated MSNs cells. The results show that MSNs from mice treated with 3-NP exhibited differences in GABA-induced dose-response currents and pharmacological responses that suggests the presence of GABAC receptors in MSNs. Furthermore, there was a reduction in the effect of the GABAC antagonist that demonstrates a lessening of this GABA receptor subtype activity as a result of mitochondria inhibition.

Basal Ganglia Neuromodulation Over Multiple Temporal and Structural Scales-Simulations of Direct Pathway MSNs Investigate the Fast Onset of Dopaminergic Effects and Predict the Role of Kv4.2

The basal ganglia are involved in the motivational and habitual control of motor and cognitive behaviors. Striatum, the largest basal ganglia input stage, integrates cortical and thalamic inputs in functionally segregated cortico-basal ganglia-thalamic loops, and in addition the basal ganglia output nuclei control targets in the brainstem. Striatal function depends on the balance between the direct pathway medium spiny neurons (D1-MSNs) that express D1 dopamine receptors and the indirect pathway MSNs that express D2 dopamine receptors. The striatal microstructure is also divided into striosomes and matrix compartments, based on the differential expression of several proteins. Dopaminergic afferents from the midbrain and local cholinergic interneurons play crucial roles for basal ganglia function, and striatal signaling via the striosomes in turn regulates the midbrain dopaminergic system directly and via the lateral habenula. Consequently, abnormal functions of the basal ganglia neuromodulatory system underlie many neurological and psychiatric disorders. Neuromodulation acts on multiple structural levels, ranging from the subcellular level to behavior, both in health and disease. For example, neuromodulation affects membrane excitability and controls synaptic plasticity and thus learning in the basal ganglia. However, it is not clear on what time scales these different effects are implemented. Phosphorylation of ion channels and the resulting membrane effects are typically studied over minutes while it has been shown that neuromodulation can affect behavior within a few hundred milliseconds. So how do these seemingly contradictory effects fit together? Here we first briefly review neuromodulation of the basal ganglia, with a focus on dopamine. We furthermore use biophysically detailed multi-compartmental models to integrate experimental data regarding dopaminergic effects on individual membrane conductances with the aim to explain the resulting cellular level dopaminergic effects. In particular we predict dopaminergic effects on Kv4.2 in D1-MSNs. Finally, we also explore dynamical aspects of the onset of neuromodulation effects in multi-scale computational models combining biochemical signaling cascades and multi-compartmental neuron models.

Corrigendum to "Dopaminergic Modulation of Striatal Inhibitory Transmission and Long-Term Plasticity"

[This corrects the article DOI: 10.1155/2015/789502.].