Effects of local connectivity on striatal function: stimulation and analysis of a model

Striatal network modeling in Huntington's Disease

Medium spiny neurons (MSNs) comprise over 90% of cells in the striatum. In vivo MSNs display coherent burst firing cell assembly activity patterns, even though isolated MSNs do not burst fire intrinsically. This activity is important for the learning and execution of action sequences and is characteristically dysregulated in Huntington's Disease (HD). However, how dysregulation is caused by the various neural pathologies affecting MSNs in HD is unknown. Previous modeling work using simple cell models has shown that cell assembly activity patterns can emerge as a result of MSN inhibitory network interactions. Here, by directly estimating MSN network model parameters from single unit spiking data, we show that a network composed of much more physiologically detailed MSNs provides an excellent quantitative fit to wild type (WT) mouse spiking data, but only when network parameters are appropriate for the striatum. We find the WT MSN network is situated in a regime close to a transition from stable to strongly fluctuating network dynamics. This regime facilitates the generation of low-dimensional slowly varying coherent activity patterns and confers high sensitivity to variations in cortical driving. By re-estimating the model on HD spiking data we discover network parameter modifications are consistent across three very different types of HD mutant mouse models (YAC128, Q175, R6/2). In striking agreement with the known pathophysiology we find feedforward excitatory drive is reduced in HD compared to WT mice, while recurrent inhibition also shows phenotype dependency. We show that these modifications shift the HD MSN network to a sub-optimal regime where higher dimensional incoherent rapidly fluctuating activity predominates. Our results provide insight into a diverse range of experimental findings in HD, including cognitive and motor symptoms, and may suggest new avenues for treatment.

From space to time: Spatial inhomogeneities lead to the emergence of spatiotemporal sequences in spiking neuronal networks

Spatio-temporal sequences of neuronal activity are observed in many brain regions in a variety of tasks and are thought to form the basis of meaningful behavior. However, mechanisms by which a neuronal network can generate spatio-temporal activity sequences have remained obscure. Existing models are biologically untenable because they either require manual embedding of a feedforward network within a random network or supervised learning to train the connectivity of a network to generate sequences. Here, we propose a biologically plausible, generative rule to create spatio-temporal activity sequences in a network of spiking neurons with distance-dependent connectivity. We show that the emergence of spatio-temporal activity sequences requires: (1) individual neurons preferentially project a small fraction of their axons in a specific direction, and (2) the preferential projection direction of neighboring neurons is similar. Thus, an anisotropic but correlated connectivity of neuron groups suffices to generate spatio-temporal activity sequences in an otherwise random neuronal network model.

Exploring the role of striatal D1 and D2 medium spiny neurons in action selection using a virtual robotic framework

The basal ganglia have been hypothesized to be involved in action selection, i.e. resolving competition between simultaneously activated motor programs. It has been shown that the direct pathway facilitates action execution whereas the indirect pathway inhibits it. However, as the pathways are both active during an action, it remains unclear whether their role is co-operative or competitive. In order to investigate this issue, we developed a striatal model consisting of D1 and D2 medium spiny neurons (MSNs) and interfaced it to a simulated robot moving in an environment. We demonstrate that this model is able to reproduce key behavioral features of several experiments involving optogenetic manipulation of the striatum, such as freezing and ambulation. We then investigate the interaction of D1- and D2-MSNs. We find that their fundamental relationship is co-operative within a channel and competitive between channels; this turns out to be crucial for action selection. However, individual pairs of D1- and D2-MSNs may exhibit predominantly competition or co-operation depending on their distance, and D1- and D2-MSNs population activity can alternate between co-operation and competition modes during a stimulation. Additionally, our results show that D2-D2 connectivity between channels is necessary for effective resolution of competition; in its absence, a conflict of two motor programs typically results in neither being selected.

Reduced striatal dopamine release during motor skill acquisition in Parkinson's disease

Background

Striatal dopamine is functionally important for the acquisition of motor skills. However, it remains controversial as to whether intrinsic processing of motor learning is impaired in patients with Parkinson’s disease (PD), and if yes, whether the impairment is associated with altered striatal dopamine release. Additionally, most neuro-imaging studies of patients with PD have focused on motor sequence learning. In contrast, skill acquisition, specifically, the reconstruction of muscle control of isolated movements, has barely been studied.

Method

In this study, we used a repetitive skill training task to measure the peak acceleration of left thumb movement during a process to achieve fine tuning of motor skill. Using ¹¹C-raclopride (RAC) positron emission tomography, we investigated changes in striatal dopamine levels in two conditions of a skill acquisition task: initial skill training (Day 1) and acquired condition (Day 2) with eight patients with PD and age-matched healthy subjects (HS).

Result

In HS, the mean acceleration of each session improved through repeated training sessions on Day 1. However, in patients with PD, the training-associated increase was less than that for HS, and this suggests that repetitive skill training does not result in the effective improvement of motor performance. The regions of interest (ROI) analysis revealed that the RAC-binding potential (BP) was significantly reduced in the right putamen on Day 1 compared with Day 2 in HS. In patients with PD, BP within the right putamen was unchanged. Further, we found that patients with PD had increased dopamine levels within the right ventral striatum (VST) and right caudate (CAU) on Day 2, which was greater than that in HS. These results suggest the impaired activation of the putamen during skill acquisition in patients with PD and compensated hyperactivation of the VST and CAU for the reduced dopamine release within the dorsal putamen (DPU).

Conclusion

Our findings suggest that patients with PD had insufficiency in the process to improve motor skills. Different patterns of striatal dopamine release are relevant to the impairment of these motor functions in patients with PD, at the early stage of the disease.

Activity Dynamics and Signal Representation in a Striatal Network Model with Distance-Dependent Connectivity

The striatum is the main input nucleus of the basal ganglia. Characterizing striatal activity dynamics is crucial to understanding mechanisms underlying action selection, initiation, and execution. Here, we studied the effects of spatial network connectivity on the spatiotemporal structure of striatal activity. We show that a striatal network with nonmonotonically changing distance-dependent connectivity (according to a gamma distribution) can exhibit a wide repertoire of spatiotemporal dynamics, ranging from spatially homogeneous, asynchronous-irregular (AI) activity to a state with stable, spatially localized activity bumps, as in “winner-take-all” (WTA) dynamics. Among these regimes, the unstable activity bumps [transition activity (TA)] regime closely resembles the experimentally observed spatiotemporal activity dynamics and neuronal assemblies in the striatum. In contrast, striatal networks with monotonically decreasing distance-dependent connectivity (in a Gaussian fashion) can exhibit only an AI state. Thus, given the observation of spatially compact neuronal clusters in the striatum, our model suggests that recurrent connectivity among striatal projection neurons should vary nonmonotonically. In brain disorders such as Parkinson’s disease, increased cortical inputs and high striatal firing rates are associated with a reduction in stimulus sensitivity. Consistent with this, our model suggests that strong cortical inputs drive the striatum to a WTA state, leading to low stimulus sensitivity and high variability. In contrast, the AI and TA states show high stimulus sensitivity and reliability. Thus, based on these results, we propose that in a healthy state the striatum operates in a AI/TA state and that lack of dopamine pushes it into a WTA state.

Representation of the body in the lateral striatum of the freely moving rat: Fast Spiking Interneurons respond to stimulation of individual body parts

Numerous studies have shown that certain types of striatal interneurons play a crucial role in selection and regulation of striatal output. Striatal Fast-Spiking Interneurons (FSIs) are parvalbumin positive, GABAergic interneurons that constitute less than 1% of the total striatal population. It is becoming increasingly evident that these sparsely distributed neurons exert a strong inhibitory effect on Medium Spiny projection Neurons (MSNs). MSNs in lateral striatum receive direct synaptic input from regions of cortex representing discrete body parts, and show phasic increases in activity during touch or movement of specific body parts. In the present study, we sought to determine whether lateral striatal FSIs identified by their electrophysiological properties, i.e., short-duration spike and fast firing rate (FR), display body part sensitivity similar to that exhibited by MSNs. During video recorded somatosensorimotor exams, each individual body part was stimulated and responses of single neurons were observed and quantified. Individual FSIs displayed patterns of activity related selectively to stimulation of a discrete body part. Most patterns of activity were similar to those exhibited by typical MSNs, but some phasic decreases were observed. These results serve as evidence that some striatal FSIs process information related to discrete body parts and participate in sensorimotor processing by striatal networks that contribute to motor output.

STATEMENT OF SIGNIFICANCE

Parvalbumin positive, striatal FSIs are hypothesized to play an important role in behavior by inhibiting MSNs. We asked a fundamental question regarding information processed during behavior by FSIs: whether FSIs, which preferentially occupy the sensorimotor portion of the striatum, process activity of discrete body parts. Our finding that they do, in a selective manner similar to MSNs, begins to reveal the types of phasic signals that FSI feed forward to projection neurons during striatal processing of cortical input regarding a specific sensorimotor event. These findings suggest new avenues for testing feed-forward inhibition theory as applied to striatum in naturalistic conditions, such as whether FSI decreases facilitate excitation of MSNs related to the current movement while FSI increases silence MSNs unrelated to the current movement.

Signatures of Value Comparison in Ventral Striatum Neurons

The ventral striatum (VS), like its cortical afferents, is closely associated with processing of rewards, but the relative contributions of striatal and cortical reward systems remains unclear. Most theories posit distinct roles for these structures, despite their similarities. We compared responses of VS neurons to those of ventromedial prefrontal cortex (vmPFC) Area 14 neurons, recorded in a risky choice task. Five major response patterns observed in vmPFC were also observed in VS: (1) offer value encoding, (2) value difference encoding, (3) preferential encoding of chosen relative to unchosen value, (4) a correlation between residual variance in responses and choices, and (5) prominent encoding of outcomes. We did observe some differences as well; in particular, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, the close match between vmPFC and VS suggests that cortex and its striatal targets make overlapping contributions to economic choice.

A study of single neurons in the ventral striatum reveals signatures of value comparison and selection during a risky choice task, suggesting that the cortex and its striatal targets make overlapping contributions to the choice process. Read the accompanying Primer.

The neural calculations underlying reward-based choice are closely associated with a network of brain areas including the ventral striatum (VS) and ventromedial prefrontal cortex (vmPFC). Most theories ascribe distinct roles to these two structures during choice, but these differences have yet to be confirmed at the level of single neurons. We compared responses of VS neurons to those of vmPFC neurons recorded in rhesus macaques choosing between potential gambles for water rewards. We found widespread similarities in the way that VS and vmPFC neurons fire during the choice process. Neurons in both areas encoded the value of the offered gamble, the difference in value between offered gambles, and the gamble outcome. Additionally, both areas showed stronger coding for the chosen gamble than for the unchosen one and predicted choice even when we controlled for offer value. Interestingly, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, similarities between vmPFC and VS suggest that cortex and its striatal targets make overlapping contributions to reward-based choice.

Endocannabinoid-dependent modulation of phasic dopamine signaling encodes external and internal reward-predictive cues

The mesolimbic dopamine (DA) system plays an integral role in incentive motivation and reward seeking and a growing body of evidence identifies signal transduction at cannabinoid receptors as a critical modulator of this system. Indeed, administration of exogenous cannabinoids results in burst firing of DA neurons of the ventral tegmental area and increases extracellular DA in the nucleus accumbens (NAcc). Implementation of fast-scan cyclic voltammetry (FSCV) confirms the ability of cannabinoids to augment DA within the NAcc on a subsecond timescale. The use of FSCV along with newly developed highly selective pharmacological compounds advances our understanding of how cannabinoids influence DA transmission and highlights a role for endocannabinoid-modulated subsecond DAergic activation in the incentive motivational properties of not only external, but also internal reward-predictive cues. For example, our laboratory has recently demonstrated that in mice responding under a fixed-interval (FI) schedule for food reinforcement, fluctuations in NAcc DA signal the principal cue predictive of reinforcer availability - time. That is, as the interval progresses, NAcc DA levels decline leading to accelerated food seeking and the resulting characteristic FI scallop pattern of responding. Importantly, administration of WIN 55,212-2, a synthetic cannabinoid agonist, or JZL184, an indirect cannabinoid agonist, increases DA levels during the interval and disrupts this pattern of responding. Along with a wealth of other reports, these results illustrate the role of cannabinoid receptor activation in the regulation of DA transmission and the control of temporally guided reward seeking. The current review will explore the striatal beat frequency model of interval timing as it pertains to cannabinoid signaling and propose a neurocircuitry through which this system modulates interoceptive time cues.

Inhibitory synapses between striatal projection neurons support efficient enhancement of cortical signals: a computational model

The function of lateral inhibitory synapses between striatal projection neurons is currently poorly understood. This paper puts forward a model suggesting that inhibitory collaterals can be used to enhance the incoming cortical signals. In particular, we propose that lateral inhibition between projection neurons performs a signal-enhancing process that resembles the image processing technique of "unsharp masking", where a blurred copy is used to enhance and sharpen an input image. The paper also presents the results of computer simulations deomsntrating that the proposed mechanisms is compatible with known properties of striatal projection neurons, and outperforms alternative models of lateral inhibition. Finally, this paper illustrates the advantages of the proposed model and discusses the relevance of these conclusions for existing computational models of the basal ganglia and their role in cognition.

Molecular nexopathies: a new paradigm of neurodegenerative disease

Neural networks provide candidate substrates for the spread of proteinopathies causing neurodegeneration, and emerging data suggest that macroscopic signatures of network disintegration differentiate diseases. However, how do protein abnormalities produce network signatures? The answer may lie with 'molecular nexopathies': specific, coherent conjunctions of pathogenic protein and intrinsic network characteristics that define network signatures of neurodegenerative pathologies. Key features of the paradigm that we propose here include differential intrinsic network vulnerability to propagating protein abnormalities, in part reflecting developmental structural and functional factors; differential vulnerability of neural connection types (e.g., clustered versus distributed connections) to particular pathogenic proteins; and differential impact of molecular effects (e.g., toxic-gain-of-function versus loss-of-function) on gradients of network damage. The paradigm has implications for understanding and predicting neurodegenerative disease biology.

Context, emotion, and the strategic pursuit of goals: interactions among multiple brain systems controlling motivated behavior

Motivated behavior exhibits properties that change with experience and partially dissociate among a number of brain structures. Here, we review evidence from rodent experiments demonstrating that multiple brain systems acquire information in parallel and either cooperate or compete for behavioral control. We propose a conceptual model of systems interaction wherein a ventral emotional memory network involving ventral striatum (VS), amygdala, ventral hippocampus, and ventromedial prefrontal cortex triages behavioral responding to stimuli according to their associated affective outcomes. This system engages autonomic and postural responding (avoiding, ignoring, approaching) in accordance with associated stimulus valence (negative, neutral, positive), but does not engage particular operant responses. Rather, this emotional system suppresses or invigorates actions that are selected through competition between goal-directed control involving dorsomedial striatum (DMS) and habitual control involving dorsolateral striatum (DLS). The hippocampus provides contextual specificity to the emotional system, and provides an information rich input to the goal-directed system for navigation and discriminations involving ambiguous contexts, complex sensory configurations, or temporal ordering. The rapid acquisition and high capacity for episodic associations in the emotional system may unburden the more complex goal-directed system and reduce interference in the habit system from processing contingencies of neutral stimuli. Interactions among these systems likely involve inhibitory mechanisms and neuromodulation in the striatum to form a dominant response strategy. Innate traits, training methods, and task demands contribute to the nature of these interactions, which can include incidental learning in non-dominant systems. Addition of these features to reinforcement learning models of decision-making may better align theoretical predictions with behavioral and neural correlates in animals.

Multi-scale modeling of complex neuronal networks: a view towards striatal cholinergic pattern formations

The phenomena related to brain function occur as the interplay of various modules at different spatial and temporal scales. Particularly, the integration of the dynamical behavior of cells within the complex brain topology reveals a heterogeneous multi-scale problem, which has, to date, mainly been addressed by methods of statistical physics such as mean-field approximations. In contrast, the present study introduces an abstract mathematical model of a deterministic nature that provides a robust integral transformation of the microscopic activities into macroscopic spatiotemporal patterns. The existence of the transformation operator is guaranteed by the convergence of a repetitive patching of the network domain with its fundamental domains that express the local topologies of the tissue. Depending on the choice of the local connectivity function, this framework represents a computationally efficient generalization of the classical Kirchhoff's, Hebbian, and Hopfield's approaches. The capabilities of this multi-scale method have been evaluated within the structure of the dorsal striatum of rats, a brain region with major involvement in motor and cognitive information processing. Numerical simulations suggest the formation of characteristic spatiotemporal patterns due to the activation of cholinergic interneurons.

Changes in striatal dopamine release associated with human motor-skill acquisition

The acquisition of new motor skills is essential throughout daily life and involves the processes of learning new motor sequence and encoding elementary aspects of new movement. Although previous animal studies have suggested a functional importance for striatal dopamine release in the learning of new motor sequence, its role in encoding elementary aspects of new movement has not yet been investigated. To elucidate this, we investigated changes in striatal dopamine levels during initial skill-training (Day 1) compared with acquired conditions (Day 2) using ¹¹C-raclopride positron-emission tomography. Ten volunteers learned to perform brisk contractions using their non-dominant left thumbs with the aid of visual feedback. On Day 1, the mean acceleration of each session was improved through repeated training sessions until performance neared asymptotic levels, while improved motor performance was retained from the beginning on Day 2. The ¹¹C-raclopride binding potential (BP) in the right putamen was reduced during initial skill-training compared with under acquired conditions. Moreover, voxel-wise analysis revealed that ¹¹C-raclopride BP was particularly reduced in the right antero-dorsal to the lateral part of the putamen. Based on findings from previous fMRI studies that show a gradual shift of activation within the striatum during the initial processing of motor learning, striatal dopamine may play a role in the dynamic cortico-striatal activation during encoding of new motor memory in skill acquisition.

Expectancy, ambiguity, and behavioral flexibility: separable and complementary roles of the orbital frontal cortex and amygdala in processing reward expectancies

Appetitive goal-directed behavior can be associated with a cue-triggered expectancy that it will lead to a particular reward, a process thought to depend on the OFC and basolateral amygdala complex. We developed a biologically informed neural network model of this system to investigate the separable and complementary roles of these areas as the main components of a flexible expectancy system. These areas of interest are part of a neural network with additional subcortical areas, including the central nucleus of amygdala, ventral (limbic) and dorsomedial (associative) striatum. Our simulations are consistent with the view that the amygdala maintains Pavlovian associations through incremental updating of synaptic strength and that the OFC supports flexibility by maintaining an activation-based working memory of the recent reward history. Our model provides a mechanistic explanation for electrophysiological evidence that cue-related firing in OFC neurons is nonselectively early after a contingency change and why this nonselective firing is critical for promoting plasticity in the amygdala. This ambiguous activation results from the simultaneous maintenance of recent outcomes and obsolete Pavlovian contingencies in working memory. Furthermore, at the beginning of reversal, the OFC is critical for supporting responses that are no longer inappropriate. This result is inconsistent with an exclusive inhibitory account of OFC function.

Basal ganglia shapes predict social, communication, and motor dysfunctions in boys with autism spectrum disorder

OBJECTIVE

Basal ganglia abnormalities have been suggested as contributing to motor, social, and communicative impairments in autism spectrum disorder (ASD). Volumetric analyses offer limited ability to detect localized differences in basal ganglia structure. Our objective was to investigate basal ganglia shape abnormalities and their association with behavioral features of ASD, which may involve multiple frontal-subcortical circuits.

METHOD

Basal ganglia were manually delineated from MR images of 32 boys with ASD and 45 typically developing (TD) boys. Large deformation diffeomorphic metric mapping (LDDMM) was used to assess between-group differences in basal ganglia shape and to examine associations with motor, praxis, and reciprocal social and communicative impairments in ASD.

RESULTS

Boys with ASD showed changes in right basal ganglia shape as compared with TD boys; surface deformation was present in the caudate, putamen, and globus pallidus but did not stand up to correction for multiple comparisons. Brain-behavior correlation findings were more robust; analyses accounting for multiple comparisons revealed, in boys with ASD, surface inward deformation of the right posterior putamen predicted poorer motor skill, whereas surface inward deformation of the bilateral anterior and posterior putamen predicted poorer praxis. Surface outward deformation in the bilateral medial caudate head predicted greater reciprocal social and communicative impairment.

CONCLUSIONS

Motor, social, and communicative impairments in boys with ASD are associated with shape abnormalities in the basal ganglia. The findings suggest abnormalities within parallel frontal-subcortical circuits are differentially associated with impaired acquisition of motor and reciprocal social and communicative skills in ASD.

Dopamine-modulated dynamic cell assemblies generated by the GABAergic striatal microcircuit

The striatum, the principal input structure of the basal ganglia, is crucial to both motor control and learning. It receives convergent input from all over the neocortex, hippocampal formation, amygdala and thalamus, and is the primary recipient of dopamine in the brain. Within the striatum is a GABAergic microcircuit that acts upon these inputs, formed by the dominant medium-spiny projection neurons (MSNs) and fast-spiking interneurons (FSIs). There has been little progress in understanding the computations it performs, hampered by the non-laminar structure that prevents identification of a repeating canonical microcircuit. We here begin the identification of potential dynamically-defined computational elements within the striatum. We construct a new three-dimensional model of the striatal microcircuit's connectivity, and instantiate this with our dopamine-modulated neuron models of the MSNs and FSIs. A new model of gap junctions between the FSIs is introduced and tuned to experimental data. We introduce a novel multiple spike-train analysis method, and apply this to the outputs of the model to find groups of synchronised neurons at multiple time-scales. We find that, with realistic in vivo background input, small assemblies of synchronised MSNs spontaneously appear, consistent with experimental observations, and that the number of assemblies and the time-scale of synchronisation is strongly dependent on the simulated concentration of dopamine. We also show that feed-forward inhibition from the FSIs counter-intuitively increases the firing rate of the MSNs. Such small cell assemblies forming spontaneously only in the absence of dopamine may contribute to motor control problems seen in humans and animals following a loss of dopamine cells.

Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system

The prefrontal cortex (PFC) has long been thought to serve as an 'executive' that controls the selection of actions and cognitive functions more generally. However, the mechanistic basis of this executive function has not been clearly specified often amounting to a homunculus. This paper reviews recent attempts to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying the executive functions of the PFC. The overall approach builds upon existing mechanistic models of the basal ganglia (BG) and frontal systems known to play a critical role in motor control and action selection, where the BG provide a 'Go' versus 'NoGo' modulation of frontal action representations. In our model, the BG modulate working memory representations in prefrontal areas to support more abstract executive functions. We have developed a computational model of this system that is capable of developing human-like performance on working memory and executive control tasks through trial-and-error learning. This learning is based on reinforcement learning mechanisms associated with the midbrain dopaminergic system and its activation via the BG and amygdala. Finally, we briefly describe various empirical tests of this framework.

Feedforward and feedback inhibition in neostriatal GABAergic spiny neurons

There are two distinct inhibitory GABAergic circuits in the neostriatum. The feedforward circuit consists of a relatively small population of GABAergic interneurons that receives excitatory input from the neocortex and exerts monosynaptic inhibition onto striatal spiny projection neurons. The feedback circuit comprises the numerous spiny projection neurons and their interconnections via local axon collaterals. This network has long been assumed to provide the majority of striatal GABAergic inhibition and to sharpen and shape striatal output through lateral inhibition, producing increased activity in the most strongly excited spiny cells at the expense of their less strongly excited neighbors. Recent results, mostly from recording experiments of synaptically connected pairs of neurons, have revealed that the two GABAergic circuits differ markedly in terms of the total number of synapses made by each, the strength of the postsynaptic response detected at the soma, the extent of presynaptic convergence and divergence and the net effect of the activation of each circuit on the postsynaptic activity of the spiny neuron. These data have revealed that the feedforward inhibition is powerful and widespread, with spiking in a single interneuron being capable of significantly delaying or even blocking the generation of spikes in a large number of postsynaptic spiny neurons. In contrast, the postsynaptic effects of spiking in a single presynaptic spiny neuron on postsynaptic spiny neurons are weak when measured at the soma, and unable to significantly affect spike timing or generation. Further, reciprocity of synaptic connections between spiny neurons is only rarely observed. These results suggest that the bulk of the fast inhibition that has the strongest effects on spiny neuron spike timing comes from the feedforward interneuronal system whereas the axon collateral feedback system acts principally at the dendrites to control local excitability as well as the overall level of activity of the spiny neuron.

Reward prediction error computation in the pedunculopontine tegmental nucleus neurons

In this article, we address the role of neuronal activity in the pathways of the brainstem-midbrain circuit in reward and the basis for believing that this circuit provides advantages over previous reinforcement learning theory. Several lines of evidence support the reward-based learning theory proposing that midbrain dopamine (DA) neurons send a teaching signal (the reward prediction error signal) to control synaptic plasticity of the projection area. However, the underlying mechanism of where and how the reward prediction error signal is computed still remains unclear. Since the pedunculopontine tegmental nucleus (PPTN) in the brainstem is one of the strongest excitatory input sources to DA neurons, we hypothesized that the PPTN may play an important role in activating DA neurons and reinforcement learning by relaying necessary signals for reward prediction error computation to DA neurons. To investigate the involvement of the PPTN neurons in computation of reward prediction error, we used a visually guided saccade task (VGST) during recording of neuronal activity in monkeys. Here, we predict that PPTN neurons may relay the excitatory component of tonic reward prediction and phasic primary reward signals, and derive a new computational theory of the reward prediction error in DA neurons.

Computational models simulating electrophysiological activity in the basal ganglia

Modeling of the basal ganglia has played a substantial role in gaining insight into the mechanisms involved in the computational processes performed by this elusive group of nuclei. Models of the basal ganglia have undergone revolutionary changes over the last twenty years due to the rapid accumulation of neuroscientific data. In this chapter, we present distinct modeling approaches that can be used to enhance our understanding of the functional dynamics of information processing within the basal ganglia, and their interactions with the rest of the brain. Specific examples of recently developed models dealing with the analysis of computational processing issues at different structural levels of the basal ganglia are discussed.

Simulation of GABA function in the basal ganglia: computational models of GABAergic mechanisms in basal ganglia function

This chapter outlines current interpretation of computational aspects of GABAergic circuits of the striatum. Recent hypotheses and controversial matters are reviewed. Quantitative aspects of striatal synaptology relevant to computational models are considered, with estimates of the connectivity of the spiny projection neurons and fast-spiking interneurons. Against this background, insights into the computational properties of inhibitory circuits based on analysis and simulation of simple models are discussed. The paper concludes with suggestions for further theoretical and experimental studies.

Changes in subcellular distribution and phosphorylation of GluR1 in lesioned striatum of 6-hydroxydopamine-lesioned and l-dopa-treated rats

Recent evidence has linked striatal amino-3-hydroxy-5-methyl-4-isoxazole propionic acid (AMPA) receptor function to the adverse effects of long-term dopaminergic treatment in Parkinson's disease. The phosphorylation of AMPA subunit, GluR1, reflects AMPA receptor activity. To determine whether serine phosphorylation of GluR1 subunit by activation of Ca(2+)/calmodulin-dependent protein kinase II (CaMKII) contributes to the process, we examined the effects of unilateral nigrostriatal depletion with 6-hydroxydopamine and subsequent L: -dopa treatment on motor responses and phosphorylation states. Three weeks of L: -dopa administration to rats shortened the duration of the rotational response. We found a significant reduction in the abundance of both phosphorylated GluR1 at serine-831 site (pGluR1S831) and GluR1 in the cell plasma membrane of lesioned striatum. Chronic treatment of lesioned rats with L: -dopa markedly upregulated the phosphorylation of GluR1 in lesioned striatum with a concomitant normalization of the plasma membrane GluR1 abundance, which lasted at least 1 day after withdrawal of chronic L: -dopa treatment. Our immunostaining data showed that these changes were confined to parvalbumin-positive neurons where GluR1 subunits are exclusively expressed. Both the altered motor response duration and the degree of pGluR1S831 were attenuated by the intrastriatal administration of CaMKII inhibitor KN-93. These findings suggest that activation of CaMKII contributes to both development and maintenance of motor response duration alterations, through a mechanism that involves an increase in pGluR1S831 within parvalbumin-positive neurons.

Altered cortico-striatal synaptic plasticity and related behavioural impairments in reeler mice

Reelin-deficient mice have been used to investigate the role of this extracellular protein in cortico-striatal plasticity and striatum-related behaviours. Here we show that a repetitive electrical stimulation of the cortico-striatal pathway elicited long-term potentiation (LTP) in homozygous reeler (rl/rl) mice, while causing long-term depression in their wild-type (+/+) littermates. The N-methyl-D-aspartic acid (NMDA) receptor antagonist D-(-)-2 amino-5-phosphonopentanoic acid prevented the induction of LTP in (rl/rl) mice, thus confirming that this form of synaptic plasticity was NMDA receptor-dependent. Interestingly, in the presence of tiagabine, a blocker of gamma-aminobutyric acid (GABA) re-uptake system, the probability that (rl/rl) mice showed LTP decreased significantly, thus suggesting an impaired GABAergic transmission in reeler mutants. Consistent with this view, a decreased density of parvalbumin-positive GABAergic striatal interneurons was found in (rl/rl) mice in comparison to (+/+) mice. Finally, compatible with their abnormal striatal function (rl/rl) mice exhibited procedural learning deficits. Our data, showing alterations in cortico-striatal plasticity largely depending on a depressed GABAergic tone, delineate a mechanism whereby the lack of reelin may affect cognitive functions.

Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia

The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and executive functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This article presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia, and amygdala, which together form an actor-critic architecture. The critic system learns which prefrontal representations are task relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model's performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task and other benchmark working memory tasks.

Translational Studies in Neurorehabilitation: From Bench to Bedside

Stroke is the leading cause of adult disability in the western world. Consensus has built over the last few years regarding the usefulness of training to improve motor disability resulting from stroke. Until recently, there were no accepted strategies to enhance the beneficial effects of training. However, the combination of basic and clinical science data over the last few years is changing this picture, and is highly relevant to the field of neurorehabilitation. Human studies in both healthy individuals and patients after brain damage demonstrate as a proof of principle that somatosensory input, cortical stimulation, interhemispheric interactions, and pharmacologic interventions can modulate cortical plasticity in neurorehabilitation after stroke. These findings strongly suggest directions in the development of novel strategies to enhance training effects on motor recovery. The intent of this review is to describe these strategies, the basic science principles on which they are based, and the clinical applications that have emerged so far.

Banishing the homunculus: making working memory work

The prefrontal cortex has long been thought to subserve both working memory and "executive" function, but the mechanistic basis of their integrated function has remained poorly understood, often amounting to a homunculus. This paper reviews the progress in our laboratory and others pursuing a long-term research agenda to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying these phenomena. We outline six key functional demands underlying working memory, and then describe the current state of our computational model of the prefrontal cortex and associated systems in the basal ganglia (BG). The model, called PBWM (prefrontal cortex, basal ganglia working memory model), relies on actively maintained representations in the prefrontal cortex, which are dynamically updated/gated by the basal ganglia. It is capable of developing human-like performance largely on its own by taking advantage of powerful reinforcement learning mechanisms, based on the midbrain dopaminergic system and its activation via the basal ganglia and amygdala. These learning mechanisms enable the model to learn to control both itself and other brain areas in a strategic, task-appropriate manner. The model can learn challenging working memory tasks, and has been corroborated by several important empirical studies.

Dopaminergic influences on formation of a motor memory

The ability of the central nervous system to form motor memories, a process contributing to motor learning and skill acquisition, decreases with age. Dopaminergic activity, one of the mechanisms implicated in memory formation, experiences a similar decline with aging. It is possible that restoring dopaminergic function in elderly adults could lead to improved formation of motor memories with training. We studied the influence of a single oral dose of levodopa (100mg) administered preceding training on the ability to encode an elementary motor memory in the primary motor cortex of elderly and young healthy volunteers in a randomized, double-blind, placebo-controlled design. Attention to the task and motor training kinematics were comparable across age groups and sessions. In young subjects, encoding a motor memory under placebo was more prominent than in older subjects, and the encoding process was accelerated by intake of levodopa. In the elderly group, diminished motor memory encoding under placebo was enhanced by intake of levodopa to levels present in younger subjects. Therefore, upregulation of dopaminergic activity accelerated memory formation in young subjects and restored the ability to form a motor memory in elderly subjects; possible mechanisms underlying the beneficial effects of dopaminergic agents on motor learning in neurorehabilitation.

Comparison of IPSCs evoked by spiny and fast-spiking neurons in the neostriatum

Most neurons in the neostriatum are GABAergic spiny projection neurons with extensive local axon collaterals innervating principally other spiny projection neurons. The other source of GABAergic inputs to spiny neurons derives from a small number of interneurons, of which the best characterized are the parvalbumin-containing, fast-spiking interneurons. Spiny neuron collateral inhibition was not demonstrated until recently, because the IPSPs recorded at the soma are surprisingly small. In contrast, interneuronal inhibition was readily detected, comprising much larger IPSPs. Here, we report the application of quantal analysis and compartmental modeling to compare and contrast IPSCs in spiny neurons originating from axon collaterals and interneurons. The results indicate that individual release sites at spiny and interneuron synapses have similar quantal sizes and baseline release probabilities. Interneuronal unitary IPSCs are several times larger because of their proximal location on the neuron and because they have a larger number of transmitter release sites. Despite the small amount of current they can deliver to the soma, spiny cell collateral synapses had moderately high baseline release probabilities (0.5-0.9), suggesting that they are not weak because of some form of depression or modulation. The size of unitary collateral synaptic currents increased monotonically during development. These results argue against models of competitive inhibition in neostriatum, including those in which competitive inhibition is transiently effective during development and learning, and suggest a different role for the spiny cell axon collaterals.

Differential contribution of dopamine D2S and D2L receptors in the modulation of glutamate and GABA transmission in the striatum

Compelling evidence indicates that the long (D2L) and the short (D2S) isoform of dopamine (DA) D2 receptors serve distinct physiological functions in vivo. To address the involvement of these isoforms in the control of synaptic transmission in the striatum, we measured the sensitivity to D2 receptor stimulation of glutamate- and GABA-mediated currents recorded from striatal neurons of three mutant mice, in which the expression of D2L and D2S receptors was either ablated or variably altered. Our data indicate that both isoforms participate in the presynaptic inhibition of GABA transmission in the striatum, while the D2-receptor-dependent modulation of glutamate release preferentially involves the D2S receptor. Accordingly, the inhibitory effects of the DA D2 receptor agonist quinpirole (10 microM) on GABA(A)-mediated spontaneous inhibitory postsynaptic currents (IPSCs)correlate with the total number of D2 receptor sites in the striatum, irrespective of the specific receptor isoform expressed. In contrast, glutamate-mediated spontaneous excitatory postsynaptic currents (EPSCs) were significantly inhibited by quinpirole only when the total number of D2 receptor sites, normally composed by both D2L and D2S receptors in a ratio favoring the D2L isoform, was modified to express only the D2S isoform at higher than normal levels. Understanding the physiological roles of DA D2 receptors in the striatum is essential for the treatment of several neuropsychiatric conditions, such as Parkinson's disease, Tourette's syndrome, schizophrenia, and drug addiction.

Neurobiology of Tourette's syndrome: concepts of neuroanatomic localization and neurochemical abnormalities

Despite a preponderance of evidence suggesting an organic rather than psychogenic origin for Tourette syndrome, the precise neurobiological abnormality remains speculative. Neuroanatomically, there is expanding confirmation that cortico-striato-thalamo-cortical pathways represent the site of origin for tics and accompanying neuropsychiatric problems. Pathophysiological hypothesis are generally defined based on involvement of (1) a specific anatomical site (striato-thalamic circuits, striatal compartments), (2) physiologic abnormality (excess thalamic excitation, impaired intracortical inhibition), or (3) involvement of a specific neurotransmitter or synaptic component. This review provides information essential for understanding current and future proposals pertaining to the neurobiology of this intriguing disorder.

Receptor subtypes involved in the presynaptic and postsynaptic actions of dopamine on striatal interneurons

By stimulating distinct receptor subtypes, dopamine (DA) exerts presynaptic and postsynaptic actions on both large aspiny (LA) cholinergic and fast-spiking (FS) parvalbumin-positive interneurons of the striatum. Lack of receptor- and isoform-specific pharmacological agents, however, has hampered the progress toward a detailed identification of the specific DA receptors involved in these actions. To overcome this issue, in the present study we used four different mutant mice in which the expression of specific DA receptors was ablated. In D1 receptor null mice, D1R-/-, DA dose-dependently depolarized both LA and FS interneurons. Interestingly, SCH 233390 (10 microm), a D1-like (D1 and D5) receptor antagonist, but not l-sulpiride (3-10 microm), a D2-like (D2, D3, D4) receptor blocker, prevented this effect, implying D5 receptors in this action. Accordingly, immunohistochemical analyses in both wild-type and D1R-/- mice confirmed the expression of D5 receptors in both cholinergic and parvalbumin-positive interneurons of the striatum. In mice lacking D2 receptors, D2R-/-, the DA-dependent inhibition of GABA transmission was lost in both interneuron populations. Both isoforms of D2 receptor, D2L and D2S, were very likely involved in this inhibitory action, as revealed by the electrophysiological analysis of the effect of the DA D2-like receptor agonist quinpirole in two distinct mutants lacking D2L receptors and expressing variable contents of D2S receptors. The identification of the receptor subtypes involved in the actions of DA on different populations of striatal cells is essential to understand the circuitry of the basal ganglia and to develop pharmacological strategies able to interfere selectively with specific neuronal functions.

When inhibition goes incognito: feedback interaction between spiny projection neurons in striatal function

Theories about basal ganglia function have always been driven by our knowledge about the spiny projection neurons of the striatum. At the core of these theories lies the question of how, precisely, spiny projection neurons process cortical inputs. Most recently, studies demonstrating the role of spiny projection neurons in local synaptic GABA transmission have provided several new avenues for exploring striatal dynamics. They have also suggested new experimental directives for examining the specific ways in which spiny projection neurons both compete and cooperate through their local axon collaterals during cortical input processing.

Fast synaptic transmission between striatal spiny projection neurons

Striatal inhibition plays an important role in models of cortex-basal ganglia function and is altered in many basal ganglia diseases. The gamma-aminobutyric acid ergic spiny projection neuron comprises >95% of striatal neurons, but despite strong anatomical evidence, the electrophysiological properties and functions of their local axon collaterals are unknown. We simultaneously recorded from adjacent spiny projection neurons (<5-10 microm) in whole-cell patch mode and demonstrated a fast synaptic connection between 2669 pairs in cortex-striatum-substantia nigra organotypic cultures and 538 pairs in acute striatal slices. The synapse, which was blocked by gamma-aminobutyric acid type A antagonists, displayed a wide range of failure rates, was depolarizing at rest, and reversed above -60 mV. Presynaptic bursts of action potentials were highly correlated with total postsynaptic depolarization at rest. Synaptic transmission was optimized for burst discharge >14 Hz and showed considerable short-term plasticity, including paired-pulse depression at intervals <25 ms, intraburst facilitation, and interburst augmentation. This activity-dependent collateral interaction provides the basis for a new class of basal ganglia models in which striatal neurons cooperate as well as compete during processing of cortical inputs.

Interactions between frontal cortex and basal ganglia in working memory: a computational model

The frontal cortex and the basal ganglia interact via a relatively well understood and elaborate system of interconnections. In the context of motor function, these interconnections can be understood as disinhibiting, or "releasing the brakes," on frontal motor action plans: The basal ganglia detect appropriate contexts for performing motor actions and enable the frontal cortex to execute such actions at the appropriate time. We build on this idea in the domain of working memory through the use of computational neural network models of this circuit. In our model, the frontal cortex exhibits robust active maintenance, whereas the basal ganglia contribute a selective, dynamic gating function that enables frontal memory representations to be rapidly updated in a task-relevant manner. We apply the model to a novel version of the continuous performance task that requires subroutine-like selective working memory updating and compare and contrast our model with other existing models and theories of frontal-cortex-basal-ganglia interactions.

D(1) dopamine receptor activation reduces GABA(A) receptor currents in neostriatal neurons through a PKA/DARPP-32/PP1 signaling cascade

Dopamine is a critical determinant of neostriatal function, but its impact on intrastriatal GABAergic signaling is poorly understood. The role of D(1) dopamine receptors in the regulation of postsynaptic GABA(A) receptors was characterized using whole cell voltage-clamp recordings in acutely isolated, rat neostriatal medium spiny neurons. Exogenous application of GABA evoked a rapidly desensitizing current that was blocked by bicuculline. Application of the D(1) dopamine receptor agonist SKF 81297 reduced GABA-evoked currents in most medium spiny neurons. The D(1) dopamine receptor antagonist SCH 23390 blocked the effect of SKF 81297. Membrane-permeant cAMP analogues mimicked the effect of D(1) dopamine receptor stimulation, whereas an inhibitor of protein kinase A (PKA; Rp-8-chloroadenosine 3',5' cyclic monophosphothioate) attenuated the response to D(1) dopamine receptor stimulation or cAMP analogues. Inhibitors of protein phosphatase 1/2A potentiated the modulation by cAMP analogues. Single-cell RT-PCR profiling revealed consistent expression of mRNA for the beta1 subunit of the GABA(A) receptor-a known substrate of PKA-in medium spiny neurons. Immunoprecipitation assays of radiolabeled proteins revealed that D(1) dopamine receptor stimulation increased phosphorylation of GABA(A) receptor beta1/beta3 subunits. The D(1) dopamine receptor-induced phosphorylation of beta1/beta3 subunits was attenuated significantly in neostriata from DARPP-32 mutants. Voltage-clamp recordings corroborated these results, revealing that the efficacy of the D(1) dopamine receptor modulation of GABA(A) currents was reduced in DARPP-32-deficient medium spiny neurons. These results argue that D(1) dopamine receptor stimulation in neostriatal medium spiny neurons reduces postsynaptic GABA(A) receptor currents by activating a PKA/DARPP-32/protein phosphatase 1 signaling cascade targeting GABA(A) receptor beta1 subunits.

Inhibitory control of neostriatal projection neurons by GABAergic interneurons

The basal ganglia are a highly interconnected network of nuclei essential for the modulation and execution of voluntary behavior. The neostriatum is the principal input and one of the principal controllers of the output of the basal ganglia. Neostriatal projection neurons seem to be dynamically and powerfully controlled by GABAergic inputs, but the source(s) and physiological properties of these inputs remain unclear. Here we use paired whole-cell recordings to show that this inhibition derives from small populations of GABAergic interneurons that are themselves interconnected through functional electrotonic synapses. Inhibitory synaptic potentials generated from single interneurons are sufficiently powerful to delay or entirely block the generation of action potentials in a large number of projection neurons simultaneously.

Effect of dopamine denervation and dopamine agonist administration on serine phosphorylation of striatal NMDA receptor subunits

Sensitization of striatal N-methyl-d-aspartate (NMDA) receptors has been implicated in the pathogenesis of the response alterations associated with dopaminomimetic treatment of parkinsonian animals and patients. To determine whether serine phosphorylation of NMDA receptor subunits by activation of Ca2+/calmodulin-dependent protein-kinase II (CaMKII) contributes to this process, we examined the effects of unilateral nigrostriatal ablation with 6-hydroxydopamine and subsequent treatment with levodopa, SKF 38393 (D1-preferring dopamine agonist), or quinpirole (D2-preferring agonist) on motor responses and phosphorylation states. Three weeks of twice-daily levodopa administration to rats shortened the duration of their rotational response to levodopa or SKF 38393 challenge, but prolonged the duration of quinpirole-induced rotation. At the same time, levodopa treatment elevated serine phosphorylation of striatal NR2A (p<0.02), but not that of NR2B subunits, without associated changes in subunit protein levels. Chronic treatment with SKF 38393 increased NR2A (p<0.0001) but decreased NR2B (p<0.004) serine phosphorylation. In contrast, chronic quinpirole treatment had no effect on NR2A but increased NR2B phosphorylation (p<0.0001). The acute intrastriatal injection of the CaMKII inhibitor KN93 (1.0 micrograms) not only normalized the levodopa-induced motor response alterations but also attenuated the D1 and D2 receptor-mediated serine phosphorylation of NR2A and NR2B subunits, respectively (p<0.02). These results suggest that a CaMKII-mediated rise in serine phosphorylation of NMDA receptor subunits induced by intermittent stimulation of D1 or D2 dopaminergic receptors contributes to the apparent enhancement in striatal NMDA receptor sensitivity and thus to the dopaminergic response plasticity in levodopa-treated parkinsonian rats.

Enhanced tyrosine phosphorylation of striatal NMDA receptor subunits: effect of dopaminergic denervation and L-DOPA administration

Sensitization of striatal N-methyl-D-aspartate receptors (NMDAR) has been linked to events leading to the motor response changes associated with the administration of dopaminomimetics to parkinsonian animals and patients. To determine whether tyrosine phosphorylation of NMDAR subunits contributes to the apparent long-term enhancement in synaptic efficacy of these receptors, we examined the effect of unilateral nigrostriatal dopamine system ablation with 6-hydroxydopamine followed by twice-daily treatment with l-DOPA on the phosphorylation state of rat striatal NR2A and NR2B subunits. Three weeks of intermittent l-DOPA administration produced a shortening in the duration of the rotational response to dopaminergic challenge and other changes mimicking those occurring in patients with Parkinson's disease. Concurrently, tyrosine phosphorylation of NR2A and especially of NR2B subunits increased ipsilateral to the lesion (20+/-5% and 46+/-7% of intact striatum, respectively; p<0.01) without attendant changes in subunit protein levels. Selective blockade of NR2B subunits with ACEA 10-1244, but not of NR2A subunits with MDL 100,453, reversed the l-DOPA-induced response alterations. The intrastriatal injection of a tyrosine kinase inhibitor, genistein, at a dose (2.0 microg) that normalized the response shortening, attenuated the NR2A and NR2B phosphorylation increase by about 12% and 24%, respectively (p<0.01). Taken together, these results suggest that augmented tyrosine phosphorylation of NR2B subunits, alone or in combination with the smaller rise in NR2A subunit phosphorylation, contributes to the apparent enhancement in striatal NMDAR sensitivity and thus to the plastic alterations in dopaminergic responses in l-DOPA-treated parkinsonian rats.

Network models of the basal ganglia

Over the past year, a number of conceptual and mathematical models of the basal ganglia and their interactions with other areas of the brain have appeared in the literature. Even though the models each differ in significant ways, several computational principles, such as convergence, recurrence and competition, appear to have emerged as common themes of information processing in the basal ganglia. Simulation studies of these models have provoked new types of questions at the many levels of inquiry linking biophysics to behavior.

Interactions of glutamate and dopamine in a computational model of the striatum

A network model of simplified striatal principal neurons with mutual inhibition was used to investigate possible interactions between cortical glutamatergic and nigral dopaminergic afferents in the neostriatum. Glutamatergic and dopaminergic inputs were represented by an excitatory synaptic conductance and a slow membrane potassium conductance, respectively. Neuronal activity in the model was characterized by episodes of increased action potential firing rates of variable duration and frequency. Autocorrelation histograms constructed from the action potential activity of striatal model neurons showed that reducing peak excitatory conductance had the effect of increasing interspike intervals. On the other hand, the maximum value of the dopamine-sensitive potassium conductance was inversely related to the duration of firing episodes and the maximal firing rates. A smaller potassium conductance restored normal firing rates in the most active neurons at the expense of a larger proportion of neurons showing reduced activity. Thus, a homogeneous network with mutual inhibition can produce equally complex dynamics as have been proposed to occur in a striatal network with two neuron populations that are oppositely regulated by dopamine. Even without mutual inhibition it appears that increased dopamine concentrations could partially compensate for the effects of reduced glutamatergic input in individual neurons.