Selective encoding of reward predictions and prediction errors by globus pallidus subpopulations

Basal ganglia (BG) circuits help guide and invigorate actions using predictions of future rewards (values). Within the BG, the globus pallidus pars externa (GPe) may play an essential role in aggregating and distributing value information. We recorded from the GPe in unrestrained rats performing both Pavlovian and instrumental tasks to obtain rewards and distinguished neuronal subtypes by their firing properties across the wake/sleep cycle and optogenetic tagging. In both tasks, the parvalbumin-positive (PV+), faster-firing "prototypical" neurons showed strong, sustained modulation by value, unlike other subtypes, including the "arkypallidal" cells that project back to striatum. Furthermore, we discovered that a distinct minority (7%) of GP cells display slower, pacemaker-like firing and encode reward prediction errors (RPEs) almost identically to midbrain dopamine neurons. These cell-specific forms of GPe value representation help define the circuit mechanisms by which the BG contribute to motivation and reinforcement learning.

An emergent temporal basis set robustly supports cerebellar time-series learning

The cerebellum is considered a "learning machine" essential for time interval estimation underlying motor coordination and other behaviors. Theoretical work has proposed that the cerebellum's input recipient structure, the granule cell layer (GCL), performs pattern separation of inputs that facilitates learning in Purkinje cells (P-cells). However, the relationship between input reformatting and learning has remained debated, with roles emphasized for pattern separation features from sparsification to decorrelation. We took a novel approach by training a minimalist model of the cerebellar cortex to learn complex time-series data from time-varying inputs, typical during movements. The model robustly produced temporal basis sets from these inputs, and the resultant GCL output supported better learning of temporally complex target functions than mossy fibers alone. Learning was optimized at intermediate threshold levels, supporting relatively dense granule cell activity, yet the key statistical features in GCL population activity that drove learning differed from those seen previously for classification tasks. These findings advance testable hypotheses for mechanisms of temporal basis set formation and predict that moderately dense population activity optimizes learning.NEW & NOTEWORTHY During movement, mossy fiber inputs to the cerebellum relay time-varying information with strong intrinsic relationships to ongoing movement. Are such mossy fibers signals sufficient to support Purkinje signals and learning? In a model, we show how the GCL greatly improves Purkinje learning of complex, temporally dynamic signals relative to mossy fibers alone. Learning-optimized GCL population activity was moderately dense, which retained intrinsic input variance while also performing pattern separation.

Basal ganglia output to the thalamus: still a paradox

The basal ganglia (BG)-recipient thalamus controls motor output but it remains unclear how its activity is regulated. Several studies report that thalamic activation occurs via disinhibition during pauses in the firing of inhibitory pallidal inputs from the BG. Other studies indicate that thalamic spiking is triggered by pallidal inputs via post-inhibitory 'rebound' calcium spikes. Finally excitatory cortical inputs can drive thalamic activity, which becomes entrained, or time-locked, to pallidal spikes. We present a unifying framework where these seemingly distinct results arise from a continuum of thalamic firing 'modes' controlled by excitatory inputs. We provide a mechanistic explanation for paradoxical pallidothalamic coactivations observed during behavior that raises new questions about what information is integrated in the thalamus to control behavior.

Integration of cortical and pallidal inputs in the basal ganglia-recipient thalamus of singing birds

The basal ganglia-recipient thalamus receives inhibitory inputs from the pallidum and excitatory inputs from cortex, but it is unclear how these inputs interact during behavior. We recorded simultaneously from thalamic neurons and their putative synaptically connected pallidal inputs in singing zebra finches. We find, first, that each pallidal spike produces an extremely brief (∼5 ms) pulse of inhibition that completely suppresses thalamic spiking. As a result, thalamic spikes are entrained to pallidal spikes with submillisecond precision. Second, we find that the number of thalamic spikes that discharge within a single pallidal interspike interval (ISI) depends linearly on the duration of that interval but does not depend on pallidal activity prior to the interval. In a detailed biophysical model, our results were not easily explained by the postinhibitory "rebound" mechanism previously observed in anesthetized birds and in brain slices, nor could most of our data be characterized as "gating" of excitatory transmission by inhibitory pallidal input. Instead, we propose a novel "entrainment" mechanism of pallidothalamic transmission that highlights the importance of an excitatory conductance that drives spiking, interacting with brief pulses of pallidal inhibition. Building on our recent finding that cortical inputs can drive syllable-locked rate modulations in thalamic neurons during singing, we report here that excitatory inputs affect thalamic spiking in two ways: by shortening the latency of a thalamic spike after a pallidal spike and by increasing thalamic firing rates within individual pallidal ISIs. We present a unifying biophysical model that can reproduce all known modes of pallidothalamic transmission--rebound, gating, and entrainment--depending on the amount of excitation the thalamic neuron receives.

Biophysical basis of the phase response curve of subthalamic neurons with generalization to other cell types

Experimental evidence indicates that the response of subthalamic neurons to excitatory postsynaptic potentials (EPSPs) is well described by their infinitesimal phase response curves (iPRC). However, the factors controlling the shape of that iPRC, and hence controlling the way subthalamic neurons respond to synaptic input, are unclear. We developed a biophysical model of subthalamic neurons to aid in the understanding of their iPRCs; this model exhibited an iPRC type common to many subthalamic cells. We devised a method for deriving its iPRC from its biophysical properties that clarifies how these different properties interact to shape the iPRC. This method revealed why the response of subthalamic neurons is well approximated by their iPRCs and how that approximation becomes less accurate under strong fluctuating input currents. It also connected iPRC structure to aspects of cellular physiology that could be estimated in simple current-clamp experiments. This allowed us to directly compare the iPRC predicted by our theory with the iPRC estimated from the response to EPSPs or current pulses in individual cells. We found that theoretically predicted iPRCs agreed well with estimates derived from synaptic stimuli, but not with those estimated from the response to somatic current injection. The difference between synaptic currents and those applied experimentally at the soma may arise from differences in the dynamics of charge redistribution on the dendrites and axon. Ultimately, our approach allowed us to identify novel ways in which voltage-dependent conductances interact with AHP conductances to influence synaptic integration that will apply to a wide range of cell types.

Phase response curves of subthalamic neurons measured with synaptic input and current injection

Infinitesimal phase response curves (iPRCs) provide a simple description of the response of repetitively firing neurons and may be used to predict responses to any pattern of synaptic input. Their simplicity makes them useful for understanding the dynamics of neurons when certain conditions are met. For example, the sizes of evoked phase shifts should scale linearly with stimulus strength, and the form of the iPRC should remain relatively constant as firing rate varies. We measured the PRCs of rat subthalamic neurons in brain slices using corticosubthalamic excitatory postsynaptic potentials (EPSPs; mediated by both AMPA- and NMDA-type receptors) and injected current pulses and used them to calculate the iPRC. These were relatively insensitive to both the size of the stimulus and the cell's firing rate, suggesting that the iPRC can predict the response of subthalamic nucleus cells to extrinsic inputs. However, the iPRC calculated using EPSPs differed from that obtained using current pulses. EPSPs (normalized for charge) were much more effective at altering the phase of subthalamic neurons than current pulses. The difference was not attributable to the extended time course of NMDA receptor-mediated currents, being unaffected by blockade of NMDA receptors. The iPRC provides a good description of subthalamic neurons' response to input, but iPRCs are best estimated using synaptic inputs rather than somatic current injection.

Organization of the songbird basal ganglia, including area X

Area X is a songbird basal ganglia nucleus that is required for vocal learning. Both Area X and its immediate surround, the medial striatum (MSt), contain cells displaying either striatal or pallidal characteristics. We used pathway-tracing techniques to compare directly the targets of Area X and MSt with those of the lateral striatum (LSt) and globus pallidus (GP). We found that the zebra finch LSt projects to the GP, substantia nigra pars reticulata (SNr) and pars compacta (SNc), but not the thalamus. The GP is reciprocally connected with the subthalamic nucleus (STN) and projects to the SNr and motor thalamus analog, the ventral intermediate area (VIA). In contrast to the LSt, Area X and surrounding MSt project to the ventral pallidum (VP) and dorsal thalamus via pallidal-like neurons. A dorsal strip of the MSt contains spiny neurons that project to the VP. The MSt, but not Area X, projects to the ventral tegmental area (VTA) and SNc, but neither MSt nor Area X projects to the SNr. Largely distinct populations of SNc and VTA dopaminergic neurons innervate Area X and surrounding the MSt. Finally, we provide evidence consistent with an indirect pathway from the cerebellum to the basal ganglia, including Area X. Area X projections thus differ from those of the GP and LSt, but are similar to those of the MSt. These data clarify the relationships among different portions of the oscine basal ganglia as well as among the basal ganglia of birds and mammals.

Reinforcement learning with modulated spike timing dependent synaptic plasticity

Spike timing-dependent synaptic plasticity (STDP) has emerged as the preferred framework linking patterns of pre- and postsynaptic activity to changes in synaptic strength. Although synaptic plasticity is widely believed to be a major component of learning, it is unclear how STDP itself could serve as a mechanism for general purpose learning. On the other hand, algorithms for reinforcement learning work on a wide variety of problems, but lack an experimentally established neural implementation. Here, we combine these paradigms in a novel model in which a modified version of STDP achieves reinforcement learning. We build this model in stages, identifying a minimal set of conditions needed to make it work. Using a performance-modulated modification of STDP in a two-layer feedforward network, we can train output neurons to generate arbitrarily selected spike trains or population responses. Furthermore, a given network can learn distinct responses to several different input patterns. We also describe in detail how this model might be implemented biologically. Thus our model offers a novel and biologically plausible implementation of reinforcement learning that is capable of training a neural population to produce a very wide range of possible mappings between synaptic input and spiking output.

Electrophysiological properties of neurons in the basal ganglia of the domestic chick: conservation and divergence in the evolution of the avian basal ganglia

Although the basal ganglia of birds and mammals share an enormous number of anatomical, histochemical, and electrophysiological characteristics, studies in songbirds have revealed some important differences. Specifically, a specialized region of songbird striatum (the input structure of the basal ganglia) has an anatomical projection and a physiologically defined cell type that are characteristic of the globus pallidus. At present, it is not clear if these differences result from adaptations specific to songbirds and perhaps a few other avian taxa or are common to all birds. We shed some light on this issue by characterizing the morphology and electrophysiological properties of basal ganglia neurons in an avian species that is only distantly related to songbirds: the domestic chick. We recorded neurons in chick basal ganglia in a brain slice preparation, using the whole cell technique. We found that chick striatum, like songbird striatum, contains a pallidum-like cell type never reported in mammalian striatum, supporting the hypothesis that this feature is common to all birds. We also discovered that spiny neurons, the most common cell type in the striatum of all amniotes, possess a diverse set of physiological properties in chicks that distinguish them from both mammals and songbirds. This study revealed an unexpectedly complex pattern of conservation and divergence in the properties of neurons recorded in avian striatum.

Evidence for ?direct? and ?indirect? pathways through the song system basal ganglia

Song learning in oscine birds relies on a circuit known as the "anterior forebrain pathway," which includes a specialized region of the avian basal ganglia. This region, area X, is embedded within a telencephalic structure considered homologous to the striatum, the input structure of the mammalian basal ganglia. Area X has many features in common with the mammalian striatum, yet has distinctive traits, including largely aspiny projection neurons that directly innervate the thalamus and a cell type that physiologically resembles neurons recorded in the mammalian globus pallidus. We have proposed that area X is a mixture of striatum and globus pallidus and has the same functional organization as circuits in the mammalian basal ganglia. Using electrophysiological and anatomical approaches, we found that area X contains a functional analog of the "direct" striatopallidothalamic pathway of mammals: axons of the striatal spiny neurons make close contacts on the somata and dendrites of pallidal cells. A subset of pallidal neurons project directly to the thalamus. Surprisingly, we found evidence that many pallidal cells may not project to the thalamus, but rather participate in a functional analog of the mammalian "indirect" pathway, which may oppose the effects of the direct pathway. Our results deepen our understanding of how information flows through area X and provide more support for the notion that song learning in oscines employs physiological mechanisms similar to basal ganglia-dependent forms of motor learning in mammals.

The avian song system in comparative perspective

The song system of oscine birds has become a versatile model system that is used to study diverse problems in neurobiology. Because the song system is often studied with the intention of applying the results to mammalian systems, it is important to place song system brain nuclei in a broader context and to understand the relationships between these avian structures and regions of the mammalian brain. This task has been impeded by the distinctiveness of the song system and the vast apparent differences between the forebrains of birds and mammals. Fortunately, accumulating data on the development, histochemistry, and anatomical organization of avian and mammalian brains has begun to shed light on this issue. We now know that the forebrains of birds and mammals are more alike than they first appeared, even though many questions remain unanswered. Furthermore, the song system is not as singular as it seemed-it has much in common with other neural systems in birds and mammals. These data provide a firmer foundation for extrapolating knowledge of the song system to mammalian systems and suggest how the song system might have evolved.

Presynaptic depression of glutamatergic synaptic transmission by D1-like dopamine receptor activation in the avian basal ganglia

Vocal behavior in songbirds exemplifies a rich integration of motor, cognitive, and social functions that are shared among vertebrates. As a part of the underlying neural substrate, the song system, the anterior forebrain pathway (AFP) is required for song learning and maintenance. The AFP resembles the mammalian basal ganglia-thalamocortical loop in its macroscopic organization, neuronal intrinsic properties, and microcircuitry. Area X, the first station in the AFP, is a part of the basal ganglia essential for vocal learning. It receives glutamatergic inputs from pallial structures and sends GABAergic outputs to thalamic structures. It also receives dense dopaminergic innervation from the midbrain. The role of this innervation is essentially unknown. Here we provide evidence that dopamine (DA) can modulate the glutamatergic inputs to spiny neurons in area X. In whole-cell voltage-clamp recordings from neurons in brain slices of adult zebra finches, we found that activation of D1-like DA receptors depresses ionotropic glutamatergic synaptic current in area X spiny neurons. This effect is mediated by a presynaptic site of action, mimicked by activation of adenylyl cyclase, and blocked by protein kinase A inhibitor and an adenosine A1 receptor antagonist. These results suggest that, in addition to altering the input-output function of spiny neurons by modulating their excitability, as we have shown previously, DA can directly influence the excitatory inputs to these neurons as well. Thus, DA can exert fine control over information processing through spiny neurons in area X, the dynamics of the AFP output, and ultimately song learning and maintenance.

The oscine song system considered in the context of the avian brain: lessons learned from comparative neurobiology

The oscine song system has emerged as one of the leading model systems for studying motor learning in vertebrates, combining an easily recorded behavior with a discrete neural substrate. That neural substrate seems to be distinct from other structures in the avian brain and thus is often studied in isolation. However, the song system is unlikely to have evolved ex nihilo, and should share some features with the parts of the avian brain from which it evolved. Identification of its evolutionary precursors should help us apply what we know about the song system to other vertebrate motor systems, and vice versa. Here, I review the homologies between parts of the avian and mammalian telencephala and explain where the song system nuclei reside in this context. The organization of the song system is then compared to other parts of the avian brain and the brains of nonoscine birds. Study of the nonoscine brain has revealed a 'general motor pathway' from caudolateral neostriatum (NCL) to intermediate archistriatum (Ai) that resembles the song system motor pathway in its anatomical organization. No part of this motor pathway projects directly to brainstem vocal or respiratory centers in nonoscines, but it does innervate a wide variety of motor and premotor neuron populations that mediate other behaviors. This general motor pathway may be accompanied by an 'anterior forebrain pathway', suggesting that the song system is simply a specialization of a part of this preexisting circuit. This hypothesis has implications for how accessory structures of the song system (e.g. HVc shelf, LMAN shell) are regarded, can help explain how the forebrain vocal control systems of three avian taxa (parrots, hummingbirds, and songbirds) could have evolved independently yet be so similar in organization, and makes testable predictions concerning the anatomy of the song system and the nonoscine brain.

Electrophysiological analysis of a songbird basal ganglia circuit essential for vocal plasticity

The discrete, interconnected nuclei of the songbird brain, collectively termed the song system, underlie the learning and production of song. Two main forebrain pathways have been identified that contribute to song production, learning, and adult plasticity. A posterior "motor pathway" including nucleus HVc (used as the proper name), the robust nucleus of the archistriatum (RA) and descending projections to the brainstem, is essential for song production. An "anterior forebrain pathway," arising from HVc, passing through area X of the lobus parolfactorius, the medial portion of the dorsolateral nucleus of the anterior thalamus and the lateral magnocellular nucleus of the anterior neostriatum, and finally terminating in RA, is essential for song learning and adult plasticity. The fact that the lobus parolfactorius is thought to form a part of the avian striatum implies several predictions for the connections of area X and for the properties of its neurons. Here, we review the existing anatomical and electrophysiological data bearing on the nature of area X as a striatal structure. In general, the data strongly favor the notion that area X is striatal. One set of observations, however, is at odds with that idea, and we provide and partially test the hypothesis that area X also contains a pallidal component. We discuss further tests of this idea and implications for thinking of the song system as a basal ganglia loop similar to that described in mammals.

Complementary 'bottom-up' and 'top-down' approaches to basal ganglia function

Recently, two quite different approaches exemplifying 'bottom-up' and 'top-down' philosophies have shed new light on basal ganglia function. In vitro work using organotypic co-cultures has implicated the subthalamic nucleus (STN) and the external segment of the globus pallidus (GP(e)) as pacemakers for low-frequency bursting that is reminiscent of the activity produced in Parkinsonian tremor. A circuit essential for avian song learning has been identified as part of the basal ganglia with surprisingly well conserved cellular details; investigation of this system may help to address general issues of basal ganglia function.

Electrophysiological properties of avian basal ganglia neurons recorded in vitro

The forebrains of mammals and birds appear quite different in their gross morphology, making it difficult to identify homologies between them and to assess how far they have diverged in organization. Nevertheless one set of forebrain structures, the basal ganglia, has been successfully compared in mammals and birds. Anatomical, histochemical, and molecular data have identified the avian homologues of the mammalian basal ganglia and indicate that they are very similar in organization, suggesting that they perform similar functions in the two classes. However, the physiological properties of the avian basal ganglia have not been studied, and these properties are critical for inferring functional similarity. We have used a zebra finch brain slice preparation to characterize the intrinsic physiological properties of neurons in the avian basal ganglia, particularly in the input structure of the basal ganglia, the striatum. We found that avian striatum contains a cell type that closely resembles the medium spiny neuron, the principal cell type of mammalian striatum. Avian striatum also contains a rare cell type that is very similar to an interneuron class found in mammalian striatum, the low-threshold spike cell. On the other hand, we found an aspiny, fast-firing cell type in avian striatum that is distinct from all known classes of mammalian striatal neuron. These neurons usually fired spontaneously at 10 Hz or more and were capable of sustained firing at very high rates when injected with depolarizing current. The existence of this cell type represents an important difference between avian striatum and mammalian dorsal striatum. Our data support the general idea that the organization and functional properties of the basal ganglia have been largely conserved in mammals and birds, but they imply that avian striatum is not identical to mammalian dorsal striatum.