Advance cueing produces enhanced action-boundary patterns of spike activity in the sensorimotor striatum

Striosomes Target Nigral Dopamine-Containing Neurons via Direct-D1 and Indirect-D2 Pathways Paralleling Classic Direct-Indirect Basal Ganglia Systems

The classic output pathways of the basal ganglia are known as the direct-D1 and indirect-D2, or Go/No-Go, pathways. Balance of the activity in these canonical direct-indirect pathways is considered a core requirement for normal movement control, and their imbalance is a major etiologic factor in movement disorders including Parkinsons disease. We present evidence for a conceptually equivalent parallel system of direct-D1 and indirect-D2 pathways that arise from striatal projection neurons (SPNs) of the striosome compartment rather than from the matrix. These striosomal direct (S-D1) and indirect (S-D2) pathways, as a pair, target dopamine-containing neurons of the substantia nigra (SNpc) instead of the motor output nuclei of the basal ganglia. The novel anatomically and functionally distinct indirect-D2 striosomal pathway targets dopaminergic SNpc cells indirectly via a core region of the external pallidum (GPe). We demonstrate that these S-D1 and S-D2 pathways oppositely modulate striatal dopamine release in freely behaving mice under open-field conditions and oppositely modulate locomotor and other movements. These S-D1 and S-D2 pathways further exhibit different, time-dependent responses during performance of a probabilistic decision-making maze task and respond differently to rewarding and aversive stimuli. These contrasts depend on mediolateral and anteroposterior striatal locations of the SPNs as are the classic direct and indirect pathways. The effects of S-D1 and S-D2 stimulation on striatal dopamine release and voluntary locomotion are nearly opposite. The parallelism of the direct-indirect circuit design motifs of the striosomal S-D and S-D2 circuits and canonical matrix M-D1 and M-D2, and their contrasting behavioral effects, call for a major reformulation of the classic direct-indirect pathway model of basal ganglia function. Given that some striosomes receive limbic and association cortical inputs, the S-D1 and S-D2 circuits likely influence motivation for action and behavioral learning, complementing and possibly reorienting the motoric activities of the canonical matrix pathways. At a fundamental level, these findings suggest a unifying framework for aligning two sets of circuits that share the organizational motif of opponent D1 and D2 regulation, but that have different outputs and can even have opposite polarities in their targets and effects, albeit conditioned by striatal topography. Our findings further delineate a potentially therapeutically important set of pathways influencing dopamine, including a D2 receptor-linked S-D2 pathway likely unknowingly targeted by administration of many therapeutic drugs including those for Parkinsons disease. The novel parallel pathway model that we propose here could help to account for the normally integrated modulatory influence of the basal ganglia on motivation for actions as well as the actions themselves.

Striatum expresses region-specific plasticity consistent with distinct memory abilities

The striatum mediates two learning modalities: goal-directed behavior in dorsomedial (DMS) and habits in dorsolateral (DLS) striata. The synaptic bases of these learnings are still elusive. Indeed, while ample research has described DLS plasticity, little remains known about DMS plasticity and its involvement in procedural learning. Here, we find symmetric and asymmetric anti-Hebbian spike-timing-dependent plasticity (STDP) in DMS and DLS, respectively, with opposite plasticity dominance upon increasing corticostriatal activity. During motor-skill learning, plasticity is engaged in DMS and striatonigral DLS neurons only during early learning stages, whereas striatopallidal DLS neurons are mobilized only during late phases. With a mathematical modeling approach, we find that symmetric anti-Hebbian STDP favors memory flexibility, while asymmetric anti-Hebbian STDP favors memory maintenance, consistent with memory processes at play in procedural learning.

Concurrent Thalamostriatal and Corticostriatal Spike-Timing-Dependent Plasticity and Heterosynaptic Interactions Shape Striatal Plasticity Map

The striatum integrates inputs from the cortex and thalamus, which display concomitant or sequential activity. The striatum assists in forming memory, with acquisition of the behavioral repertoire being associated with corticostriatal (CS) plasticity. The literature has mainly focused on that CS plasticity, and little remains known about thalamostriatal (TS) plasticity rules or CS and TS plasticity interactions. We undertook here the study of these plasticity rules. We found bidirectional Hebbian and anti-Hebbian spike-timing-dependent plasticity (STDP) at the thalamic and cortical inputs, respectively, which were driving concurrent changes at the striatal synapses. Moreover, TS- and CS-STDP induced heterosynaptic plasticity. We developed a calcium-based mathematical model of the coupled TS and CS plasticity, and simulations predict complex changes in the CS and TS plasticity maps depending on the precise cortex–thalamus–striatum engram. These predictions were experimentally validated using triplet-based STDP stimulations, which revealed the significant remodeling of the CS-STDP map upon TS activity, which is notably the induction of the LTD areas in the CS-STDP for specific timing regimes. TS-STDP exerts a greater influence on CS plasticity than CS-STDP on TS plasticity. These findings highlight the major impact of precise timing in cortical and thalamic activity for the memory engram of striatal synapses.

Ensemble encoding of action speed by striatal fast-spiking interneurons

Striatal fast-spiking interneurons (FSIs) potently inhibit the output neurons of the striatum and, as such, powerfully modulate action learning. Through electrical synaptic coupling, FSIs are theorized to temporally coordinate their activity. This has important implications for their ability to temporally summate inhibition on downstream striatal projection neurons. While some in vivo single-unit electrophysiological recordings of putative FSIs support coordinated firing, others do not. Moreover, it is unclear as to what aspect of action FSIs encode. To address this, we used in vivo calcium imaging of genetically identified FSIs in freely moving mice and applied machine learning analyses to decipher the relationship between FSI activity and movement. We report that FSIs exhibit ensemble activity that encodes the speed of action sub-components, including ambulation and head movements. These results suggest FSI population dynamics fit within a Hebbian model for ensemble inhibition of striatal output guiding action.

Bridging the gap between striatal plasticity and learning

The striatum, the main input nucleus of the basal ganglia, controls goal-directed behavior and procedural learning. Striatal projection neurons integrate glutamatergic inputs from cortex and thalamus together with neuromodulatory systems, and are subjected to plasticity. Striatal projection neurons exhibit bidirectional plasticity (LTP and LTD) when exposed to Hebbian paradigms. Importantly, correlative and even causal links between procedural learning and striatal plasticity have recently been shown. This short review summarizes the current view on striatal plasticity (with a focus on spike-timing-dependent plasticity), recent studies aiming at bridging in vivo skill acquisition and striatal plasticity, the temporal credit-assignment problem, and the gaps that remain to be filled.

Dopamine-endocannabinoid interactions mediate spike-timing-dependent potentiation in the striatum

Dopamine modulates striatal synaptic plasticity, a key substrate for action selection and procedural learning. Thus, characterizing the repertoire of activity-dependent plasticity in striatum and its dependence on dopamine is of crucial importance. We recently unraveled a striatal spike-timing-dependent long-term potentiation (tLTP) mediated by endocannabinoids (eCBs) and induced with few spikes (~5–15). Whether this eCB-tLTP interacts with the dopaminergic system remains to be investigated. Here, we report that eCB-tLTP is impaired in a rodent model of Parkinson’s disease and rescued by L-DOPA. Dopamine controls eCB-tLTP via dopamine type-2 receptors (D₂R) located presynaptically in cortical terminals. Dopamine–endocannabinoid interactions via D₂R are required for the emergence of tLTP in response to few coincident pre- and post-synaptic spikes and control eCB-plasticity by modulating the long-term potentiation (LTP)/depression (LTD) thresholds. While usually considered as a depressing synaptic function, our results show that eCBs in the presence of dopamine constitute a versatile system underlying bidirectional plasticity implicated in basal ganglia pathophysiology.

Dopamine tightly regulates plasticity at corticostriatal synapses. Here, the authors report that endocannabinoid dependent LTP induced with few spikes in the striatum is impaired in a rodent model of Parkinson’s disease, requires dopamine through presynaptic D2 receptors located on corticostriatal inputs.

No Discrete Start/Stop Signals in the Dorsal Striatum of Mice Performing a Learned Action

A popular hypothesis is that the dorsal striatum generates discrete "traffic light" signals that initiate, maintain, and terminate the execution of learned actions. Alternatively, the striatum may continuously monitor the dynamics of movements associated with action execution by processing inputs from somatosensory and motor cortices. Here, we recorded the activity of striatal neurons in mice performing a run-and-stop task and characterized the diversity of firing rate modulations relative to run performance (tuning curves) across neurons. We found that the tuning curves could not be statistically clustered in discrete functional groups (start or stop neurons). Rather, their shape varied continuously according to the movement dynamics of the task. Moreover, striatal spiking activity correlated with running speed on a run-by-run basis and was modulated by task-related non-locomotor movements, such as licking. We hypothesize that such moment-to-moment movement monitoring by the dorsal striatum contributes to the learning of adaptive actions and/or updating their kinematics.

Two-photon imaging in mice shows striosomes and matrix have overlapping but differential reinforcement-related responses

Striosomes were discovered several decades ago as neurochemically identified zones in the striatum, yet technical hurdles have hampered the study of the functions of these striatal compartments. Here we used 2-photon calcium imaging in neuronal birthdate-labeled Mash1-CreER;Ai14 mice to image simultaneously the activity of striosomal and matrix neurons as mice performed an auditory conditioning task. With this method, we identified circumscribed zones of tdTomato-labeled neuropil that correspond to striosomes as verified immunohistochemically. Neurons in both striosomes and matrix responded to reward-predicting cues and were active during or after consummatory licking. However, we found quantitative differences in response strength: striosomal neurons fired more to reward-predicting cues and encoded more information about expected outcome as mice learned the task, whereas matrix neurons were more strongly modulated by recent reward history. These findings open the possibility of harnessing in vivo imaging to determine the contributions of striosomes and matrix to striatal circuit function.

Background activity and visual responsiveness of caudate nucleus neurons in halothane anesthetized and in awake, behaving cats

This study focuses on the important question whether brain activity recorded from anesthetized, paralyzed animals is comparable to that recorded from awake, behaving ones. We compared neuronal activity recorded from the caudate nucleus (CN) of two halothane-anesthetized, paralyzed and two awake, behaving cats. In both models, extracellular recordings were made from the CN during static and dynamic visual stimulation. The anesthesia was maintained during the recordings by a gaseous mixture of air and halothane (1.0%). The behaving animals were trained to perform a visual fixation task. Based on their electrophysiological properties, the recorded CN neurons were separated into three different classes: phasically active (PANs), high firing (HFNs), and tonically active (TANs) neurons. Halothane anesthesia significantly decreased the background activity of the CN neurons in all three classes. The anesthesia had the most remarkable suppressive effect on PANs, where the background activity was consistently under 1 spike/s. The analysis of these responses was almost impossible due to the extremely low activity. The evoked responses during both static and dynamic visual stimulation were obvious in the behaving cats. On the other hand, only weak visual responses were found in some neurons of halothane anesthetized cats. These results show that halothane gas anesthesia has a marked suppressive effect on the feline CN. We suggest that for the purposes of the visual and related multisensory/sensorimotor electrophysiological exploration of the CN, behaving animal models are preferable over anesthetized ones.

Vicarious trial and error

When rats come to a decision point, they sometimes pause and look back and forth as if deliberating over the choice; at other times, they proceed as if they have already made their decision. In the 1930s, this pause-and-look behaviour was termed 'vicarious trial and error' (VTE), with the implication that the rat was 'thinking about the future'. The discovery in 2007 that the firing of hippocampal place cells gives rise to alternating representations of each of the potential path options in a serial manner during VTE suggested a possible neural mechanism that could underlie the representations of future outcomes. More-recent experiments examining VTE in rats suggest that there are direct parallels to human processes of deliberative decision making, working memory and mental time travel.

Activity of Caudate Nucleus Neurons in a Visual Fixation Paradigm in Behaving Cats

Beside its motor functions, the caudate nucleus (CN), the main input structure of the basal ganglia, is also sensitive to various sensory modalities. The goal of the present study was to investigate the effects of visual stimulation on the CN by using a behaving, head-restrained, eye movement-controlled feline model developed recently for this purpose. Extracellular multielectrode recordings were made from the CN of two cats in a visual fixation paradigm applying static and dynamic stimuli. The recorded neurons were classified in three groups according to their electrophysiological properties: phasically active (PAN), tonically active (TAN) and high-firing (HFN) neurons. The response characteristics were investigated according to this classification. The PAN and TAN neurons were sensitive primarily to static stimuli, while the HFN neurons responded primarily to changes in the visual environment i.e. to optic flow and the offset of the stimuli. The HFNs were the most sensitive to visual stimulation; their responses were stronger than those of the PANs and TANs. The majority of the recorded units were insensitive to the direction of the optic flow, regardless of group, but a small number of direction-sensitive neurons were also found. Our results demonstrate that both the static and the dynamic components of the visual information are represented in the CN. Furthermore, these results provide the first piece of evidence on optic flow processing in the CN, which, in more general terms, indicates the possible role of this structure in dynamic visual information processing.

The striatum: where skills and habits meet

After more than a century of work concentrating on the motor functions of the basal ganglia, new ideas have emerged, suggesting that the basal ganglia also have major functions in relation to learning habits and acquiring motor skills. We review the evidence supporting the role of the striatum in optimizing behavior by refining action selection and in shaping habits and skills as a modulator of motor repertoires. These findings challenge the notion that striatal learning processes are limited to the motor domain. The learning mechanisms supported by striatal circuitry generalize to other domains, including cognitive skills and emotion-related patterns of action.

Endocannabinoids mediate bidirectional striatal spike-timing-dependent plasticity

KEY POINTS

Although learning can arise from few or even a single trial, synaptic plasticity is commonly assessed under prolonged activation. Here, we explored the existence of rapid responsiveness of synaptic plasticity at corticostriatal synapses in a major synaptic learning rule, spike-timing-dependent plasticity (STDP). We found that spike-timing-dependent depression (tLTD) progressively disappears when the number of paired stimulations (below 50 pairings) is decreased whereas spike-timing-dependent potentiation (tLTP) displays a biphasic profile: tLTP is observed for 75-100 pairings, is absent for 25-50 pairings and re-emerges for 5-10 pairings. This tLTP induced by low numbers of pairings (5-10) depends on activation of the endocannabinoid system, type-1 cannabinoid receptor and the transient receptor potential vanilloid type-1. Endocannabinoid-tLTP may represent a physiological mechanism operating during the rapid learning of new associative memories and behavioural rules characterizing the flexible behaviour of mammals or during the initial stages of habit learning.

ABSTRACT

Synaptic plasticity, a main substrate for learning and memory, is commonly assessed with prolonged stimulations. Since learning can arise from few or even a single trial, synaptic strength is expected to adapt rapidly. However, whether synaptic plasticity occurs in response to limited event occurrences remains elusive. To answer this question, we investigated whether a low number of paired stimulations can induce plasticity in a major synaptic learning rule, spike-timing-dependent plasticity (STDP). It is known that 100 pairings induce bidirectional STDP, i.e. spike-timing-dependent potentiation (tLTP) and depression (tLTD) at most central synapses. In rodent striatum, we found that tLTD progressively disappears when the number of paired stimulations is decreased (below 50 pairings) whereas tLTP displays a biphasic profile: tLTP is observed for 75-100 pairings, absent for 25-50 pairings and re-emerges for 5-10 pairings. This tLTP, induced by very few pairings (∼5-10) depends on the endocannabinoid (eCB) system. This eCB-dependent tLTP (eCB-tLTP) involves postsynaptic endocannabinoid synthesis, requires paired activity (post- and presynaptic) and the activation of type-1 cannabinoid receptor (CB1R) and transient receptor potential vanilloid type-1 (TRPV1). eCB-tLTP occurs in both striatopallidal and striatonigral medium-sized spiny neurons (MSNs) and is dopamine dependent. Lastly, we show that eCB-LTP and eCB-LTD can be induced sequentially in the same neuron, depending on the cellular conditioning protocol. Thus, while endocannabinoids are usually thought simply to depress synaptic function, they also constitute a versatile system underlying bidirectional plasticity. Our results reveal a novel form of synaptic plasticity, eCB-tLTP, which may underlie rapid learning capabilities characterizing behavioural flexibility.

Selective effects of dopamine depletion and L-DOPA therapy on learning-related firing dynamics of striatal neurons

Despite evidence that dopamine neurotransmission in the striatum is critical for learning as well as for movement control, little is yet known about how the learning-related dynamics of striatal activity are affected by dopamine depletion, a condition faced in Parkinson's disease. We made localized intrastriatal 6-hydroxydopamine lesions in rats and recorded within the dopamine-depleted sensorimotor striatal zone and its contralateral correspondent as the animals learned a conditional maze task. Rather than producing global, nonspecific elevations in firing rate across the task, the dopamine depletion altered striatal projection neuron activity and fast-spiking interneuron activity selectively, with sharply task-specific and cell type-specific effects, and often, with learning-stage selective effects as well. Striatal projection neurons with strong responses during the maze runs had especially elevated responsiveness during the maze runs. Projection neurons that, instead, fired most strongly before maze running showed elevated pre-start firing rates, but not during maze running, as learning progressed. The intrastriatal dopamine depletion severely affected the learning-related patterning of fast-spiking interneuron ensembles, especially during maze running and after extended training. Remarkably, L-DOPA treatment almost entirely reversed the depletion-induced elevations in pre-run firing of the projection neurons, and elevated their responses around start and end of maze runs. By contrast, L-DOPA failed to normalize fast-spiking interneuron activity. Thus the effects of striatal dopamine depletion and restoration on striatal activity are highly dependent not only on cell type, as previously shown, but also on the behavioral activity called for and the state of behavioral learning achieved.

Hierarchical reinforcement learning and decision making

The hierarchical structure of human and animal behavior has been of critical interest in neuroscience for many years. Yet understanding the neural processes that give rise to such structure remains an open challenge. In recent research, a new perspective on hierarchical behavior has begun to take shape, inspired by ideas from machine learning, and in particular the framework of hierarchical reinforcement learning. Hierarchical reinforcement learning builds on traditional reinforcement learning mechanisms, extending them to accommodate temporally extended behaviors or subroutines. The resulting computational paradigm has begun to influence both theoretical and empirical work in neuroscience, conceptually aligning the study of hierarchical behavior with research on other aspects of learning and decision making, and giving rise to some thought-provoking new findings.

Parallel contributions of cerebellar, striatal and M1 mechanisms to motor sequence learning

When learning a new motor sequence, we must execute the correct order of movements while simultaneously optimizing sensorimotor parameters such as trajectory, timing, velocity and force. Neurophysiological studies in animals and humans have identified the major brain regions involved in sequence learning, including the motor cortex (M1), basal ganglia (BG) and cerebellum. Current models link these regions to different stages of learning (early vs. late) or different components of performance (spatial vs. sensorimotor). At the same time, research in motor control has given rise to the concept that internal models at different levels of the motor system may contribute to learning. The goal of this review is to develop a new framework for motor sequence learning that combines stage and component models within the context of internal models. To do this, we review behavioral and neuroimaging studies in humans and neurophysiological studies in animals. Based on this evidence, we present a model proposing that sequence learning is underwritten by parallel, interacting processes, including internal model formation and sequence representation, that are instantiated in specific cerebellar, BG or M1 mechanisms depending on task demands and the stage of learning. The striatal system learns predictive stimulus-response associations and is critical for motor chunking. The role of the cerebellum is to acquire the optimal internal model for sequence performance in a particular context, and to contribute to error correction and control of on-going movement. M1 acts to store the representation of a learned sequence, likely as part of a distributed network including the parietal lobe and premotor cortex.

Neural systems analysis of decision making during goal-directed navigation

The ability to make adaptive decisions during goal-directed navigation is a fundamental and highly evolved behavior that requires continual coordination of perceptions, learning and memory processes, and the planning of behaviors. Here, a neurobiological account for such coordination is provided by integrating current literatures on spatial context analysis and decision-making. This integration includes discussions of our current understanding of the role of the hippocampal system in experience-dependent navigation, how hippocampal information comes to impact midbrain and striatal decision making systems, and finally the role of the striatum in the implementation of behaviors based on recent decisions. These discussions extend across cellular to neural systems levels of analysis. Not only are key findings described, but also fundamental organizing principles within and across neural systems, as well as between neural systems functions and behavior, are emphasized. It is suggested that studying decision making during goal-directed navigation is a powerful model for studying interactive brain systems and their mediation of complex behaviors.

The role of the basal ganglia in learning and memory: insight from Parkinson's disease

It has long been known that memory is not a single process. Rather, there are different kinds of memory that are supported by distinct neural systems. This idea stemmed from early findings of dissociable patterns of memory impairments in patients with selective damage to different brain regions. These studies highlighted the role of the basal ganglia in non-declarative memory, such as procedural or habit learning, contrasting it with the known role of the medial temporal lobes in declarative memory. In recent years, major advances across multiple areas of neuroscience have revealed an important role for the basal ganglia in motivation and decision making. These findings have led to new discoveries about the role of the basal ganglia in learning and highlighted the essential role of dopamine in specific forms of learning. Here we review these recent advances with an emphasis on novel discoveries from studies of learning in patients with Parkinson's disease. We discuss how these findings promote the development of current theories away from accounts that emphasize the verbalizability of the contents of memory and towards a focus on the specific computations carried out by distinct brain regions. Finally, we discuss new challenges that arise in the face of accumulating evidence for dynamic and interconnected memory systems that jointly contribute to learning.