Shifting responsibly: the importance of striatal modularity to reinforcement learning in uncertain environments

Striatal parvalbumin interneurons are activated in a mouse model of cerebellar dystonia

Dystonia is thought to arise from abnormalities in the motor loop of the basal ganglia; however, there is an ongoing debate regarding cerebellar involvement. We adopted an established cerebellar dystonia mouse model by injecting ouabain to examine the contribution of the cerebellum. Initially, we examined whether the entopeduncular nucleus (EPN), substantia nigra pars reticulata (SNr), globus pallidus externus (GPe) and striatal neurons were activated in the model. Next, we examined whether administration of a dopamine D1 receptor agonist and dopamine D2 receptor antagonist or selective ablation of striatal parvalbumin (PV, encoded by Pvalb)-expressing interneurons could modulate the involuntary movements of the mice. The cerebellar dystonia mice had a higher number of cells positive for c-fos (encoded by Fos) in the EPN, SNr and GPe, as well as a higher positive ratio of c-fos in striatal PV interneurons, than those in control mice. Furthermore, systemic administration of combined D1 receptor agonist and D2 receptor antagonist and selective ablation of striatal PV interneurons relieved the involuntary movements of the mice. Abnormalities in the motor loop of the basal ganglia could be crucially involved in cerebellar dystonia, and modulating PV interneurons might provide a novel treatment strategy.

Therapeutic Effects of Dual Dopaminergic Modulation With l-DOPA and Chlorpromazine in Patients With Idiopathic Cervical Dystonia

Objectives

Imbalanced activities between dopamine D1 and D2 signals in striatal striosome-matrix system have been proposed as a cause of dystonia symptoms. The aim of this study was to assess the therapeutic effects of dual dopaminergic modulation (DDM) with l-DOPA and chlorpromazine (CPZ) in patients with idiopathic cervical dystonia (CD).

Methods

We enrolled 21 patients with CD who responded poorly to botulinum toxin treatment. The severities of CD motor symptoms and CD-associated pain were determined using the Toronto Western Spasmodic Torticollis Rating Scale and the visual analog scale, respectively.

Results

In patients with CD (n = 7), oral administration of l-DOPA combined with CPZ significantly attenuated both CD motor symptoms and CD-associated pain in a dose-related manner. By contrast, there was no improvement of CD symptoms in patients (n = 7) who ingested l-DOPA alone nor in those (n = 7) who ingested CPZ alone.

Discussion

DDM with l-DOPA and CPZ may be an effective tool to treat dystonia symptoms in patients with botulinum toxin-resistant idiopathic CD. Our results may also indicate that CD dystonia symptoms could be attenuated through DDM inducing an increase in striosomal D1-signaling.

Classification of Evidence

This study provides Class III evidence that treatment of botulinum toxin-resistant idiopathic cervical dystonia with l-DOPA and chlorpromazine is superior to either one alone.

Having multiple selves helps learning agents explore and adapt in complex changing worlds

Satisfying a variety of conflicting needs in a changing environment is a fundamental challenge for any adaptive agent. Here, we show that designing an agent in a modular fashion as a collection of subagents, each dedicated to a separate need, powerfully enhanced the agent's capacity to satisfy its overall needs. We used the formalism of deep reinforcement learning to investigate a biologically relevant multiobjective task: continually maintaining homeostasis of a set of physiologic variables. We then conducted simulations in a variety of environments and compared how modular agents performed relative to standard monolithic agents (i.e., agents that aimed to satisfy all needs in an integrated manner using a single aggregate measure of success). Simulations revealed that modular agents a) exhibited a form of exploration that was intrinsic and emergent rather than extrinsically imposed; b) were robust to changes in nonstationary environments, and c) scaled gracefully in their ability to maintain homeostasis as the number of conflicting objectives increased. Supporting analysis suggested that the robustness to changing environments and increasing numbers of needs were due to intrinsic exploration and efficiency of representation afforded by the modular architecture. These results suggest that the normative principles by which agents have adapted to complex changing environments may also explain why humans have long been described as consisting of "multiple selves."

Striosomes and Matrisomes: Scaffolds for Dynamic Coupling of Volition and Action

Striosomes form neurochemically specialized compartments of the striatum embedded in a large matrix made up of modules called matrisomes. Striosome-matrix architecture is multiplexed with the canonical direct-indirect organization of the striatum. Striosomal functions remain to be fully clarified, but key information is emerging. First, striosomes powerfully innervate nigral dopamine-containing neurons and can completely shut down their activity, with a following rebound excitation. Second, striosomes receive limbic and cognition-related corticostriatal afferents and are dynamically modulated in relation to value-based actions. Third, striosomes are spatially interspersed among matrisomes and interneurons and are influenced by local and global neuromodulatory and oscillatory activities. Fourth, striosomes tune engagement and the motivation to perform reinforcement learning, to manifest stereotypical behaviors, and to navigate valence conflicts and valence discriminations. We suggest that, at an algorithmic level, striosomes could serve as distributed scaffolds to provide formats of the striatal computations generated through development and refined through learning. We propose that striosomes affect subjective states. By transforming corticothalamic and other inputs to the functional formats of the striatum, they could implement state transitions in nigro-striato-nigral circuits to affect bodily and cognitive actions according to internal motives whose functions are compromised in neuropsychiatric conditions.

A dual dopaminergic therapy with L-3,4-dihydroxyphenylalanine and chlorpromazine for the treatment of blepharospasm, a focal dystonia: Possible implications for striosomal D1 signaling

Impairment of balanced activity between dopamine D1 and D2 receptor functions in the striatum, particularly in striatal functional subdivisions (i.e., striosome and matrix compartments), has been proposed to underlie dystonia genesis. This study was undertaken to examine the therapeutic effect of dual dopaminergic modulation with L-3,4-dihydroxyphenylalanine (L-DOPA) and chlorpromazine (CPZ) in patients with blepharospasm, a focal dystonia. For this purpose, Dopacol tablets™ (L-DOPA 50 mg plus carbidopa 5 mg) and Wintermin™ (CPZ phenolphthalinate 180 mg/g) were used. Clinical evaluations were performed before and after an 8-week drug treatment interval using the Visual Analog Scale (VAS), Blepharospasm Disability Index (BSDI), modified VAS (mVAS), and Jankovic Rating Scale (JRS). The data were analyzed using non-parametric statistics. Results showed that in patients (n = 7) with blepharospasm, dystonia symptoms were significantly alleviated by the administration of both Dopacol tablets™ (one tablet × 3/day) and CPZ (5 mg × 3/day), as determined using the VAS, BSDI, mVAS, and JRS. In contrast, there was no improvement of dystonia symptoms in patients (n = 7) who ingested Dopacol tablets™ (one tablet × 3/day) alone, nor in those (n = 7) who ingested CPZ (5 mg × 3/day) alone. Thus, dual pharmacotherapy with L-DOPA and CPZ can exert a therapeutic effect on blepharospasm, suggesting that dystonia symptoms can be attenuated through dopaminergic modulation with inducing an increase in striatal D1-signals. Since dopamine D1 receptors are heavily enriched in the striosome compartment in the “human” striatum, our results also suggest that striosomal loss of D1-signaling may be important in the pathogenesis of dystonia.

Multiplexed action-outcome representation by striatal striosome-matrix compartments detected with a mouse cost-benefit foraging task

Learning about positive and negative outcomes of actions is crucial for survival and underpinned by conserved circuits including the striatum. How associations between actions and outcomes are formed is not fully understood, particularly when the outcomes have mixed positive and negative features. We developed a novel foraging (‘bandit’) task requiring mice to maximize rewards while minimizing punishments. By 2-photon Ca⁺⁺ imaging, we monitored activity of visually identified anterodorsal striatal striosomal and matrix neurons. We found that action-outcome associations for reward and punishment were encoded in parallel in partially overlapping populations. Single neurons could, for one action, encode outcomes of opposing valence. Striosome compartments consistently exhibited stronger representations of reinforcement outcomes than matrix, especially for high reward or punishment prediction errors. These findings demonstrate multiplexing of action-outcome contingencies by single identified striatal neurons and suggest that striosomal neurons are particularly important in action-outcome learning.

The role that the striatum plays in tracking the association between actions and combinations of rewarding and aversive outcomes remains unclear. Here, by using both calcium imaging in mice and reinforcement learning models, the authors find that individual striatal neurons can encode associations between actions and multiple, sometimes conflicting, outcomes.

From Progenitors to Progeny: Shaping Striatal Circuit Development and Function

Understanding how neurons of the striatum are formed and integrate into complex synaptic circuits is essential to provide insight into striatal function in health and disease. In this review, we summarize our current understanding of the development of striatal neurons and associated circuits with a focus on their embryonic origin. Specifically, we address the role of distinct types of embryonic progenitors, found in the proliferative zones of the ganglionic eminences in the ventral telencephalon, in the generation of diverse striatal interneurons and projection neurons. Indeed, recent evidence would suggest that embryonic progenitor origin dictates key characteristics of postnatal cells, including their neurochemical content, their location within striatum, and their long-range synaptic inputs. We also integrate recent observations regarding embryonic progenitors in cortical and other regions and discuss how this might inform future research on the ganglionic eminences. Last, we examine how embryonic progenitor dysfunction can alter striatal formation, as exemplified in Huntington's disease and autism spectrum disorder, and how increased understanding of embryonic progenitors can have significant implications for future research directions and the development of improved therapeutic options.SIGNIFICANCE STATEMENT This review highlights recently defined novel roles for embryonic progenitor cells in shaping the functional properties of both projection neurons and interneurons of the striatum. It outlines the developmental mechanisms that guide neuronal development from progenitors in the embryonic ganglionic eminences to progeny in the striatum. Where questions remain open, we integrate observations from cortex and other regions to present possible avenues for future research. Last, we provide a progenitor-centric perspective onto both Huntington's disease and autism spectrum disorder. We suggest that future investigations and manipulations of embryonic progenitor cells in both research and clinical settings will likely require careful consideration of their great intrinsic diversity and neurogenic potential.

Dystonia and Cerebellum: From Bench to Bedside

Dystonia pathogenesis remains unclear; however, findings from basic and clinical research suggest the importance of the interaction between the basal ganglia and cerebellum. After the discovery of disynaptic pathways between the two, much attention has been paid to the cerebellum. Basic research using various dystonia rodent models and clinical studies in dystonia patients continues to provide new pieces of knowledge regarding the role of the cerebellum in dystonia genesis. Herein, we review basic and clinical articles related to dystonia focusing on the cerebellum, and clarify the current understanding of the role of the cerebellum in dystonia pathogenesis. Given the recent evidence providing new hypotheses regarding dystonia pathogenesis, we discuss how the current evidence answers the unsolved clinical questions.

Altered Spontaneous Neural Activity and Functional Connectivity in Parkinson's Disease With Subthalamic Microlesion

Background

Transient improvement in motor symptoms are immediately observed in patients with Parkinson's disease (PD) after an electrode has been implanted into the subthalamic nucleus (STN) for deep brain stimulation (DBS). This phenomenon is known as the microlesion effect (MLE). However, the underlying mechanisms of MLE is poorly understood.

Purpose

We utilized resting state functional MRI (rs-fMRI) to evaluate changes in spontaneous brain activity and networks in PD patients during the microlesion period after DBS.

Method

Overall, 37 PD patients and 13 gender- and age-matched healthy controls (HCs) were recruited for this study. Rs-MRI information was collected from PD patients three days before DBS and one day after DBS, whereas the HCs group was scanned once. We utilized the amplitude of low-frequency fluctuation (ALFF) method in order to analyze differences in spontaneous whole-brain activity among all subjects. Furthermore, functional connectivity (FC) was applied to investigate connections between other brain regions and brain areas with significantly different ALFF before and after surgery in PD patients.

Result

Relative to the PD-Pre-DBS group, the PD-Post-DBS group had higher ALFF in the right putamen, right inferior frontal gyrus, right precentral gyrus and lower ALFF in right angular gyrus, right precuneus, right posterior cingulate gyrus (PCC), left insula, left middle temporal gyrus (MTG), bilateral middle frontal gyrus and bilateral superior frontal gyrus (dorsolateral). Functional connectivity analysis revealed that these brain regions with significantly different ALFF scores demonstrated abnormal FC, largely in the temporal, prefrontal cortices and default mode network (DMN).

Conclusion

The subthalamic microlesion caused by DBS in PD was found to not only improve the activity of the basal ganglia-thalamocortical circuit, but also reduce the activity of the DMN and executive control network (ECN) related brain regions. Results from this study provide new insights into the mechanism of MLE.

Causal Evidence for Induction of Pessimistic Decision-Making in Primates by the Network of Frontal Cortex and Striosomes

Clinical studies have shown that patients with anxiety disorders exhibited coactivation of limbic cortices and basal ganglia, which together form a large-scale brain network. The mechanisms by which such a large-scale network could induce or modulate anxiety-like states are largely unknown. This article reviews our experimental program in macaques demonstrating a causal involvement of local striatal and frontal cortical sites in inducing pessimistic decision-making that underlies anxiety. Where relevant, we related these findings to the wider literature. To identify such sites, we have made a series of methodologic advances, including the combination of causal evidence for behavioral modification of pessimistic decisions with viral tracing methods. Critically, we introduced a version of the classic approach-avoidance (Ap-Av) conflict task, modified for use in non-human primates. We performed microstimulation of limbic-related cortical regions and the striatum, focusing on the pregenual anterior cingulate cortex (pACC), the caudal orbitofrontal cortex (cOFC), and the caudate nucleus (CN). Microstimulation of localized sites within these regions induced pessimistic decision-making by the monkeys, supporting the idea that the focal activation of these regions could induce an anxiety-like state, which subsequently influences decision-making. We further performed combined microstimulation and tract-tracing experiments by injecting anterograde viral tracers into focal regions, at which microstimulation induced increased avoidance. We found that effective stimulation sites in both pACC and cOFC zones projected preferentially to striosomes in the anterior striatum. Experiments in rodents have shown that the striosomes in the anterior striatum project directly to the dopamine-containing cells in the substantia nigra, and we have found evidence for a functional connection between striosomes and the lateral habenular region in which responses to reward are inhibitory. We present here further evidence for network interactions: we show that the pACC and cOFC project to common structures, including not only the anterior parts of the striosome compartment but also the tail of the CN, the subgenual ACC, the amygdala, and the thalamus. Together, our findings suggest that networks having pACC and cOFC as nodes share similar features in their connectivity patterns. We here hypothesize, based on these results, that the brain sites related to pessimistic judgment are mediated by a large-scale brain network that regulates dopaminergic functions and includes striosomes and striosome-projecting cortical regions.

Learning to select actions shapes recurrent dynamics in the corticostriatal system

Learning to select appropriate actions based on their values is fundamental to adaptive behavior. This form of learning is supported by fronto-striatal systems. The dorsal-lateral prefrontal cortex (dlPFC) and the dorsal striatum (dSTR), which are strongly interconnected, are key nodes in this circuitry. Substantial experimental evidence, including neurophysiological recordings, have shown that neurons in these structures represent key aspects of learning. The computational mechanisms that shape the neurophysiological responses, however, are not clear. To examine this, we developed a recurrent neural network (RNN) model of the dlPFC-dSTR circuit and trained it on an oculomotor sequence learning task. We compared the activity generated by the model to activity recorded from monkey dlPFC and dSTR in the same task. This network consisted of a striatal component which encoded action values, and a prefrontal component which selected appropriate actions. After training, this system was able to autonomously represent and update action values and select actions, thus being able to closely approximate the representational structure in corticostriatal recordings. We found that learning to select the correct actions drove action-sequence representations further apart in activity space, both in the model and in the neural data. The model revealed that learning proceeds by increasing the distance between sequence-specific representations. This makes it more likely that the model will select the appropriate action sequence as learning develops. Our model thus supports the hypothesis that learning in networks drives the neural representations of actions further apart, increasing the probability that the network generates correct actions as learning proceeds. Altogether, this study advances our understanding of how neural circuit dynamics are involved in neural computation, revealing how dynamics in the corticostriatal system support task learning.

A Genetically Defined Compartmentalized Striatal Direct Pathway for Negative Reinforcement

The striosome compartment within the dorsal striatum has been implicated in reinforcement learning and regulation of motivation, but how striosomal neurons contribute to these functions remains elusive. Here, we show that a genetically identified striosomal population, which expresses the Teashirt family zinc finger 1 (Tshz1) and belongs to the direct pathway, drives negative reinforcement and is essential for aversive learning in mice. Contrasting a "conventional" striosomal direct pathway, the Tshz1 neurons cause aversion, movement suppression, and negative reinforcement once activated, and they receive a distinct set of synaptic inputs. These neurons are predominantly excited by punishment rather than reward and represent the anticipation of punishment or the motivation for avoidance. Furthermore, inhibiting these neurons impairs punishment-based learning without affecting reward learning or movement. These results establish a major role of striosomal neurons in behaviors reinforced by punishment and moreover uncover functions of the direct pathway unaccounted for in classic models.

Computational Characteristics of the Striatal Dopamine System Described by Reinforcement Learning With Fast Generalization

Generalization is the ability to apply past experience to similar but non-identical situations. It not only affects stimulus-outcome relationships, as observed in conditioning experiments, but may also be essential for adaptive behaviors, which involve the interaction between individuals and their environment. Computational modeling could potentially clarify the effect of generalization on adaptive behaviors and how this effect emerges from the underlying computation. Recent neurobiological observation indicated that the striatal dopamine system achieves generalization and subsequent discrimination by updating the corticostriatal synaptic connections in differential response to reward and punishment. In this study, we analyzed how computational characteristics in this neurobiological system affects adaptive behaviors. We proposed a novel reinforcement learning model with multilayer neural networks in which the synaptic weights of only the last layer are updated according to the prediction error. We set fixed connections between the input and hidden layers to maintain the similarity of inputs in the hidden-layer representation. This network enabled fast generalization of reward and punishment learning, and thereby facilitated safe and efficient exploration of spatial navigation tasks. Notably, it demonstrated a quick reward approach and efficient punishment aversion in the early learning phase, compared to algorithms that do not show generalization. However, disturbance of the network that causes noisy generalization and impaired discrimination induced maladaptive valuation. These results suggested the advantage and potential drawback of computation by the striatal dopamine system with regard to adaptive behaviors.

Functionally matched domains in parietal-frontal cortex of monkeys project to overlapping regions of the striatum

Projections of small regions (domains) of primary motor cortex (M1), premotor cortex (PMC) and posterior parietal cortex (PPC) to the striatum of squirrel monkeys were revealed by restricted injections of anterograde tracers. As many as 8 classes of action-specific domains can be identified in PPC, as well as in PMC and M1, and some have been identified for injections by the action evoked by 0.5 s trains of electrical microstimulation. Injections of domains in all three cortical regions labeled dense patches of terminations in the matrix of the ipsilateral putamen, while providing sparse or no projections to corresponding regions of the contralateral putamen. When two or three of these domains were injected with different tracers, projection fields in the putamen were highly overlapped for injections in functionally matched domains across cortical areas, but were highly segregated for injections placed in functionally mismatched domains. While not all classes of domains were studied, the results suggest that the striatum potentially has separate representations of eight or more classes of actions that receive inputs from domains in three or more cortical regions in sensorimotor cortex. The overlap/segregation of cortico-striatal projections correlates with the strength of cortico-cortical connections between injected motor areas.

Network organization during probabilistic learning via taste outcomes

Reinforcement learning guides food decisions, yet how the brain learns from taste in humans is not fully understood. Existing research examines reinforcement learning from taste using passive condition paradigms, but response-dependent instrumental conditioning better reflects natural eating behavior. Here, we examined brain response during a taste-motivated reinforcement learning task and how measures of task-based network structure were related to behavioral outcomes. During a functional MRI scan, 85 participants completed a probabilistic selection task with feedback via sweet taste or bitter taste. Whole brain response and functional network topology measures, including identification of communities and community segregation, were examined during choice, sweet taste, and bitter taste conditions. Relative to the bitter taste, sweet taste was associated with increased whole brain response in the hippocampus, oral somatosensory cortex, and orbitofrontal cortex. Sweet taste was also related to differential community assignment of the ventromedial prefrontal cortex and ventrolateral prefrontal cortex compared to bitter taste. During choice, increasing segregation of a community containing the amygdala, hippocampus, and right fusiform gyrus was associated with increased sensitivity to punishment on the task's posttest. Further, normal BMI was associated with differential community structure compared to overweight and obese BMI, where high BMI reflected increased connectivity of visual regions. Together, results demonstrate that network topology of learning and memory regions during choice is related to avoiding a bitter taste, and that BMI is associated with increased connectivity of area involved in processing external stimuli. Network organization and topology provide unique insight into individual differences in brain response to instrumental conditioning via taste reinforcers.

Bipolar oscillations between positive and negative mood states in a computational model of Basal Ganglia

Bipolar disorder is characterized by mood swings-oscillations between manic and depressive states. The swings (oscillations) mark the length of an episode in a patient's mood cycle (period), and can vary from hours to years. The proposed modeling study uses decision making framework to investigate the role of basal ganglia network in generating bipolar oscillations. In this model, the basal ganglia system performs a two-arm bandit task in which one of the arms (action responses) leads to a positive outcome, while the other leads to a negative outcome. We explore the dynamics of key reward and risk related parameters in the system while the model agent receives various outcomes. Particularly, we study the system using a model that represents the fast dynamics of decision making, and a module to capture the slow dynamics that describe the variation of some meta-parameters of fast dynamics over long time scales. The model is cast at three levels of abstraction: (1) a two-dimensional dynamical system model, that is a simple two variable model capable of showing bistability for rewarding and punitive outcomes; (2) a phenomenological basal ganglia model, to extend the implications from the reduced model to a cortico-basal ganglia setup; (3) a detailed network model of basal ganglia, that incorporates detailed cellular level models for a more realistic understanding. In healthy conditions, the model chooses positive action and avoids negative one, whereas under bipolar conditions, the model exhibits slow oscillations in its choice of positive or negative outcomes, reminiscent of bipolar oscillations. Phase-plane analyses on the simple reduced dynamical system with two variables reveal the essential parameters that generate pathological 'bipolar-like' oscillations. Phenomenological and network models of the basal ganglia extend that logic, and interpret bipolar oscillations in terms of the activity of dopaminergic and serotonergic projections on the cortico-basal ganglia network dynamics. The network's dysfunction, specifically in terms of reward and risk sensitivity, is shown to be responsible for the pathological bipolar oscillations. The study proposes a computational model that explores the effects of impaired serotonergic neuromodulation on the dynamics of the cortico basal ganglia network, and relates this impairment to abstract mood states (manic and depressive episodes) and oscillations of bipolar disorder.

Cingulate-basal ganglia-thalamo-cortical aspects of catatonia and implications for treatment

The catatonic syndrome is an example of a multifactorial neurobehavioral disorder that causes much morbidity and mortality but also has the potential to unlock the mystery of how motivation and movement interact to produce behavior. In this chapter, an attempt is made to understand better the catatonic syndrome through the lens of neurobiology and neuropathophysiology updated by recent studies in molecular biology, genomics, inflammasomics, neuroimaging, neural network theory, and neuropsychopathology. This will result in a neurostructural model for the catatonic syndrome that centers on paralimbic regions including the anterior and midcingulate cortices, as they interface with striatal and thalamic nodes in the salience decision-making network. Examination of neurologic disorders like the abulic syndrome, which includes in its extreme catatonic form, akinetic mutism, will identify the cingulate cortex and paralimbic neighbors as regions of interest. This exploration has the potential to unlock mysteries of the brain cascade from motivation to movement and to clarify catatonia therapeutics. Such a synthesis may also help us discern meaning inherent in this complex neurobehavioral syndrome.

Compartmental function and modulation of the striatum

The striatum plays a central role in guiding numerous complex behaviors, ranging from motor control to action selection and reward learning. The diverse responsibilities of the striatum are reflected by the complexity of its organization. In this review, we will summarize what is currently known about the compartmental layout of the striatum, an organizational principle that is crucial for allowing the striatum to guide such a diverse array of behaviors. We will focus on the anatomical and functional properties of striosome (patch) and matrix compartments of the striatum, and how the engagement of these compartments is uniquely controlled by their afferents, intrinsic properties, and neuromodulation. We will give examples of how advances in technology have opened the door to functionally dissecting the striatum's compartmental design, and close by offering thoughts on the future and relevance for human disease.

Predominant Striatal Input to the Lateral Habenula in Macaques Comes from Striosomes

Striosomes, neurochemically specialized modules in the striatum, are thought to be nodes in circuits extending, via basal ganglia pathways, from mood-related neocortical regions to dopamine-containing neurons of the substantia nigra. Yet striosomes have remained beyond the reach of electrophysiological methods to identify them, especially in non-human primates. Such work is needed for translational as well as for basic science. Here we introduce a method to identify striosomes on-line in awake, behaving macaques. We combined electrical microstimulation of the striatum with simultaneous electrophysiological recording in the lateral habenula (LHb) followed by immunohistochemistry. We demonstrate that striosomes provide the predominant striatal input to the macaque pallido-habenular circuit, which is known to function in relation to reinforcement signaling. Further, our experiments suggest that striosomes from different striatal regions may convergently influence the lateral habenula. This work now opens the way to defining the functions of striosomes in behaving primates in relation to mood, motivation, and action.

Striatal Microstimulation Induces Persistent and Repetitive Negative Decision-Making Predicted by Striatal Beta-Band Oscillation

Persistent thoughts inducing irrationally pessimistic and repetitive decisions are often symptoms of mood and anxiety disorders. Regional neural hyperactivities have been associated with these disorders, but it remains unclear whether there is a specific brain region causally involved in these persistent valuations. Here, we identified potential sources of such persistent states by microstimulating the striatum of macaques performing a task by which we could quantitatively estimate their subjective pessimistic states using their choices to accept or reject conflicting offers. We found that this microstimulation induced irrationally repetitive choices with negative evaluations. Local field potentials recorded in the same microstimulation sessions exhibited modulations of beta-band oscillatory activity that paralleled the persistent negative states influencing repetitive decisions. These findings demonstrate that local striatal zones can causally affect subjective states influencing persistent negative valuation and that abnormal beta-band oscillations can be associated with persistency in valuation accompanied by an anxiety-like state.

Radial Glial Lineage Progression and Differential Intermediate Progenitor Amplification Underlie Striatal Compartments and Circuit Organization

The circuitry of the striatum is characterized by two organizational plans: the division into striosome and matrix compartments, thought to mediate evaluation and action, and the direct and indirect pathways, thought to promote or suppress behavior. The developmental origins of these organizations and their developmental relationships are unknown, leaving a conceptual gap in understanding the cortico-basal ganglia system. Through genetic fate mapping, we demonstrate that striosome-matrix compartmentalization arises from a lineage program embedded in lateral ganglionic eminence radial glial progenitors mediating neurogenesis through two distinct types of intermediate progenitors (IPs). The early phase of this program produces striosomal spiny projection neurons (SPNs) through fate-restricted apical IPs (aIPSs) with limited capacity; the late phase produces matrix SPNs through fate-restricted basal IPs (bIPMs) with expanded capacity. Notably, direct and indirect pathway SPNs arise within both aIPS and bIPM pools, suggesting that striosome-matrix architecture is the fundamental organizational plan of basal ganglia circuitry.

The cannabinoid-1 receptor is abundantly expressed in striatal striosomes and striosome-dendron bouquets of the substantia nigra

Presynaptic cannabinoid-1 receptors (CB₁-R) bind endogenous and exogenous cannabinoids to modulate neurotransmitter release. CB₁-Rs are expressed throughout the basal ganglia, including striatum and substantia nigra, where they play a role in learning and control of motivated actions. However, the pattern of CB₁-R expression across different striatal compartments, microcircuits and efferent targets, and the contribution of different CB₁-R-expressing neurons to this pattern, are unclear. We use a combination of conventional techniques and novel genetic models to evaluate CB₁-R expression in striosome (patch) and matrix compartments of the striatum, and in nigral targets of striatal medium spiny projection neurons (MSNs). CB₁-R protein and mRNA follow a descending dorsolateral-to-ventromedial intensity gradient in the caudal striatum, with elevated expression in striosomes relative to the surrounding matrix. The lateral predominance of striosome CB₁-Rs contrasts with that of the classical striosomal marker, the mu opioid receptor (MOR), which is expressed most prominently in rostromedial striosomes. The dorsolateral-to-ventromedial CB₁-R gradient is similar to Drd2 dopamine receptor immunoreactivity and opposite to Substance P. This topology of CB₁-R expression is maintained downstream in the globus pallidus and substantia nigra. Dense CB₁-R-expressing striatonigral fibers extend dorsally within the substantia nigra pars reticulata, and colocalize with bundles of ventrally extending, striosome-targeted, dendrites of dopamine-containing neurons in the substantia nigra pars compacta (striosome-dendron bouquets). Within striatum, CB₁-Rs colocalize with fluorescently labeled MSN collaterals within the striosomes. Cre recombinase-mediated deletion of CB₁-Rs from cortical projection neurons or MSNs, and MSN-selective reintroduction of CB₁-Rs in knockout mice, demonstrate that the principal source of CB₁-Rs in dorsolateral striosomes is local MSN collaterals. These data suggest a role for CB₁-Rs in caudal dorsolateral striosome collaterals and striosome-dendron bouquet projections to lateral substantia nigra, where they are anatomically poised to mediate presynaptic disinhibition of both striosomal MSNs and midbrain dopamine neurons in response to endocannabinoids and cannabinomimetics.

Two-photon imaging in mice shows striosomes and matrix have overlapping but differential reinforcement-related responses

Striosomes were discovered several decades ago as neurochemically identified zones in the striatum, yet technical hurdles have hampered the study of the functions of these striatal compartments. Here we used 2-photon calcium imaging in neuronal birthdate-labeled Mash1-CreER;Ai14 mice to image simultaneously the activity of striosomal and matrix neurons as mice performed an auditory conditioning task. With this method, we identified circumscribed zones of tdTomato-labeled neuropil that correspond to striosomes as verified immunohistochemically. Neurons in both striosomes and matrix responded to reward-predicting cues and were active during or after consummatory licking. However, we found quantitative differences in response strength: striosomal neurons fired more to reward-predicting cues and encoded more information about expected outcome as mice learned the task, whereas matrix neurons were more strongly modulated by recent reward history. These findings open the possibility of harnessing in vivo imaging to determine the contributions of striosomes and matrix to striatal circuit function.

Immediate-Early Genes Modulation by Antipsychotics: Translational Implications for a Putative Gateway to Drug-Induced Long-Term Brain Changes

An increasing amount of research aims at recognizing the molecular mechanisms involved in long-lasting brain architectural changes induced by antipsychotic treatments. Although both structural and functional modifications have been identified following acute antipsychotic administration in humans, currently there is scarce knowledge on the enduring consequences of these acute changes. New insights in immediate-early genes (IEGs) modulation following acute or chronic antipsychotic administration may help to fill the gap between primary molecular response and putative long-term changes. Moreover, a critical appraisal of the spatial and temporal patterns of IEGs expression may shed light on the functional "signature" of antipsychotics, such as the propensity to induce motor side effects, the potential neurobiological mechanisms underlying the differences between antipsychotics beyond D2 dopamine receptor affinity, as well as the relevant effects of brain region-specificity in their mechanisms of action. The interest for brain IEGs modulation after antipsychotic treatments has been revitalized by breakthrough findings such as the role of early genes in schizophrenia pathophysiology, the involvement of IEGs in epigenetic mechanisms relevant for cognition, and in neuronal mapping by means of IEGs expression profiling. Here we critically review the evidence on the differential modulation of IEGs by antipsychotics, highlighting the association between IEGs expression and neuroplasticity changes in brain regions impacted by antipsychotics, trying to elucidate the molecular mechanisms underpinning the effects of this class of drugs on psychotic, cognitive and behavioral symptoms.

Neuroplastin deletion in glutamatergic neurons impairs selective brain functions and calcium regulation: implication for cognitive deterioration

The cell adhesion molecule neuroplastin (Np) is a novel candidate to influence human intelligence. Np-deficient mice display complex cognitive deficits and reduced levels of Plasma Membrane Ca²⁺ ATPases (PMCAs), an essential regulator of the intracellular Ca²⁺ concentration ([iCa²⁺]) and neuronal activity. We show abundant expression and conserved cellular and molecular features of Np in glutamatergic neurons in human hippocampal-cortical pathways as characterized for the rodent brain. In Nptn^{lox/loxEmx1Cre} mice, glutamatergic neuron-selective Np ablation resulted in behavioral deficits indicating hippocampal, striatal, and sensorimotor dysfunction paralleled by highly altered activities in hippocampal CA1 area, sensorimotor cortex layers I-III/IV, and the striatal sensorimotor domain detected by single-photon emission computed tomography. Altered hippocampal and cortical activities correlated with reduction of distinct PMCA paralogs in Nptn^{lox/loxEmx1Cre} mice and increased [iCa²⁺] in cultured mutant neurons. Human and rodent Np enhanced the post-transcriptional expression of and co-localized with PMCA paralogs in the plasma membrane of transfected cells. Our results indicate Np as essential for PMCA expression in glutamatergic neurons allowing proper [iCa²⁺] regulation and normal circuit activity. Neuron-type-specific Np ablation empowers the investigation of circuit-coded learning and memory and identification of causal mechanisms leading to cognitive deterioration.

A Biologically Plausible Architecture of the Striatum to Solve Context-Dependent Reinforcement Learning Tasks

Basal ganglia circuit is an important subcortical system of the brain thought to be responsible for reward-based learning. Striatum, the largest nucleus of the basal ganglia, serves as an input port that maps cortical information. Microanatomical studies show that the striatum is a mosaic of specialized input-output structures called striosomes and regions of the surrounding matrix called the matrisomes. We have developed a computational model of the striatum using layered self-organizing maps to capture the center-surround structure seen experimentally and explain its functional significance. We believe that these structural components could build representations of state and action spaces in different environments. The striatum model is then integrated with other components of basal ganglia, making it capable of solving reinforcement learning tasks. We have proposed a biologically plausible mechanism of action-based learning where the striosome biases the matrisome activity toward a preferred action. Several studies indicate that the striatum is critical in solving context dependent problems. We build on this hypothesis and the proposed model exploits the modularity of the striatum to efficiently solve such tasks.

Clinicopathological Phenotype and Genetics of X-Linked Dystonia-Parkinsonism (XDP; DYT3; Lubag)

X-linked dystonia–parkinsonism (XDP; OMIM314250), also referred to as DYT3 dystonia or “Lubag” disease, was first described as an endemic disease in the Philippine island of Panay. XDP is an adult-onset movement disorder characterized by progressive and severe dystonia followed by overt parkinsonism in the later years of life. Among the primary monogenic dystonias, XDP has been identified as a transcriptional dysregulation syndrome with impaired expression of the TAF1 (TATA box-binding protein associated factor 1) gene, which is a critical component of the cellular transcription machinery. The major neuropathology of XDP is progressive neuronal loss in the neostriatum (i.e., the caudate nucleus and putamen). XDP may be used as a human disease model to elucidate the pathomechanisms by which striatal neurodegeneration leads to dystonia symptoms. In this article, we introduce recent advances in the understanding of the interplay between pathophysiology and genetics in XDP.

Striatal Vulnerability in Huntington's Disease: Neuroprotection Versus Neurotoxicity

Huntington’s disease (HD) is an autosomal dominant neurodegenerative disease caused by the expansion of a CAG trinucleotide repeat encoding an abnormally long polyglutamine tract (PolyQ) in the huntingtin (Htt) protein. In HD, striking neuropathological changes occur in the striatum, including loss of medium spiny neurons and parvalbumin-expressing interneurons accompanied by neurodegeneration of the striosome and matrix compartments, leading to progressive impairment of reasoning, walking and speaking abilities. The precise cause of striatal pathology in HD is still unknown; however, accumulating clinical and experimental evidence suggests multiple plausible pathophysiological mechanisms underlying striatal neurodegeneration in HD. Here, we review and discuss the characteristic neurodegenerative patterns observed in the striatum of HD patients and consider the role of various huntingtin-related and striatum-enriched proteins in neurotoxicity and neuroprotection.

The Striosome and Matrix Compartments of the Striatum: A Path through the Labyrinth from Neurochemistry toward Function

The striatum is a heterogeneous structure with a diverse range of neuron types and neuromodulators. Three decades of anatomical and biochemical studies have established that the neurochemical organization of striatum is not uniformly heterogeneous, but rather, can be differentiated into neurochemically discrete compartments known as striosomes (also known as patches) and matrix. These compartments are well understood to differ in their expression of neurochemical markers, with some differences in afferent and efferent connectivity and have also been suggested to have different involvement in a range of neurological diseases. However, the functional outcomes of striosome-matrix organization are poorly understood. Now, recent findings and new experimental tools are beginning to reveal that the distinctions between striosomes and matrix have distinct consequences for striatal synapse function. Here, we review recent findings that suggest there can be distinct regulation of neural function in striosome versus matrix compartments, particularly compartment-specific neurochemical interactions. We highlight that new transgenic and viral tools are becoming available that should now accelerate the pace of advances in understanding of these long-mysterious striatal compartments.

A basal ganglia circuit for evaluating action outcomes

The basal ganglia, a group of subcortical nuclei, play a crucial role in decision making by selecting actions and evaluating their outcomes^1,2. While much is known about the function of the basal ganglia circuitry in selection^1,3,4, how these nuclei contribute to outcome evaluation is less clear. Here we show that neurons in the habenula-projecting globus pallidus (GPh) are essential for evaluating action outcomes and are regulated by a specific set of inputs from the basal ganglia. We found in a classical conditioning task that individual mouse GPh neurons bidirectionally encode whether an outcome is better or worse than expected. Mimicking these evaluation signals with optogenetic inhibition or excitation is sufficient to reinforce or discourage actions in a decision making task. Moreover, cell-type-specific synaptic manipulations revealed that the inhibitory and excitatory inputs to the GPh are necessary for mice to appropriately evaluate positive and negative feedback, respectively. Finally, using rabies virus-assisted monosynaptic tracing⁵, we discovered that the GPh is embedded in a basal ganglia circuit wherein it receives inhibitory input from both striosomal and matrix compartments of the striatum, and excitatory input from the “limbic” regions of the subthalamic nucleus (STN). Our results provide the first direct evidence that information about the selection and evaluation of actions is channelled through distinct sets of basal ganglia circuits, with the GPh representing a key locus where information of opposing valence is integrated to determine whether action outcomes are better or worse than expected.

Striosome-dendron bouquets highlight a unique striatonigral circuit targeting dopamine-containing neurons

The dopamine systems of the brain powerfully influence movement and motivation. We demonstrate that striatonigral fibers originating in striosomes form highly unusual bouquet-like arborizations that target bundles of ventrally extending dopamine-containing dendrites and clusters of their parent nigral cell bodies. Retrograde tracing showed that these clustered cell bodies in turn project to the striatum as part of the classic nigrostriatal pathway. Thus, these striosome-dendron formations, here termed "striosome-dendron bouquets," likely represent subsystems with the nigro-striato-nigral loop that are affected in human disorders including Parkinson's disease. Within the bouquets, expansion microscopy resolved many individual striosomal fibers tightly intertwined with the dopamine-containing dendrites and also with afferents labeled by glutamatergic, GABAergic, and cholinergic markers and markers for astrocytic cells and fibers and connexin 43 puncta. We suggest that the striosome-dendron bouquets form specialized integrative units within the dopamine-containing nigral system. Given evidence that striosomes receive input from cortical regions related to the control of mood and motivation and that they link functionally to reinforcement and decision-making, the striosome-dendron bouquets could be critical to dopamine-related function in health and disease.

Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity

The brain enables animals to behaviorally adapt in order to survive in a complex and dynamic environment, but how reward-oriented behaviors are achieved and computed by its underlying neural circuitry is an open question. To address this concern, we have developed a spiking model of the basal ganglia (BG) that learns to dis-inhibit the action leading to a reward despite ongoing changes in the reward schedule. The architecture of the network features the two pathways commonly described in BG, the direct (denoted D1) and the indirect (denoted D2) pathway, as well as a loop involving striatum and the dopaminergic system. The activity of these dopaminergic neurons conveys the reward prediction error (RPE), which determines the magnitude of synaptic plasticity within the different pathways. All plastic connections implement a versatile four-factor learning rule derived from Bayesian inference that depends upon pre- and post-synaptic activity, receptor type, and dopamine level. Synaptic weight updates occur in the D1 or D2 pathways depending on the sign of the RPE, and an efference copy informs upstream nuclei about the action selected. We demonstrate successful performance of the system in a multiple-choice learning task with a transiently changing reward schedule. We simulate lesioning of the various pathways and show that a condition without the D2 pathway fares worse than one without D1. Additionally, we simulate the degeneration observed in Parkinson's disease (PD) by decreasing the number of dopaminergic neurons during learning. The results suggest that the D1 pathway impairment in PD might have been overlooked. Furthermore, an analysis of the alterations in the synaptic weights shows that using the absolute reward value instead of the RPE leads to a larger change in D1.

Putaminal Mosaic Visualized by Tyrosine Hydroxylase Immunohistochemistry in the Human Neostriatum

Among the basal ganglia-thalamocortical circuits, the putamen plays a critical role in the “motor” circuits that control voluntary movements and motor learning. The human neostriatum comprises two functional subdivisions known as the striosome (patch) and matrix compartments. Accumulating evidence suggests that compartment-specific dysregulations of dopamine activity might be involved in the disease-specific pathology and symptoms of human striatal diseases including movement disorders. This study was undertaken to examine whether or how striatal dopaminergic innervations are organized into the compartmentalized architecture found in the putamen of adult human brains. For this purpose, we used a highly sensitive immunohistochemistry (IHC) technique to identify tyrosine hydroxylase (TH; EC 1.14.16.2), a marker for striatal dopaminergic axons and terminals, in formalin-fixed paraffin-embedded (FFPE) tissues obtained from autopsied human brains. Herein, we report that discrete compartmentalization of TH-labeled innervations occurs in the putamen, as in the caudate nucleus (CN), with a higher density of TH labeling in the matrix compared to the striosomes. Our results provide anatomical evidence to support the hypothesis that compartment-specific dysfunction of the striosome-matrix dopaminergic systems might contribute to the genesis of movement disorders.

Selective Increase of Auditory Cortico-Striatal Coherence during Auditory-Cued Go/NoGo Discrimination Learning

Goal directed behavior and associated learning processes are tightly linked to neuronal activity in the ventral striatum. Mechanisms that integrate task relevant sensory information into striatal processing during decision making and learning are implicitly assumed in current reinforcement models, yet they are still weakly understood. To identify the functional activation of cortico-striatal subpopulations of connections during auditory discrimination learning, we trained Mongolian gerbils in a two-way active avoidance task in a shuttlebox to discriminate between falling and rising frequency modulated tones with identical spectral properties. We assessed functional coupling by analyzing the field-field coherence between the auditory cortex and the ventral striatum of animals performing the task. During the course of training, we observed a selective increase of functional coupling during Go-stimulus presentations. These results suggest that the auditory cortex functionally interacts with the ventral striatum during auditory learning and that the strengthening of these functional connections is selectively goal-directed.

Postsynaptic Density Protein 95 in the Striosome and Matrix Compartments of the Human Neostriatum

The human neostriatum consists of two functional subdivisions referred to as the striosome (patch) and matrix compartments. The striosome-matrix dopamine systems play a central role in cortico-thalamo-basal ganglia circuits, and their involvement is thought to underlie the genesis of multiple movement and behavioral disorders, and of drug addiction. Human neuropathology also has shown that striosomes and matrix have differential vulnerability patterns in several striatal neurodegenerative diseases. Postsynaptic density protein 95 (PSD-95), also known as disks large homolog 4, is a major scaffolding protein in the postsynaptic densities of dendritic spines. PSD-95 is now known to negatively regulate not only N-methyl-D-aspartate glutamate signaling, but also dopamine D1 signals at sites of postsynaptic transmission. Accordingly, a neuroprotective role for PSD-95 against dopamine D1 receptor (D1R)-mediated neurotoxicity in striatal neurodegeneration also has been suggested. Here, we used a highly sensitive immunohistochemistry technique to show that in the human neostriatum, PSD-95 is differentially concentrated in the striosome and matrix compartments, with a higher density of PSD-95 labeling in the matrix compartment than in the striosomes. This compartment-specific distribution of PSD-95 was strikingly complementary to that of D1R. In addition to the possible involvement of PSD-95-mediated synaptic function in compartment-specific dopamine signals, we suggest that the striosomes might be more susceptible to D1R-mediated neurotoxicity than the matrix compartment. This notion may provide new insight into the compartment-specific vulnerability of MSNs in striatal neurodegeneration.

Habit Learning by Naive Macaques Is Marked by Response Sharpening of Striatal Neurons Representing the Cost and Outcome of Acquired Action Sequences

Over a century of scientific work has focused on defining the factors motivating behavioral learning. Observations in animals and humans trained on a wide range of tasks support reinforcement learning (RL) algorithms as accounting for the learning. Still unknown, however, are the signals that drive learning in naive, untrained subjects. Here, we capitalized on a sequential saccade task in which macaque monkeys acquired repetitive scanning sequences without instruction. We found that spike activity in the caudate nucleus after each trial corresponded to an integrated cost-benefit signal that was highly correlated with the degree of naturalistic untutored learning by the monkeys. Across learning, neurons encoding both cost and outcome gradually acquired increasingly sharp phasic trial-end responses that paralleled the development of the habit-like, repetitive saccade sequences. Our findings demonstrate an integrated cost-benefit signal by which RL and its neural correlates could drive naturalistic behaviors in freely behaving primates.

Substance P Weights Striatal Dopamine Transmission Differently within the Striosome-Matrix Axis

The mammalian striatum has a topographical organization of input-output connectivity, but a complex internal, nonlaminar neuronal architecture comprising projection neurons of two types interspersed among multiple interneuron types and potential local neuromodulators. From this cellular melange arises a biochemical compartmentalization of areas termed striosomes and extrastriosomal matrix. The functions of these compartments are poorly understood but might confer distinct features to striatal signal processing and be discretely governed. Dopamine transmission occurs throughout striosomes and matrix, and is reported to be modulated by the striosomally enriched neuromodulator substance P. However, reported effects are conflicting, ranging from facilitation to inhibition. We addressed whether dopamine transmission is modulated differently in striosome-matrix compartments by substance P.We paired detection of evoked dopamine release at carbon-fiber microelectrodes in mouse striatal slices with subsequent identification of the location of recording sites with respect to μ-opioid receptor-rich striosomes. Substance P had bidirectional effects on dopamine release that varied between recording sites and were prevented by inhibition of neurokinin-1 receptors. The direction of modulation was determined by location within the striosomal-matrix axis: dopamine release was boosted in striosome centers, diminished in striosomal-matrix border regions, and unaffected in the matrix. In turn, this different weighting of dopamine transmission by substance P modified the apparent center-surround contrast of striosomal dopamine signals. These data reveal that dopamine transmission can be differentially modulated within the striosomal-matrix axis, and furthermore, indicate a functionally distinct zone at the striosome-matrix interface, which may have key impacts on striatal integration.

Striatal cholinergic interneurons and cortico-striatal synaptic plasticity in health and disease

Basal ganglia disorders such as Parkinson's disease, dystonia, and Huntington's disease are characterized by a dysregulation of the basal ganglia neuromodulators (dopamine, acetylcholine, and others), which impacts cortico-striatal transmission. Basal ganglia disorders are often associated with an imbalance between the midbrain dopaminergic and striatal cholinergic systems. In contrast to the extensive research and literature on the consequences of a malfunction of midbrain dopaminergic signaling on the plasticity of the cortico-striatal synapse, very little is known about the role of striatal cholinergic interneurons in normal and pathological control of cortico-striatal transmission. In this review, we address the functional role of striatal cholinergic interneurons, also known as tonically active neurons and attempt to understand how the alteration of their functional properties in basal ganglia disorders leads to abnormal cortico-striatal synaptic plasticity. Specifically, we suggest that striatal cholinergic interneurons provide a permissive signal, which enables long-term changes in the efficacy of the cortico-striatal synapse. We further discuss how modifications in the striatal cholinergic activity pattern alter or prohibit plastic changes of the cortico-striatal synapse. Long-term cortico-striatal synaptic plasticity is the cellular substrate of procedural learning and adaptive control behavior. Hence, abnormal changes in this plasticity may underlie the inability of patients with basal ganglia disorders to adjust their behavior to situational demands. Normalization of the cholinergic modulation of cortico-striatal synaptic plasticity may be considered as a critical feature in future treatments of basal ganglia disorders.

A network model of basal ganglia for understanding the roles of dopamine and serotonin in reward-punishment-risk based decision making

There is significant evidence that in addition to reward-punishment based decision making, the Basal Ganglia (BG) contributes to risk-based decision making (Balasubramani et al., 2014). Despite this evidence, little is known about the computational principles and neural correlates of risk computation in this subcortical system. We have previously proposed a reinforcement learning (RL)-based model of the BG that simulates the interactions between dopamine (DA) and serotonin (5HT) in a diverse set of experimental studies including reward, punishment and risk based decision making (Balasubramani et al., 2014). Starting with the classical idea that the activity of mesencephalic DA represents reward prediction error, the model posits that serotoninergic activity in the striatum controls risk-prediction error. Our prior model of the BG was an abstract model that did not incorporate anatomical and cellular-level data. In this work, we expand the earlier model into a detailed network model of the BG and demonstrate the joint contributions of DA-5HT in risk and reward-punishment sensitivity. At the core of the proposed network model is the following insight regarding cellular correlates of value and risk computation. Just as DA D1 receptor (D1R) expressing medium spiny neurons (MSNs) of the striatum were thought to be the neural substrates for value computation, we propose that DA D1R and D2R co-expressing MSNs are capable of computing risk. Though the existence of MSNs that co-express D1R and D2R are reported by various experimental studies, prior existing computational models did not include them. Ours is the first model that accounts for the computational possibilities of these co-expressing D1R-D2R MSNs, and describes how DA and 5HT mediate activity in these classes of neurons (D1R-, D2R-, D1R-D2R- MSNs). Starting from the assumption that 5HT modulates all MSNs, our study predicts significant modulatory effects of 5HT on D2R and co-expressing D1R-D2R MSNs which in turn explains the multifarious functions of 5HT in the BG. The experiments simulated in the present study relates 5HT to risk sensitivity and reward-punishment learning. Furthermore, our model is shown to capture reward-punishment and risk based decision making impairment in Parkinson's Disease (PD). The model predicts that optimizing 5HT levels along with DA medications might be essential for improving the patients' reward-punishment learning deficits.

Identifying the Basal Ganglia network model markers for medication-induced impulsivity in Parkinson's disease patients

Impulsivity, i.e. irresistibility in the execution of actions, may be prominent in Parkinson's disease (PD) patients who are treated with dopamine precursors or dopamine receptor agonists. In this study, we combine clinical investigations with computational modeling to explore whether impulsivity in PD patients on medication may arise as a result of abnormalities in risk, reward and punishment learning. In order to empirically assess learning outcomes involving risk, reward and punishment, four subject groups were examined: healthy controls, ON medication PD patients with impulse control disorder (PD-ON ICD) or without ICD (PD-ON non-ICD), and OFF medication PD patients (PD-OFF). A neural network model of the Basal Ganglia (BG) that has the capacity to predict the dysfunction of both the dopaminergic (DA) and the serotonergic (5HT) neuromodulator systems was developed and used to facilitate the interpretation of experimental results. In the model, the BG action selection dynamics were mimicked using a utility function based decision making framework, with DA controlling reward prediction and 5HT controlling punishment and risk predictions. The striatal model included three pools of Medium Spiny Neurons (MSNs), with D1 receptor (R) alone, D2R alone and co-expressing D1R-D2R. Empirical studies showed that reward optimality was increased in PD-ON ICD patients while punishment optimality was increased in PD-OFF patients. Empirical studies also revealed that PD-ON ICD subjects had lower reaction times (RT) compared to that of the PD-ON non-ICD patients. Computational modeling suggested that PD-OFF patients have higher punishment sensitivity, while healthy controls showed comparatively higher risk sensitivity. A significant decrease in sensitivity to punishment and risk was crucial for explaining behavioral changes observed in PD-ON ICD patients. Our results highlight the power of computational modelling for identifying neuronal circuitry implicated in learning, and its impairment in PD. The results presented here not only show that computational modelling can be used as a valuable tool for understanding and interpreting clinical data, but they also show that computational modeling has the potential to become an invaluable tool to predict the onset of behavioral changes during disease progression.

Targeted expression of μ-opioid receptors in a subset of striatal direct-pathway neurons restores opiate reward

Mu-Opioid Receptors (MOR) are necessary for the analgesic and addictive effects of opioids such as morphine, but the MOR-expressing neuronal populations that mediate the distinct opiate effects remain elusive. Here we devised a novel conditional BAC rescue strategy to show that mice with targeted MOR expression in a subpopulation of striatal direct-pathway neurons enriched in the striosome and nucleus accumbens, in an otherwise MOR-null background, restore opiate reward, opiate-induced striatal dopamine release, and partially restore motivation to self-administer opiates. However, they lack opiate analgesia or withdrawal. Importantly, we used Cre-mediated deletion of the rescued MOR transgene to establish that striatal, rather than a few extrastriatal sites of MOR transgene expression, is needed for the restoration of opiate reward. Together, our study demonstrates that a subpopulation of striatal direct-pathway neurons is sufficient to support opiate reward-driven behaviors and provides a novel intersectional genetic approach to dissect neurocircuit-specific gene function in vivo.

A computational model of altered gait patterns in parkinson's disease patients negotiating narrow doorways

We present a computational model of altered gait velocity patterns in Parkinson's Disease (PD) patients. PD gait is characterized by short shuffling steps, reduced walking speed, increased double support time and sometimes increased cadence. The most debilitating symptom of PD gait is the context dependent cessation in gait known as freezing of gait (FOG). Cowie et al. (2010) and Almeida and Lebold (2010) investigated FOG as the changes in velocity profiles of PD gait, as patients walked through a doorway with variable width. The former reported a sharp dip in velocity, a short distance from the doorway that was greater for narrower doorways. They compared the gait performance in PD freezers at ON and OFF dopaminergic medication. In keeping with this finding, the latter also reported the same for ON medicated PD freezers and non-freezers. In the current study, we sought to simulate these gait changes using a computational model of Basal Ganglia based on Reinforcement Learning, coupled with a spinal rhythm mimicking central pattern generator (CPG) model. In the model, a simulated agent was trained to learn a value profile over a corridor leading to the doorway by repeatedly attempting to pass through the doorway. Temporal difference error in value, associated with dopamine signal, was appropriately constrained in order to reflect the dopamine-deficient conditions of PD. Simulated gait under PD conditions exhibited a sharp dip in velocity close to the doorway, with PD OFF freezers showing the largest decrease in velocity compared to PD ON freezers and controls. PD ON and PD OFF freezers both showed sensitivity to the doorway width, with narrow door producing the least velocity/ stride length. Step length variations were also captured with PD freezers producing smaller steps and larger step-variability than PD non-freezers and controls. In addition this model is the first to explain the non-dopamine dependence for FOG giving rise to several other possibilities for its etiology.

Medial-lateral organization of the orbitofrontal cortex

Emerging evidence suggests that specific cognitive functions localize to different subregions of OFC, but the nature of these functional distinctions remains unclear. One prominent theory, derived from human neuroimaging, proposes that different stimulus valences are processed in separate orbital regions, with medial and lateral OFC processing positive and negative stimuli, respectively. Thus far, neurophysiology data have not supported this theory. We attempted to reconcile these accounts by recording neural activity from the full medial-lateral extent of the orbital surface in monkeys receiving rewards and punishments via gain or loss of secondary reinforcement. We found no convincing evidence for valence selectivity in any orbital region. Instead, we report differences between neurons in central OFC and those on the inferior-lateral orbital convexity, in that they encoded different sources of value information provided by the behavioral task. Neurons in inferior convexity encoded the value of external stimuli, whereas those in OFC encoded value information derived from the structure of the behavioral task. We interpret these results in light of recent theories of OFC function and propose that these distinctions, not valence selectivity, may shed light on a fundamental organizing principle for value processing in orbital cortex.

Independent circuits in the basal ganglia for the evaluation and selection of actions

The basal ganglia are critical for selecting actions and evaluating their outcome. Although the circuitry for selection is well understood, how these nuclei evaluate the outcome of actions is unknown. Here, we show in lamprey that a separate evaluation circuit, which regulates the habenula-projecting globus pallidus (GPh) neurons, exists within the basal ganglia. The GPh neurons are glutamatergic and can drive the activity of the lateral habenula, which, in turn, provides an indirect inhibitory influence on midbrain dopamine neurons. We show that GPh neurons receive inhibitory input from the striosomal compartment of the striatum. The striosomal input can reduce the excitatory drive to the lateral habenula and, consequently, decrease the inhibition onto the dopaminergic system. Dopaminergic neurons, in turn, provide feedback that inhibits the GPh. In addition, GPh neurons receive direct projections from the pallium (cortex in mammals), which can increase the GPh activity to drive the lateral habenula to increase the inhibition of the neuromodulatory systems. This circuitry, thus, differs markedly from the "direct" and "indirect" pathways that regulate the pallidal (e.g., globus pallidus) output nuclei involved in the control of motion. Our results show that a distinct reward-evaluation circuit exists within the basal ganglia, in parallel to the direct and indirect pathways, which select actions. Our results suggest that these circuits are part of the fundamental blueprint that all vertebrates use to select actions and evaluate their outcome.

Deep homology of arthropod central complex and vertebrate basal ganglia

The arthropod central complex and vertebrate basal ganglia derive from embryonic basal forebrain lineages that are specified by an evolutionarily conserved genetic program leading to interconnected neuropils and nuclei that populate the midline of the forebrain-midbrain boundary region. In the substructures of both the central complex and basal ganglia, network connectivity and neuronal activity mediate control mechanisms in which inhibitory (GABAergic) and modulatory (dopaminergic) circuits facilitate the regulation and release of adaptive behaviors. Both basal ganglia and central complex dysfunction result in behavioral defects including motor abnormalities, impaired memory formation, attention deficits, affective disorders, and sleep disturbances. The observed multitude of similarities suggests deep homology of arthropod central complex and vertebrate basal ganglia circuitries underlying the selection and maintenance of behavioral actions.