1
|
Handa T, Fukai T, Kurikawa T. Single-Trial Representations of Decision-Related Variables by Decomposed Frontal Corticostriatal Ensemble Activity. eNeuro 2024; 11:ENEURO.0172-24.2024. [PMID: 39054055 DOI: 10.1523/eneuro.0172-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2024] [Revised: 06/06/2024] [Accepted: 07/02/2024] [Indexed: 07/27/2024] Open
Abstract
The frontal cortex-striatum circuit plays a pivotal role in adaptive goal-directed behaviors. However, it remains unclear how decision-related signals are mediated through cross-regional transmission between the medial frontal cortex and the striatum by neuronal ensembles in making decision based on outcomes of past action. Here, we analyzed neuronal ensemble activity obtained through simultaneous multiunit recordings in the secondary motor cortex (M2) and dorsal striatum (DS) in rats performing an outcome-based left-or-right choice task. By adopting tensor component analysis (TCA), a single-trial-based unsupervised dimensionality reduction approach, for concatenated ensembles of M2 and DS neurons, we identified distinct three spatiotemporal neural dynamics (TCA components) at the single-trial level specific to task-relevant variables. Choice-position-selective neural dynamics reflected the positions chosen and was correlated with the trial-to-trial fluctuation of behavioral variables. Intriguingly, choice-pattern-selective neural dynamics distinguished whether the incoming choice was a repetition or a switch from the previous choice before a response choice. Other neural dynamics was selective to outcome and increased within-trial activity following response. Our results demonstrate how the concatenated ensembles of M2 and DS process distinct features of decision-related signals at various points in time. Thereby, the M2 and DS collaboratively monitor action outcomes and determine the subsequent choice, whether to repeat or switch, for action selection.
Collapse
Affiliation(s)
- Takashi Handa
- Department of Neurobiology, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima 734-8553, Japan
- Laboratory for Neural Coding and Brain Computing, RIKEN Center for Brain Science, Saitama 351-0198, Japan
| | - Tomoki Fukai
- Laboratory for Neural Coding and Brain Computing, RIKEN Center for Brain Science, Saitama 351-0198, Japan
- Neural Coding and Brain Computing Unit, Okinawa Institute of Science and Technology, Okinawa 904-0495, Japan
| | - Tomoki Kurikawa
- Laboratory for Neural Coding and Brain Computing, RIKEN Center for Brain Science, Saitama 351-0198, Japan
- Department of Complex and Intelligent Systems, Future University of Hakodate, Hokkaido 041-8655, Japan
| |
Collapse
|
2
|
Iguchi Y, Fukabori R, Kato S, Takahashi K, Eifuku S, Maejima Y, Shimomura K, Mizuma H, Mawatari A, Doi H, Cui Y, Onoe H, Hikishima K, Osanai M, Nishijo T, Momiyama T, Benton R, Kobayashi K. Chemogenetic activation of mammalian brain neurons expressing insect Ionotropic Receptors by systemic ligand precursor administration. Commun Biol 2024; 7:547. [PMID: 38714803 PMCID: PMC11076466 DOI: 10.1038/s42003-024-06223-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Accepted: 04/22/2024] [Indexed: 05/10/2024] Open
Abstract
Chemogenetic approaches employing ligand-gated ion channels are advantageous regarding manipulation of target neuronal population functions independently of endogenous second messenger pathways. Among them, Ionotropic Receptor (IR)-mediated neuronal activation (IRNA) allows stimulation of mammalian neurons that heterologously express members of the insect chemosensory IR repertoire in response to their cognate ligands. In the original protocol, phenylacetic acid, a ligand of the IR84a/IR8a complex, was locally injected into a brain region due to its low permeability of the blood-brain barrier. To circumvent this invasive injection, we sought to develop a strategy of peripheral administration with a precursor of phenylacetic acid, phenylacetic acid methyl ester, which is efficiently transferred into the brain and converted to the mature ligand by endogenous esterase activities. This strategy was validated by electrophysiological, biochemical, brain-imaging, and behavioral analyses, demonstrating high utility of systemic IRNA technology in the remote activation of target neurons in the brain.
Collapse
Affiliation(s)
- Yoshio Iguchi
- Department of Molecular Genetics, Institute of Biomedical Sciences, Fukushima Medical University School of Medicine, 1 Hikarigaoka, Fukushima, 960-1295, Japan
| | - Ryoji Fukabori
- Department of Molecular Genetics, Institute of Biomedical Sciences, Fukushima Medical University School of Medicine, 1 Hikarigaoka, Fukushima, 960-1295, Japan
| | - Shigeki Kato
- Department of Molecular Genetics, Institute of Biomedical Sciences, Fukushima Medical University School of Medicine, 1 Hikarigaoka, Fukushima, 960-1295, Japan
| | - Kazumi Takahashi
- Department of Systems Neuroscience, Fukushima Medical University School of Medicine, 1 Hikarigaoka, Fukushima, 960-1295, Japan
| | - Satoshi Eifuku
- Department of Systems Neuroscience, Fukushima Medical University School of Medicine, 1 Hikarigaoka, Fukushima, 960-1295, Japan
| | - Yuko Maejima
- Department of Bioregulation and Pharmacological Medicine, Fukushima Medical University School of Medicine, 1 Hikarigaoka, Fukushima, 960-1295, Japan
| | - Kenju Shimomura
- Department of Bioregulation and Pharmacological Medicine, Fukushima Medical University School of Medicine, 1 Hikarigaoka, Fukushima, 960-1295, Japan
| | - Hiroshi Mizuma
- Laboratory for Pathophysiological and Health Science, RIKEN Center for Biosystems Dynamics Research, 6-7-3 Minatojima-minamimachi, Chuo-ku, Kobe, 650-0047, Japan
- Department of Functional Brain Imaging, Institute for Quantum Medical Science, National Institutes for Quantum Science and Technology, 4-9-1 Anagawa, Inage-ku, Chiba, 263-8555, Japan
| | - Aya Mawatari
- Laboratory for Labeling Chemistry, RIKEN Center for Biosystems Dynamics Research, 6-7-3 Minatojima-minamimachi, Chuo-ku, Kobe, 650-0047, Japan
| | - Hisashi Doi
- Laboratory for Labeling Chemistry, RIKEN Center for Biosystems Dynamics Research, 6-7-3 Minatojima-minamimachi, Chuo-ku, Kobe, 650-0047, Japan
- Research, Institute for Drug Discovery Science, Collaborative Creation Research Center, Organization for Research Promotion, Osaka Metropolitan University, 1-1 Gakuen-cho, Naka-ku, Sakai, 599-8531, Japan
| | - Yilong Cui
- Laboratory for Biofunction Dynamics Imaging, RIKEN Center for Biosystems Dynamics Research, 6-7-3 Minatojima-minamimachi, Chuo-ku, Kobe, 650-0047, Japan
| | - Hirotaka Onoe
- Human Brain Research Center, Kyoto University Graduate School of Medicine, 54 Shogoin-Kawahara-Cho, Sakyo-ku, Kyoto, 606-8507, Japan
| | - Keigo Hikishima
- Medical Devices Research Group, Health and Medical Research Institute, National Institute of Advanced Industrial Science and Technology (AIST), 1-2-1 Namiki, Tsukuba, 305-8564, Japan
| | - Makoto Osanai
- Department of Medical Physics and Engineering, Division of Health Sciences, Osaka University Graduate School of Medicine, 1-7 Yamadaoka, Suita, 565-0871, Japan
| | - Takuma Nishijo
- Department of Pharmacology, Jikei University School of Medicine, 3-25-8 Nishi-shinbashi, Tokyo, 105-8461, Japan
- Department of Molecular Neurobiology, Institute for Developmental Research, Aichi Developmental Disability Center, 713-8 Kamiya-cho, Kasugai, 480-0392, Japan
| | - Toshihiko Momiyama
- Department of Pharmacology, Jikei University School of Medicine, 3-25-8 Nishi-shinbashi, Tokyo, 105-8461, Japan
| | - Richard Benton
- Center for Integrative Genomics, Faculty of Biology and Medicine, University of Lausanne, CH-1015, Lausanne, Switzerland
| | - Kazuto Kobayashi
- Department of Molecular Genetics, Institute of Biomedical Sciences, Fukushima Medical University School of Medicine, 1 Hikarigaoka, Fukushima, 960-1295, Japan.
| |
Collapse
|
3
|
Gmaz JM, Keller JA, Dudman JT, Gallego JA. Integrating across behaviors and timescales to understand the neural control of movement. Curr Opin Neurobiol 2024; 85:102843. [PMID: 38354477 DOI: 10.1016/j.conb.2024.102843] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 12/03/2023] [Accepted: 01/13/2024] [Indexed: 02/16/2024]
Abstract
The nervous system evolved to enable navigation throughout the environment in the pursuit of resources. Evolutionarily newer structures allowed increasingly complex adaptations but necessarily added redundancy. A dominant view of movement neuroscientists is that there is a one-to-one mapping between brain region and function. However, recent experimental data is hard to reconcile with the most conservative interpretation of this framework, suggesting a degree of functional redundancy during the performance of well-learned, constrained behaviors. This apparent redundancy likely stems from the bidirectional interactions between the various cortical and subcortical structures involved in motor control. We posit that these bidirectional connections enable flexible interactions across structures that change depending upon behavioral demands, such as during acquisition, execution or adaptation of a skill. Observing the system across both multiple actions and behavioral timescales can help isolate the functional contributions of individual structures, leading to an integrated understanding of the neural control of movement.
Collapse
Affiliation(s)
- Jimmie M Gmaz
- Department of Bioengineering, Imperial College London, London, UK. https://twitter.com/j_gmaz
| | - Jason A Keller
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn VA, USA. https://twitter.com/jakNeurd
| | - Joshua T Dudman
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn VA, USA.
| | - Juan A Gallego
- Department of Bioengineering, Imperial College London, London, UK.
| |
Collapse
|
4
|
Hart G, Burton TJ, Nolan CR, Balleine BW. Striatal dopamine release tracks the relationship between actions and their consequences. Cell Rep 2024; 43:113828. [PMID: 38386550 DOI: 10.1016/j.celrep.2024.113828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 12/05/2023] [Accepted: 02/03/2024] [Indexed: 02/24/2024] Open
Abstract
The acquisition and performance of goal-directed actions has long been argued to depend on the integration of glutamatergic inputs to the posterior dorsomedial striatum (pDMS) under the modulatory influence of dopamine. Nevertheless, relatively little is known about the dynamics of striatal dopamine during goal-directed actions. To investigate this, we chronically recorded dopamine release in the pDMS as rats acquired two actions for distinct outcomes as these action-outcome associations were incremented and then subsequently degraded or reversed. We found that bilateral dopamine release scaled with action value, whereas the lateralized dopamine signal, i.e., the difference in dopamine release ipsilaterally and contralaterally to the direction of the goal-directed action, reflected the strength of the action-outcome association independently of changes in movement. Our results establish, therefore, that striatal dopamine activity during goal-directed action reflects both bilateral moment-to-moment changes in action value and the long-term action-outcome association.
Collapse
Affiliation(s)
- G Hart
- Decision Neuroscience Laboratory, School of Psychology, UNSW Sydney, Sydney, NSW, Australia
| | - T J Burton
- Decision Neuroscience Laboratory, School of Psychology, UNSW Sydney, Sydney, NSW, Australia
| | - C R Nolan
- Decision Neuroscience Laboratory, School of Psychology, UNSW Sydney, Sydney, NSW, Australia
| | - B W Balleine
- Decision Neuroscience Laboratory, School of Psychology, UNSW Sydney, Sydney, NSW, Australia.
| |
Collapse
|
5
|
Lowet AS, Zheng Q, Meng M, Matias S, Drugowitsch J, Uchida N. An opponent striatal circuit for distributional reinforcement learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.02.573966. [PMID: 38260354 PMCID: PMC10802299 DOI: 10.1101/2024.01.02.573966] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/24/2024]
Abstract
Machine learning research has achieved large performance gains on a wide range of tasks by expanding the learning target from mean rewards to entire probability distributions of rewards - an approach known as distributional reinforcement learning (RL)1. The mesolimbic dopamine system is thought to underlie RL in the mammalian brain by updating a representation of mean value in the striatum2,3, but little is known about whether, where, and how neurons in this circuit encode information about higher-order moments of reward distributions4. To fill this gap, we used high-density probes (Neuropixels) to acutely record striatal activity from well-trained, water-restricted mice performing a classical conditioning task in which reward mean, reward variance, and stimulus identity were independently manipulated. In contrast to traditional RL accounts, we found robust evidence for abstract encoding of variance in the striatum. Remarkably, chronic ablation of dopamine inputs disorganized these distributional representations in the striatum without interfering with mean value coding. Two-photon calcium imaging and optogenetics revealed that the two major classes of striatal medium spiny neurons - D1 and D2 MSNs - contributed to this code by preferentially encoding the right and left tails of the reward distribution, respectively. We synthesize these findings into a new model of the striatum and mesolimbic dopamine that harnesses the opponency between D1 and D2 MSNs5-15 to reap the computational benefits of distributional RL.
Collapse
Affiliation(s)
- Adam S Lowet
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
- Program in Neuroscience, Harvard University, Boston, MA, USA
| | - Qiao Zheng
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Melissa Meng
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Sara Matias
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Jan Drugowitsch
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Naoshige Uchida
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| |
Collapse
|
6
|
Frost-Nylén J, Thompson WS, Robertson B, Grillner S. The Basal Ganglia Downstream Control of Action - An Evolutionarily Conserved Strategy. Curr Neuropharmacol 2024; 22:1419-1430. [PMID: 37563813 PMCID: PMC11097981 DOI: 10.2174/1570159x21666230810141746] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 01/30/2023] [Accepted: 02/05/2023] [Indexed: 08/12/2023] Open
Abstract
The motor areas of the cortex and the basal ganglia both contribute to determining which motor actions will be recruited at any moment in time, and their functions are intertwined. Here, we review the basal ganglia mechanisms underlying the selection of behavior of the downstream control of motor centers in the midbrain and brainstem and show that the basic organization of the forebrain motor system is evolutionarily conserved throughout vertebrate phylogeny. The output level of the basal ganglia (e.g. substantia nigra pars reticulata) has GABAergic neurons that are spontaneously active at rest and inhibit a number of specific motor centers, each of which can be relieved from inhibition if the inhibitory output neurons themselves become inhibited. The motor areas of the cortex act partially via the dorsolateral striatum (putamen), which has specific modules for the forelimb, hindlimb, trunk, etc. Each module operates in turn through the two types of striatal projection neurons that control the output modules of the basal ganglia and thereby the downstream motor centers. The mechanisms for lateral inhibition in the striatum are reviewed as well as other striatal mechanisms contributing to action selection. The motor cortex also exerts a direct excitatory action on specific motor centers. An overview is given of the basal ganglia control exerted on the different midbrain/brainstem motor centers, and the efference copy information fed back via the thalamus to the striatum and cortex, which is of importance for the planning of future movements.
Collapse
Affiliation(s)
| | | | - Brita Robertson
- Department of Neuroscience, Karolinska Institute, Stockholm, Sweden
| | - Sten Grillner
- Department of Neuroscience, Karolinska Institute, Stockholm, Sweden
| |
Collapse
|
7
|
Sato Y, Sakai Y, Hirata S. State-transition-free reinforcement learning in chimpanzees (Pan troglodytes). Learn Behav 2023; 51:413-427. [PMID: 37369920 DOI: 10.3758/s13420-023-00591-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/07/2023] [Indexed: 06/29/2023]
Abstract
The outcome of an action often occurs after a delay. One solution for learning appropriate actions from delayed outcomes is to rely on a chain of state transitions. Another solution, which does not rest on state transitions, is to use an eligibility trace (ET) that directly bridges a current outcome and multiple past actions via transient memories. Previous studies revealed that humans (Homo sapiens) learned appropriate actions in a behavioral task in which solutions based on the ET were effective but transition-based solutions were ineffective. This suggests that ET may be used in human learning systems. However, no studies have examined nonhuman animals with an equivalent behavioral task. We designed a task for nonhuman animals following a previous human study. In each trial, participants chose one of two stimuli that were randomly selected from three stimulus types: a stimulus associated with a food reward delivered immediately, a stimulus associated with a reward delivered after a few trials, and a stimulus associated with no reward. The presented stimuli did not vary according to the participants' choices. To maximize the total reward, participants had to learn the value of the stimulus associated with a delayed reward. Five chimpanzees (Pan troglodytes) performed the task using a touchscreen. Two chimpanzees were able to learn successfully, indicating that learning mechanisms that do not depend on state transitions were involved in the learning processes. The current study extends previous ET research by proposing a behavioral task and providing empirical data from chimpanzees.
Collapse
Grants
- 16H06283 Ministry of Education, Culture, Sports, Science, Japan Society for the Promotion of Science
- 18H05524 Ministry of Education, Culture, Sports, Science, Japan Society for the Promotion of Science
- 19J22889 Ministry of Education, Culture, Sports, Science, Japan Society for the Promotion of Science
- 26245069 Ministry of Education, Culture, Sports, Science, Japan Society for the Promotion of Science
- U04 Program for Leading Graduate Schools
Collapse
Affiliation(s)
- Yutaro Sato
- Wildlife Research Center, Kyoto University, Kyoto, Japan.
- University Administration Office, Headquarters for Management Strategy, Niigata University, Niigata, Japan.
| | - Yutaka Sakai
- Brain Science Institute, Tamagawa University, Tokyo, Japan
| | - Satoshi Hirata
- Wildlife Research Center, Kyoto University, Kyoto, Japan
| |
Collapse
|
8
|
Rusheen AE, Rojas-Cabrera J, Goyal A, Shin H, Yuen J, Jang DP, Bennet KE, Blaha CD, Lee KH, Oh Y. Deep brain stimulation alleviates tics in Tourette syndrome via striatal dopamine transmission. Brain 2023; 146:4174-4190. [PMID: 37141283 PMCID: PMC10545518 DOI: 10.1093/brain/awad142] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2022] [Revised: 03/24/2023] [Accepted: 04/14/2023] [Indexed: 05/05/2023] Open
Abstract
Tourette syndrome is a childhood-onset neuropsychiatric disorder characterized by intrusive motor and vocal tics that can lead to self-injury and deleterious mental health complications. While dysfunction in striatal dopamine neurotransmission has been proposed to underlie tic behaviour, evidence is scarce and inconclusive. Deep brain stimulation (DBS) of the thalamic centromedian parafascicular complex (CMPf), an approved surgical interventive treatment for medical refractory Tourette syndrome, may reduce tics by affecting striatal dopamine release. Here, we use electrophysiology, electrochemistry, optogenetics, pharmacological treatments and behavioural measurements to mechanistically examine how thalamic DBS modulates synaptic and tonic dopamine activity in the dorsomedial striatum. Previous studies demonstrated focal disruption of GABAergic transmission in the dorsolateral striatum of rats led to repetitive motor tics recapitulating the major symptom of Tourette syndrome. We employed this model under light anaesthesia and found CMPf DBS evoked synaptic dopamine release and elevated tonic dopamine levels via striatal cholinergic interneurons while concomitantly reducing motor tic behaviour. The improvement in tic behaviour was found to be mediated by D2 receptor activation as blocking this receptor prevented the therapeutic response. Our results demonstrate that release of striatal dopamine mediates the therapeutic effects of CMPf DBS and points to striatal dopamine dysfunction as a driver for motor tics in the pathoneurophysiology of Tourette syndrome.
Collapse
Affiliation(s)
- Aaron E Rusheen
- Medical Scientist Training Program, Mayo Clinic, Rochester, MN 55902, USA
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN 55902, USA
| | - Juan Rojas-Cabrera
- Medical Scientist Training Program, Mayo Clinic, Rochester, MN 55902, USA
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN 55902, USA
| | - Abhinav Goyal
- Medical Scientist Training Program, Mayo Clinic, Rochester, MN 55902, USA
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN 55902, USA
| | - Hojin Shin
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN 55902, USA
- Department of Biomedical Engineering, Mayo Clinic, Rochester, MN 55902, USA
| | - Jason Yuen
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN 55902, USA
- IMPACT—the Institute for Mental and Physical Health and Clinical Translation, School of Medicine, Deakin University, Barwon Health, Geelong, VIC 3216, Australia
| | - Dong-Pyo Jang
- Department of Biomedical Engineering, Hanyang University, Seoul 04763, South Korea
| | - Keven E Bennet
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN 55902, USA
- Division of Engineering, Mayo Clinic, Rochester, MN 55902, USA
| | - Charles D Blaha
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN 55902, USA
| | - Kendall H Lee
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN 55902, USA
- Department of Biomedical Engineering, Mayo Clinic, Rochester, MN 55902, USA
| | - Yoonbae Oh
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN 55902, USA
- Department of Biomedical Engineering, Mayo Clinic, Rochester, MN 55902, USA
| |
Collapse
|
9
|
Liu J, Liu D, Pu X, Zou K, Xie T, Li Y, Yao H. The Secondary Motor Cortex-striatum Circuit Contributes to Suppressing Inappropriate Responses in Perceptual Decision Behavior. Neurosci Bull 2023; 39:1544-1560. [PMID: 37253985 PMCID: PMC10533474 DOI: 10.1007/s12264-023-01073-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2022] [Accepted: 03/08/2023] [Indexed: 06/01/2023] Open
Abstract
The secondary motor cortex (M2) encodes choice-related information and plays an important role in cue-guided actions. M2 neurons innervate the dorsal striatum (DS), which also contributes to decision-making behavior, yet how M2 modulates signals in the DS to influence perceptual decision-making is unclear. Using mice performing a visual Go/No-Go task, we showed that inactivating M2 projections to the DS impaired performance by increasing the false alarm (FA) rate to the reward-irrelevant No-Go stimulus. The choice signal of M2 neurons correlated with behavioral performance, and the inactivation of M2 neurons projecting to the DS reduced the choice signal in the DS. By measuring and manipulating the responses of direct or indirect pathway striatal neurons defined by M2 inputs, we found that the indirect pathway neurons exhibited a shorter response latency to the No-Go stimulus, and inactivating their early responses increased the FA rate. These results demonstrate that the M2-to-DS pathway is crucial for suppressing inappropriate responses in perceptual decision behavior.
Collapse
Affiliation(s)
- Jing Liu
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Dechen Liu
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
| | - Xiaotian Pu
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Kexin Zou
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Taorong Xie
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
| | - Yaping Li
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
| | - Haishan Yao
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China.
- Shanghai Center for Brain Science and Brain-Inspired Intelligence Technology, Shanghai, 201210, China.
| |
Collapse
|
10
|
Li H, Jin X. Multiple dynamic interactions from basal ganglia direct and indirect pathways mediate action selection. eLife 2023; 12:RP87644. [PMID: 37751468 PMCID: PMC10522336 DOI: 10.7554/elife.87644] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/28/2023] Open
Abstract
The basal ganglia are known to be essential for action selection. However, the functional role of basal ganglia direct and indirect pathways in action selection remains unresolved. Here, by employing cell-type-specific neuronal recording and manipulation in mice trained in a choice task, we demonstrate that multiple dynamic interactions from the direct and indirect pathways control the action selection. While the direct pathway regulates the behavioral choice in a linear manner, the indirect pathway exerts a nonlinear inverted-U-shaped control over action selection, depending on the inputs and the network state. We propose a new center (direct)-surround (indirect)-context (indirect) 'Triple-control' functional model of basal ganglia, which can replicate the physiological and behavioral experimental observations that cannot be simply explained by either the traditional 'Go/No-go' or more recent 'Co-activation' model. These findings have important implications on understanding the basal ganglia circuitry and action selection in health and disease.
Collapse
Affiliation(s)
- Hao Li
- Molecular Neurobiology Laboratory, The Salk Institute for Biological StudiesLa JollaUnited States
| | - Xin Jin
- Molecular Neurobiology Laboratory, The Salk Institute for Biological StudiesLa JollaUnited States
- Center for Motor Control and Disease, Key Laboratory of Brain Functional Genomics, East China Normal UniversityShanghaiChina
- NYU–ECNU Institute of Brain and Cognitive Science, New York University ShanghaiShanghaiChina
| |
Collapse
|
11
|
Rios A, Nonomura S, Kato S, Yoshida J, Matsushita N, Nambu A, Takada M, Hira R, Kobayashi K, Sakai Y, Kimura M, Isomura Y. Reward expectation enhances action-related activity of nigral dopaminergic and two striatal output pathways. Commun Biol 2023; 6:914. [PMID: 37673949 PMCID: PMC10482957 DOI: 10.1038/s42003-023-05288-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2022] [Accepted: 08/25/2023] [Indexed: 09/08/2023] Open
Abstract
Neurons comprising nigrostriatal system play important roles in action selection. However, it remains unclear how this system integrates recent outcome information with current action (movement) and outcome (reward or no reward) information to achieve appropriate subsequent action. We examined how neuronal activity of substantia nigra pars compacta (SNc) and dorsal striatum reflects the level of reward expectation from recent outcomes in rats performing a reward-based choice task. Movement-related activity of direct and indirect pathway striatal projection neurons (dSPNs and iSPNs, respectively) were enhanced by reward expectation, similarly to the SNc dopaminergic neurons, in both medial and lateral nigrostriatal projections. Given the classical basal ganglia model wherein dopamine stimulates dSPNs and suppresses iSPNs through distinct dopamine receptors, dopamine might not be the primary driver of iSPN activity increasing following higher reward expectation. In contrast, outcome-related activity was affected by reward expectation in line with the classical model and reinforcement learning theory, suggesting purposive effects of reward expectation.
Collapse
Affiliation(s)
- Alain Rios
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University (TMDU), Tokyo, 113-8510, Japan.
| | - Satoshi Nonomura
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University (TMDU), Tokyo, 113-8510, Japan
- Center for the Evolutionary Origins of Human Behavior, Kyoto University, Aichi, 484-8506, Japan
| | - Shigeki Kato
- Department of Molecular Genetics, Institute of Biomedical Science, Fukushima Medical University, Fukushima, 960-1295, Japan
| | - Junichi Yoshida
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, 10461, USA
| | - Natsuki Matsushita
- Division of Laboratory Animal Research, Aichi Medical University, Aichi, 480-1195, Japan
| | - Atsushi Nambu
- Division of System Neurophysiology, National Institute of Physiological Sciences and Department of Physiological Sciences, SOKENDAI, Aichi, 444-8585, Japan
| | - Masahiko Takada
- Center for the Evolutionary Origins of Human Behavior, Kyoto University, Aichi, 484-8506, Japan
| | - Riichiro Hira
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University (TMDU), Tokyo, 113-8510, Japan
| | - Kazuto Kobayashi
- Department of Molecular Genetics, Institute of Biomedical Science, Fukushima Medical University, Fukushima, 960-1295, Japan
| | - Yutaka Sakai
- Brain Science Institute, Tamagawa University, Tokyo, 194-8610, Japan
| | - Minoru Kimura
- Brain Science Institute, Tamagawa University, Tokyo, 194-8610, Japan
| | - Yoshikazu Isomura
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University (TMDU), Tokyo, 113-8510, Japan.
- Brain Science Institute, Tamagawa University, Tokyo, 194-8610, Japan.
| |
Collapse
|
12
|
Kono A, Shikano Y, Tanaka KF, Yamaura K, Tsutsui‐Kimura I. Inhibition of the dorsomedial striatal direct pathway is essential for the execution of action sequences. Neuropsychopharmacol Rep 2023; 43:414-424. [PMID: 37553985 PMCID: PMC10496086 DOI: 10.1002/npr2.12369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 07/13/2023] [Accepted: 07/23/2023] [Indexed: 08/10/2023] Open
Abstract
Contrary to the previous notion that the dorsomedial striatum (DMS) is crucial for acquiring new learning, accumulated evidence has suggested that the DMS also plays a role in the execution of already learned action sequences. Here, we examined how the direct and indirect pathways in the DMS regulate action sequences using a task that requires animals to press a lever consecutively. Cell-type-specific bulk Ca2+ recording revealed that the direct pathway was inhibited at the time of sequence execution. The sequence-related response was blunted in trials where the sequential behaviors were disrupted. Optogenetic activation at the sequence start caused distraction of action sequences without affecting motor function or memory of the task structure. By contrast with the direct pathway, the indirect pathway was slightly activated at the start of the sequence, but the optogenetic suppression of such sequence-related signaling did not impact the behaviors. These results suggest that the inhibition of the DMS direct pathway promotes sequence execution potentially by suppressing the formation of a new association.
Collapse
Affiliation(s)
- Anna Kono
- Division of Brain SciencesInstitute for Advanced Medical Research, Keio University School of MedicineTokyoJapan
- Division of Social Pharmacy, Center for Social Pharmacy and Pharmaceutical Care SciencesKeio University Faculty of PharmacyTokyoJapan
| | - Yu Shikano
- Division of Brain SciencesInstitute for Advanced Medical Research, Keio University School of MedicineTokyoJapan
| | - Kenji F. Tanaka
- Division of Brain SciencesInstitute for Advanced Medical Research, Keio University School of MedicineTokyoJapan
| | - Katsunori Yamaura
- Division of Social Pharmacy, Center for Social Pharmacy and Pharmaceutical Care SciencesKeio University Faculty of PharmacyTokyoJapan
| | - Iku Tsutsui‐Kimura
- Division of Brain SciencesInstitute for Advanced Medical Research, Keio University School of MedicineTokyoJapan
| |
Collapse
|
13
|
Vellucci L, Ciccarelli M, Buonaguro EF, Fornaro M, D’Urso G, De Simone G, Iasevoli F, Barone A, de Bartolomeis A. The Neurobiological Underpinnings of Obsessive-Compulsive Symptoms in Psychosis, Translational Issues for Treatment-Resistant Schizophrenia. Biomolecules 2023; 13:1220. [PMID: 37627285 PMCID: PMC10452784 DOI: 10.3390/biom13081220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 07/24/2023] [Accepted: 07/31/2023] [Indexed: 08/27/2023] Open
Abstract
Almost 25% of schizophrenia patients suffer from obsessive-compulsive symptoms (OCS) considered a transdiagnostic clinical continuum. The presence of symptoms pertaining to both schizophrenia and obsessive-compulsive disorder (OCD) may complicate pharmacological treatment and could contribute to lack or poor response to the therapy. Despite the clinical relevance, no reviews have been recently published on the possible neurobiological underpinnings of this comorbidity, which is still unclear. An integrative view exploring this topic should take into account the following aspects: (i) the implication for glutamate, dopamine, and serotonin neurotransmission as demonstrated by genetic findings; (ii) the growing neuroimaging evidence of the common brain regions and dysfunctional circuits involved in both diseases; (iii) the pharmacological modulation of dopaminergic, serotoninergic, and glutamatergic systems as current therapeutic strategies in schizophrenia OCS; (iv) the recent discovery of midbrain dopamine neurons and dopamine D1- and D2-like receptors as orchestrating hubs in repetitive and psychotic behaviors; (v) the contribution of N-methyl-D-aspartate receptor subunits to both psychosis and OCD neurobiology. Finally, we discuss the potential role of the postsynaptic density as a structural and functional hub for multiple molecular signaling both in schizophrenia and OCD pathophysiology.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | - Andrea de Bartolomeis
- Section of Psychiatry, Laboratory of Translational and Molecular Psychiatry and Unit of Treatment-Resistant Psychosis, Department of Neuroscience, Reproductive Sciences and Dentistry University Medical School of Naples “Federico II”, Via Pansini 5, 80131 Naples, Italy
| |
Collapse
|
14
|
Blackwell KT, Doya K. Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks. PLoS Comput Biol 2023; 19:e1011385. [PMID: 37594982 PMCID: PMC10479916 DOI: 10.1371/journal.pcbi.1011385] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Revised: 09/05/2023] [Accepted: 07/25/2023] [Indexed: 08/20/2023] Open
Abstract
A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.
Collapse
Affiliation(s)
- Kim T Blackwell
- Department of Bioengineering, Volgenau School of Engineering, George Mason University, Fairfax, Virginia, United States of America
| | - Kenji Doya
- Neural Computation Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan
| |
Collapse
|
15
|
Yoshida A, Hikosaka O. Opposing functions of glutamatergic inputs between the globus pallidus external segment and substantia nigra pars reticulata. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.07.25.550377. [PMID: 37546868 PMCID: PMC10402021 DOI: 10.1101/2023.07.25.550377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/08/2023]
Abstract
The indirect pathway of the basal ganglia, including the subthalamic nucleus (STN) and globus pallidus external segment (GPe), is believed to play a crucial role in suppressing involuntary movements. However, recent evidence suggests the STN and GPe also facilitate voluntary movements. This study hypothesized that excitatory inputs from the STN to the GPe contribute to this facilitation, and that excitatory projections to the substantia nigra pars reticulata (SNr) are involved in the inhibition. To disrupt the STN-GPe or STN-SNr projections in monkeys during choice and fixation tasks, glutamate receptor inhibitors were injected into the GPe or SNr, which induced delayed saccade latencies toward good choices in the choice task (GPe) and caused frequent reflexive saccades to objects in the fixation task (SNr). Our findings suggest excitatory inputs to the GPe and SNr work in opposing manners, providing new insights that redefine our understanding of the functions of basal ganglia pathways.
Collapse
Affiliation(s)
- Atsushi Yoshida
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
| | - Okihide Hikosaka
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
| |
Collapse
|
16
|
Li H, Jin X. Multiple dynamic interactions from basal ganglia direct and indirect pathways mediate action selection. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.03.20.533567. [PMID: 36993546 PMCID: PMC10055198 DOI: 10.1101/2023.03.20.533567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/31/2023]
Abstract
The basal ganglia are known to be essential for action selection. However, the functional role of basal ganglia direct and indirect pathways in action selection remains unresolved. Here by employing cell-type-specific neuronal recording and manipulation in mice trained in a choice task, we demonstrate that multiple dynamic interactions from the direct and indirect pathways control the action selection. While the direct pathway regulates the behavioral choice in a linear manner, the indirect pathway exerts a nonlinear inverted-U-shaped control over action selection, depending on the inputs and the network state. We propose a new center (direct) - surround (indirect) - context (indirect) "Triple-control" functional model of basal ganglia, which can replicate the physiological and behavioral experimental observations that cannot be simply explained by either the traditional "Go/No-go" or more recent "Co-activation" model. These findings have important implications on understanding the basal ganglia circuitry and action selection in health and disease.
Collapse
Affiliation(s)
- Hao Li
- Molecular Neurobiology Laboratory, The Salk Institute for Biological Studies, 10010 North Torrey Pines Road, La Jolla, CA 92037, USA
| | - Xin Jin
- Molecular Neurobiology Laboratory, The Salk Institute for Biological Studies, 10010 North Torrey Pines Road, La Jolla, CA 92037, USA
- Center for Motor Control and Disease, Key Laboratory of Brain Functional Genomics, East China Normal University, 3663 North Zhongshan Road, Shanghai 200062, China
- NYU–ECNU Institute of Brain and Cognitive Science, New York University Shanghai, 3663 North Zhongshan Road, Shanghai 200062, China
- Lead Contact
| |
Collapse
|
17
|
Soma S, Ohara S, Nonomura S, Suematsu N, Yoshida J, Pastalkova E, Sakai Y, Tsutsui KI, Isomura Y. Rat hippocampal CA1 region represents learning-related action and reward events with shorter latency than the lateral entorhinal cortex. Commun Biol 2023; 6:584. [PMID: 37258700 DOI: 10.1038/s42003-023-04958-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2022] [Accepted: 05/20/2023] [Indexed: 06/02/2023] Open
Abstract
The hippocampus and entorhinal cortex are deeply involved in learning and memory. However, little is known how ongoing events are processed in the hippocampal-entorhinal circuit. By recording from head-fixed rats during action-reward learning, here we show that the action and reward events are represented differently in the hippocampal CA1 region and lateral entorhinal cortex (LEC). Although diverse task-related activities developed after learning in both CA1 and LEC, phasic activities related to action and reward events differed in the timing of behavioral event representation. CA1 represented action and reward events almost instantaneously, whereas the superficial and deep layers of the LEC showed a delayed representation of the same events. Interestingly, we also found that ramping activity towards spontaneous action was correlated with waiting time in both regions and exceeded that in the motor cortex. Such functional activities observed in the entorhinal-hippocampal circuits may play a crucial role for animals in utilizing ongoing information to dynamically optimize their behaviors.
Collapse
Affiliation(s)
- Shogo Soma
- Brain Science Institute, Tamagawa University, Tokyo, Japan.
- Department of Molecular Cell Physiology, Kyoto Prefectural University of Medicine, Kyoto, Japan.
| | - Shinya Ohara
- Laboratory of Systems Neuroscience, Tohoku University Graduate School of Life Sciences, Sendai, Japan
- PRESTO, Japan Science and Technology Agency (JST), Kawaguchi, Japan
| | - Satoshi Nonomura
- Brain Science Institute, Tamagawa University, Tokyo, Japan
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan
- Center for the Evolutionary Origins of Human Behavior, Kyoto University, Aichi, Japan
| | - Naofumi Suematsu
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Junichi Yoshida
- Brain Science Institute, Tamagawa University, Tokyo, Japan
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA
| | - Eva Pastalkova
- Department of Clinical Psychology, Pacifica Graduate Institute, Carpinteria, CA, USA
| | - Yutaka Sakai
- Brain Science Institute, Tamagawa University, Tokyo, Japan
| | - Ken-Ichiro Tsutsui
- Laboratory of Systems Neuroscience, Tohoku University Graduate School of Life Sciences, Sendai, Japan
| | - Yoshikazu Isomura
- Brain Science Institute, Tamagawa University, Tokyo, Japan.
- Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan.
| |
Collapse
|
18
|
Maith O, Baladron J, Einhäuser W, Hamker FH. Exploration behavior after reversals is predicted by STN-GPe synaptic plasticity in a basal ganglia model. iScience 2023; 26:106599. [PMID: 37250300 PMCID: PMC10214406 DOI: 10.1016/j.isci.2023.106599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Revised: 02/02/2023] [Accepted: 03/29/2023] [Indexed: 05/31/2023] Open
Abstract
Humans can quickly adapt their behavior to changes in the environment. Classical reversal learning tasks mainly measure how well participants can disengage from a previously successful behavior but not how alternative responses are explored. Here, we propose a novel 5-choice reversal learning task with alternating position-reward contingencies to study exploration behavior after a reversal. We compare human exploratory saccade behavior with a prediction obtained from a neuro-computational model of the basal ganglia. A new synaptic plasticity rule for learning the connectivity between the subthalamic nucleus (STN) and external globus pallidus (GPe) results in exploration biases to previously rewarded positions. The model simulations and human data both show that during experimental experience exploration becomes limited to only those positions that have been rewarded in the past. Our study demonstrates how quite complex behavior may result from a simple sub-circuit within the basal ganglia pathways.
Collapse
Affiliation(s)
- Oliver Maith
- Department of Computer Science, Chemnitz University of Technology, Chemnitz, Germany
| | - Javier Baladron
- Department of Computer Science, Chemnitz University of Technology, Chemnitz, Germany
- Departamento de Ingeniería Informática, Universidad de Santiago de Chile, Santiago, Chile
| | - Wolfgang Einhäuser
- Institute of Physics, Chemnitz University of Technology, Chemnitz, Germany
| | - Fred H. Hamker
- Department of Computer Science, Chemnitz University of Technology, Chemnitz, Germany
| |
Collapse
|
19
|
Basanisi R, Marche K, Combrisson E, Apicella P, Brovelli A. Beta Oscillations in Monkey Striatum Encode Reward Prediction Error Signals. J Neurosci 2023; 43:3339-3352. [PMID: 37015808 PMCID: PMC10162459 DOI: 10.1523/jneurosci.0952-22.2023] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Revised: 02/22/2023] [Accepted: 03/17/2023] [Indexed: 04/06/2023] Open
Abstract
Reward prediction error (RPE) signals are crucial for reinforcement learning and decision-making as they quantify the mismatch between predicted and obtained rewards. RPE signals are encoded in the neural activity of multiple brain areas, such as midbrain dopaminergic neurons, prefrontal cortex, and striatum. However, it remains unclear how these signals are expressed through anatomically and functionally distinct subregions of the striatum. In the current study, we examined to which extent RPE signals are represented across different striatal regions. To do so, we recorded local field potentials (LFPs) in sensorimotor, associative, and limbic striatal territories of two male rhesus monkeys performing a free-choice probabilistic learning task. The trial-by-trial evolution of RPE during task performance was estimated using a reinforcement learning model fitted on monkeys' choice behavior. Overall, we found that changes in beta band oscillations (15-35 Hz), after the outcome of the animal's choice, are consistent with RPE encoding. Moreover, we provide evidence that the signals related to RPE are more strongly represented in the ventral (limbic) than dorsal (sensorimotor and associative) part of the striatum. To conclude, our results suggest a relationship between striatal beta oscillations and the evaluation of outcomes based on RPE signals and highlight a major contribution of the ventral striatum to the updating of learning processes.SIGNIFICANCE STATEMENT Reward prediction error (RPE) signals are crucial for reinforcement learning and decision-making as they quantify the mismatch between predicted and obtained rewards. Current models suggest that RPE signals are encoded in the neural activity of multiple brain areas, including the midbrain dopaminergic neurons, prefrontal cortex and striatum. However, it remains elusive whether RPEs recruit anatomically and functionally distinct subregions of the striatum. Our study provides evidence that RPE-related modulations in local field potential (LFP) power are dominant in the striatum. In particular, they are stronger in the rostro-ventral rather than the caudo-dorsal striatum. Our findings contribute to a better understanding of the role of striatal territories in reward-based learning and may be relevant for neuropsychiatric and neurologic diseases that affect striatal circuits.
Collapse
Affiliation(s)
- Ruggero Basanisi
- Institut de Neurosciences de la Timone, Aix Marseille Université, Unité Mixte de Recherche 7289 Centre National de la Recherche Scientifique, Marseille 13005, France
| | - Kevin Marche
- Institut de Neurosciences de la Timone, Aix Marseille Université, Unité Mixte de Recherche 7289 Centre National de la Recherche Scientifique, Marseille 13005, France
- Wellcome Center for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX3 9DU, United Kingdom
| | - Etienne Combrisson
- Institut de Neurosciences de la Timone, Aix Marseille Université, Unité Mixte de Recherche 7289 Centre National de la Recherche Scientifique, Marseille 13005, France
| | - Paul Apicella
- Institut de Neurosciences de la Timone, Aix Marseille Université, Unité Mixte de Recherche 7289 Centre National de la Recherche Scientifique, Marseille 13005, France
| | - Andrea Brovelli
- Institut de Neurosciences de la Timone, Aix Marseille Université, Unité Mixte de Recherche 7289 Centre National de la Recherche Scientifique, Marseille 13005, France
| |
Collapse
|
20
|
Nishioka T, Attachaipanich S, Hamaguchi K, Lazarus M, de Kerchove d'Exaerde A, Macpherson T, Hikida T. Error-related signaling in nucleus accumbens D2 receptor-expressing neurons guides inhibition-based choice behavior in mice. Nat Commun 2023; 14:2284. [PMID: 37085502 PMCID: PMC10121661 DOI: 10.1038/s41467-023-38025-3] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2021] [Accepted: 04/12/2023] [Indexed: 04/23/2023] Open
Abstract
Learned associations between environmental cues and the outcomes they predict (cue-outcome associations) play a major role in behavioral control, guiding not only which responses we should perform, but also which we should inhibit, in order to achieve a specific goal. The encoding of such cue-outcome associations, as well as the performance of cue-guided choice behavior, is thought to involve dopamine D1 and D2 receptor-expressing medium spiny neurons (D1-/D2-MSNs) of the nucleus accumbens (NAc). Here, using a visual discrimination task in male mice, we assessed the role of NAc D1-/D2-MSNs in cue-guided inhibition of inappropriate responding. Cell-type specific neuronal silencing and in-vivo imaging revealed NAc D2-MSNs to contribute to inhibiting behavioral responses, with activation of NAc D2-MSNs following response errors playing an important role in optimizing future choice behavior. Our findings indicate that error-signaling by NAc D2-MSNs contributes to the ability to use environmental cues to inhibit inappropriate behavior.
Collapse
Affiliation(s)
- Tadaaki Nishioka
- Laboratory for Advanced Brain Functions, Institute for Protein Research, Osaka University, Suita, Japan.
- Laboratory for Developing Minds, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| | - Suthinee Attachaipanich
- Laboratory for Advanced Brain Functions, Institute for Protein Research, Osaka University, Suita, Japan
| | - Kosuke Hamaguchi
- Department of Biological Sciences, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Michael Lazarus
- International Institute for Integrative Sleep Medicine (WPI-IIIS) and Faculty of Medicine, University of Tsukuba, Tsukuba, Japan
| | | | - Tom Macpherson
- Laboratory for Advanced Brain Functions, Institute for Protein Research, Osaka University, Suita, Japan.
| | - Takatoshi Hikida
- Laboratory for Advanced Brain Functions, Institute for Protein Research, Osaka University, Suita, Japan.
| |
Collapse
|
21
|
Ishino S, Kamada T, Sarpong GA, Kitano J, Tsukasa R, Mukohira H, Sun F, Li Y, Kobayashi K, Naoki H, Oishi N, Ogawa M. Dopamine error signal to actively cope with lack of expected reward. SCIENCE ADVANCES 2023; 9:eade5420. [PMID: 36897945 PMCID: PMC10005178 DOI: 10.1126/sciadv.ade5420] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Accepted: 02/06/2023] [Indexed: 06/17/2023]
Abstract
To obtain more of a particular uncertain reward, animals must learn to actively overcome the lack of reward and adjust behavior to obtain it again. The neural mechanisms underlying such coping with reward omission remain unclear. Here, we developed a task in rats to monitor active behavioral switch toward the next reward after no reward. We found that some dopamine neurons in the ventral tegmental area exhibited increased responses to unexpected reward omission and decreased responses to unexpected reward, following the opposite responses of the well-known dopamine neurons that signal reward prediction error (RPE). The dopamine increase reflected in the nucleus accumbens correlated with behavioral adjustment to actively overcome unexpected no reward. We propose that these responses signal error to actively cope with lack of expected reward. The dopamine error signal thus cooperates with the RPE signal, enabling adaptive and robust pursuit of uncertain reward to ultimately obtain more reward.
Collapse
Affiliation(s)
- Seiya Ishino
- Medical Innovation Center/SK Project, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
- Department of Neuroscience, Graduate School of Medicine, Kyoto University, Kyoto 606-8501, Japan
- Department of Developmental Physiology, National Institute for Physiological Sciences, Okazaki, Aichi 444-8585, Japan
| | - Taisuke Kamada
- Medical Innovation Center/SK Project, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
| | - Gideon A. Sarpong
- Medical Innovation Center/SK Project, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
| | - Julia Kitano
- Medical Innovation Center/SK Project, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
| | - Reo Tsukasa
- Medical Innovation Center/SK Project, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
| | - Hisa Mukohira
- Medical Innovation Center/SK Project, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
| | - Fangmiao Sun
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- Peking-Tsinghua Center for Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Yulong Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- Peking-Tsinghua Center for Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Kenta Kobayashi
- Section of Viral Vector Development, National Institute for Physiological Sciences, Okazaki, Aichi 444-8585, Japan
- SOKENDAI (The Graduate University for Advanced Studies), Okazaki, Aichi 444-8585, Japan
| | - Honda Naoki
- Laboratory of Data-driven Biology, Graduate School of Integrated Sciences for Life, Hiroshima University, Higashi-Hiroshima, Hiroshima 739-8526, Japan
- Theoretical Biology Research Group, Exploratory Research Center on Life and Living Systems (ExCELLS), National Institutes of Natural Sciences, Okazaki, Aichi 444-8787, Japan
- Laboratory of Theoretical Biology, Graduate School of Biostudies, Kyoto University, Kyoto 606-8315, Japan
- Kansei-Brain Informatics Group, Center for Brain, Mind and Kansei Sciences Research (BMK Center), Hiroshima University, Kasumi, Minami-ku, Hiroshima 734-8551, Japan
| | - Naoya Oishi
- Medical Innovation Center/SK Project, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
| | - Masaaki Ogawa
- Medical Innovation Center/SK Project, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
- Department of Neuroscience, Graduate School of Medicine, Kyoto University, Kyoto 606-8501, Japan
- Department of Developmental Physiology, National Institute for Physiological Sciences, Okazaki, Aichi 444-8585, Japan
| |
Collapse
|
22
|
Matsushita N, Kato S, Nishizawa K, Sugawara M, Takeuchi K, Miyasaka Y, Mashimo T, Kobayashi K. Highly selective transgene expression through the flip-excision switch system by using a unilateral spacer sequence. CELL REPORTS METHODS 2023; 3:100393. [PMID: 36936079 PMCID: PMC10014282 DOI: 10.1016/j.crmeth.2022.100393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Revised: 11/09/2022] [Accepted: 12/28/2022] [Indexed: 01/19/2023]
Abstract
The flip-excision switch (FLEX) system with an adeno-associated viral (AAV) vector allows expression of transgenes in specific cell populations having Cre recombinase. A significant issue with this system is non-specific expression of transgenes in tissues after vector injection. We show here that Cre-independent recombination events in the AAV genome carrying the FLEX sequence occur mainly during the production of viral vectors in packaging cells, which results in transgene expression in off-target populations. Introduction of a relatively longer nucleotide sequence between two recognition sites at the unilateral side of the transgene cassette, termed a unilateral spacer sequence (USS), is useful to suppress the recombination in the viral genome, leading to the protection of non-specific transgene expression with enhanced gene expression selectivity. Our FLEX/USS system offers a powerful strategy for highly specific Cre-dependent transgene expression, aiming at various applications for structural and functional analyses of target cell populations.
Collapse
Affiliation(s)
- Natsuki Matsushita
- Division of Laboratory Animal Research, Aichi Medical University School of Medicine, Nagakute, Aichi 480-1195, Japan
| | - Shigeki Kato
- Department of Molecular Genetics, Institute of Biomedical Sciences, Fukushima Medical University School of Medicine, Fukushima 960-1295, Japan
| | - Kayo Nishizawa
- Department of Molecular Genetics, Institute of Biomedical Sciences, Fukushima Medical University School of Medicine, Fukushima 960-1295, Japan
| | - Masateru Sugawara
- Department of Molecular Genetics, Institute of Biomedical Sciences, Fukushima Medical University School of Medicine, Fukushima 960-1295, Japan
| | - Kosei Takeuchi
- Department of Medical Cell Biology, Aichi Medical University School of Medicine, Nagakute, Aichi 480-1195, Japan
| | - Yoshiki Miyasaka
- Laboratory of Reproductive Engineering, Institute of Experimental Animal Sciences, Osaka University Medical School, Suita 565-0871, Japan
| | - Tomoji Mashimo
- Division of Animal Genetics, Laboratory Animal Research Center, Institute of Medical Science, The University of Tokyo, Tokyo 108-8639, Japan
| | - Kazuto Kobayashi
- Department of Molecular Genetics, Institute of Biomedical Sciences, Fukushima Medical University School of Medicine, Fukushima 960-1295, Japan
| |
Collapse
|
23
|
Morita K, Shimomura K, Kawaguchi Y. Opponent Learning with Different Representations in the Cortico-Basal Ganglia Circuits. eNeuro 2023; 10:ENEURO.0422-22.2023. [PMID: 36653187 PMCID: PMC9884109 DOI: 10.1523/eneuro.0422-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 12/06/2022] [Accepted: 01/03/2023] [Indexed: 01/20/2023] Open
Abstract
The direct and indirect pathways of the basal ganglia (BG) have been suggested to learn mainly from positive and negative feedbacks, respectively. Since these pathways unevenly receive inputs from different cortical neuron types and/or regions, they may preferentially use different state/action representations. We explored whether such a combined use of different representations, coupled with different learning rates from positive and negative reward prediction errors (RPEs), has computational benefits. We modeled animal as an agent equipped with two learning systems, each of which adopted individual representation (IR) or successor representation (SR) of states. With varying the combination of IR or SR and also the learning rates from positive and negative RPEs in each system, we examined how the agent performed in a dynamic reward navigation task. We found that combination of SR-based system learning mainly from positive RPEs and IR-based system learning mainly from negative RPEs could achieve a good performance in the task, as compared with other combinations. In such a combination of appetitive SR-based and aversive IR-based systems, both systems show activities of comparable magnitudes with opposite signs, consistent with the suggested profiles of the two BG pathways. Moreover, the architecture of such a combination provides a novel coherent explanation for the functional significance and underlying mechanism of diverse findings about the cortico-BG circuits. These results suggest that particularly combining different representations with appetitive and aversive learning could be an effective learning strategy in certain dynamic environments, and it might actually be implemented in the cortico-BG circuits.
Collapse
Affiliation(s)
- Kenji Morita
- Physical and Health Education, Graduate School of Education, The University of Tokyo, Tokyo 113-0033, Japan
- International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo, Tokyo 113-0033, Japan
| | - Kanji Shimomura
- Physical and Health Education, Graduate School of Education, The University of Tokyo, Tokyo 113-0033, Japan
- Department of Behavioral Medicine, National Institute of Mental Health, National Center of Neurology and Psychiatry, Kodaira 187-8551, Japan
| | - Yasuo Kawaguchi
- Brain Science Institute, Tamagawa University, Machida 194-8610, Japan
- National Institute for Physiological Sciences (NIPS), Okazaki 444-8787, Japan
| |
Collapse
|
24
|
Continuous cholinergic-dopaminergic updating in the nucleus accumbens underlies approaches to reward-predicting cues. Nat Commun 2022; 13:7924. [PMID: 36564387 PMCID: PMC9789106 DOI: 10.1038/s41467-022-35601-x] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Accepted: 12/13/2022] [Indexed: 12/25/2022] Open
Abstract
The ability to learn Pavlovian associations from environmental cues predicting positive outcomes is critical for survival, motivating adaptive behaviours. This cued-motivated behaviour depends on the nucleus accumbens (NAc). NAc output activity mediated by spiny projecting neurons (SPNs) is regulated by dopamine, but also by cholinergic interneurons (CINs), which can release acetylcholine and glutamate via the activity of the vesicular acetylcholine transporter (VAChT) or the vesicular glutamate transporter (VGLUT3), respectively. Here we investigated behavioural and neurochemical changes in mice performing a touchscreen Pavlovian approach task by recording dopamine, acetylcholine, and calcium dynamics from D1- and D2-SPNs using fibre photometry in control, VAChT or VGLUT3 mutant mice to understand how these signals cooperate in the service of approach behaviours toward reward-predicting cues. We reveal that NAc acetylcholine-dopaminergic signalling is continuously updated to regulate striatal output underlying the acquisition of Pavlovian approach learning toward reward-predicting cues.
Collapse
|
25
|
Nishimaru H, Matsumoto J, Setogawa T, Nishijo H. Neuronal structures controlling locomotor behavior during active and inactive motor states. Neurosci Res 2022; 189:83-93. [PMID: 36549389 DOI: 10.1016/j.neures.2022.12.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Accepted: 12/16/2022] [Indexed: 12/23/2022]
Abstract
Animal behaviors can be divided into two states according to their motor activity: the active motor state, which involves significant body movements, and the inactive motor state, which refers to when the animal is stationary. The timing and duration of these states are determined by the activity of the neuronal circuits involved in motor control. Among these motor circuits, those that generate locomotion are some of the most studied neuronal networks and are widely distributed from the spinal cord to the cerebral cortex. In this review, we discuss recent discoveries, mainly in rodents using state-of-the-art experimental approaches, of the neuronal mechanisms underlying the initiation and termination of locomotion in the brainstem, basal ganglia, and prefrontal cortex. These findings is discussed with reference to studies on the neuronal mechanism of motor control during sleep and the modulation of cortical states in these structures. Accumulating evidence has unraveled the complex yet highly structured network that controls the transition between motor states.
Collapse
Affiliation(s)
- Hiroshi Nishimaru
- System Emotional Science, Faculty of Medicine, University of Toyama, Toyama 930-0194, Japan; Graduate school of Innovative Life Science, University of Toyama, Toyama 930-0194, Japan; Research Center for Idling Brain Science (RCIBS), University of Toyama, Toyama 930-0194, Japan.
| | - Jumpei Matsumoto
- System Emotional Science, Faculty of Medicine, University of Toyama, Toyama 930-0194, Japan; Graduate school of Innovative Life Science, University of Toyama, Toyama 930-0194, Japan; Research Center for Idling Brain Science (RCIBS), University of Toyama, Toyama 930-0194, Japan
| | - Tsuyoshi Setogawa
- System Emotional Science, Faculty of Medicine, University of Toyama, Toyama 930-0194, Japan; Graduate school of Innovative Life Science, University of Toyama, Toyama 930-0194, Japan; Research Center for Idling Brain Science (RCIBS), University of Toyama, Toyama 930-0194, Japan
| | - Hisao Nishijo
- System Emotional Science, Faculty of Medicine, University of Toyama, Toyama 930-0194, Japan; Graduate school of Innovative Life Science, University of Toyama, Toyama 930-0194, Japan; Research Center for Idling Brain Science (RCIBS), University of Toyama, Toyama 930-0194, Japan
| |
Collapse
|
26
|
The orphan receptor GPR88 controls impulsivity and is a risk factor for Attention-Deficit/Hyperactivity Disorder. Mol Psychiatry 2022; 27:4662-4672. [PMID: 36075963 PMCID: PMC9936886 DOI: 10.1038/s41380-022-01738-w] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Revised: 08/08/2022] [Accepted: 08/10/2022] [Indexed: 02/02/2023]
Abstract
The neural orphan G protein coupled receptor GPR88 is predominant in the striatum and cortex of both rodents and humans, and considered a potential target for brain disorders. Previous studies have shown multiple behavioral phenotypes in Gpr88 knockout mice, and human genetic studies have reported association with psychosis. Here we tested the possibility that GPR88 contributes to Attention Deficit Hyperactivity Disorder (ADHD). In the mouse, we tested Gpr88 knockout mice in three behavioral paradigms, best translatable between rodents and humans, and found higher motor impulsivity and reduced attention together with the reported hyperactivity. Atomoxetine, a typical ADHD drug, reduced impulsivity in mutant mice. Conditional Gpr88 knockout mice in either D1R-type or D2R-type medium spiny neurons revealed distinct implications of the two receptor populations in waiting and stopping impulsivity. Thus, animal data demonstrate that deficient GPR88 activity causally promotes ADHD-like behaviors, and identify circuit mechanisms underlying GPR88-regulated impulsivity. In humans, we performed a family-based genetic study including 567 nuclear families with DSM-IV diagnosis of ADHD. There was a minor association for SNP rs2036212 with diagnosis, treatment response and cognition. A stronger association was found for SNP rs2809817 upon patient stratification, suggesting that the T allele is a risk factor when prenatal stress is involved. Human data therefore identify GPR88 variants associated with the disease, and highlight a potential role of life trajectories to modulate GPR88 function. Overall, animal and human data concur to suggest that GPR88 signaling should be considered a key factor for diagnostic and treatment of ADHD.
Collapse
|
27
|
Mitani K, Kawabata M, Isomura Y, Sakai Y. Automated and parallelized spike collision tests to identify spike signal projections. iScience 2022; 25:105071. [PMID: 36157577 PMCID: PMC9490030 DOI: 10.1016/j.isci.2022.105071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 08/09/2022] [Accepted: 08/30/2022] [Indexed: 10/28/2022] Open
Abstract
The spike collision test is a highly reliable technique to identify the axonal projection of a neuron recorded electrophysiologically for investigating functional spike information among brain areas. It is potentially applicable to more neuronal projections by combining multi-channel recording with optogenetic stimulation. Yet, it remains inefficient and laborious because an experimenter must visually select spikes in every channel and manually repeat spike collision tests for each neuron serially. Here, we automated spike collision tests for all channels in parallel (Multi-Linc analysis) in a multi-channel real-time processing system. The rat cortical neurons identified with this technique displayed physiological spike features consistent with excitatory projection neurons. Their antidromic spikes were similar in shape but slightly larger in amplitude compared with spontaneous spikes. In addition, we demonstrated simultaneous identification of reciprocal or bifurcating projections among cortical areas. Thus, our Multi-Linc analysis will be a powerful approach to elucidate interareal spike communication.
Collapse
Affiliation(s)
- Keita Mitani
- Brain Science Institute, Tamagawa University, Machida, Tokyo, Japan.,Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan
| | - Masanori Kawabata
- Brain Science Institute, Tamagawa University, Machida, Tokyo, Japan.,Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan
| | - Yoshikazu Isomura
- Brain Science Institute, Tamagawa University, Machida, Tokyo, Japan.,Department of Physiology and Cell Biology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan
| | - Yutaka Sakai
- Brain Science Institute, Tamagawa University, Machida, Tokyo, Japan
| |
Collapse
|
28
|
Sakai Y, Sakai Y, Abe Y, Narumoto J, Tanaka SC. Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior. Cell Rep 2022; 40:111275. [PMID: 36044850 DOI: 10.1016/j.celrep.2022.111275] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Revised: 06/09/2022] [Accepted: 08/05/2022] [Indexed: 11/03/2022] Open
Abstract
We may view most of our daily activities as rational action selections; however, we sometimes reinforce maladaptive behaviors despite having explicit environmental knowledge. In this study, we model obsessive-compulsive disorder (OCD) symptoms as implicitly learned maladaptive behaviors. Simulations in the reinforcement learning framework show that agents implicitly learn to respond to intrusive thoughts when the memory trace signal for past actions decays differently for positive and negative prediction errors. Moreover, this model extends our understanding of therapeutic effects of behavioral therapy in OCD. Using empirical data, we confirm that patients with OCD show extremely imbalanced traces, which are normalized by serotonin enhancers. We find that healthy participants also vary in their obsessive-compulsive tendencies, consistent with the degree of imbalanced traces. These behavioral characteristics can be generalized to variations in the healthy population beyond the spectrum of clinical phenotypes.
Collapse
Affiliation(s)
- Yuki Sakai
- ATR Brain Information Communication Research Laboratory Group, 2-2-2 Hikaridai Seika-Cho, Soraku-Gun, Kyoto 619-0288, Japan; Department of Psychiatry, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, 465 Kajii-Cho, Kawaramachi-Hirokoji, Kamigyo-Ku, Kyoto 602-8566, Japan
| | - Yutaka Sakai
- Brain Science Institute, Tamagawa University, 6-1-1, Tamagawa-Gakuen, Machida, Tokyo 194-8610, Japan
| | - Yoshinari Abe
- Department of Psychiatry, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, 465 Kajii-Cho, Kawaramachi-Hirokoji, Kamigyo-Ku, Kyoto 602-8566, Japan
| | - Jin Narumoto
- Department of Psychiatry, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, 465 Kajii-Cho, Kawaramachi-Hirokoji, Kamigyo-Ku, Kyoto 602-8566, Japan
| | - Saori C Tanaka
- ATR Brain Information Communication Research Laboratory Group, 2-2-2 Hikaridai Seika-Cho, Soraku-Gun, Kyoto 619-0288, Japan; Division of Information Science, Graduate School of Science and Technology, Nara Institute of Science and Technology, 8916-5 Takayama-Cho, Ikoma, Nara 630-0192, Japan.
| |
Collapse
|
29
|
Delevich K, Hoshal B, Zhou LZ, Zhang Y, Vedula S, Lin WC, Chase J, Collins AGE, Wilbrecht L. Activation, but not inhibition, of the indirect pathway disrupts choice rejection in a freely moving, multiple-choice foraging task. Cell Rep 2022; 40:111129. [PMID: 35905722 PMCID: PMC10481643 DOI: 10.1016/j.celrep.2022.111129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2020] [Revised: 03/25/2022] [Accepted: 07/01/2022] [Indexed: 11/20/2022] Open
Abstract
The dorsomedial striatum (DMS) plays a key role in action selection, but less is known about how direct and indirect pathway spiny projection neurons (dSPNs and iSPNs, respectively) contribute to choice rejection in freely moving animals. Here, we use pathway-specific chemogenetic manipulation during a serial choice foraging task to test the role of dSPNs and iSPNs in learned choice rejection. We find that chemogenetic activation, but not inhibition, of iSPNs disrupts rejection of nonrewarded choices, contrary to predictions of a simple "select/suppress" heuristic. Our findings suggest that iSPNs' role in stopping and freezing does not extend in a simple fashion to choice rejection in an ethological, freely moving context. These data may provide insights critical for the successful design of interventions for addiction or other conditions in which it is desirable to strengthen choice rejection.
Collapse
Affiliation(s)
- Kristen Delevich
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA; Department of Integrative Physiology and Neuroscience, College of Veterinary Medicine, Washington State University, Pullman, WA 99164, USA
| | - Benjamin Hoshal
- Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Lexi Z Zhou
- Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Yuting Zhang
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Satya Vedula
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Wan Chen Lin
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA; Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Juliana Chase
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Anne G E Collins
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA; Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Linda Wilbrecht
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA; Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA.
| |
Collapse
|
30
|
Holly EN, Diáz-Hernández E, Fuccillo MV. A blueprint for examining striatal control of cognition. Trends Neurosci 2022; 45:649-650. [PMID: 35690496 DOI: 10.1016/j.tins.2022.05.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Accepted: 05/26/2022] [Indexed: 10/18/2022]
Abstract
In a recent study, Bolkan, Stone, and colleagues demonstrated that direct and indirect striatal pathways in mice exert opponent control over choice behavior in a task- and state-dependent manner. This work highlights the need for rigorously controlled behavioral experiments and novel behavioral modeling in investigations of the neural mechanisms of decision making.
Collapse
|
31
|
Liang B, Zhang L, Zhang Y, Werner CT, Beacher NJ, Denman AJ, Li Y, Chen R, Gerfen CR, Barbera G, Lin DT. Striatal direct pathway neurons play leading roles in accelerating rotarod motor skill learning. iScience 2022; 25:104245. [PMID: 35494244 PMCID: PMC9046249 DOI: 10.1016/j.isci.2022.104245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Revised: 03/08/2022] [Accepted: 04/07/2022] [Indexed: 10/27/2022] Open
Abstract
Dorsal striatum is important for movement control and motor skill learning. However, it remains unclear how the spatially and temporally distributed striatal medium spiny neuron (MSN) activity in the direct and indirect pathways (D1 and D2 MSNs, respectively) encodes motor skill learning. Combining miniature fluorescence microscopy with an accelerating rotarod procedure, we identified two distinct MSN subpopulations involved in accelerating rotarod learning. In both D1 and D2 MSNs, we observed neurons that displayed activity tuned to acceleration during early stages of trials, as well as movement speed during late stages of trials. We found a distinct evolution trajectory for early-stage neurons during motor skill learning, with the evolution of D1 MSNs correlating strongly with performance improvement. Importantly, optogenetic inhibition of the early-stage neural activity in D1 MSNs, but not D2 MSNs, impaired accelerating rotarod learning. Together, this study provides insight into striatal D1 and D2 MSNs encoding motor skill learning.
Collapse
Affiliation(s)
- Bo Liang
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
- School of Electrical Engineering & Computer Science, College of Engineering & Mines, University of North Dakota, Grand Forks, ND 58202, USA
| | - Lifeng Zhang
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
| | - Yan Zhang
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
| | - Craig T. Werner
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
| | - Nicholas J. Beacher
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
| | - Alex J. Denman
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
| | - Yun Li
- Department of Zoology and Physiology, University of Wyoming, 1000 E University Avenue, Laramie, WY 82071, USA
| | - Rong Chen
- Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, 100 N Greene St, Baltimore, MD 21201, USA
| | - Charles R. Gerfen
- Intramural Research Program, National Institute of Mental Health, National Institutes of Health, Building 49, Room 5A60, Bethesda, MD 20814, USA
| | - Giovanni Barbera
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
| | - Da-Ting Lin
- Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, 333 Cassell Drive, Baltimore, MD 21224, USA
- The Solomon H. Snyder Department of Neuroscience, Johns Hopkins University School of Medicine, 725 N. Wolfe Street, Baltimore, MD 21205, USA
| |
Collapse
|
32
|
Barakchian Z, Vahabie AH, Nili Ahmadabadi M. Implicit Counterfactual Effect in Partial Feedback Reinforcement Learning: Behavioral and Modeling Approach. Front Neurosci 2022; 16:631347. [PMID: 35620668 PMCID: PMC9127865 DOI: 10.3389/fnins.2022.631347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2020] [Accepted: 03/28/2022] [Indexed: 11/13/2022] Open
Abstract
Context remarkably affects learning behavior by adjusting option values according to the distribution of available options. Displaying counterfactual outcomes, the outcomes of the unchosen option alongside the chosen one (i.e., providing complete feedback), would increase the contextual effect by inducing participants to compare the two outcomes during learning. However, when the context only consists of the juxtaposition of several options and there is no such explicit counterfactual factor (i.e., only partial feedback is provided), it is not clear whether and how the contextual effect emerges. In this research, we employ Partial and Complete feedback paradigms in which options are associated with different reward distributions. Our modeling analysis shows that the model that uses the outcome of the chosen option for updating the values of both chosen and unchosen options in opposing directions can better account for the behavioral data. This is also in line with the diffusive effect of dopamine on the striatum. Furthermore, our data show that the contextual effect is not limited to probabilistic rewards, but also extends to magnitude rewards. These results suggest that by extending the counterfactual concept to include the effect of the chosen outcome on the unchosen option, we can better explain why there is a contextual effect in situations in which there is no extra information about the unchosen outcome.
Collapse
Affiliation(s)
- Zahra Barakchian
- Department of Cognitive Neuroscience, Institute for Research in Fundamental Sciences, Tehran, Iran
- *Correspondence: Zahra Barakchian
| | - Abdol-Hossein Vahabie
- Cognitive Systems Laboratory, Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran
- Department of Psychology, Faculty of Psychology and Education, University of Tehran, Tehran, Iran
| | - Majid Nili Ahmadabadi
- Cognitive Systems Laboratory, Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran
| |
Collapse
|
33
|
Badreddine N, Zalcman G, Appaix F, Becq G, Tremblay N, Saudou F, Achard S, Fino E. Spatiotemporal reorganization of corticostriatal networks encodes motor skill learning. Cell Rep 2022; 39:110623. [PMID: 35385722 DOI: 10.1016/j.celrep.2022.110623] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Revised: 01/21/2022] [Accepted: 03/15/2022] [Indexed: 11/28/2022] Open
Abstract
Motor skill learning requires the activity of the dorsal striatum, with a differential global implication of the dorsomedial and dorsolateral territories. We investigate here whether and how specific striatal neurons encode the acquisition and consolidation of a motor skill. Using ex vivo two-photon calcium imaging after rotarod training, we report that highly active (HA) striatal populations arise from distinct spatiotemporal reorganization in the dorsomedial (DMS) and dorsolateral (DLS) striatum networks and are correlated with learning performance. The DMS overall activity decreases in early training, with few and sparsely distributed HA cells, while the DLS shows a progressive and long-lasting formation of HA cell clusters. These reorganizations result from reinforcement of synaptic connections to the DMS and anatomical rearrangements to the DLS. Targeted silencing of DMS or DLS HA cells with the cFos-TRAP strategy strongly impairs individual performance. Our data reveal that discrete domains of striatal populations encode acquisition and long-lasting retention of a motor skill.
Collapse
Affiliation(s)
- Nagham Badreddine
- Université Grenoble Alpes, INSERM, U1216, CHU Grenoble Alpes, CNRS, Grenoble Institut Neurosciences, 38000 Grenoble, France
| | - Gisela Zalcman
- Université Grenoble Alpes, INSERM, U1216, CHU Grenoble Alpes, CNRS, Grenoble Institut Neurosciences, 38000 Grenoble, France
| | - Florence Appaix
- Université Grenoble Alpes, INSERM, U1216, CHU Grenoble Alpes, CNRS, Grenoble Institut Neurosciences, 38000 Grenoble, France
| | - Guillaume Becq
- Université Grenoble Alpes, CNRS, Grenoble INP, GIPSA-Lab, 38000 Grenoble, France
| | - Nicolas Tremblay
- Université Grenoble Alpes, CNRS, Grenoble INP, GIPSA-Lab, 38000 Grenoble, France
| | - Frédéric Saudou
- Université Grenoble Alpes, INSERM, U1216, CHU Grenoble Alpes, CNRS, Grenoble Institut Neurosciences, 38000 Grenoble, France
| | - Sophie Achard
- Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, 38000 Grenoble, France
| | - Elodie Fino
- Université Grenoble Alpes, INSERM, U1216, CHU Grenoble Alpes, CNRS, Grenoble Institut Neurosciences, 38000 Grenoble, France.
| |
Collapse
|
34
|
Bloem B, Huda R, Amemori KI, Abate AS, Krishna G, Wilson AL, Carter CW, Sur M, Graybiel AM. Multiplexed action-outcome representation by striatal striosome-matrix compartments detected with a mouse cost-benefit foraging task. Nat Commun 2022; 13:1541. [PMID: 35318343 PMCID: PMC8941061 DOI: 10.1038/s41467-022-28983-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2021] [Accepted: 02/15/2022] [Indexed: 11/17/2022] Open
Abstract
Learning about positive and negative outcomes of actions is crucial for survival and underpinned by conserved circuits including the striatum. How associations between actions and outcomes are formed is not fully understood, particularly when the outcomes have mixed positive and negative features. We developed a novel foraging (‘bandit’) task requiring mice to maximize rewards while minimizing punishments. By 2-photon Ca++ imaging, we monitored activity of visually identified anterodorsal striatal striosomal and matrix neurons. We found that action-outcome associations for reward and punishment were encoded in parallel in partially overlapping populations. Single neurons could, for one action, encode outcomes of opposing valence. Striosome compartments consistently exhibited stronger representations of reinforcement outcomes than matrix, especially for high reward or punishment prediction errors. These findings demonstrate multiplexing of action-outcome contingencies by single identified striatal neurons and suggest that striosomal neurons are particularly important in action-outcome learning. The role that the striatum plays in tracking the association between actions and combinations of rewarding and aversive outcomes remains unclear. Here, by using both calcium imaging in mice and reinforcement learning models, the authors find that individual striatal neurons can encode associations between actions and multiple, sometimes conflicting, outcomes.
Collapse
Affiliation(s)
- Bernard Bloem
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.,Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.,Sinopia Biosciences, 600W Broadway, Suite 700, San Diego, CA, 92101, USA
| | - Rafiq Huda
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.,Picower Institute for Learning and Memory, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.,Department of Cell Biology and Neuroscience, WM Keck Center for Collaborative Neuroscience, Rutgers University, 604 Allison Rd, Piscataway, NJ, 08854, USA
| | - Ken-Ichi Amemori
- Institute for the Advanced Study of Human Biology, Kyoto University, Yoshida Konoe-cho, Sakyo-ku, Kyoto, 606-8501, Japan
| | - Alex S Abate
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.,Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA
| | - Gayathri Krishna
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.,Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA
| | - Anna L Wilson
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.,Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA
| | - Cody W Carter
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.,Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA
| | - Mriganka Sur
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.,Picower Institute for Learning and Memory, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA
| | - Ann M Graybiel
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA. .,Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 43 Vassar Street, Cambridge, MA, 02139, USA.
| |
Collapse
|
35
|
Bolkan SS, Stone IR, Pinto L, Ashwood ZC, Iravedra Garcia JM, Herman AL, Singh P, Bandi A, Cox J, Zimmerman CA, Cho JR, Engelhard B, Pillow JW, Witten IB. Opponent control of behavior by dorsomedial striatal pathways depends on task demands and internal state. Nat Neurosci 2022; 25:345-357. [PMID: 35260863 PMCID: PMC8915388 DOI: 10.1038/s41593-022-01021-9] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2020] [Accepted: 01/21/2022] [Indexed: 11/27/2022]
Abstract
A classic view of the striatum holds that activity in direct and indirect pathways oppositely modulates motor output. Whether this involves direct control of movement, or reflects a cognitive process underlying movement, remains unresolved. Here we find that strong, opponent control of behavior by the two pathways of the dorsomedial striatum depends on the cognitive requirements of a task. Furthermore, a latent state model (a hidden Markov model with generalized linear model observations) reveals that-even within a single task-the contribution of the two pathways to behavior is state dependent. Specifically, the two pathways have large contributions in one of two states associated with a strategy of evidence accumulation, compared to a state associated with a strategy of repeating previous choices. Thus, both the demands imposed by a task, as well as the internal state of mice when performing a task, determine whether dorsomedial striatum pathways provide strong and opponent control of behavior.
Collapse
Affiliation(s)
- Scott S Bolkan
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Iris R Stone
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Lucas Pinto
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Zoe C Ashwood
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Alison L Herman
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Priyanka Singh
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Akhil Bandi
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Julia Cox
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Jounhong Ryan Cho
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Ben Engelhard
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Jonathan W Pillow
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
- Department of Psychology, Princeton University, Princeton, NJ, USA.
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
- Department of Psychology, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
36
|
Korkutata M, Agrawal L, Lazarus M. Allosteric Modulation of Adenosine A 2A Receptors as a New Therapeutic Avenue. Int J Mol Sci 2022; 23:ijms23042101. [PMID: 35216213 PMCID: PMC8880556 DOI: 10.3390/ijms23042101] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2022] [Revised: 02/11/2022] [Accepted: 02/11/2022] [Indexed: 12/26/2022] Open
Abstract
The therapeutic potential of targeting adenosine A2A receptors (A2ARs) is immense due to their broad expression in the body and central nervous system. The role of A2ARs in cardiovascular function, inflammation, sleep/wake behaviors, cognition, and other primary nervous system functions has been extensively studied. Numerous A2AR agonist and antagonist molecules are reported, many of which are currently in clinical trials or have already been approved for treatment. Allosteric modulators can selectively elicit a physiologic response only where and when the orthosteric ligand is released, which reduces the risk of an adverse effect resulting from A2AR activation. Thus, these allosteric modulators have a potential therapeutic advantage over classical agonist and antagonist molecules. This review focuses on the recent developments regarding allosteric A2AR modulation, which is a promising area for future pharmaceutical research because the list of existing allosteric A2AR modulators and their physiologic effects is still short.
Collapse
Affiliation(s)
- Mustafa Korkutata
- Department of Neurology, Division of Sleep Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA 02215, USA;
| | - Lokesh Agrawal
- Molecular Neuroscience Unit, Okinawa Institute of Science and Technology Graduate University, Kunigami-gun, Onna 904-0412, Japan;
| | - Michael Lazarus
- International Institute for Integrative Sleep Medicine (WPI-IIIS), University of Tsukuba, Tsukuba 305-8575, Japan
- Correspondence: ; Tel.: +81-29-853-3681
| |
Collapse
|
37
|
Kaneko T, Komatsu M, Yamamori T, Ichinohe N, Okano H. Cortical neural dynamics unveil the rhythm of natural visual behavior in marmosets. Commun Biol 2022; 5:108. [PMID: 35115680 PMCID: PMC8814246 DOI: 10.1038/s42003-022-03052-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Accepted: 01/13/2022] [Indexed: 01/13/2023] Open
Abstract
Numerous studies have shown that the visual system consists of functionally distinct ventral and dorsal streams; however, its exact spatial-temporal dynamics during natural visual behavior remain to be investigated. Here, we report cerebral neural dynamics during active visual exploration recorded by an electrocorticographic array covering the entire lateral surface of the marmoset cortex. We found that the dorsal stream was activated before the primary visual cortex with saccades and followed by the alteration of suppression and activation signals along the ventral stream. Similarly, the signal that propagated from the dorsal to ventral visual areas was accompanied by a travelling wave of low frequency oscillations. Such signal dynamics occurred at an average of 220 ms after saccades, which corresponded to the timing when whole-brain activation returned to background levels. We also demonstrated that saccades could occur at any point of signal flow, indicating the parallel computation of motor commands. Overall, this study reveals the neural dynamics of active vision, which are efficiently linked to the natural rhythms of visual exploration.
Collapse
Affiliation(s)
- Takaaki Kaneko
- Laboratory for Marmoset Neural Architecture, RIKEN Center for Brain Science, Saitama, Japan. .,Systems Neuroscience Section, Primate Research Institute, Kyoto University, Aichi, Japan.
| | - Misako Komatsu
- Laboratory for Molecular Analysis of Higher Brain Function, Center for Brain Science, RIKEN, Saitama, Japan
| | - Tetsuo Yamamori
- Laboratory for Molecular Analysis of Higher Brain Function, Center for Brain Science, RIKEN, Saitama, Japan
| | - Noritaka Ichinohe
- Department of Ultrastructural Research, National Institute of Neuroscience, National Center of Neurology and Psychiatry, Tokyo, Japan
| | - Hideyuki Okano
- Laboratory for Marmoset Neural Architecture, RIKEN Center for Brain Science, Saitama, Japan. .,Department of Physiology, Keio University School of Medicine, Tokyo, Japan.
| |
Collapse
|
38
|
Srisathan WA, Naruetharadhol P. A COVID-19 disruption: The great acceleration of digitally planned and transformed behaviors in Thailand. TECHNOLOGY IN SOCIETY 2022; 68:101912. [PMID: 35125560 PMCID: PMC8800537 DOI: 10.1016/j.techsoc.2022.101912] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Revised: 01/24/2022] [Accepted: 01/24/2022] [Indexed: 05/12/2023]
Abstract
The Coronavirus Disease 2019 (COVID-19) pandemic has reshaped human behaviors and switched communication systems from face-to-face to digital communication technologies. This study aimed to examine how digital transformation practices affect human behavioral change digitally, and how perceived COVID-19 severity affects digital transformation practices and behavioral decisions. We use the traditional theory of planned behavior (TPB) to determine new behavioral roles in the digital era, namely digitally planned and transformed behavior. The quantitative survey method was designed to collect cross-sectional data from 550 Thai citizens to provide the conceptual evidence of key proximal measures of digital attitude, digital social norms, digital behavioral control perception, and the digital behavioral decision to predict digitally planned and transformed behavior. The results show that people are more likely to digitalize than before, which predicts the decision to behave digitally at 93.9% of the variability, more than 75% of the predictive power of the total variance suggested by Hair, Ringle, and Sarstedt [1]. However, the higher the COVID-19 severity, the more likely digital transformation is impactful (β = 0.481). This study provides interesting evidence that people struggle to transform their digital behavior during the pandemic. We demonstrate that digital transformation can offer the desired consequences by cultivating digital attitudes, promoting digital social norms, increasing digital behavioral control perception, and enhancing digital behavioral decisions.
Collapse
Affiliation(s)
- Wutthiya A Srisathan
- International College, Khon Kaen University, 123 Mitrphap Road, Khon Kaen, 40002, Thailand
- Center for Sustainable, Innovation and Society, International College, Khon Kaen University, 123 Mitrphap Road, Khon Kaen, 40002, Thailand
| | - Phaninee Naruetharadhol
- International College, Khon Kaen University, 123 Mitrphap Road, Khon Kaen, 40002, Thailand
- Center for Sustainable, Innovation and Society, International College, Khon Kaen University, 123 Mitrphap Road, Khon Kaen, 40002, Thailand
| |
Collapse
|
39
|
Averbeck B, O'Doherty JP. Reinforcement-learning in fronto-striatal circuits. Neuropsychopharmacology 2022; 47:147-162. [PMID: 34354249 PMCID: PMC8616931 DOI: 10.1038/s41386-021-01108-0] [Citation(s) in RCA: 34] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Revised: 07/06/2021] [Accepted: 07/09/2021] [Indexed: 01/03/2023]
Abstract
We review the current state of knowledge on the computational and neural mechanisms of reinforcement-learning with a particular focus on fronto-striatal circuits. We divide the literature in this area into five broad research themes: the target of the learning-whether it be learning about the value of stimuli or about the value of actions; the nature and complexity of the algorithm used to drive the learning and inference process; how learned values get converted into choices and associated actions; the nature of state representations, and of other cognitive machinery that support the implementation of various reinforcement-learning operations. An emerging fifth area focuses on how the brain allocates or arbitrates control over different reinforcement-learning sub-systems or "experts". We will outline what is known about the role of the prefrontal cortex and striatum in implementing each of these functions. We then conclude by arguing that it will be necessary to build bridges from algorithmic level descriptions of computational reinforcement-learning to implementational level models to better understand how reinforcement-learning emerges from multiple distributed neural networks in the brain.
Collapse
Affiliation(s)
| | - John P O'Doherty
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA.
| |
Collapse
|
40
|
Martinez MC, Zold CL, Coletti MA, Murer MG, Belluscio MA. Dorsal striatum coding for the timely execution of action sequences. eLife 2022; 11:74929. [PMID: 36426715 PMCID: PMC9699698 DOI: 10.7554/elife.74929] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2021] [Accepted: 10/27/2022] [Indexed: 11/27/2022] Open
Abstract
The automatic initiation of actions can be highly functional. But occasionally these actions cannot be withheld and are released at inappropriate times, impulsively. Striatal activity has been shown to participate in the timing of action sequence initiation and it has been linked to impulsivity. Using a self-initiated task, we trained adult male rats to withhold a rewarded action sequence until a waiting time interval has elapsed. By analyzing neuronal activity we show that the striatal response preceding the initiation of the learned sequence is strongly modulated by the time subjects wait before eliciting the sequence. Interestingly, the modulation is steeper in adolescent rats, which show a strong prevalence of impulsive responses compared to adults. We hypothesize this anticipatory striatal activity reflects the animals’ subjective reward expectation, based on the elapsed waiting time, while the steeper waiting modulation in adolescence reflects age-related differences in temporal discounting, internal urgency states, or explore–exploit balance.
Collapse
Affiliation(s)
- Maria Cecilia Martinez
- Universidad de Buenos Aires, Facultad de Ciencias Exactas y Naturales, Departamento de Fisiología, Biología Molecular y Celular “Dr. Héctor Maldonado”Buenos AiresArgentina,Universidad de Buenos Aires - CONICET, Instituto de Fisiología y Biofísica “Dr. Bernardo Houssay” (IFIBIO-Houssay), Grupo de Neurociencia de SistemasBuenos AiresArgentina
| | - Camila Lidia Zold
- Universidad de Buenos Aires - CONICET, Instituto de Fisiología y Biofísica “Dr. Bernardo Houssay” (IFIBIO-Houssay), Grupo de Neurociencia de SistemasBuenos AiresArgentina,Universidad de Buenos Aires, Facultad de Ciencias Médicas, Departamento de FisiologíaBuenos AiresArgentina
| | - Marcos Antonio Coletti
- Universidad de Buenos Aires - CONICET, Instituto de Fisiología y Biofísica “Dr. Bernardo Houssay” (IFIBIO-Houssay), Grupo de Neurociencia de SistemasBuenos AiresArgentina,Universidad de Buenos Aires, Facultad de Ciencias Médicas, Departamento de FisiologíaBuenos AiresArgentina
| | - Mario Gustavo Murer
- Universidad de Buenos Aires - CONICET, Instituto de Fisiología y Biofísica “Dr. Bernardo Houssay” (IFIBIO-Houssay), Grupo de Neurociencia de SistemasBuenos AiresArgentina,Universidad de Buenos Aires, Facultad de Ciencias Médicas, Departamento de FisiologíaBuenos AiresArgentina
| | - Mariano Andrés Belluscio
- Universidad de Buenos Aires - CONICET, Instituto de Fisiología y Biofísica “Dr. Bernardo Houssay” (IFIBIO-Houssay), Grupo de Neurociencia de SistemasBuenos AiresArgentina,Universidad de Buenos Aires, Facultad de Ciencias Médicas, Departamento de FisiologíaBuenos AiresArgentina
| |
Collapse
|
41
|
Lee J, Sabatini BL. Striatal indirect pathway mediates exploration via collicular competition. Nature 2021; 599:645-649. [PMID: 34732888 DOI: 10.1038/s41586-021-04055-4] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2020] [Accepted: 09/27/2021] [Indexed: 11/09/2022]
Abstract
The ability to suppress actions that lead to a negative outcome and explore alternative actions is necessary for optimal decision making. Although the basal ganglia have been implicated in these processes1-5, the circuit mechanisms underlying action selection and exploration remain unclear. Here, using a simple lateralized licking task, we show that indirect striatal projection neurons (iSPN) in the basal ganglia contribute to these processes through modulation of the superior colliculus (SC). Optogenetic activation of iSPNs suppresses contraversive licking and promotes ipsiversive licking. Activity in lateral superior colliculus (lSC), a region downstream of the basal ganglia, is necessary for task performance and predicts lick direction. Furthermore, iSPN activation suppresses ipsilateral lSC, but surprisingly excites contralateral lSC, explaining the emergence of ipsiversive licking. Optogenetic inactivation reveals inter-collicular competition whereby each hemisphere of the superior colliculus inhibits the other, thus allowing the indirect pathway to disinhibit the contralateral lSC and trigger licking. Finally, inactivating iSPNs impairs suppression of devalued but previously rewarded licking and reduces exploratory licking. Our results reveal that iSPNs engage the competitive interaction between lSC hemispheres to trigger a motor action and suggest a general circuit mechanism for exploration during action selection.
Collapse
Affiliation(s)
- Jaeeon Lee
- Howard Hughes Medical Institute, Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Bernardo L Sabatini
- Howard Hughes Medical Institute, Department of Neurobiology, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
42
|
Torigoe M, Islam T, Kakinuma H, Fung CCA, Isomura T, Shimazaki H, Aoki T, Fukai T, Okamoto H. Zebrafish capable of generating future state prediction error show improved active avoidance behavior in virtual reality. Nat Commun 2021; 12:5712. [PMID: 34588436 PMCID: PMC8481257 DOI: 10.1038/s41467-021-26010-7] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2020] [Accepted: 09/06/2021] [Indexed: 11/08/2022] Open
Abstract
Animals make decisions under the principle of reward value maximization and surprise minimization. It is still unclear how these principles are represented in the brain and are reflected in behavior. We addressed this question using a closed-loop virtual reality system to train adult zebrafish for active avoidance. Analysis of the neural activity of the dorsal pallium during training revealed neural ensembles assigning rules to the colors of the surrounding walls. Additionally, one third of fish generated another ensemble that becomes activated only when the real perceived scenery shows discrepancy from the predicted favorable scenery. The fish with the latter ensemble escape more efficiently than the fish with the former ensembles alone, even though both fish have successfully learned to escape, consistent with the hypothesis that the latter ensemble guides zebrafish to take action to minimize this prediction error. Our results suggest that zebrafish can use both principles of goal-directed behavior, but with different behavioral consequences depending on the repertoire of the adopted principles.
Collapse
Affiliation(s)
- Makio Torigoe
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
| | - Tanvir Islam
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
- RIKEN CBS-Kao Collaboration Center, Wako, Saitama, 351-0198, Japan
| | - Hisaya Kakinuma
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
- RIKEN CBS-Kao Collaboration Center, Wako, Saitama, 351-0198, Japan
| | - Chi Chung Alan Fung
- Neural Coding and Brain Computing Unit, Okinawa Institute of Science and Technology, Onna-son, Okinawa, 904-0495, Japan
| | - Takuya Isomura
- Brain Intelligence Theory Unit, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
| | - Hideaki Shimazaki
- Center for Human Nature, Artificial Intelligence, and Neuroscience (CHAIN), Hokkaido University, Sapporo, Hokkaido, 060-0812, Japan
| | - Tazu Aoki
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
| | - Tomoki Fukai
- Neural Coding and Brain Computing Unit, Okinawa Institute of Science and Technology, Onna-son, Okinawa, 904-0495, Japan
| | - Hitoshi Okamoto
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan.
- RIKEN CBS-Kao Collaboration Center, Wako, Saitama, 351-0198, Japan.
| |
Collapse
|
43
|
A Role of BDNF in the Depression Pathogenesis and a Potential Target as Antidepressant: The Modulator of Stress Sensitivity "Shati/Nat8l-BDNF System" in the Dorsal Striatum. Pharmaceuticals (Basel) 2021; 14:ph14090889. [PMID: 34577589 PMCID: PMC8469819 DOI: 10.3390/ph14090889] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2021] [Revised: 08/27/2021] [Accepted: 08/27/2021] [Indexed: 12/12/2022] Open
Abstract
Depression is one of the most common mental diseases, with increasing numbers of patients globally each year. In addition, approximately 30% of patients with depression are resistant to any treatment and do not show an expected response to first-line antidepressant drugs. Therefore, novel antidepressant agents and strategies are required. Although depression is triggered by post-birth stress, while some individuals show the pathology of depression, others remain resilient. The molecular mechanisms underlying stress sensitivity remain unknown. Brain-derived neurotrophic factor (BDNF) has both pro- and anti-depressant effects, dependent on brain region. Considering the strong region-specific contribution of BDNF to depression pathogenesis, the regulation of BDNF in the whole brain is not a beneficial strategy for the treatment of depression. We reviewed a novel finding of BDNF function in the dorsal striatum, which induces vulnerability to social stress, in addition to recent research progress regarding the brain regional functions of BDNF, including the prefrontal cortex, hippocampus, and nucleus accumbens. Striatal BDNF is regulated by Shati/Nat8l, an N-acetyltransferase through epigenetic regulation. Targeting of Shati/Nat8l would allow BDNF to be striatum-specifically regulated, and the striatal Shati/Nat8l-BDNF pathway could be a promising novel therapeutic agent for the treatment of depression by modulating sensitivity to stress.
Collapse
|
44
|
Handa T, Harukuni R, Fukai T. Concomitant Processing of Choice and Outcome in Frontal Corticostriatal Ensembles Correlates with Performance of Rats. Cereb Cortex 2021; 31:4357-4375. [PMID: 33914862 PMCID: PMC8328202 DOI: 10.1093/cercor/bhab091] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2020] [Revised: 03/19/2021] [Accepted: 03/20/2021] [Indexed: 11/30/2022] Open
Abstract
The frontal cortex-basal ganglia network plays a pivotal role in adaptive goal-directed behaviors. Medial frontal cortex (MFC) encodes information about choices and outcomes into sequential activation of neural population, or neural trajectory. While MFC projects to the dorsal striatum (DS), whether DS also displays temporally coordinated activity remains unknown. We studied this question by simultaneously recording neural ensembles in the MFC and DS of rodents performing an outcome-based alternative choice task. We found that the two regions exhibited highly parallel evolution of neural trajectories, transforming choice information into outcome-related information. When the two trajectories were highly correlated, spike synchrony was task-dependently modulated in some MFC-DS neuron pairs. Our results suggest that neural trajectories concomitantly process decision-relevant information in MFC and DS with increased spike synchrony between these regions.
Collapse
Affiliation(s)
- Takashi Handa
- Department of Behavior and Brain Organization, Center Advanced European Study and Research (Caesar), Bonn 53175, Germany
- Department of Neurobiology, Graduate School of Biomedical and Health Sciences (Medicine), Hiroshima University, Hiroshima 734-8553, Japan
- Laboratory for Neural Coding and Brain Computing, RIKEN Center for Brain Science, Wako, Saitama 351-0198, Japan
| | - Rie Harukuni
- Laboratory for Neural Coding and Brain Computing, RIKEN Center for Brain Science, Wako, Saitama 351-0198, Japan
| | - Tomoki Fukai
- Laboratory for Neural Coding and Brain Computing, RIKEN Center for Brain Science, Wako, Saitama 351-0198, Japan
- Neural Coding and Brain Computing Unit, Okinawa Institute of Science and Technology, Okinawa 904-0495, Japan
| |
Collapse
|
45
|
Over-representation of fundamental decision variables in the prefrontal cortex underlies decision bias. Neurosci Res 2021; 173:1-13. [PMID: 34274406 DOI: 10.1016/j.neures.2021.07.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Revised: 06/15/2021] [Accepted: 07/13/2021] [Indexed: 11/24/2022]
Abstract
The brain is organized into anatomically distinct structures consisting of a variety of projection neurons. While such evolutionarily conserved neural circuit organization underlies the innate ability of animals to swiftly adapt to environments, they can cause biased cognition and behavior. Although recent studies have begun to address the causal importance of projection-neuron types as distinct computational units, it remains unclear how projection types are functionally organized in encoding variables during cognitive tasks. This review focuses on the neural computation of decision making in the prefrontal cortex and discusses what decision variables are encoded by single neurons, neuronal populations, and projection type, alongside how specific projection types constrain decision making. We focus particularly on "over-representations" of distinct decision variables in the prefrontal cortex that reflect the biological and subjective significance of the variables for the decision makers. We suggest that task-specific over-representation in the prefrontal cortex involves the refinement of the given decision making, while generalized over-representation of fundamental decision variables is associated with suboptimal decision biases, including pathological ones such as those in patients with psychiatric disorders. Such over-representation of the fundamental decision variables in the prefrontal cortex appear to be tightly constrained by afferent and efferent connections that can be optogenetically intervened on. These ideas may provide critical insights into potential therapeutic targets for psychiatric disorders, including addiction and depression.
Collapse
|
46
|
Hori Y, Nagai Y, Mimura K, Suhara T, Higuchi M, Bouret S, Minamimoto T. D1- and D2-like receptors differentially mediate the effects of dopaminergic transmission on cost-benefit evaluation and motivation in monkeys. PLoS Biol 2021; 19:e3001055. [PMID: 34197448 PMCID: PMC8248602 DOI: 10.1371/journal.pbio.3001055] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Accepted: 05/27/2021] [Indexed: 11/29/2022] Open
Abstract
It has been widely accepted that dopamine (DA) plays a major role in motivation, yet the specific contribution of DA signaling at D1-like receptor (D1R) and D2-like receptor (D2R) to cost–benefit trade-off remains unclear. Here, by combining pharmacological manipulation of DA receptors (DARs) and positron emission tomography (PET) imaging, we assessed the relationship between the degree of D1R/D2R blockade and changes in benefit- and cost-based motivation for goal-directed behavior of macaque monkeys. We found that the degree of blockade of either D1R or D2R was associated with a reduction of the positive impact of reward amount and increasing delay discounting. Workload discounting was selectively increased by D2R antagonism. In addition, blocking both D1R and D2R had a synergistic effect on delay discounting but an antagonist effect on workload discounting. These results provide fundamental insight into the distinct mechanisms of DA action in the regulation of the benefit- and cost-based motivation, which have important implications for motivational alterations in both neurological and psychiatric disorders. Using quantitatively controlled pharmacological manipulations, this study teases apart the role of D1- and D2-like dopamine receptors in motivation and goal-directed behavior in monkeys, revealing complementary roles of two dopamine receptor subtypes in the computation of the cost/benefit trade-off to guide action.
Collapse
Affiliation(s)
- Yukiko Hori
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and Technology, Chiba, Japan
| | - Yuji Nagai
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and Technology, Chiba, Japan
| | - Koki Mimura
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and Technology, Chiba, Japan
| | - Tetsuya Suhara
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and Technology, Chiba, Japan
| | - Makoto Higuchi
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and Technology, Chiba, Japan
| | - Sebastien Bouret
- Team Motivation Brain & Behavior, Institut du Cerveau et de la Moelle épinière (ICM), Centre National de la Recherche Scientifique (CNRS), Hôpital Pitié Salpêtrière, Paris, France
| | - Takafumi Minamimoto
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and Technology, Chiba, Japan
- * E-mail:
| |
Collapse
|
47
|
Ohta H, Satori K, Takarada Y, Arake M, Ishizuka T, Morimoto Y, Takahashi T. The asymmetric learning rates of murine exploratory behavior in sparse reward environments. Neural Netw 2021; 143:218-229. [PMID: 34157646 DOI: 10.1016/j.neunet.2021.05.030] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2021] [Revised: 04/16/2021] [Accepted: 05/26/2021] [Indexed: 11/29/2022]
Abstract
Goal-oriented behaviors of animals can be modeled by reinforcement learning algorithms. Such algorithms predict future outcomes of selected actions utilizing action values and updating those values in response to the positive and negative outcomes. In many models of animal behavior, the action values are updated symmetrically based on a common learning rate, that is, in the same way for both positive and negative outcomes. However, animals in environments with scarce rewards may have uneven learning rates. To investigate the asymmetry in learning rates in reward and non-reward, we analyzed the exploration behavior of mice in five-armed bandit tasks using a Q-learning model with differential learning rates for positive and negative outcomes. The positive learning rate was significantly higher in a scarce reward environment than in a rich reward environment, and conversely, the negative learning rate was significantly lower in the scarce environment. The positive to negative learning rate ratio was about 10 in the scarce environment and about 2 in the rich environment. This result suggests that when the reward probability was low, the mice tend to ignore failures and exploit the rare rewards. Computational modeling analysis revealed that the increased learning rates ratio could cause an overestimation of and perseveration on rare-rewarding events, increasing total reward acquisition in the scarce environment but disadvantaging impartial exploration.
Collapse
Affiliation(s)
- Hiroyuki Ohta
- Department of Pharmacology, National Defense Medical College, Saitama, 359-8513, Japan.
| | | | - Yu Takarada
- Tokyo Denki University, Saitama, 350-0394, Japan
| | - Masashi Arake
- Department of Physiology, National Defense Medical College, Saitama, 359-8513, Japan
| | - Toshiaki Ishizuka
- Department of Pharmacology, National Defense Medical College, Saitama, 359-8513, Japan
| | - Yuji Morimoto
- Department of Physiology, National Defense Medical College, Saitama, 359-8513, Japan
| | | |
Collapse
|
48
|
Rangel-Barajas C, Boehm SL, Logrip ML. Altered excitatory transmission in striatal neurons after chronic ethanol consumption in selectively bred crossed high alcohol-preferring mice. Neuropharmacology 2021; 190:108564. [PMID: 33857521 PMCID: PMC8293703 DOI: 10.1016/j.neuropharm.2021.108564] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2021] [Revised: 03/24/2021] [Accepted: 04/06/2021] [Indexed: 12/12/2022]
Abstract
Genetic predisposition to heavy drinking is a risk factor for alcohol misuse. We used selectively bred crossed high alcohol-preferring (cHAP) mice to study sex differences in alcohol drinking and its effect on glutamatergic activity in dorsolateral (DLS) and dorsomedial (DMS) striatum. We performed whole-cell patch-clamp recording in neurons from male and female cHAP mice with 5-week alcohol drinking history and alcohol-naïve controls. In DMS, alcohol-naïve males' neurons displayed lower cell capacitance and higher membrane resistance than females' neurons, both effects reversed by drinking. Conversely, in DLS neurons, drinking history increased capacitance only in males and changed membrane resistance only in females. Altered biophysical membrane properties were accompanied by disrupted glutamatergic transmission. Drinking history increased spontaneous excitatory postsynaptic current (sEPSC) amplitude in DMS and frequency in DLS female neurons, compared to alcohol-naïve females, without effect in males. Acute ethanol differentially impacted DMS and DLS neurons by sex and drinking history. In DMS, acute alcohol significantly increased sEPSC frequency only in neurons from alcohol-naïve females, an effect that disappeared after drinking history. In DLS, acute alcohol had opposing effects in males and females based on drinking history. Estrous cycle also impacted DMS and DLS neurons differently: sEPSC amplitudes were higher in DMS cells from drinking history than alcohol-naïve females, whereas estrous cycle, not drinking history, modified DLS firing rate. Our data show sex differences in cHAP ethanol consumption and neurophysiology, suggesting differential dysregulation of glutamatergic drive onto DMS and DLS after chronic ethanol consumption.
Collapse
Affiliation(s)
- Claudia Rangel-Barajas
- Department of Psychology, Indiana University - Purdue University Indianapolis, Indianapolis, IN, 46202, USA
| | - Stephen L Boehm
- Department of Psychology, Indiana University - Purdue University Indianapolis, Indianapolis, IN, 46202, USA; Indiana Alcohol Research Center, Indiana University School of Medicine, Indianapolis, IN, 46202, USA; Stark Neurosciences Research Institute, Indiana University School of Medicine, Indianapolis, IN, 46202, USA
| | - Marian L Logrip
- Department of Psychology, Indiana University - Purdue University Indianapolis, Indianapolis, IN, 46202, USA; Indiana Alcohol Research Center, Indiana University School of Medicine, Indianapolis, IN, 46202, USA; Stark Neurosciences Research Institute, Indiana University School of Medicine, Indianapolis, IN, 46202, USA.
| |
Collapse
|
49
|
Cataldi S, Stanley AT, Miniaci MC, Sulzer D. Interpreting the role of the striatum during multiple phases of motor learning. FEBS J 2021; 289:2263-2281. [PMID: 33977645 DOI: 10.1111/febs.15908] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Revised: 03/28/2021] [Accepted: 04/30/2021] [Indexed: 01/11/2023]
Abstract
The synaptic pathways in the striatum are central to basal ganglia functions including motor control, learning and organization, action selection, acquisition of motor skills, cognitive function, and emotion. Here, we review the role of the striatum and its connections in motor learning and performance. The development of new techniques to record neuronal activity and animal models of motor disorders using neurotoxin, pharmacological, and genetic manipulations are revealing pathways that underlie motor performance and motor learning, as well as how they are altered by pathophysiological mechanisms. We discuss approaches that can be used to analyze complex motor skills, particularly in rodents, and identify specific questions central to understanding how striatal circuits mediate motor learning.
Collapse
Affiliation(s)
- Stefano Cataldi
- Departments of Psychiatry, Neurology, Pharmacology, Biology, Columbia University, New York, NY, USA.,Division of Molecular Therapeutics, New York State Psychiatric Institute, NY, USA
| | - Adrien T Stanley
- Departments of Psychiatry, Neurology, Pharmacology, Biology, Columbia University, New York, NY, USA.,Division of Molecular Therapeutics, New York State Psychiatric Institute, NY, USA
| | | | - David Sulzer
- Departments of Psychiatry, Neurology, Pharmacology, Biology, Columbia University, New York, NY, USA.,Division of Molecular Therapeutics, New York State Psychiatric Institute, NY, USA
| |
Collapse
|
50
|
Mukherjee D, Gonzales BJ, Ashwal-Fluss R, Turm H, Groysman M, Citri A. Egr2 induction in spiny projection neurons of the ventrolateral striatum contributes to cocaine place preference in mice. eLife 2021; 10:65228. [PMID: 33724178 PMCID: PMC8057818 DOI: 10.7554/elife.65228] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2020] [Accepted: 03/15/2021] [Indexed: 12/16/2022] Open
Abstract
Drug addiction develops due to brain-wide plasticity within neuronal ensembles, mediated by dynamic gene expression. Though the most common approach to identify such ensembles relies on immediate early gene expression, little is known of how the activity of these genes is linked to modified behavior observed following repeated drug exposure. To address this gap, we present a broad-to-specific approach, beginning with a comprehensive investigation of brain-wide cocaine-driven gene expression, through the description of dynamic spatial patterns of gene induction in subregions of the striatum, and finally address functionality of region-specific gene induction in the development of cocaine preference. Our findings reveal differential cell-type specific dynamic transcriptional recruitment patterns within two subdomains of the dorsal striatum following repeated cocaine exposure. Furthermore, we demonstrate that induction of the IEG Egr2 in the ventrolateral striatum, as well as the cells within which it is expressed, are required for the development of cocaine seeking. The human brain is ever changing, constantly rewiring itself in response to new experiences, knowledge or information from the environment. Addictive drugs such as cocaine can hijack the genetic mechanisms responsible for this plasticity, creating dangerous, obsessive drug-seeking and consuming behaviors. Cocaine-induced plasticity is difficult to apprehend, however, as brain regions or even cell populations can react differently to the compound. For instance, sub-regions in the striatum – the brain area that responds to rewards and helps to plan movement – show distinct responses during progressive exposure to cocaine. And while researchers know that the drug immediately changes how neurons switch certain genes on and off, it is still unclear how these genetic modifications later affect behavior. Mukherjee, Gonzales et al. explored these questions at different scales, first focusing on how progressive cocaine exposure changed the way various gene programs were activated across the entire brain. This revealed that programs in the striatum were the most affected by the drug. Examining this region more closely showed that cocaine switches on genes in specific ‘spiny projection’ neuron populations, depending on where these cells are located and the drug history of the mouse. Finally, Mukherjee, Gonzales et al. used genetically modified mice to piece together cocaine exposure, genetic changes and modifications in behavior. These experiments revealed that the drive to seek cocaine depended on activation of the Egr2 gene in populations of spiny projection neurons in a specific sub-region of the striatum. The gene, which codes for a protein that regulates how genes are switched on and off, was itself strongly activated by cocaine intake. Cocaine addiction can have devastating consequences for individuals. Grasping how this drug alters the brain could pave the way for new treatments, while also providing information on the basic mechanisms underlying brain plasticity.
Collapse
Affiliation(s)
- Diptendu Mukherjee
- The Edmond and Lily Safra Center for Brain Sciences, Jerusalem, Israel.,Institute of Life Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Ben Jerry Gonzales
- The Edmond and Lily Safra Center for Brain Sciences, Jerusalem, Israel.,Institute of Life Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Reut Ashwal-Fluss
- The Edmond and Lily Safra Center for Brain Sciences, Jerusalem, Israel
| | - Hagit Turm
- The Edmond and Lily Safra Center for Brain Sciences, Jerusalem, Israel.,Institute of Life Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Maya Groysman
- The Edmond and Lily Safra Center for Brain Sciences, Jerusalem, Israel
| | - Ami Citri
- The Edmond and Lily Safra Center for Brain Sciences, Jerusalem, Israel.,Institute of Life Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel.,Program in Child and Brain Development, Canadian Institute for Advanced Research, MaRS Centre, Toronto, Canada
| |
Collapse
|