1
|
Lee H, Kim HF, Hikosaka O. Implication of regional selectivity of dopamine deficits in impaired suppressing of involuntary movements in Parkinson's disease. Neurosci Biobehav Rev 2024; 162:105719. [PMID: 38759470 PMCID: PMC11167649 DOI: 10.1016/j.neubiorev.2024.105719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 04/26/2024] [Accepted: 05/09/2024] [Indexed: 05/19/2024]
Abstract
To improve the initiation and speed of intended action, one of the crucial mechanisms is suppressing unwanted movements that interfere with goal-directed behavior, which is observed relatively aberrant in Parkinson's disease patients. Recent research has highlighted that dopamine deficits in Parkinson's disease predominantly occur in the caudal lateral part of the substantia nigra pars compacta (SNc) in human patients. We previously found two parallel circuits within the basal ganglia, primarily divided into circuits mediated by the rostral medial part and caudal lateral part of the SNc dopamine neurons. We have further discovered that the indirect pathway in caudal basal ganglia circuits, facilitated by the caudal lateral part of the SNc dopamine neurons, plays a critical role in suppressing unnecessary involuntary movements when animals perform voluntary goal-directed actions. We thus explored recent research in humans and non-human primates focusing on the distinct functions and networks of the caudal lateral part of the SNc dopamine neurons to elucidate the mechanisms involved in the impairment of suppressing involuntary movements in Parkinson's disease patients.
Collapse
Affiliation(s)
- Hyunchan Lee
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892-4435, USA.
| | - Hyoung F Kim
- School of Biological Sciences, College of Natural Sciences, Seoul National University (SNU), Seoul 08826, Republic of Korea
| | - Okihide Hikosaka
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892-4435, USA
| |
Collapse
|
2
|
Resnik Robida K, Politakis VA, Oblak A, Ozimič AS, Burger H, Pirtošek Z, Bon J. Detecting Subtle Cognitive Impairment in Patients with Parkinson's Disease and Normal Cognition: A Novel Cognitive Control Challenge Task (C3T). Brain Sci 2023; 13:961. [PMID: 37371439 DOI: 10.3390/brainsci13060961] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Revised: 06/07/2023] [Accepted: 06/14/2023] [Indexed: 06/29/2023] Open
Abstract
Patients with Parkinson's disease (PD) often show early deficits in cognitive control, with primary difficulties in flexibility and relatively intact stable representations. The aim of our study was to assess executive function using an ecologically valid approach that combines measures of stability and flexibility. Fourteen patients without cognitive deficits and sixteen comparable control subjects completed a standardized neuropsychological test battery and a newly developed cognitive control challenge task (C3T). We found that the accuracy of C3T performance decreased with age in healthy participants and remained impaired in PD patients regardless of age. In addition, PD patients showed significantly lower overall performance for cognitive control tasks than healthy controls, even when they scored in the normal range on standardized neuropsychological tests. PD Patients responded significantly faster than healthy control subjects regarding flexible cognitive control tasks due to their impulsivity. Correlations showed that the C3T task targets multiple cognitive systems, including working memory, inhibition, and task switching, providing a reliable measure of complex cognitive control. C3T could be a valuable tool for characterizing cognitive deficits associated with PD and appears to be a more sensitive measure than standardized neuropsychological tests. A different assessment approach could potentially detect early signs of the disease and identify opportunities for early intervention with neuroprotective therapies.
Collapse
Affiliation(s)
- Karmen Resnik Robida
- University Rehabilitation Institute, University of Ljubljana, SI-1000 Ljubljana, Slovenia
| | - Vida Ana Politakis
- University Rehabilitation Institute, University of Ljubljana, SI-1000 Ljubljana, Slovenia
| | - Aleš Oblak
- University Psychiatric Clinic Ljubljana, SI-1260 Ljubljana, Slovenia
| | - Anka Slana Ozimič
- Department of Psychology, Faculty of Arts, University of Ljubljana, SI-1000 Ljubljana, Slovenia
| | - Helena Burger
- University Rehabilitation Institute, University of Ljubljana, SI-1000 Ljubljana, Slovenia
- Department of Physical Medicine and Rehabilitation, Faculty of Medicine, University of Ljubljana, SI-1000 Ljubljana, Slovenia
| | - Zvezdan Pirtošek
- Department of Neurology, University Medical Centre Ljubljana, SI-1000 Ljubljana, Slovenia
- MRD Center for Active and Healthy Ageing, University of Ljubljana, SI-1000 Ljubljana, Slovenia
| | - Jurij Bon
- University Psychiatric Clinic Ljubljana, SI-1260 Ljubljana, Slovenia
- Department of Neurology, University Medical Centre Ljubljana, SI-1000 Ljubljana, Slovenia
- Department of Psychiatry, Faculty of Medicine, University of Ljubljana, SI-1000 Ljubljana, Slovenia
| |
Collapse
|
3
|
Lee H, Hikosaka O. Lateral habenula neurons signal step-by-step changes of reward prediction. iScience 2022; 25:105440. [DOI: 10.1016/j.isci.2022.105440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 08/15/2022] [Accepted: 10/20/2022] [Indexed: 11/06/2022] Open
|
4
|
Ortu D, Bugg RM. Response Systems, Antagonistic Responses, and the Behavioral Repertoire. Front Behav Neurosci 2022; 15:778420. [PMID: 35095436 PMCID: PMC8792759 DOI: 10.3389/fnbeh.2021.778420] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2021] [Accepted: 12/14/2021] [Indexed: 11/13/2022] Open
Abstract
While response systems are often mentioned in the behavioral and physiological literature, an explicit discussion of what response systems are is lacking. Here we argue that response systems can be understood as an interaction between anatomically constrained behavioral topographies occasioned by currently present stimuli and a history of reinforcement. “New” response systems can develop during the lifetime as the organism gains instrumental control of new fine-grained topographies. Within this framework, antagonistic responses compete within each response system based on environmental stimulation, and competition is resolved at the striatum-thalamo-cortical loops level. While response systems can be by definition independent from one another, separate systems are often recruited at the same time to engage in complex responses, which themselves may be selected by reinforcement as functional units.
Collapse
|
5
|
Subramanian A, Chitlangia S, Baths V. Reinforcement learning and its connections with neuroscience and psychology. Neural Netw 2021; 145:271-287. [PMID: 34781215 DOI: 10.1016/j.neunet.2021.10.003] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Revised: 09/26/2021] [Accepted: 10/01/2021] [Indexed: 11/19/2022]
Abstract
Reinforcement learning methods have recently been very successful at performing complex sequential tasks like playing Atari games, Go and Poker. These algorithms have outperformed humans in several tasks by learning from scratch, using only scalar rewards obtained through interaction with their environment. While there certainly has been considerable independent innovation to produce such results, many core ideas in reinforcement learning are inspired by phenomena in animal learning, psychology and neuroscience. In this paper, we comprehensively review a large number of findings in both neuroscience and psychology that evidence reinforcement learning as a promising candidate for modeling learning and decision making in the brain. In doing so, we construct a mapping between various classes of modern RL algorithms and specific findings in both neurophysiological and behavioral literature. We then discuss the implications of this observed relationship between RL, neuroscience and psychology and its role in advancing research in both AI and brain science.
Collapse
Affiliation(s)
- Ajay Subramanian
- Department of Psychology, New York University, New York, New York, 10003, USA; Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| | - Sharad Chitlangia
- Amazon; Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| | - Veeky Baths
- Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India; Department of Biological Sciences, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| |
Collapse
|
6
|
Gabitov E, Lungu O, Albouy G, Doyon J. Movement errors during skilled motor performance engage distinct prediction error mechanisms. Commun Biol 2020; 3:763. [PMID: 33311566 PMCID: PMC7732826 DOI: 10.1038/s42003-020-01465-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Accepted: 11/06/2020] [Indexed: 12/23/2022] Open
Abstract
The brain detects deviations from intended behaviors by estimating the mismatch between predicted and actual outcomes. Axiomatic to these computations are salience and valence prediction error signals, which alert the brain to the occurrence and value of unexpected events. Despite the theoretical assertion of these prediction error signals, it is unknown whether and how brain mechanisms underlying their computations support error processing during skilled motor behavior. Here we demonstrate, with functional magnetic resonance imaging, that internal detection, i.e., without externally-provided feedback, of self-generated movement errors evokes instantaneous activity increases within the salience network and delayed lingering decreases within the nucleus accumbens - a key structure in the reward valuation pathway. A widespread suppression within the sensorimotor network was also observed. Our findings suggest that neural computations of salience and valence prediction errors during skilled motor behaviors operate on different time-scales and, therefore, may contribute differentially to immediate and longer-term adaptive processes.
Collapse
Affiliation(s)
- Ella Gabitov
- McConnell Brain Imaging Center, Montreal Neurological Institute, Montreal, QC, H3A 2B4, Canada. .,Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, QC, H3A 2B4, Canada.
| | - Ovidiu Lungu
- McConnell Brain Imaging Center, Montreal Neurological Institute, Montreal, QC, H3A 2B4, Canada.,Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, QC, H3A 2B4, Canada
| | - Geneviève Albouy
- Movement Control and Neuroplasticity Research Group, Department of Movement Sciences, KU Leuven, 3000, Leuven, Belgium
| | - Julien Doyon
- McConnell Brain Imaging Center, Montreal Neurological Institute, Montreal, QC, H3A 2B4, Canada. .,Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, QC, H3A 2B4, Canada.
| |
Collapse
|
7
|
Lin WC, Delevich K, Wilbrecht L. A role for adaptive developmental plasticity in learning and decision making. Curr Opin Behav Sci 2020; 36:48-54. [DOI: 10.1016/j.cobeha.2020.07.010] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
|
8
|
Balice-Gordon R, Honey GD, Chatham C, Arce E, Duvvuri S, Naylor MG, Liu W, Xie Z, DeMartinis N, Harel BT, Braley GH, Kozak R, Park L, Gray DL. A Neurofunctional Domains Approach to Evaluate D1/D5 Dopamine Receptor Partial Agonism on Cognition and Motivation in Healthy Volunteers With Low Working Memory Capacity. Int J Neuropsychopharmacol 2020; 23:287-299. [PMID: 32055822 PMCID: PMC7251631 DOI: 10.1093/ijnp/pyaa007] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/24/2019] [Revised: 01/02/2020] [Accepted: 02/12/2020] [Indexed: 12/21/2022] Open
Abstract
BACKGROUND Dopamine D1 receptor signaling plays key roles in core domains of neural function, including cognition and reward processing; however, many questions remain about the functions of circuits modulated by dopamine D1 receptor, largely because clinically viable, selective agonists have yet to be tested in humans. METHODS Using a novel, exploratory neurofunctional domains study design, we assessed the safety, tolerability, pharmacodynamics, and pharmacokinetics of PF-06412562, a selective D1/D5R partial agonist, in healthy male volunteers who met prespecified criteria for low working memory capacity. Functional magnetic resonance imaging, electrophysiologic endpoints, and behavioral paradigms were used to assess working memory, executive function, and motivation/reward processing following multiple-dose administration of PF-06412562. A total of 77 patients were assigned PF-06412562 (3 mg twice daily and 15 mg twice daily) or placebo administered for 5 to 7 days. Due to the exploratory nature of the study, it was neither powered for any specific treatment effect nor corrected for multiple comparisons. RESULTS Nominally significant improvements from baseline in cognitive endpoints were observed in all 3 groups; however, improvements in PF-06412562-treated patients were less than in placebo-treated participants. Motivation/reward processing endpoints were variable. PF-06412562 was safe and well tolerated, with no serious adverse events, severe adverse events, or adverse events leading to dose reduction or temporary discontinuation except for 1 permanent discontinuation due to increased orthostatic heart rate. CONCLUSIONS PF-06412562, in the dose range and patient population explored in this study, did not improve cognitive function or motivation/reward processing more than placebo over the 5- to 7-day treatment period. CLINICALTRIALS.GOV IDENTIFIER NCT02306876.
Collapse
Affiliation(s)
| | - Garry D Honey
- Pfizer Worldwide Research and Development, Cambridge, MA
| | | | - Estibaliz Arce
- Pfizer Worldwide Research and Development, Cambridge, MA
| | | | | | - Wenlei Liu
- Pfizer Worldwide Research and Development, Cambridge, MA
| | - Zhiyong Xie
- Pfizer Worldwide Research and Development, Cambridge, MA
| | | | - Brian T Harel
- Pfizer Worldwide Research and Development, Cambridge, MA
| | | | - Rouba Kozak
- Pfizer Worldwide Research and Development, Cambridge, MA
| | - Lovingly Park
- California Clinical Trials Medical Group/PAREXEL International, Glendale, CA
| | - David L Gray
- Pfizer Worldwide Research and Development, Cambridge, MA
| |
Collapse
|
9
|
Rubin JE, Vich C, Clapp M, Noneman K, Verstynen T. The credit assignment problem in cortico‐basal ganglia‐thalamic networks: A review, a problem and a possible solution. Eur J Neurosci 2020; 53:2234-2253. [DOI: 10.1111/ejn.14745] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2020] [Revised: 03/23/2020] [Accepted: 03/25/2020] [Indexed: 12/21/2022]
Affiliation(s)
- Jonathan E. Rubin
- Department of Mathematics Center for the Neural Basis of Cognition University of Pittsburgh Pittsburgh PA USA
| | - Catalina Vich
- Department de Matemàtiques i Informàtica Institute of Applied Computing and Community Code Universitat de les Illes Balears Palma Spain
| | - Matthew Clapp
- Carnegie Mellon Neuroscience Institute Carnegie Mellon University Pittsburgh PA USA
| | - Kendra Noneman
- Micron School of Materials Science and Engineering Boise State University Boise ID USA
| | - Timothy Verstynen
- Carnegie Mellon Neuroscience Institute Carnegie Mellon University Pittsburgh PA USA
- Department of Psychology Center for the Neural Basis of Cognition Carnegie Mellon University Pittsburgh PA USA
| |
Collapse
|
10
|
Multiple neuronal circuits for variable object-action choices based on short- and long-term memories. Proc Natl Acad Sci U S A 2019; 116:26313-26320. [PMID: 31871157 DOI: 10.1073/pnas.1902283116] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/28/2023] Open
Abstract
At each time in our life, we choose one or few behaviors, while suppressing many other behaviors. This is the basic mechanism in the basal ganglia, which is done by tonic inhibition and selective disinhibition. Dysfunctions of the basal ganglia then cause 2 types of disorders (difficulty in initiating necessary actions and difficulty in suppressing unnecessary actions) that occur in Parkinson's disease. The basal ganglia generate such opposite outcomes through parallel circuits: The direct pathway for initiation and indirect pathway for suppression. Importantly, the direct pathway processes good information and the indirect pathway processes bad information, which enables the choice of good behavior and the rejection of bad behavior. This is mainly enabled by dopaminergic inputs to these circuits. However, the value judgment is complex because the world is complex. Sometimes, the value must be based on recent events, thus is based on short-term memories. Or, the value must be based on historical events, thus is based on long-term memories. Such memory-based value judgment is generated by another parallel circuit originating from the caudate head and caudate tail. These circuit-information mechanisms allow other brain areas (e.g., prefrontal cortex) to contribute to decisions by sending information to these basal ganglia circuits. Moreover, the basal ganglia mechanisms (i.e., what to choose) are associated with cerebellum mechanisms (i.e., when to choose). Overall, multiple levels of parallel circuits in and around the basal ganglia are essential for coordinated behaviors. Understanding these circuits is useful for creating clinical treatments of disorders resulting from the failure of these circuits.
Collapse
|
11
|
Möller M, Bogacz R. Learning the payoffs and costs of actions. PLoS Comput Biol 2019; 15:e1006285. [PMID: 30818357 PMCID: PMC6413954 DOI: 10.1371/journal.pcbi.1006285] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2018] [Revised: 03/12/2019] [Accepted: 01/15/2019] [Indexed: 11/19/2022] Open
Abstract
A set of sub-cortical nuclei called basal ganglia is critical for learning the values of actions. The basal ganglia include two pathways, which have been associated with approach and avoid behavior respectively and are differentially modulated by dopamine projections from the midbrain. Inspired by the influential opponent actor learning model, we demonstrate that, under certain circumstances, these pathways may represent learned estimates of the positive and negative consequences (payoffs and costs) of individual actions. In the model, the level of dopamine activity encodes the motivational state and controls to what extent payoffs and costs enter the overall evaluation of actions. We show that a set of previously proposed plasticity rules is suitable to extract payoffs and costs from a prediction error signal if they occur at different moments in time. For those plasticity rules, successful learning requires differential effects of positive and negative outcome prediction errors on the two pathways and a weak decay of synaptic weights over trials. We also confirm through simulations that the model reproduces drug-induced changes of willingness to work, as observed in classical experiments with the D2-antagonist haloperidol. The basal ganglia are structures underneath the surface of the vertebrate brain, associated with error-driven learning. Much is known about the anatomical and biological features of the basal ganglia; scientists now try to understand the algorithms implemented by these structures. Numerous models aspire to capture the learning functionality, but many of them only cover some specific aspect of the algorithm. Instead of further adding to that pool of partial models, we unify two existing ones—one which captures what the basal ganglia learn, and one that describes the learning mechanism itself. The first model suggests that the basal ganglia weigh positive against negative consequences of actions according to the motivational state. It hints how payoff and cost might be represented, but does not explain how those representations arise. The other model consists of biologically plausible plasticity rules, which describe how learning takes place, but not how the brain makes use of what is learned. We show that the two theories are compatible. Together, they form a model of learning and decision making that integrates the motivational state as well as the learned payoffs and costs of opportunities.
Collapse
Affiliation(s)
- Moritz Möller
- MRC Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Rafal Bogacz
- MRC Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
- * E-mail:
| |
Collapse
|
12
|
Chronic treatment with galantamine rescues reversal learning in an attentional set-shifting test after experimental brain trauma. Exp Neurol 2019; 315:32-41. [PMID: 30711647 DOI: 10.1016/j.expneurol.2019.01.019] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2018] [Revised: 01/12/2019] [Accepted: 01/30/2019] [Indexed: 12/19/2022]
Abstract
Approximately 10 million new cases of traumatic brain injury (TBI) are reported each year worldwide with many of these injuries resulting in higher order cognitive impairments. Galantamine (GAL), an acetylcholine esterase inhibitor (AChEI) and positive allosteric modulator of nicotinic acetylcholine receptors (nAChRs), has been reported to ameliorate cognitive deficits after clinical TBI. Previously, we demonstrated that controlled cortical impact (CCI) injury to rats resulted in significant executive function impairments as measured by the attentional set-shifting test (AST), a complex cognitive task analogous to the Wisconsin Card Sorting Test (WCST). We hypothesized that chronic administration of GAL would normalize performance on the AST post-TBI. Isoflurane-anesthetized adult male rats were subjected to moderate CCI (2.8 mm tissue deformation at 4 m/s) or sham injury. Rats were then randomized into one of three treatment groups (i.e., 1 mg/kg GAL, 2 mg/kg GAL, or 1 mL/kg saline vehicle; VEH) or their respective sham controls. GAL or VEH was administered intraperitoneally daily commencing 24 hours post-surgery and until AST testing at 4 weeks post-injury. The AST data revealed significant impairments in the first reversal stage after TBI, seen as increased trials to reach criterion and elevated total errors (p < 0.05). These behavioral flexibility deficits were equally normalized by the administration of both doses of GAL (p < 0.05). Additionally, the higher dose of GAL (2 mg/kg) also significantly reduced cortical lesion volume compared to TBI + VEH controls (p < 0.05). In summary, daily GAL administration provides an efficacious treatment for cognitive deficits and histological recovery after experimental brain trauma. Clinically, these findings are promising considering robust results were attained using a pharmacotherapy already used in the clinic to treat mild dementia.
Collapse
|
13
|
Hikosaka O, Kim HF, Amita H, Yasuda M, Isoda M, Tachibana Y, Yoshida A. Direct and indirect pathways for choosing objects and actions. Eur J Neurosci 2018; 49:637-645. [PMID: 29473660 DOI: 10.1111/ejn.13876] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2017] [Revised: 02/16/2018] [Accepted: 02/19/2018] [Indexed: 01/23/2023]
Abstract
A prominent target of the basal ganglia is the superior colliculus (SC) which controls gaze orientation (saccadic eye movement in primates) to an important object. This 'object choice' is crucial for choosing an action on the object. SC is innervated by the substantia nigra pars reticulata (SNr) which is controlled mainly by the caudate nucleus (CD). This CD-SNr-SC circuit is sensitive to the values of individual objects and facilitates saccades to good objects. The object values are processed differently in two parallel circuits: flexibly by the caudate head (CDh) and stably by the caudate tail (CDt). To choose good objects, we need to reject bad objects. In fact, these contrasting functions are accomplished by the circuit originating from CDt: The direct pathway focuses on good objects and facilitates saccades to them; the indirect pathway focuses on bad objects and suppresses saccades to them. Inactivation of CDt deteriorated the object choice, because saccades to bad objects were no longer suppressed. This suggests that the indirect pathway is important for object choice. However, the direct and indirect pathways for 'object choice', which aim at the same action (i.e., saccade), may not work for 'action choice'. One possibility is that circuits controlling different actions are connected through the indirect pathway. Additional connections of the indirect pathway with brain areas outside the basal ganglia may also provide a wider range of behavioral choice. In conclusion, basal ganglia circuits are composed of the basic direct/indirect pathways and additional connections and thus have acquired multiple functions.
Collapse
Affiliation(s)
- Okihide Hikosaka
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, 49 Convent Drive, Bldg. 49, Rm. 2A50, Bethesda, MD, 20892-4435, USA
| | - Hyoung F Kim
- Department of Biomedical Engineering, Sungkyunkwan University (SKKU), Suwon, Korea.,Center for Neuroscience Imaging Research, Institute for Basic Science (IBS), Suwon, Korea
| | - Hidetoshi Amita
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, 49 Convent Drive, Bldg. 49, Rm. 2A50, Bethesda, MD, 20892-4435, USA
| | - Masaharu Yasuda
- Department of Physiology, Kansai Medical University, Hirakata, Japan
| | - Masaki Isoda
- Division of Behavioral Development, Department of System Neuroscience, National Institute for Physiological Sciences, Okazaki, Aichi, Japan
| | - Yoshihisa Tachibana
- Division of System Neuroscience, Kobe University Graduate School of Medicine, Kobe, Hyogo, Japan
| | - Atsushi Yoshida
- Functional Architecture Imaging team, RIKEN Center for Life Science Technologies, Kobe, Hyogo, Japan
| |
Collapse
|
14
|
Xiao L, Chattree G, Oscos FG, Cao M, Wanat MJ, Roberts TF. A Basal Ganglia Circuit Sufficient to Guide Birdsong Learning. Neuron 2018; 98:208-221.e5. [PMID: 29551492 DOI: 10.1016/j.neuron.2018.02.020] [Citation(s) in RCA: 79] [Impact Index Per Article: 13.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2017] [Revised: 01/28/2018] [Accepted: 02/20/2018] [Indexed: 11/28/2022]
Abstract
Learning vocal behaviors, like speech and birdsong, is thought to rely on continued performance evaluation. Whether candidate performance evaluation circuits in the brain are sufficient to guide vocal learning is not known. Here, we test the sufficiency of VTA projections to the vocal basal ganglia in singing zebra finches, a songbird species that learns to produce a complex and stereotyped multi-syllabic courtship song during development. We optogenetically manipulate VTA axon terminals in singing birds contingent on how the pitch of an individual song syllable is naturally performed. We find that optical inhibition and excitation of VTA terminals are each sufficient to reliably guide learned changes in song. Inhibition and excitation have opponent effects on future performances of targeted song syllables, consistent with positive and negative reinforcement of performance outcomes. These findings define a central role for reinforcement mechanisms in learning vocalizations and demonstrate minimal circuit elements for learning vocal behaviors. VIDEO ABSTRACT.
Collapse
Affiliation(s)
- Lei Xiao
- Department of Neuroscience, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Gaurav Chattree
- Department of Neuroscience, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Francisco Garcia Oscos
- Department of Neuroscience, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Mou Cao
- Department of Pediatrics, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Matthew J Wanat
- Department of Biology, Neuroscience Institute, University of Texas at San Antonio, San Antonio, TX 78249, USA
| | - Todd F Roberts
- Department of Neuroscience, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| |
Collapse
|
15
|
Kim LH, Sharma S, Sharples SA, Mayr KA, Kwok CHT, Whelan PJ. Integration of Descending Command Systems for the Generation of Context-Specific Locomotor Behaviors. Front Neurosci 2017; 11:581. [PMID: 29093660 PMCID: PMC5651258 DOI: 10.3389/fnins.2017.00581] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2017] [Accepted: 10/04/2017] [Indexed: 11/23/2022] Open
Abstract
Over the past decade there has been a renaissance in our understanding of spinal cord circuits; new technologies are beginning to provide key insights into descending circuits which project onto spinal cord central pattern generators. By integrating work from both the locomotor and animal behavioral fields, we can now examine context-specific control of locomotion, with an emphasis on descending modulation arising from various regions of the brainstem. Here we examine approach and avoidance behaviors and the circuits that lead to the production and arrest of locomotion.
Collapse
Affiliation(s)
- Linda H Kim
- Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada.,Department of Neuroscience, University of Calgary, Calgary, AB, Canada
| | - Sandeep Sharma
- Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada.,Department of Comparative Biology and Experimental Medicine, University of Calgary, Calgary, AB, Canada
| | - Simon A Sharples
- Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada.,Department of Neuroscience, University of Calgary, Calgary, AB, Canada
| | - Kyle A Mayr
- Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada.,Department of Neuroscience, University of Calgary, Calgary, AB, Canada
| | - Charlie H T Kwok
- Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada.,Department of Comparative Biology and Experimental Medicine, University of Calgary, Calgary, AB, Canada
| | - Patrick J Whelan
- Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada.,Department of Neuroscience, University of Calgary, Calgary, AB, Canada.,Department of Comparative Biology and Experimental Medicine, University of Calgary, Calgary, AB, Canada
| |
Collapse
|
16
|
Combining Dopaminergic Facilitation with Robot-Assisted Upper Limb Therapy in Stroke Survivors: A Focused Review. Am J Phys Med Rehabil 2017; 95:459-74. [PMID: 26829074 PMCID: PMC4866584 DOI: 10.1097/phm.0000000000000438] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
Despite aggressive conventional therapy, lasting hemiplegia persists in a large percentage of stroke survivors. The aim of this article is to critically review the rationale behind targeting multiple sites along the motor learning network by combining robotic therapy with pharmacotherapy and virtual reality–based reward learning to alleviate upper extremity impairment in stroke survivors. Methods for personalizing pharmacologic facilitation to each individual’s unique biology are also reviewed. At the molecular level, treatment with levodopa was shown to induce long-term potentiation-like and practice-dependent plasticity. Clinically, trials combining conventional therapy with levodopa in stroke survivors yielded statistically significant but clinically unconvincing outcomes because of limited personalization, standardization, and reproducibility. Robotic therapy can induce neuroplasticity by delivering intensive, reproducible, and functionally meaningful interventions that are objective enough for the rigors of research. Robotic therapy also provides an apt platform for virtual reality, which boosts learning by engaging reward circuits. The future of stroke rehabilitation should target distinct molecular, synaptic, and cortical sites through personalized multimodal treatments to maximize motor recovery.
Collapse
|
17
|
Nasser HM, Calu DJ, Schoenbaum G, Sharpe MJ. The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning. Front Psychol 2017; 8:244. [PMID: 28275359 PMCID: PMC5319959 DOI: 10.3389/fpsyg.2017.00244] [Citation(s) in RCA: 50] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2016] [Accepted: 02/07/2017] [Indexed: 12/31/2022] Open
Abstract
Phasic activity of midbrain dopamine neurons is currently thought to encapsulate the prediction-error signal described in Sutton and Barto’s (1981) model-free reinforcement learning algorithm. This phasic signal is thought to contain information about the quantitative value of reward, which transfers to the reward-predictive cue after learning. This is argued to endow the reward-predictive cue with the value inherent in the reward, motivating behavior toward cues signaling the presence of reward. Yet theoretical and empirical research has implicated prediction-error signaling in learning that extends far beyond a transfer of quantitative value to a reward-predictive cue. Here, we review the research which demonstrates the complexity of how dopaminergic prediction errors facilitate learning. After briefly discussing the literature demonstrating that phasic dopaminergic signals can act in the manner described by Sutton and Barto (1981), we consider how these signals may also influence attentional processing across multiple attentional systems in distinct brain circuits. Then, we discuss how prediction errors encode and promote the development of context-specific associations between cues and rewards. Finally, we consider recent evidence that shows dopaminergic activity contains information about causal relationships between cues and rewards that reflect information garnered from rich associative models of the world that can be adapted in the absence of direct experience. In discussing this research we hope to support the expansion of how dopaminergic prediction errors are thought to contribute to the learning process beyond the traditional concept of transferring quantitative value.
Collapse
Affiliation(s)
- Helen M Nasser
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore MD, USA
| | - Donna J Calu
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore MD, USA
| | - Geoffrey Schoenbaum
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, BaltimoreMD, USA; Cellular Neurobiology Research Branch, National Institute on Drug Abuse Intramural Research Program, BaltimoreMD, USA; Solomon H. Snyder Department of Neuroscience, Johns Hopkins University, BaltimoreMD, USA
| | - Melissa J Sharpe
- Cellular Neurobiology Research Branch, National Institute on Drug Abuse Intramural Research Program, BaltimoreMD, USA; Princeton Neuroscience Institute, Princeton University, PrincetonNJ, USA
| |
Collapse
|
18
|
Gadagkar V, Puzerey PA, Chen R, Baird-Daniel E, Farhang AR, Goldberg JH. Dopamine neurons encode performance error in singing birds. Science 2016; 354:1278-1282. [PMID: 27940871 PMCID: PMC5464363 DOI: 10.1126/science.aah6837] [Citation(s) in RCA: 132] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2016] [Accepted: 10/24/2016] [Indexed: 12/20/2022]
Abstract
Many behaviors are learned through trial and error by matching performance to internal goals. Yet neural mechanisms of performance evaluation remain poorly understood. We recorded basal ganglia-projecting dopamine neurons in singing zebra finches as we controlled perceived song quality with distorted auditory feedback. Dopamine activity was phasically suppressed after distorted syllables, consistent with a worse-than-predicted outcome, and was phasically activated at the precise moment of the song when a predicted distortion did not occur, consistent with a better-than-predicted outcome. Error response magnitude depended on distortion probability. Thus, dopaminergic error signals can evaluate behaviors that are not learned for reward and are instead learned by matching performance outcomes to internal goals.
Collapse
Affiliation(s)
- Vikram Gadagkar
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Pavel A Puzerey
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Ruidong Chen
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Eliza Baird-Daniel
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Alexander R Farhang
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Jesse H Goldberg
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA.
| |
Collapse
|
19
|
Abstract
Unidirectional connections from the cortex to the matrix of the corpus striatum initiate the cortico-basal ganglia (BG)-thalamocortical loop, thought to be important in momentary action selection and in longer-term fine tuning of behavioural repertoire; a discrete set of striatal compartments, striosomes, has the complementary role of registering or anticipating reward that shapes corticostriatal plasticity. Re-entrant signals traversing the cortico-BG loop impact predominantly frontal cortices, conveyed through topographically ordered output channels; by contrast, striatal input signals originate from a far broader span of cortex, and are far more divergent in their termination. The term 'disclosed loop' is introduced to describe this organisation: a closed circuit that is open to outside influence at the initial stage of cortical input. The closed circuit component of corticostriatal afferents is newly dubbed 'operative', as it is proposed to establish the bid for action selection on the part of an incipient cortical action plan; the broader set of converging corticostriatal afferents is described as contextual. A corollary of this proposal is that every unit of the striatal volume, including the long, C-shaped tail of the caudate nucleus, should receive a mandatory component of operative input, and hence include at least one area of BG-recipient cortex amongst the sources of its corticostriatal afferents. Individual operative afferents contact twin classes of GABAergic striatal projection neuron (SPN), distinguished by their neurochemical character, and onward circuitry. This is the basis of the classic direct and indirect pathway model of the cortico-BG loop. Each pathway utilises a serial chain of inhibition, with two such links, or three, providing positive and negative feedback, respectively. Operative co-activation of direct and indirect SPNs is, therefore, pictured to simultaneously promote action, and to restrain it. The balance of this rival activity is determined by the contextual inputs, which summarise the external and internal sensory environment, and the state of ongoing behavioural priorities. Notably, the distributed sources of contextual convergence upon a striatal locus mirror the transcortical network harnessed by the origin of the operative input to that locus, thereby capturing a similar set of contingencies relevant to determining action. The disclosed loop formulation of corticostriatal and subsequent BG loop circuitry, as advanced here, refines the operating rationale of the classic model and allows the integration of more recent anatomical and physiological data, some of which can appear at variance with the classic model. Equally, it provides a lucid functional context for continuing cellular studies of SPN biophysics and mechanisms of synaptic plasticity.
Collapse
|
20
|
Klanker M, Sandberg T, Joosten R, Willuhn I, Feenstra M, Denys D. Phasic dopamine release induced by positive feedback predicts individual differences in reversal learning. Neurobiol Learn Mem 2015; 125:135-45. [PMID: 26343836 DOI: 10.1016/j.nlm.2015.08.011] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2015] [Revised: 08/20/2015] [Accepted: 08/24/2015] [Indexed: 01/01/2023]
Abstract
Striatal dopamine (DA) is central to reward-based learning. Less is known about the contribution of DA to the ability to adapt previously learned behavior in response to changes in the environment, such as a reversal of response-reward contingencies. We hypothesized that DA is involved in the rapid updating of response-reward information essential for successful reversal learning. We trained rats to discriminate between two levers, where lever availability was signaled by a non-discriminative cue. Pressing one lever was always rewarded, whereas the other lever was never rewarded. After reaching stable discrimination performance, a reversal was presented, so that the previously non-rewarded lever was now rewarded and vice versa. We used fast-scan cyclic voltammetry to monitor DA release in the ventromedial striatum. During discrimination performance (pre-reversal), cue presentation induced phasic DA release, whereas reward delivery did not. The opposite pattern was observed post-reversal: Striatal DA release emerged after reward delivery, while cue-induced release diminished. Trial-by-trial analysis showed rapid reinstatement of cue-induced DA release on trials immediately following initial correct responses. This effect of positive feedback was observed in animals that learned the reversal, but not in 'non-learners'. In contrast, neither pre-reversal responding and DA signaling, nor post-reversal DA signaling in response to negative feedback differed between learners and non-learners. Together, we show that phasic DA dynamics in the ventromedial striatum encoding reward-predicting cues are associated with positive feedback during reversal learning. Furthermore, these signals predict individual differences in learning that are not present prior to reversal, suggesting a distinct role for dopamine in the adaptation of previously learned behavior.
Collapse
Affiliation(s)
- Marianne Klanker
- Netherlands Institute for Neuroscience, Institute of the Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, The Netherlands; Department of Psychiatry, Academic Medical Center, University of Amsterdam, Postbus 22660, 1100 DD Amsterdam, The Netherlands.
| | - Tessa Sandberg
- Netherlands Institute for Neuroscience, Institute of the Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, The Netherlands
| | - Ruud Joosten
- Netherlands Institute for Neuroscience, Institute of the Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, The Netherlands
| | - Ingo Willuhn
- Netherlands Institute for Neuroscience, Institute of the Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, The Netherlands; Department of Psychiatry, Academic Medical Center, University of Amsterdam, Postbus 22660, 1100 DD Amsterdam, The Netherlands
| | - Matthijs Feenstra
- Netherlands Institute for Neuroscience, Institute of the Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, The Netherlands; Department of Psychiatry, Academic Medical Center, University of Amsterdam, Postbus 22660, 1100 DD Amsterdam, The Netherlands
| | - Damiaan Denys
- Netherlands Institute for Neuroscience, Institute of the Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, The Netherlands; Department of Psychiatry, Academic Medical Center, University of Amsterdam, Postbus 22660, 1100 DD Amsterdam, The Netherlands
| |
Collapse
|
21
|
Morita K, Kawaguchi Y. Computing reward-prediction error: an integrated account of cortical timing and basal-ganglia pathways for appetitive and aversive learning. Eur J Neurosci 2015; 42:2003-21. [PMID: 26095906 PMCID: PMC5034842 DOI: 10.1111/ejn.12994] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2015] [Revised: 06/11/2015] [Accepted: 06/17/2015] [Indexed: 12/12/2022]
Abstract
There are two prevailing notions regarding the involvement of the corticobasal ganglia system in value‐based learning: (i) the direct and indirect pathways of the basal ganglia are crucial for appetitive and aversive learning, respectively, and (ii) the activity of midbrain dopamine neurons represents reward‐prediction error. Although (ii) constitutes a critical assumption of (i), it remains elusive how (ii) holds given (i), with the basal‐ganglia influence on the dopamine neurons. Here we present a computational neural‐circuit model that potentially resolves this issue. Based on the latest analyses of the heterogeneous corticostriatal neurons and connections, our model posits that the direct and indirect pathways, respectively, represent the values of upcoming and previous actions, and up‐regulate and down‐regulate the dopamine neurons via the basal‐ganglia output nuclei. This explains how the difference between the upcoming and previous values, which constitutes the core of reward‐prediction error, is calculated. Simultaneously, it predicts that blockade of the direct/indirect pathway causes a negative/positive shift of reward‐prediction error and thereby impairs learning from positive/negative error, i.e. appetitive/aversive learning. Through simulation of reward‐reversal learning and punishment‐avoidance learning, we show that our model could indeed account for the experimentally observed features that are suggested to support notion (i) and could also provide predictions on neural activity. We also present a behavioral prediction of our model, through simulation of inter‐temporal choice, on how the balance between the two pathways relates to the subject's time preference. These results indicate that our model, incorporating the heterogeneity of the cortical influence on the basal ganglia, is expected to provide a closed‐circuit mechanistic understanding of appetitive/aversive learning.
Collapse
Affiliation(s)
- Kenji Morita
- Physical and Health Education, Graduate School of Education, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Yasuo Kawaguchi
- Division of Cerebral Circuitry, National Institute for Physiological Sciences, Okazaki, Japan.,Department of Physiological Sciences, SOKENDAI (The Graduate University for Advanced Studies), Okazaki, Japan.,Japan Science and Technology Agency, Core Research for Evolutional Science and Technology, Tokyo, Japan
| |
Collapse
|
22
|
Yager LM, Garcia AF, Wunsch AM, Ferguson SM. The ins and outs of the striatum: role in drug addiction. Neuroscience 2015; 301:529-41. [PMID: 26116518 DOI: 10.1016/j.neuroscience.2015.06.033] [Citation(s) in RCA: 271] [Impact Index Per Article: 30.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2015] [Revised: 06/05/2015] [Accepted: 06/18/2015] [Indexed: 10/23/2022]
Abstract
Addiction is a chronic relapsing disorder characterized by the loss of control over drug intake, high motivation to obtain the drug, and a persistent craving for the drug. Accumulating evidence implicates cellular and molecular alterations within cortico-basal ganglia-thalamic circuitry in the development and persistence of this disease. The striatum is a heterogeneous structure that sits at the interface of this circuit, receiving input from a variety of brain regions (e.g., prefrontal cortex, ventral tegmental area) to guide behavioral output, including motor planning, decision-making, motivation and reward. However, the vast interconnectivity of this circuit has made it difficult to isolate how individual projections and cellular subtypes within this circuit modulate each of the facets of addiction. Here, we review the use of new technologies, including optogenetics and DREADDs (Designer Receptors Exclusively Activated by Designer Drugs), in unraveling the role of the striatum in addiction. In particular, we focus on the role of striatal cell populations (i.e., direct and indirect pathway medium spiny neurons) and striatal dopaminergic and glutamatergic afferents in addiction-related plasticity and behaviors.
Collapse
Affiliation(s)
- L M Yager
- Center for Integrative Brain Research, Seattle Children's Research Institute, Seattle, WA, United States
| | - A F Garcia
- Center for Integrative Brain Research, Seattle Children's Research Institute, Seattle, WA, United States; Neuroscience Graduate Program, University of Washington, Seattle, WA, United States
| | - A M Wunsch
- Center for Integrative Brain Research, Seattle Children's Research Institute, Seattle, WA, United States; Neuroscience Graduate Program, University of Washington, Seattle, WA, United States
| | - S M Ferguson
- Center for Integrative Brain Research, Seattle Children's Research Institute, Seattle, WA, United States; Neuroscience Graduate Program, University of Washington, Seattle, WA, United States; Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, United States.
| |
Collapse
|
23
|
Kim HF, Hikosaka O. Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards. Brain 2015; 138:1776-800. [PMID: 25981958 DOI: 10.1093/brain/awv134] [Citation(s) in RCA: 104] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2014] [Accepted: 04/14/2015] [Indexed: 11/13/2022] Open
Abstract
The basal ganglia control body movements, value processing and decision-making. Many studies have shown that the inputs and outputs of each basal ganglia structure are topographically organized, which suggests that the basal ganglia consist of separate circuits that serve distinct functions. A notable example is the circuits that originate from the rostral (head) and caudal (tail) regions of the caudate nucleus, both of which target the superior colliculus. These two caudate regions encode the reward values of visual objects differently: flexible (short-term) values by the caudate head and stable (long-term) values by the caudate tail. These value signals in the caudate guide the orienting of gaze differently: voluntary saccades by the caudate head circuit and automatic saccades by the caudate tail circuit. Moreover, separate groups of dopamine neurons innervate the caudate head and tail and may selectively guide the flexible and stable learning/memory in the caudate regions. Studies focusing on manual handling of objects also suggest that rostrocaudally separated circuits in the basal ganglia control the action differently. These results suggest that the basal ganglia contain parallel circuits for two steps of goal-directed behaviour: finding valuable objects and manipulating the valuable objects. These parallel circuits may underlie voluntary behaviour and automatic skills, enabling animals (including humans) to adapt to both volatile and stable environments. This understanding of the functions and mechanisms of the basal ganglia parallel circuits may inform the differential diagnosis and treatment of basal ganglia disorders.
Collapse
Affiliation(s)
- Hyoung F Kim
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
| | - Okihide Hikosaka
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
| |
Collapse
|
24
|
Matsumoto M. Dopamine signals and physiological origin of cognitive dysfunction in Parkinson's disease. Mov Disord 2015; 30:472-83. [PMID: 25773863 DOI: 10.1002/mds.26177] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2014] [Revised: 01/08/2015] [Accepted: 01/19/2015] [Indexed: 11/12/2022] Open
Abstract
The pathological hallmark of Parkinson's disease (PD) is the degeneration of midbrain dopamine neurons. Cognitive dysfunction is a feature of PD patients even at the early stages of the disease. Electrophysiological studies on dopamine neurons in awake animals provide contradictory accounts of the role of dopamine. These studies have established that dopamine neurons convey a unique signal associated with rewards rather than cognitive functions. Emphasizing their role in reward processing leads to difficulty in developing hypothesis as to how cognitive impairments in PD are associated with the degeneration of dopamine circuitry. A hint to resolve this contradiction came from recent electrophysiological studies reporting that dopamine neurons transmit more diverse signals than previously thought. These studies suggest that dopamine neurons are divided into at least two functional subgroups, one signaling "motivational value" and the other signaling "salience." The former subgroup fits well with the conventional reward theory, whereas the latter subgroup has been shown to transmit signals related to salient but non-rewarding experiences such as aversive stimulations and cognitively demanding situations. This article reviews recent advances in understanding the non-reward functions of dopamine, and then discusses the possibility that cognitive dysfunction in PD is at least partially caused by the degeneration of the dopamine neuron subgroup signaling the salience of events in the environment.
Collapse
Affiliation(s)
- Masayuki Matsumoto
- Laboratory of Cognitive and Behavioral Neuroscience, Division of Biomedical Science, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| |
Collapse
|
25
|
Sagheddu C, Muntoni AL, Pistis M, Melis M. Endocannabinoid Signaling in Motivation, Reward, and Addiction. INTERNATIONAL REVIEW OF NEUROBIOLOGY 2015; 125:257-302. [DOI: 10.1016/bs.irn.2015.10.004] [Citation(s) in RCA: 28] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
|
26
|
Yasuda M, Hikosaka O. Functional territories in primate substantia nigra pars reticulata separately signaling stable and flexible values. J Neurophysiol 2014; 113:1681-96. [PMID: 25540224 DOI: 10.1152/jn.00674.2014] [Citation(s) in RCA: 44] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Gaze is strongly attracted to visual objects that have been associated with rewards. Key to this function is a basal ganglia circuit originating from the caudate nucleus (CD), mediated by the substantia nigra pars reticulata (SNr), and aiming at the superior colliculus (SC). Notably, subregions of CD encode values of visual objects differently: stably by CD tail [CD(T)] vs. flexibly by CD head [CD(H)]. Are the stable and flexible value signals processed separately throughout the CD-SNr-SC circuit? To answer this question, we identified SNr neurons by their inputs from CD and outputs to SC and examined their sensitivity to object values. The direct input from CD was identified by SNr neuron's inhibitory response to electrical stimulation of CD. We found that SNr neurons were separated into two groups: 1) neurons inhibited by CD(T) stimulation, located in the caudal-dorsal-lateral SNr (cdlSNr), and 2) neurons inhibited by CD(H) stimulation, located in the rostral-ventral-medial SNr (rvmSNr). Most of CD(T)-recipient SNr neurons encoded stable values, whereas CD(H)-recipient SNr neurons tended to encode flexible values. The output to SC was identified by SNr neuron's antidromic response to SC stimulation. Among the antidromically activated neurons, many encoded only stable values, while some encoded only flexible values. These results suggest that CD(T)-cdlSNr-SC circuit and CD(H)-rvmSNr-SC circuit transmit stable and flexible value signals, largely separately, to SC. The speed of signal transmission was faster through CD(T)-cdlSNr-SC circuit than through CD(H)-rvmSNr-SC circuit, which may reflect automatic and controlled gaze orienting guided by these circuits.
Collapse
Affiliation(s)
- Masaharu Yasuda
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland
| | - Okihide Hikosaka
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland
| |
Collapse
|
27
|
Wanjerkhede SM, Bapi RS, Mytri VD. Reinforcement learning and dopamine in the striatum: A modeling perspective. Neurocomputing 2014. [DOI: 10.1016/j.neucom.2013.02.061] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
28
|
Abstract
There are a growing number of roles that midbrain dopamine (DA) neurons assume, such as, reward, aversion, alerting and vigor. Here I propose a theory that may be able to explain why the suggested functions of DA came about. It has been suggested that largely parallel cortico-basal ganglia-thalamo-cortico loops exist to control different aspects of behavior. I propose that (1) the midbrain DA system is organized in a similar manner, with different groups of DA neurons corresponding to these parallel neural pathways (NPs). The DA system can be viewed as the "manager" of these parallel NPs in that it recruits and activates only the task-relevant NPs when they are needed. It is likely that the functions of those NPs that have been consistently activated by the corresponding DA groups are facilitated. I also propose that (2) there are two levels of DA roles: the How and What roles. The How role is encoded in tonic and phasic DA neuron firing patterns and gives a directive to its target NP: how vigorously its function needs to be carried out. The tonic DA firing is to provide the needed level of DA in the target NPs to support their expected behavioral and mental functions; it is only when a sudden unexpected boost or suppression of activity is required by the relevant target NP that DA neurons in the corresponding NP act in a phasic manner. The What role is the implementational aspect of the role of DA in the target NP, such as binding to D1 receptors to boost working memory. This What aspect of DA explains why DA seems to assume different functions depending on the region of the brain in which it is involved. In terms of the role of the lateral habenula (LHb), the LHb is expected to suppress maladaptive behaviors and mental processes by controlling the DA system. The demand-based smart management by the DA system may have given animals an edge in evolution with adaptive behaviors and a better survival rate in resource-scarce situations.
Collapse
Affiliation(s)
- Simon Hong
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology Cambridge, MA, USA
| |
Collapse
|
29
|
Klanker M, Feenstra M, Denys D. Dopaminergic control of cognitive flexibility in humans and animals. Front Neurosci 2013; 7:201. [PMID: 24204329 PMCID: PMC3817373 DOI: 10.3389/fnins.2013.00201] [Citation(s) in RCA: 148] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2013] [Accepted: 10/11/2013] [Indexed: 12/21/2022] Open
Abstract
Striatal dopamine (DA) is thought to code for learned associations between cues and reinforcers and to mediate approach behavior toward a reward. Less is known about the contribution of DA to cognitive flexibility—the ability to adapt behavior in response to changes in the environment. Altered reward processing and impairments in cognitive flexibility are observed in psychiatric disorders such as obsessive compulsive disorder (OCD). Patients with this disorder show a disruption of functioning in the frontostriatal circuit and alterations in DA signaling. In this review we summarize findings from animal and human studies that have investigated the involvement of striatal DA in cognitive flexibility. These findings may provide a better understanding of the role of dopaminergic dysfunction in cognitive inflexibility in psychiatric disorders, such as OCD.
Collapse
Affiliation(s)
- Marianne Klanker
- Netherlands Institute for Neuroscience, Institute of the Royal Netherlands Academy of Arts and Sciences Amsterdam, Netherlands ; Department of Psychiatry, Academic Medical Center, University of Amsterdam Amsterdam, Netherlands
| | | | | |
Collapse
|
30
|
Dopaminergic control of motivation and reinforcement learning: a closed-circuit account for reward-oriented behavior. J Neurosci 2013; 33:8866-90. [PMID: 23678129 DOI: 10.1523/jneurosci.4614-12.2013] [Citation(s) in RCA: 36] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
Humans and animals take actions quickly when they expect that the actions lead to reward, reflecting their motivation. Injection of dopamine receptor antagonists into the striatum has been shown to slow such reward-seeking behavior, suggesting that dopamine is involved in the control of motivational processes. Meanwhile, neurophysiological studies have revealed that phasic response of dopamine neurons appears to represent reward prediction error, indicating that dopamine plays central roles in reinforcement learning. However, previous attempts to elucidate the mechanisms of these dopaminergic controls have not fully explained how the motivational and learning aspects are related and whether they can be understood by the way the activity of dopamine neurons itself is controlled by their upstream circuitries. To address this issue, we constructed a closed-circuit model of the corticobasal ganglia system based on recent findings regarding intracortical and corticostriatal circuit architectures. Simulations show that the model could reproduce the observed distinct motivational effects of D1- and D2-type dopamine receptor antagonists. Simultaneously, our model successfully explains the dopaminergic representation of reward prediction error as observed in behaving animals during learning tasks and could also explain distinct choice biases induced by optogenetic stimulation of the D1 and D2 receptor-expressing striatal neurons. These results indicate that the suggested roles of dopamine in motivational control and reinforcement learning can be understood in a unified manner through a notion that the indirect pathway of the basal ganglia represents the value of states/actions at a previous time point, an empirically driven key assumption of our model.
Collapse
|
31
|
Porter-Stransky KA, Seiler JL, Day JJ, Aragona BJ. Development of behavioral preferences for the optimal choice following unexpected reward omission is mediated by a reduction of D2-like receptor tone in the nucleus accumbens. Eur J Neurosci 2013; 38:2572-88. [PMID: 23692625 DOI: 10.1111/ejn.12253] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2013] [Revised: 04/28/2013] [Accepted: 04/12/2013] [Indexed: 11/30/2022]
Abstract
To survive in a dynamic environment, animals must identify changes in resource availability and rapidly apply adaptive strategies to obtain resources that promote survival. We have utilised a behavioral paradigm to assess differences in foraging strategy when resource (reward) availability unexpectedly changes. When reward magnitude was reduced by 50% (receive one reward pellet instead of two), male and female rats developed a preference for the optimal choice by the second session. However, when an expected reward was omitted (receive no reward pellets instead of one), subjects displayed a robust preference for the optimal choice during the very first session. Previous research shows that, when an expected reward is omitted, dopamine neurons phasically decrease their firing rate, which is hypothesised to decrease dopamine release preferentially affecting D2-like receptors. As robust changes in behavioral preference were specific to reward omission, we tested this hypothesis and the functional role of D1- and D2-like receptors in the nucleus accumbens in mediating the rapid development of a behavioral preference for the rewarded option during reward omission in male rats. Blockade of both receptor types had no effect on this behavior; however, holding D2-like, but not D1-like, receptor tone via infusion of dopamine receptor agonists prevented the development of the preference for the rewarded option during reward omission. These results demonstrate that avoiding an outcome that has been tagged with aversive motivational properties is facilitated through decreased dopamine transmission and subsequent functional disruption of D2-like, but not D1-like, receptor tone in the nucleus accumbens.
Collapse
Affiliation(s)
- Kirsten A Porter-Stransky
- Department of Psychology, Biopsychology Area, University of Michigan, 530 Church Street, Ann Arbor, 48109 MI, USA.
| | | | | | | |
Collapse
|
32
|
Chen LL, Hung LY, Quinet J, Kosek K. Cognitive regulation of saccadic velocity by reward prospect. Eur J Neurosci 2013; 38:2434-44. [PMID: 23668781 DOI: 10.1111/ejn.12247] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2012] [Revised: 02/25/2013] [Accepted: 04/03/2013] [Indexed: 11/25/2022]
Abstract
It is known that expectation of reward speeds up saccades. Past studies have also shown the presence of a saccadic velocity bias in the orbit, resulting from a biomechanical regulation over varying eccentricities. Nevertheless, whether and how reward expectation interacts with the biomechanical regulation of saccadic velocities over varying eccentricities remains unknown. We addressed this question by conducting a visually guided double-step saccade task. The role of reward expectation was tested in monkeys performing two consecutive horizontal saccades, one associated with reward prospect and the other not. To adequately assess saccadic velocity and avoid adaptation, we systematically varied initial eye positions, saccadic directions and amplitudes. Our results confirmed the existence of a velocity bias in the orbit, i.e., saccadic peak velocity decreased linearly as the initial eye position deviated in the direction of the saccade. The slope of this bias increased as saccadic amplitudes increased. Nevertheless, reward prospect facilitated velocity to a greater extent for saccades away from than for saccades toward the orbital centre, rendering an overall reduction in the velocity bias. The rate (slope) and magnitude (intercept) of reward modulation over this velocity bias were linearly correlated with amplitudes, similar to the amplitude-modulated velocity bias without reward prospect, which presumably resulted from a biomechanical regulation. Small-amplitude (≤ 5°) saccades received little modulation. These findings together suggest that reward expectation modulated saccadic velocity not as an additive signal but as a facilitating mechanism that interacted with the biomechanical regulation.
Collapse
Affiliation(s)
- Lewis L Chen
- Department of Otolaryngology and Communicative Sciences, University of Mississippi Medical Center, Jackson, MS 39211, USA.
| | | | | | | |
Collapse
|
33
|
The influence of motivational salience on saccade latencies. Exp Brain Res 2012; 224:35-47. [DOI: 10.1007/s00221-012-3284-4] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2012] [Accepted: 09/23/2012] [Indexed: 10/27/2022]
|
34
|
Ray NJ, Strafella AP. The neurobiology and neural circuitry of cognitive changes in Parkinson's disease revealed by functional neuroimaging. Mov Disord 2012; 27:1484-92. [PMID: 23038645 DOI: 10.1002/mds.25173] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2012] [Revised: 06/29/2012] [Accepted: 08/03/2012] [Indexed: 11/12/2022] Open
Abstract
Patients with Parkinson's disease (PD) often develop a spectrum of cognitive symptoms that can evolve into dementia. Dopamine (DA) replacement medications, though improving motor symptoms, can exert both positive and negative effects on cognitive ability, depending on the severity of the disease and the specific skill being tested. By considering the behavioral and clinical aspects of disease- and treatment-mediated changes in cognition alongside the pathophysiology of PD, an understanding of the factors that govern the heterogeneous expression of cognitive impairment in PD is beginning to emerge. Here, we review the neuroimaging studies revealing the neural correlates of cognitive changes after DA loss and DA replacement as well as those that may accompany the conversion from milder stages of cognitive impairment to frank dementia.
Collapse
Affiliation(s)
- Nicola J Ray
- Morton and Gloria Shulman Movement Disorder Unit & EJ Safra Parkinson Disease Program, Toronto Western Hospital, University Health Network, University of Toronto, Toronto, Ontario, Canada
| | | |
Collapse
|
35
|
The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning. Neuron 2012; 74:874-86. [PMID: 22681691 DOI: 10.1016/j.neuron.2012.04.018] [Citation(s) in RCA: 105] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/03/2012] [Indexed: 11/22/2022]
Abstract
Dopamine is thought to play a major role in learning. However, while dopamine D1 receptors (D1Rs) in the prefrontal cortex (PFC) have been shown to modulate working memory-related neural activity, their role in the cellular basis of learning is unknown. We recorded activity from multiple electrodes while injecting the D1R antagonist SCH23390 in the lateral PFC as monkeys learned visuomotor associations. Blocking D1Rs impaired learning of novel associations and decreased cognitive flexibility but spared performance of already familiar associations. This suggests a greater role for prefrontal D1Rs in learning new, rather than performing familiar, associations. There was a corresponding greater decrease in neural selectivity and increase in alpha and beta oscillations in local field potentials for novel than for familiar associations. Our results suggest that weak stimulation of D1Rs observed in aging and psychiatric disorders may impair learning and PFC function by reducing neural selectivity and exacerbating neural oscillations associated with inattention and cognitive deficits.
Collapse
|
36
|
Abstract
Reinforcement learning is an adaptive process in which an animal utilizes its previous experience to improve the outcomes of future choices. Computational theories of reinforcement learning play a central role in the newly emerging areas of neuroeconomics and decision neuroscience. In this framework, actions are chosen according to their value functions, which describe how much future reward is expected from each action. Value functions can be adjusted not only through reward and penalty, but also by the animal's knowledge of its current environment. Studies have revealed that a large proportion of the brain is involved in representing and updating value functions and using them to choose an action. However, how the nature of a behavioral task affects the neural mechanisms of reinforcement learning remains incompletely understood. Future studies should uncover the principles by which different computational elements of reinforcement learning are dynamically coordinated across the entire brain.
Collapse
Affiliation(s)
- Daeyeol Lee
- Department of Neurobiology, Kavli Institute for Neuroscience, Yale University School of Medicine, New Haven, Connecticut 06510, USA.
| | | | | |
Collapse
|
37
|
Roesch MR, Bryden DW. Impact of size and delay on neural activity in the rat limbic corticostriatal system. Front Neurosci 2011; 5:130. [PMID: 22363252 PMCID: PMC3277262 DOI: 10.3389/fnins.2011.00130] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2011] [Accepted: 11/04/2011] [Indexed: 11/17/2022] Open
Abstract
A number of factors influence an animal’s economic decisions. Two most commonly studied are the magnitude of and delay to reward. To investigate how these factors are represented in the firing rates of single neurons, we devised a behavioral task that independently manipulated the expected delay to and size of reward. Rats perceived the differently delayed and sized rewards as having different values and were more motivated under short delay and big-reward conditions than under long delay and small reward conditions as measured by percent choice, accuracy, and reaction time. Since the creation of this task, we have recorded from several different brain areas including, orbitofrontal cortex, striatum, amygdala, substantia nigra pars reticulata, and midbrain dopamine neurons. Here, we review and compare those data with a substantial focus on those areas that have been shown to be critical for performance on classic time discounting procedures and provide a potential mechanism by which they might interact when animals are deciding between differently delayed rewards. We found that most brain areas in the cortico-limbic circuit encode both the magnitude and delay to reward delivery in one form or another, but only a few encode them together at the single neuron level.
Collapse
Affiliation(s)
- Matthew R Roesch
- Department of Psychology and Program in Neuroscience and Cognitive Science, University of Maryland College Park, MD, USA
| | | |
Collapse
|
38
|
Fee MS, Goldberg JH. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 2011; 198:152-70. [PMID: 22015923 DOI: 10.1016/j.neuroscience.2011.09.069] [Citation(s) in RCA: 145] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2011] [Revised: 09/30/2011] [Accepted: 09/30/2011] [Indexed: 01/08/2023]
Abstract
Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence can be learned through an RL-like strategy. Interestingly, like motor sequence learning in mammals, song learning in birds requires a basal ganglia (BG)-thalamocortical loop, suggesting common neural mechanisms. Here, we outline a specific working hypothesis for how BG-forebrain circuits could utilize an internally computed reinforcement signal to direct song learning. Our model includes a number of general concepts borrowed from the mammalian BG literature, including a dopaminergic reward prediction error and dopamine-mediated plasticity at corticostriatal synapses. We also invoke a number of conceptual advances arising from recent observations in the songbird. Specifically, there is evidence for a specialized cortical circuit that adds trial-to-trial variability to stereotyped cortical motor programs, and a role for the BG in "biasing" this variability to improve behavioral performance. This BG-dependent "premotor bias" may in turn guide plasticity in downstream cortical synapses to consolidate recently learned song changes. Given the similarity between mammalian and songbird BG-thalamocortical circuits, our model for the role of the BG in this process may have broader relevance to mammalian BG function.
Collapse
Affiliation(s)
- M S Fee
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA.
| | | |
Collapse
|
39
|
The contribution of NMDA receptor signaling in the corticobasal ganglia reward network to appetitive Pavlovian learning. J Neurosci 2011; 31:11362-9. [PMID: 21813695 DOI: 10.1523/jneurosci.2411-11.2011] [Citation(s) in RCA: 33] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022] Open
Abstract
NMDA receptors (NMDARs) contribute to phasic transmission and synaptic plasticity and are thought to be important for learning. To better understand where NMDAR signaling is necessary for learning, we combined viral genetic strategies with genetic mouse models to investigate the contribution of NMDARs in the dopamine system to appetitive Pavlovian conditioning. NMDAR signaling in dopamine neurons was not required for Pavlovian conditioning; however, NMDARs in D(1) dopamine receptor (D(1)R)-expressing medium spiny neurons (MSNs), which receive input from dopamine neurons, were critical for this type of learning. NMDAR signaling was also required in brain regions that project to dopamine neurons, because removing NMDARs from afferent neurons to the ventral tegmental area (VTA) also prevented learning. This effect was likely attributable to loss of NMDAR signaling in the neurons of the prefrontal cortex (PFC), because learning could be restored in these animals by rescuing NMDAR expression in the PFC. Moreover, removing NMDARs exclusively from the PFC also prevented learning. Our findings suggest that NMDARs in neurons that project to and receive projections from the VTA are necessary for Pavlovian conditioning and specifically implicate the PFC and D(1)R-expressing MSNs in associative learning.
Collapse
|