1
|
Jin S, Hull C. Reward-driven cerebellar climbing fiber activity influences both neural and behavioral learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.10.09.617466. [PMID: 39416023 PMCID: PMC11482817 DOI: 10.1101/2024.10.09.617466] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/19/2024]
Abstract
The cerebellum plays a key role in motor coordination and learning. In contrast with classical supervised learning models, recent work has revealed that CFs can signal reward-predictive information in some behaviors. This raises the question of whether CFs may also operate according to principles similar to those described by reinforcement learning models. To test how CFs operate during reward-guided behavior, and evaluate the role of reward-related CF activity in learning, we have measured CF responses in Purkinje cells of the lateral cerebellum during a Pavlovian task using 2-photon calcium imaging. Specifically, we have performed multi-stimulus experiments to determine whether CF activity meets the requirements of a reward prediction error (rPE) signal for transfer from an unexpected reward to a reward-predictive cue. We find that once CF activity is transferred to a conditioned stimulus, and there is no longer a response to reward, CFs cannot generate learned responses to a second conditioned stimulus that carries the same reward prediction. In addition, by expressing the inhibitory opsin GtACR2 in neurons of the inferior olive, and optically inhibiting these neurons across behavioral training at the time of unexpected reward, we find that the transfer of CF signals to the conditioned stimulus is impaired. Moreover, this optogenetic inhibition also impairs learning, resulting in a deficit in anticipatory lick timing. Together, these results indicate that CF signals can exhibit several characteristics in common with rPEs during reinforcement learning, and that the cerebellum can harness these learning signals to generate accurately timed motor behavior.
Collapse
|
2
|
Vignali C, Mutersbaugh M, Hull C. Cerebellar climbing fibers signal flexible, rapidly adapting reward predictions. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.10.09.617467. [PMID: 39416163 PMCID: PMC11482763 DOI: 10.1101/2024.10.09.617467] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/19/2024]
Abstract
Classical models of cerebellar computation posit that climbing fibers (CFs) operate according to supervised learning rules, correcting movements by signaling the occurrence of motor errors. However, recent findings suggest that in some behaviors, CF activity can exhibit features that resemble the instructional signals necessary for reinforcement learning, namely reward prediction errors (rPEs). Despite these initial observations, many key properties of reward-related CF responses remain unclear, thus limiting our understanding of how they operate to guide cerebellar learning. Here, we have measured the postsynaptic responses of CFs onto cerebellar Purkinje cells using two-photon calcium imaging to test how they respond to learned stimuli that either do or do not predict reward. We find that CFs can develop generalized responses to similar cues of the same modality, regardless of whether they are reward predictive. However, this generalization depends on temporal context, and does not extend across sensory modalities. Further, learned CF responses are flexible, and can be rapidly updated according to new reward contingencies. Together these results suggest that CFs can generate learned, reward-predictive responses that flexibly adapt to the current environment in a context-sensitive manner.
Collapse
|
3
|
Manto M, Adamaszek M, Apps R, Carlson E, Guarque-Chabrera J, Heleven E, Kakei S, Khodakhah K, Kuo SH, Lin CYR, Joshua M, Miquel M, Mitoma H, Larry N, Péron JA, Pickford J, Schutter DJLG, Singh MK, Tan T, Tanaka H, Tsai P, Van Overwalle F, Yamashiro K. Consensus Paper: Cerebellum and Reward. CEREBELLUM (LONDON, ENGLAND) 2024; 23:2169-2192. [PMID: 38769243 DOI: 10.1007/s12311-024-01702-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 05/06/2024] [Indexed: 05/22/2024]
Abstract
Cerebellum is a key-structure for the modulation of motor, cognitive, social and affective functions, contributing to automatic behaviours through interactions with the cerebral cortex, basal ganglia and spinal cord. The predictive mechanisms used by the cerebellum cover not only sensorimotor functions but also reward-related tasks. Cerebellar circuits appear to encode temporal difference error and reward prediction error. From a chemical standpoint, cerebellar catecholamines modulate the rate of cerebellar-based cognitive learning, and mediate cerebellar contributions during complex behaviours. Reward processing and its associated emotions are tuned by the cerebellum which operates as a controller of adaptive homeostatic processes based on interoceptive and exteroceptive inputs. Lobules VI-VII/areas of the vermis are candidate regions for the cortico-subcortical signaling pathways associated with loss aversion and reward sensitivity, together with other nodes of the limbic circuitry. There is growing evidence that the cerebellum works as a hub of regional dysconnectivity across all mood states and that mental disorders involve the cerebellar circuitry, including mood and addiction disorders, and impaired eating behaviors where the cerebellum might be involved in longer time scales of prediction as compared to motor operations. Cerebellar patients exhibit aberrant social behaviour, showing aberrant impulsivity/compulsivity. The cerebellum is a master-piece of reward mechanisms, together with the striatum, ventral tegmental area (VTA) and prefrontal cortex (PFC). Critically, studies on reward processing reinforce our view that a fundamental role of the cerebellum is to construct internal models, perform predictions on the impact of future behaviour and compare what is predicted and what actually occurs.
Collapse
Affiliation(s)
- Mario Manto
- Service de Neurologie, Médiathèque Jean Jacquy, CHU-Charleroi, 6000, Charleroi, Belgium.
- Service Des Neurosciences, Université de Mons, 7000, Mons, Belgium.
- Unité Des Ataxies Cérébelleuses, CHU-Charleroi, Service Des Neurosciences, University of Mons, 7000, Mons, Belgium.
| | - Michael Adamaszek
- Department of Clinical and Cognitive Neurorehabilitation, Klinik Bavaria Kreischa, 01731, Kreischa, Germany
| | - Richard Apps
- School of Physiology, Pharmacology & Neuroscience, Faculty of Life Sciences, University of Bristol, Bristol, BS8 1TD, UK
| | - Erik Carlson
- Department of Psychiatry and Behavioural Sciences, University of Washington, Seattle, WA, 98108, USA
- Geriatric Research, Education and Clinical Center, Veterans Affairs Puget Sound Health Care System, Seattle, WA, 98108, USA
| | - Julian Guarque-Chabrera
- Área de Psicobiología, Facultat de Ciències de La Salut, Universitat Jaume I, 12071, Castellón de La Plana, Spain
- Dominick Purpura Department of Neuroscience, Albert Einstein College of Medicine, New York, 10461, USA
| | - Elien Heleven
- Faculty of Psychology and Center for Neuroscience, Vrije Universiteit Brussel, 1050, Brussels, Belgium
| | - Shinji Kakei
- Department of Anatomy and Physiology, Jissen Women's University, Tokyo, 191-8510, Japan
| | - Kamran Khodakhah
- Dominick Purpura Department of Neuroscience, Albert Einstein College of Medicine, New York, 10461, USA
| | - Sheng-Han Kuo
- Department of Neurology, Columbia University Medical Center, New York, NY, 10032, USA
- Initiative of Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, 10032, USA
| | - Chi-Ying R Lin
- Alzheimer's Disease and Memory Disorders Center, Department of Neurology, Baylor College of Medicine, Houston, 77030 TX, USA
- Parkinson's Disease Center and Movement Disorders Clinic, Department of Neurology, Baylor College of Medicine, Houston, 77030 TX, USA
| | - Mati Joshua
- Edmond and Lily Safra Center for Brain Sciences, The Hebrew University, Jerusalem, Israel
| | - Marta Miquel
- Área de Psicobiología, Facultat de Ciències de La Salut, Universitat Jaume I, 12071, Castellón de La Plana, Spain
- Dominick Purpura Department of Neuroscience, Albert Einstein College of Medicine, New York, 10461, USA
| | - Hiroshi Mitoma
- Department of Medical Education, Tokyo Medical University, Tokyo, 160-8402, Japan
| | - Noga Larry
- Edmond and Lily Safra Center for Brain Sciences, The Hebrew University, Jerusalem, Israel
| | - Julie Anne Péron
- Clinical and Experimental Neuropsychology Laboratory, Department of Psychology and Educational Sciences, University of Geneva, 1205, Geneva, Switzerland
| | - Jasmine Pickford
- School of Physiology, Pharmacology & Neuroscience, Faculty of Life Sciences, University of Bristol, Bristol, BS8 1TD, UK
| | - Dennis J L G Schutter
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands
| | - Manpreet K Singh
- Psychiatry and Behavioral Sciences, University of California Davis, 2230 Stockton Blvd, Sacramento, CA, 95817, USA
| | - Tommy Tan
- Department of Neurology, UT Southwestern Medical Center, Dallas, TX, 75235, USA
| | - Hirokazu Tanaka
- Faculty of Information Technology, Tokyo City University, Tokyo, 158-8557, Japan
| | - Peter Tsai
- Department of Neurology, UT Southwestern Medical Center, Dallas, TX, 75235, USA
- Departments of Neuroscience, Pediatrics, Psychiatry, UT Southwestern Medical Center, Dallas, TX, 75235, USA
| | - Frank Van Overwalle
- Faculty of Psychology and Center for Neuroscience, Vrije Universiteit Brussel, 1050, Brussels, Belgium
| | - Kunihiko Yamashiro
- Department of Neurology, UT Southwestern Medical Center, Dallas, TX, 75235, USA
| |
Collapse
|
4
|
Ipata AE, Fascianelli V, De Zeeuw CI, Sendhilnathan N, Fusi S, Goldberg ME. Purkinje cells in Crus I and II encode the visual stimulus and the impending choice as monkeys learn a reinforcement based visuomotor association task. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.09.13.612926. [PMID: 39314292 PMCID: PMC11419136 DOI: 10.1101/2024.09.13.612926] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/25/2024]
Abstract
Visuomotor association involves linking an arbitrary visual cue to a well-learned movement. Transient inactivation of Crus I/II impairs primates' ability to learn new associations and delays motor responses without affecting the kinematics of the movement. The simple spikes of Purkinje cells in the Crus regions signal cognitive errors as monkeys learn to associate specific fractal stimuli with movements of the left or right hand. Here we show that as learning progresses, the simple spike activity of individual neurons becomes more selective for stimulus-response associations, with selectivity developing closer to the appearance of visual stimuli. Initially, most neurons respond to both associations, irrespective of the identity of the stimulus and the associated movement, but as learning advances, more neurons distinguish between specific stimulus-hand associations. Using a linear decoder, it was found that in early learning stages, the visual stimulus can be decoded only when the choice can also be decoded. As learning improves, the visual stimulus is decoded earlier than the choice. A simple model can replicate the observed simple spike signals and the monkeys' behavior in both the early and late learning stages.
Collapse
|
5
|
Nicholas J, Amlang C, Lin CYR, Montaser-Kouhsari L, Desai N, Pan MK, Kuo SH, Shohamy D. The Role of the Cerebellum in Learning to Predict Reward: Evidence from Cerebellar Ataxia. CEREBELLUM (LONDON, ENGLAND) 2024; 23:1355-1368. [PMID: 38066397 PMCID: PMC11161554 DOI: 10.1007/s12311-023-01633-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/02/2023] [Indexed: 01/25/2024]
Abstract
Recent findings in animals have challenged the traditional view of the cerebellum solely as the site of motor control, suggesting that the cerebellum may also be important for learning to predict reward from trial-and-error feedback. Yet, evidence for the role of the cerebellum in reward learning in humans is lacking. Moreover, open questions remain about which specific aspects of reward learning the cerebellum may contribute to. Here we address this gap through an investigation of multiple forms of reward learning in individuals with cerebellum dysfunction, represented by cerebellar ataxia cases. Nineteen participants with cerebellar ataxia and 57 age- and sex-matched healthy controls completed two separate tasks that required learning about reward contingencies from trial-and-error. To probe the selectivity of reward learning processes, the tasks differed in their underlying structure: while one task measured incremental reward learning ability alone, the other allowed participants to use an alternative learning strategy based on episodic memory alongside incremental reward learning. We found that individuals with cerebellar ataxia were profoundly impaired at reward learning from trial-and-error feedback on both tasks, but retained the ability to learn to predict reward based on episodic memory. These findings provide evidence from humans for a specific and necessary role for the cerebellum in incremental learning of reward associations based on reinforcement. More broadly, the findings suggest that alongside its role in motor learning, the cerebellum likely operates in concert with the basal ganglia to support reinforcement learning from reward.
Collapse
Affiliation(s)
- Jonathan Nicholas
- Department of Psychology, Columbia University, New York, NY, USA
- Zuckerman Mind Brain Behavior Institute, Columbia University, Quad 3D, 3227 Broadway, New York, NY, 10027, USA
| | - Christian Amlang
- Department of Neurology, Columbia University Medical Center, 650 W. 168th St, Rm 305, New York, NY, 10032, USA
- Initiative for Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, USA
| | - Chi-Ying R Lin
- Department of Neurology, Baylor College of Medicine, Houston, TX, USA
| | | | - Natasha Desai
- Department of Neurology, Columbia University Medical Center, 650 W. 168th St, Rm 305, New York, NY, 10032, USA
- Initiative for Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, USA
| | - Ming-Kai Pan
- Department of Medical Research, National Taiwan University Hospital, 100, Taipei, Taiwan
- Department and Graduate Institute of Pharmacology, National Taiwan University College of Medicine, 100, Taipei, Taiwan
- Cerebellar Research Center, National Taiwan University Hospital, Yun-Lin Branch, Yun-Lin, Taiwan
| | - Sheng-Han Kuo
- Department of Neurology, Columbia University Medical Center, 650 W. 168th St, Rm 305, New York, NY, 10032, USA.
- Initiative for Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, USA.
| | - Daphna Shohamy
- Department of Psychology, Columbia University, New York, NY, USA.
- Zuckerman Mind Brain Behavior Institute, Columbia University, Quad 3D, 3227 Broadway, New York, NY, 10027, USA.
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
6
|
White CM, Snow EC, Therrien AS. Reinforcement Motor Learning After Cerebellar Damage Is Related to State Estimation. CEREBELLUM (LONDON, ENGLAND) 2024; 23:1061-1073. [PMID: 37828231 DOI: 10.1007/s12311-023-01615-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 10/03/2023] [Indexed: 10/14/2023]
Abstract
Recent work showed that individuals with cerebellar degeneration could leverage intact reinforcement learning (RL) to alter their movement. However, there was marked inter-individual variability in learning, and the factors underlying it were unclear. Cerebellum-dependent sensory prediction may contribute to RL in motor contexts by enhancing body state estimates, which are necessary to solve the credit-assignment problem. The objective of this study was to test the relationship between the predictive component of state estimation and RL in individuals with cerebellar degeneration. Individuals with cerebellar degeneration and neurotypical control participants completed two tasks: an RL task that required them to alter the angle of reaching movements and a state estimation task that tested the somatosensory perception of active and passive movement. The state estimation task permitted the calculation of the active benefit shown by each participant, which is thought to reflect the cerebellum-dependent predictive component of state estimation. We found that the cerebellar and control groups showed similar magnitudes of learning with reinforcement and active benefit on average, but there was substantial variability across individuals. Using multiple regression, we assessed potential predictors of RL. Our analysis included active benefit, somatosensory acuity, clinical ataxia severity, movement variability, movement speed, and age. We found a significant relationship in which greater active benefit predicted better learning with reinforcement in the cerebellar, but not the control group. No other variables showed significant relationships with learning. Overall, our results support the hypothesis that the integrity of sensory prediction is a strong predictor of RL after cerebellar damage.
Collapse
Affiliation(s)
- Christopher M White
- Moss Rehabilitation Research Institute, Medical Arts Building, Suite 100, 50 Township Line Rd, Elkins Park, PA, USA
| | - Evan C Snow
- Moss Rehabilitation Research Institute, Medical Arts Building, Suite 100, 50 Township Line Rd, Elkins Park, PA, USA
| | - Amanda S Therrien
- Moss Rehabilitation Research Institute, Medical Arts Building, Suite 100, 50 Township Line Rd, Elkins Park, PA, USA.
- Department of Rehabilitation Medicine, Thomas Jefferson University, Philadelphia, PA, USA.
| |
Collapse
|
7
|
Sendhilnathan N, Bostan AC, Strick PL, Goldberg ME. A cerebro-cerebellar network for learning visuomotor associations. Nat Commun 2024; 15:2519. [PMID: 38514616 PMCID: PMC10957870 DOI: 10.1038/s41467-024-46281-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 02/16/2024] [Indexed: 03/23/2024] Open
Abstract
Consensus is rapidly building to support a role for the cerebellum beyond motor function, but its contributions to non-motor learning remain poorly understood. Here, we provide behavioral, anatomical and computational evidence to demonstrate a causal role for the primate posterior lateral cerebellum in learning new visuomotor associations. Reversible inactivation of the posterior lateral cerebellum of male monkeys impeded the learning of new visuomotor associations, but had no effect on movement parameters, or on well-practiced performance of the same task. Using retrograde transneuronal transport of rabies virus, we identified a distinct cerebro-cerebellar network linking Purkinje cells in the posterior lateral cerebellum with a region of the prefrontal cortex that is critical in learning visuomotor associations. Together, these results demonstrate a causal role for the primate posterior lateral cerebellum in non-motor, reinforcement learning.
Collapse
Affiliation(s)
- Naveen Sendhilnathan
- Doctoral program in Neurobiology and Behavior, Columbia University, New York, NY, USA.
- Dept. of Neuroscience, Mahoney Center for Brain and Behavior Research, Zuckerman Mind, Brain, and Behavior Institute, Columbia University, New York, NY, USA.
| | - Andreea C Bostan
- Department of Neurobiology, Systems Neuroscience Center, and Brain Institute, University of Pittsburgh, Pittsburgh, PA, USA.
| | - Peter L Strick
- Department of Neurobiology, Systems Neuroscience Center, and Brain Institute, University of Pittsburgh, Pittsburgh, PA, USA
| | - Michael E Goldberg
- Dept. of Neuroscience, Mahoney Center for Brain and Behavior Research, Zuckerman Mind, Brain, and Behavior Institute, Columbia University, New York, NY, USA
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA
- Dept. of Neurology, Psychiatry, and Ophthalmology, Columbia University College of Physicians and Surgeons, New York, NY, USA
| |
Collapse
|
8
|
Ge F, Wang Z, Yu W, Yuan X, Cai Q, Wang G, Li X, Xu X, Yang P, Fan Y, Chang J, Guan X. Activating Lobule VI PC TH+-Med Pathway in Cerebellum Blocks the Acquisition of Methamphetamine Conditioned Place Preference in Mice. J Neurosci 2024; 44:e1312232024. [PMID: 38331582 PMCID: PMC10941241 DOI: 10.1523/jneurosci.1312-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Revised: 01/26/2024] [Accepted: 01/31/2024] [Indexed: 02/10/2024] Open
Abstract
Cerebellum has been implicated in drug addiction; however, its underlying cellular populations and neuronal circuitry remain largely unknown. In the current study, we identified a neural pathway from tyrosine hydroxylase (TH)-positive Purkinje cells (PCTH+) in cerebellar lobule VI to calcium/calmodulin-dependent protein kinase II (CaMKII)-positive glutamatergic neurons in the medial cerebellar nucleus (MedCaMKII), forming the lobule VI PCTH+-MedCaMKII pathway in male mice. In naive male mice, inhibition of PCTH+ neurons activated Med neurons. During conditioned place preference (CPP) training, exposure to methamphetamine (METH) inhibited lobule VI PCTH+ neurons while excited MedCaMKII neurons in mice. Silencing MedCaMKII using a tetanus toxin light chain (tettox) suppressed the acquisition of METH CPP in mice but resulted in motor coordination deficits in naive mice. In contrast, activating lobule VI PCTH+ terminals within Med inhibited the activity of Med neurons and subsequently blocked the acquisition of METH CPP in mice without affecting motor coordination, locomotor activity, and sucrose reinforcements in naive mice. Our findings identified a novel lobule VI PCTH+-MedCaMKII pathway within the cerebellum and explored its role in mediating the acquisition of METH-preferred behaviors.
Collapse
Affiliation(s)
- Feifei Ge
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Zilin Wang
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Wen Yu
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Xiya Yuan
- The first Affiliated Hospital of Nanjing University of Chinese Medicine, Nanjing 210029, China
| | - Qinglong Cai
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Guanxiong Wang
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Xiang Li
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Xing Xu
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Ping Yang
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Yu Fan
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Jiasong Chang
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| | - Xiaowei Guan
- Department of Human Anatomy and Histoembryology, Nanjing University of Chinese Medicine, Nanjing 210023, China
| |
Collapse
|
9
|
Hu Y, Wang H, Joshua M, Yang Y. Sensorimotor-linked reward modulates smooth pursuit eye movements in monkeys. Front Neurosci 2024; 17:1297914. [PMID: 38264498 PMCID: PMC10803645 DOI: 10.3389/fnins.2023.1297914] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Accepted: 12/18/2023] [Indexed: 01/25/2024] Open
Abstract
Reward is essential for shaping behavior. Using sensory cues to imply forthcoming rewards, previous studies have demonstrated powerful effects of rewards on behavior. Nevertheless, the impact of reward on the sensorimotor transformation, particularly when reward is linked to behavior remains uncertain. In this study, we investigated how reward modulates smooth pursuit eye movements in monkeys. Three distinct associations between reward and eye movements were conducted in independent blocks. Results indicated that reward increased eye velocity during the steady-state pursuit, rather than during the initiation. The influence depended on the particular association between behavior and reward: a faster eye velocity was linked with reward. Neither rewarding slower eye movements nor randomizing rewards had a significant effect on behavior. The findings support the existence of distinct mechanisms involved in the initiation and steady-state phases of pursuit, and contribute to a deeper understanding of how reward interacts with these two periods of pursuit.
Collapse
Affiliation(s)
- Yongxiang Hu
- Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
| | - Huan Wang
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
| | - Mati Joshua
- Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Yan Yang
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
- Sino-Danish College, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
10
|
Hoang H, Tsutsumi S, Matsuzaki M, Kano M, Kawato M, Kitamura K, Toyama K. Dynamic organization of cerebellar climbing fiber response and synchrony in multiple functional components reduces dimensions for reinforcement learning. eLife 2023; 12:e86340. [PMID: 37712651 PMCID: PMC10531405 DOI: 10.7554/elife.86340] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Accepted: 09/13/2023] [Indexed: 09/16/2023] Open
Abstract
Cerebellar climbing fibers convey diverse signals, but how they are organized in the compartmental structure of the cerebellar cortex during learning remains largely unclear. We analyzed a large amount of coordinate-localized two-photon imaging data from cerebellar Crus II in mice undergoing 'Go/No-go' reinforcement learning. Tensor component analysis revealed that a majority of climbing fiber inputs to Purkinje cells were reduced to only four functional components, corresponding to accurate timing control of motor initiation related to a Go cue, cognitive error-based learning, reward processing, and inhibition of erroneous behaviors after a No-go cue. Changes in neural activities during learning of the first two components were correlated with corresponding changes in timing control and error learning across animals, indirectly suggesting causal relationships. Spatial distribution of these components coincided well with boundaries of Aldolase-C/zebrin II expression in Purkinje cells, whereas several components are mixed in single neurons. Synchronization within individual components was bidirectionally regulated according to specific task contexts and learning stages. These findings suggest that, in close collaborations with other brain regions including the inferior olive nucleus, the cerebellum, based on anatomical compartments, reduces dimensions of the learning space by dynamically organizing multiple functional components, a feature that may inspire new-generation AI designs.
Collapse
Affiliation(s)
- Huu Hoang
- ATR Neural Information Analysis LaboratoriesKyotoJapan
| | | | | | - Masanobu Kano
- Department of Neurophysiology, The University of TokyoTokyoJapan
- International Research Center for Neurointelligence (WPI-IRCN), The University of TokyoTokyoJapan
| | - Mitsuo Kawato
- ATR Brain Information Communication Research Laboratory GroupKyotoJapan
| | - Kazuo Kitamura
- Department of Neurophysiology, University of YamanashiKofuJapan
| | | |
Collapse
|
11
|
Sendhilnathan N, Goldberg ME, Ipata AE. Mixed Selectivity in the Cerebellar Purkinje-Cell Response during Visuomotor Association Learning. J Neurosci 2022; 42:3847-3855. [PMID: 35351828 PMCID: PMC9087720 DOI: 10.1523/jneurosci.1771-21.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Revised: 02/18/2022] [Accepted: 03/18/2022] [Indexed: 11/21/2022] Open
Abstract
Although the cerebellum has been traditionally considered to be exclusively involved in motor control, recent anatomic and clinical studies show that it also has a role in reward-processing. However, the way in which the movement-related and the reward-related neural activity interact at the level of the cerebellar cortex and contribute toward learning is still unclear. Here, we studied the simple spike activity of Purkinje cells in the mid-lateral cerebellum when 2 male monkeys learned to associate a right or left-hand movement with one of two visual symbolic cues. These cells had distinctly different discharge patterns between an overtrained symbol-hand association and a novel symbol-hand association, responding in association with the movement of both hands, although the kinematics of the movement did not change between the two conditions. The activity change was not related to the pattern of the visual symbols, the movement kinematics, the monkeys' reaction times, or the novelty of the visual symbols. The simple spike activity changed throughout the learning process, but the concurrent complex spikes did not instruct that change. Although these neurons also have reward-related activity, the reward-related and movement-related signals were independent. We suggest that this mixed selectivity may facilitate the flexible learning of difficult reinforcement learning problems.SIGNIFICANCE STATEMENT The cerebellum receives both motor-related and reward-related information. However, it is unclear how these two signals interact at the level of cerebellar cortex and contribute to learning nonmotor skills. Here we show that in the mid-lateral cerebellum, the reward information is encoded independently from the motor information such that during reward-based learning, only the reward information carried by the Purkinje cells inform learning while the motor information remains unchanged with learning.
Collapse
Affiliation(s)
- Naveen Sendhilnathan
- Doctoral Program in Neurobiology and Behavior, Columbia University, New York 10027, New York
- Department of Neuroscience, Columbia University, New York 10027, New York
- Mahoney Center for Brain and Behavior Research, Columbia University, New York 10032, New York
- Zuckerman Mind, Brain, and Behavior Institute, Columbia University, New York 10027, New York
| | - Michael E Goldberg
- Department of Neuroscience, Columbia University, New York 10027, New York
- Mahoney Center for Brain and Behavior Research, Columbia University, New York 10032, New York
- Kavli Institute for Brain Science, Columbia University, New York 10027, New York
- Zuckerman Mind, Brain, and Behavior Institute, Columbia University, New York 10027, New York
- Department of Neurology, Psychiatry, and Ophthalmology, Columbia University College of Physicians and Surgeons, New York 10032, New York
| | - Anna E Ipata
- Department of Neuroscience, Columbia University, New York 10027, New York
- Mahoney Center for Brain and Behavior Research, Columbia University, New York 10032, New York
- Zuckerman Mind, Brain, and Behavior Institute, Columbia University, New York 10027, New York
| |
Collapse
|
12
|
Kostadinov D, Häusser M. Reward signals in the cerebellum: origins, targets, and functional implications. Neuron 2022; 110:1290-1303. [PMID: 35325616 DOI: 10.1016/j.neuron.2022.02.015] [Citation(s) in RCA: 39] [Impact Index Per Article: 19.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2021] [Revised: 12/22/2021] [Accepted: 02/16/2022] [Indexed: 12/14/2022]
Abstract
The cerebellum has long been proposed to play a role in cognitive function, although this has remained controversial. This idea has received renewed support with the recent discovery that signals associated with reward can be observed in the cerebellar circuitry, particularly in goal-directed learning tasks involving an interplay between the cerebellar cortex, basal ganglia, and cerebral cortex. Remarkably, a wide range of reward contingencies-including reward expectation, delivery, size, and omission-can be encoded by specific circuit elements in a manner that reflects the microzonal organization of the cerebellar cortex. The facts that reward signals have been observed in both the mossy fiber and climbing fiber input pathways to the cerebellar cortex and that their convergence may trigger plasticity in Purkinje cells suggest that these interactions may be crucial for the role of the cerebellar cortex in learned behavior. These findings strengthen the emerging consensus that the cerebellum plays a pivotal role in shaping cognitive processing and suggest that the cerebellum may combine both supervised learning and reinforcement learning to optimize goal-directed action. We make specific predictions about how cerebellar circuits can work in concert with the basal ganglia to guide different stages of learning.
Collapse
Affiliation(s)
- Dimitar Kostadinov
- Wolfson Institute for Biomedical Research and Department of Neuroscience, Physiology and Pharmacology, University College London, London, UK.
| | - Michael Häusser
- Wolfson Institute for Biomedical Research and Department of Neuroscience, Physiology and Pharmacology, University College London, London, UK.
| |
Collapse
|