1
|
White CM, Snow EC, Therrien AS. Reinforcement Motor Learning After Cerebellar Damage Is Related to State Estimation. CEREBELLUM (LONDON, ENGLAND) 2024; 23:1061-1073. [PMID: 37828231 DOI: 10.1007/s12311-023-01615-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 10/03/2023] [Indexed: 10/14/2023]
Abstract
Recent work showed that individuals with cerebellar degeneration could leverage intact reinforcement learning (RL) to alter their movement. However, there was marked inter-individual variability in learning, and the factors underlying it were unclear. Cerebellum-dependent sensory prediction may contribute to RL in motor contexts by enhancing body state estimates, which are necessary to solve the credit-assignment problem. The objective of this study was to test the relationship between the predictive component of state estimation and RL in individuals with cerebellar degeneration. Individuals with cerebellar degeneration and neurotypical control participants completed two tasks: an RL task that required them to alter the angle of reaching movements and a state estimation task that tested the somatosensory perception of active and passive movement. The state estimation task permitted the calculation of the active benefit shown by each participant, which is thought to reflect the cerebellum-dependent predictive component of state estimation. We found that the cerebellar and control groups showed similar magnitudes of learning with reinforcement and active benefit on average, but there was substantial variability across individuals. Using multiple regression, we assessed potential predictors of RL. Our analysis included active benefit, somatosensory acuity, clinical ataxia severity, movement variability, movement speed, and age. We found a significant relationship in which greater active benefit predicted better learning with reinforcement in the cerebellar, but not the control group. No other variables showed significant relationships with learning. Overall, our results support the hypothesis that the integrity of sensory prediction is a strong predictor of RL after cerebellar damage.
Collapse
Affiliation(s)
- Christopher M White
- Moss Rehabilitation Research Institute, Medical Arts Building, Suite 100, 50 Township Line Rd, Elkins Park, PA, USA
| | - Evan C Snow
- Moss Rehabilitation Research Institute, Medical Arts Building, Suite 100, 50 Township Line Rd, Elkins Park, PA, USA
| | - Amanda S Therrien
- Moss Rehabilitation Research Institute, Medical Arts Building, Suite 100, 50 Township Line Rd, Elkins Park, PA, USA.
- Department of Rehabilitation Medicine, Thomas Jefferson University, Philadelphia, PA, USA.
| |
Collapse
|
2
|
Vassiliadis P, Beanato E, Popa T, Windel F, Morishita T, Neufeld E, Duque J, Derosiere G, Wessel MJ, Hummel FC. Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills. Nat Hum Behav 2024:10.1038/s41562-024-01901-z. [PMID: 38811696 DOI: 10.1038/s41562-024-01901-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 04/23/2024] [Indexed: 05/31/2024]
Abstract
Reinforcement feedback can improve motor learning, but the underlying brain mechanisms remain underexplored. In particular, the causal contribution of specific patterns of oscillatory activity within the human striatum is unknown. To address this question, we exploited a recently developed non-invasive deep brain stimulation technique called transcranial temporal interference stimulation (tTIS) during reinforcement motor learning with concurrent neuroimaging, in a randomized, sham-controlled, double-blind study. Striatal tTIS applied at 80 Hz, but not at 20 Hz, abolished the benefits of reinforcement on motor learning. This effect was related to a selective modulation of neural activity within the striatum. Moreover, 80 Hz, but not 20 Hz, tTIS increased the neuromodulatory influence of the striatum on frontal areas involved in reinforcement motor learning. These results show that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.
Collapse
Affiliation(s)
- Pierre Vassiliadis
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
- Institute of Neuroscience, Université Catholique de Louvain, Brussels, Belgium
| | - Elena Beanato
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
| | - Traian Popa
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
| | - Fabienne Windel
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
| | - Takuya Morishita
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
| | - Esra Neufeld
- Foundation for Research on Information Technologies in Society, Zurich, Switzerland
| | - Julie Duque
- Institute of Neuroscience, Université Catholique de Louvain, Brussels, Belgium
| | - Gerard Derosiere
- Institute of Neuroscience, Université Catholique de Louvain, Brussels, Belgium
- Lyon Neuroscience Research Center, Impact Team, Inserm U1028, CNRS UMR5292, Lyon 1 University, Bron, France
| | - Maximilian J Wessel
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
- Department of Neurology, University Hospital Würzburg, Würzburg, Germany
| | - Friedhelm C Hummel
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland.
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland.
- Clinical Neuroscience, University of Geneva Medical School, Geneva, Switzerland.
| |
Collapse
|
3
|
Yang X, Song Y, Zou Y, Li Y, Zeng J. Neural correlates of prediction error in patients with schizophrenia: evidence from an fMRI meta-analysis. Cereb Cortex 2024; 34:bhad471. [PMID: 38061699 DOI: 10.1093/cercor/bhad471] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 11/07/2023] [Accepted: 11/09/2023] [Indexed: 01/19/2024] Open
Abstract
Abnormal processes of learning from prediction errors, i.e. the discrepancies between expectations and outcomes, are thought to underlie motivational impairments in schizophrenia. Although dopaminergic abnormalities in the mesocorticolimbic reward circuit have been found in patients with schizophrenia, the pathway through which prediction error signals are processed in schizophrenia has yet to be elucidated. To determine the neural correlates of prediction error processing in schizophrenia, we conducted a meta-analysis of whole-brain neuroimaging studies that investigated prediction error signal processing in schizophrenia patients and healthy controls. A total of 14 studies (324 schizophrenia patients and 348 healthy controls) using the reinforcement learning paradigm were included. Our meta-analysis showed that, relative to healthy controls, schizophrenia patients showed increased activity in the precentral gyrus and middle frontal gyrus and reduced activity in the mesolimbic circuit, including the striatum, thalamus, amygdala, hippocampus, anterior cingulate cortex, insula, superior temporal gyrus, and cerebellum, when processing prediction errors. We also found hyperactivity in frontal areas and hypoactivity in mesolimbic areas when encoding prediction error signals in schizophrenia patients, potentially indicating abnormal dopamine signaling of reward prediction error and suggesting failure to represent the value of alternative responses during prediction error learning and decision making.
Collapse
Affiliation(s)
- Xun Yang
- School of Public Policy and Administration, Chongqing University, No. 174, Shazhengjie, Shapingba, Chongqing, China
| | - Yuan Song
- School of Public Policy and Administration, Chongqing University, No. 174, Shazhengjie, Shapingba, Chongqing, China
| | - Yuhan Zou
- School of Economics and Business Administration, Chongqing University, No. 174, Shazhengjie, Shapingba, Chongqing, China
| | - Yilin Li
- Psychology and Neuroscience Department, University of St Andrews, Forbes 1 DRA, Buchanan Garden, St Andrews, Fife, United Kingdom
| | - Jianguang Zeng
- School of Economics and Business Administration, Chongqing University, No. 174, Shazhengjie, Shapingba, Chongqing, China
| |
Collapse
|
4
|
Vassiliadis P, Lete A, Duque J, Derosiere G. Reward timing matters in motor learning. iScience 2022; 25:104290. [PMID: 35573187 PMCID: PMC9095742 DOI: 10.1016/j.isci.2022.104290] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Revised: 02/25/2022] [Accepted: 04/20/2022] [Indexed: 12/01/2022] Open
Abstract
Reward timing, that is, the delay after which reward is delivered following an action is known to strongly influence reinforcement learning. Here, we asked if reward timing could also modulate how people learn and consolidate new motor skills. In 60 healthy participants, we found that delaying reward delivery by a few seconds influenced motor learning. Indeed, training with a short reward delay (1 s) induced continuous improvements in performance, whereas a long reward delay (6 s) led to initially high learning rates that were followed by an early plateau in the learning curve and a lower performance at the end of training. Participants who learned the skill with a long reward delay also exhibited reduced overnight memory consolidation. Overall, our data show that reward timing affects the dynamics and consolidation of motor learning, a finding that could be exploited in future rehabilitation programs.
Collapse
Affiliation(s)
- Pierre Vassiliadis
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, 1200 Brussels, Belgium
- Defitech Chair for Clinical Neuroengineering, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology (EPFL), 1202 Geneva, Switzerland
| | - Aegryan Lete
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, 1200 Brussels, Belgium
| | - Julie Duque
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, 1200 Brussels, Belgium
| | - Gerard Derosiere
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, 1200 Brussels, Belgium
| |
Collapse
|
5
|
Vassiliadis P, Derosiere G, Dubuc C, Lete A, Crevecoeur F, Hummel FC, Duque J. Reward boosts reinforcement-based motor learning. iScience 2021; 24:102821. [PMID: 34345810 PMCID: PMC8319366 DOI: 10.1016/j.isci.2021.102821] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Revised: 06/16/2021] [Accepted: 07/02/2021] [Indexed: 11/25/2022] Open
Abstract
Besides relying heavily on sensory and reinforcement feedback, motor skill learning may also depend on the level of motivation experienced during training. Yet, how motivation by reward modulates motor learning remains unclear. In 90 healthy subjects, we investigated the net effect of motivation by reward on motor learning while controlling for the sensory and reinforcement feedback received by the participants. Reward improved motor skill learning beyond performance-based reinforcement feedback. Importantly, the beneficial effect of reward involved a specific potentiation of reinforcement-related adjustments in motor commands, which concerned primarily the most relevant motor component for task success and persisted on the following day in the absence of reward. We propose that the long-lasting effects of motivation on motor learning may entail a form of associative learning resulting from the repetitive pairing of the reinforcement feedback and reward during training, a mechanism that may be exploited in future rehabilitation protocols.
Collapse
Affiliation(s)
- Pierre Vassiliadis
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, Brussels 1200, Belgium
- Defitech Chair for Clinical Neuroengineering, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology (EPFL), Geneva 1202, Switzerland
| | - Gerard Derosiere
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, Brussels 1200, Belgium
| | - Cecile Dubuc
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, Brussels 1200, Belgium
| | - Aegryan Lete
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, Brussels 1200, Belgium
| | - Frederic Crevecoeur
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, Brussels 1200, Belgium
- Institute of Information and Communication Technologies, Electronics and Applied Mathematics, Université Catholique de Louvain, Louvain-la-Neuve 1348, Belgium
| | - Friedhelm C. Hummel
- Defitech Chair for Clinical Neuroengineering, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology (EPFL), Geneva 1202, Switzerland
- Defitech Chair for Clinical Neuroengineering, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology Sion (EPFL), Sion 1951, Switzerland
- Clinical Neuroscience, University of Geneva Medical School (HUG), Geneva 1202, Switzerland
| | - Julie Duque
- Institute of Neuroscience, Université Catholique de Louvain, 53, Avenue Mounier, Brussels 1200, Belgium
| |
Collapse
|