1
|
Vassiliadis P, Beanato E, Popa T, Windel F, Morishita T, Neufeld E, Duque J, Derosiere G, Wessel MJ, Hummel FC. Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills. Nat Hum Behav 2024:10.1038/s41562-024-01901-z. [PMID: 38811696 DOI: 10.1038/s41562-024-01901-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 04/23/2024] [Indexed: 05/31/2024]
Abstract
Reinforcement feedback can improve motor learning, but the underlying brain mechanisms remain underexplored. In particular, the causal contribution of specific patterns of oscillatory activity within the human striatum is unknown. To address this question, we exploited a recently developed non-invasive deep brain stimulation technique called transcranial temporal interference stimulation (tTIS) during reinforcement motor learning with concurrent neuroimaging, in a randomized, sham-controlled, double-blind study. Striatal tTIS applied at 80 Hz, but not at 20 Hz, abolished the benefits of reinforcement on motor learning. This effect was related to a selective modulation of neural activity within the striatum. Moreover, 80 Hz, but not 20 Hz, tTIS increased the neuromodulatory influence of the striatum on frontal areas involved in reinforcement motor learning. These results show that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.
Collapse
Affiliation(s)
- Pierre Vassiliadis
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
- Institute of Neuroscience, Université Catholique de Louvain, Brussels, Belgium
| | - Elena Beanato
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
| | - Traian Popa
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
| | - Fabienne Windel
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
| | - Takuya Morishita
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
| | - Esra Neufeld
- Foundation for Research on Information Technologies in Society, Zurich, Switzerland
| | - Julie Duque
- Institute of Neuroscience, Université Catholique de Louvain, Brussels, Belgium
| | - Gerard Derosiere
- Institute of Neuroscience, Université Catholique de Louvain, Brussels, Belgium
- Lyon Neuroscience Research Center, Impact Team, Inserm U1028, CNRS UMR5292, Lyon 1 University, Bron, France
| | - Maximilian J Wessel
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland
- Department of Neurology, University Hospital Würzburg, Würzburg, Germany
| | - Friedhelm C Hummel
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland.
- Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland.
- Clinical Neuroscience, University of Geneva Medical School, Geneva, Switzerland.
| |
Collapse
|
2
|
Leow LA, Bernheine L, Carroll TJ, Dux PE, Filmer HL. Dopamine Increases Accuracy and Lengthens Deliberation Time in Explicit Motor Skill Learning. eNeuro 2024; 11:ENEURO.0360-23.2023. [PMID: 38238069 PMCID: PMC10849023 DOI: 10.1523/eneuro.0360-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2023] [Revised: 11/23/2023] [Accepted: 11/28/2023] [Indexed: 01/23/2024] Open
Abstract
Although animal research implicates a central role for dopamine in motor skill learning, a direct causal link has yet to be established in neurotypical humans. Here, we tested if a pharmacological manipulation of dopamine alters motor learning, using a paradigm which engaged explicit, goal-directed strategies. Participants (27 females; 11 males; aged 18-29 years) first consumed either 100 mg of levodopa (n = 19), a dopamine precursor that increases dopamine availability, or placebo (n = 19). Then, during training, participants learnt the explicit strategy of aiming away from presented targets by instructed angles of varying sizes. Targets jumped mid-movement by the instructed aiming angle. Task success was thus contingent upon aiming accuracy and not speed. The effect of the dopamine manipulations on skill learning was assessed during training and after an overnight follow-up. Increasing dopamine availability at training improved aiming accuracy and lengthened reaction times, particularly for larger, more difficult aiming angles, both at training and, importantly, at follow-up, despite prominent session-by-session performance improvements in both accuracy and speed. Exogenous dopamine thus seems to result in a learnt, persistent propensity to better adhere to task goals. Results support the proposal that dopamine is important in engagement of instrumental motivation to optimize adherence to task goals, particularly when learning to execute goal-directed strategies in motor skill learning.
Collapse
Affiliation(s)
- Li-Ann Leow
- School of Psychology, The University of Queensland, St Lucia, 4072, Australia
- Centre for Sensorimotor Performance, School of Human Movement & Nutrition Sciences, St Lucia, 4067, Australia
| | - Lena Bernheine
- Centre for Sensorimotor Performance, School of Human Movement & Nutrition Sciences, St Lucia, 4067, Australia
- School of Sport Science Faculty of Sport Governance and Event Management, University of Bayreuth, 95447 Bayreuth, Germany
| | - Timothy J Carroll
- Centre for Sensorimotor Performance, School of Human Movement & Nutrition Sciences, St Lucia, 4067, Australia
| | - Paul E Dux
- School of Psychology, The University of Queensland, St Lucia, 4072, Australia
| | - Hannah L Filmer
- School of Psychology, The University of Queensland, St Lucia, 4072, Australia
| |
Collapse
|
3
|
Kalidindi HT, Crevecoeur F. Human reaching control in dynamic environments. Curr Opin Neurobiol 2023; 83:102810. [PMID: 37950956 DOI: 10.1016/j.conb.2023.102810] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Revised: 10/09/2023] [Accepted: 10/19/2023] [Indexed: 11/13/2023]
Abstract
Closed-loop models of movement control have attracted growing interest in how the nervous system transforms sensory information into motor commands, and several brain structures have been identified as neural substrates for these computational operations. Recently, several studies have focused on how these models need to be updated when environmental parameters change. Current evidence suggests that when the task changes, rapid control updates enable flexible modifications of current actions and online decisions. At the same time, when movement dynamics change, humans use different strategies based on a combination of adaptation and modulation of controller sensitivity to exogenous perturbations (robust control). This review proposes a unified framework to capture these results based on online estimation of model parameters with dynamic updates in control. The reviewed studies also identify the time scales of associated behavioral mechanisms to guide future research on the neural basis of movement control.
Collapse
Affiliation(s)
- Hari T Kalidindi
- Institute of Information and Communication Technologies, Electronics and Applied Mathematics, University of Louvain (UCLouvain), Belgium; Institute of Neuroscience, UCLouvain, Belgium
| | - Frédéric Crevecoeur
- Institute of Information and Communication Technologies, Electronics and Applied Mathematics, University of Louvain (UCLouvain), Belgium; Institute of Neuroscience, UCLouvain, Belgium.
| |
Collapse
|
4
|
De Comite A, Lefèvre P, Crevecoeur F. Continuous evaluation of cost-to-go for flexible reaching control and online decisions. PLoS Comput Biol 2023; 19:e1011493. [PMID: 37756355 PMCID: PMC10561875 DOI: 10.1371/journal.pcbi.1011493] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Revised: 10/09/2023] [Accepted: 09/05/2023] [Indexed: 09/29/2023] Open
Abstract
Humans consider the parameters linked to movement goal during reaching to adjust their control strategy online. Indeed, rapid changes in target structure or disturbances interfering with their initial plan elicit rapid changes in behavior. Here, we hypothesize that these changes could result from the continuous use of a decision variable combining motor and cognitive components. We combine an optimal feedback controller with a real-time evaluation of the expected cost-to-go, which considers target- and movement-related costs, in a common theoretical framework. This model reproduces human behaviors in presence of changes in the target structure occurring during movement and of online decisions to flexibly change target following external perturbations. It also predicts that the time taken to decide to select a novel goal after a perturbation depends on the amplitude of the disturbance and on the rewards of the different options, which is a direct result of the continuous monitoring of the cost-to-go. We show that this result was present in our previously collected dataset. Together our developments point towards a continuous evaluation of the cost-to-go during reaching to update control online and make efficient decisions about movement goal.
Collapse
Affiliation(s)
- Antoine De Comite
- Institute of Neuroscience, UCLouvain, Louvain-la-Neuve, Belgium
- Institute of Information and Communication Technologies, Electronics and Applied Mathematics, UCLouvain, Louvain-la-Neuve, Belgium
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
| | - Philippe Lefèvre
- Institute of Neuroscience, UCLouvain, Louvain-la-Neuve, Belgium
- Institute of Information and Communication Technologies, Electronics and Applied Mathematics, UCLouvain, Louvain-la-Neuve, Belgium
| | - Frédéric Crevecoeur
- Institute of Neuroscience, UCLouvain, Louvain-la-Neuve, Belgium
- Institute of Information and Communication Technologies, Electronics and Applied Mathematics, UCLouvain, Louvain-la-Neuve, Belgium
| |
Collapse
|
5
|
Al-Fawakhiri N, Ma A, Taylor JA, Kim OA. Exploring the role of task success in implicit motor adaptation. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.02.01.526533. [PMID: 36778277 PMCID: PMC9915693 DOI: 10.1101/2023.02.01.526533] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
We learn to improve our motor skills using different forms of feedback: sensory-prediction error, task success, and reward/punishment. While implicit motor adaptation is driven by sensory-prediction errors, recent work has shown that task success modulates this process. Task success is often confounded with reward, so we sought to determine if the effects of these two signals on adaptation can be dissociated. To address this question, we conducted five experiments that isolated implicit learning using error-clamp visuomotor reach adaptation paradigms. Task success was manipulated by changing the size and position of the target relative to the cursor providing visual feedback, and reward expectation was established using monetary cues and auditory feedback. We found that neither monetary cues nor auditory feedback affected implicit adaptation, suggesting that task success influences implicit adaptation via mechanisms distinct from conventional reward-related processes. Additionally, we found that changes in target size, which caused the target to either exclude or fully envelop the cursor, only affected implicit adaptation for a narrow range of error sizes, while jumping the target to overlap with the cursor more reliably and robustly affected implicit adaptation. Taken together, our data indicate that, while task success exerts a small effect on implicit adaptation, these effects are susceptible to methodological variations and unlikely to be mediated by reward.
Collapse
Affiliation(s)
| | - Ambri Ma
- Department of Psychology, Princeton University, Princeton, NJ 08544
| | - Jordan A Taylor
- Department of Psychology, Princeton University, Princeton, NJ 08544
| | - Olivia A Kim
- Department of Psychology, Princeton University, Princeton, NJ 08544
| |
Collapse
|