Learning Similar Actions by Reinforcement or Sensory-Prediction Errors Rely on Distinct Physiological Mechanisms

Reinforcement Motor Learning After Cerebellar Damage Is Related to State Estimation

Recent work showed that individuals with cerebellar degeneration could leverage intact reinforcement learning (RL) to alter their movement. However, there was marked inter-individual variability in learning, and the factors underlying it were unclear. Cerebellum-dependent sensory prediction may contribute to RL in motor contexts by enhancing body state estimates, which are necessary to solve the credit-assignment problem. The objective of this study was to test the relationship between the predictive component of state estimation and RL in individuals with cerebellar degeneration. Individuals with cerebellar degeneration and neurotypical control participants completed two tasks: an RL task that required them to alter the angle of reaching movements and a state estimation task that tested the somatosensory perception of active and passive movement. The state estimation task permitted the calculation of the active benefit shown by each participant, which is thought to reflect the cerebellum-dependent predictive component of state estimation. We found that the cerebellar and control groups showed similar magnitudes of learning with reinforcement and active benefit on average, but there was substantial variability across individuals. Using multiple regression, we assessed potential predictors of RL. Our analysis included active benefit, somatosensory acuity, clinical ataxia severity, movement variability, movement speed, and age. We found a significant relationship in which greater active benefit predicted better learning with reinforcement in the cerebellar, but not the control group. No other variables showed significant relationships with learning. Overall, our results support the hypothesis that the integrity of sensory prediction is a strong predictor of RL after cerebellar damage.

Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills

Reinforcement feedback can improve motor learning, but the underlying brain mechanisms remain underexplored. In particular, the causal contribution of specific patterns of oscillatory activity within the human striatum is unknown. To address this question, we exploited a recently developed non-invasive deep brain stimulation technique called transcranial temporal interference stimulation (tTIS) during reinforcement motor learning with concurrent neuroimaging, in a randomized, sham-controlled, double-blind study. Striatal tTIS applied at 80 Hz, but not at 20 Hz, abolished the benefits of reinforcement on motor learning. This effect was related to a selective modulation of neural activity within the striatum. Moreover, 80 Hz, but not 20 Hz, tTIS increased the neuromodulatory influence of the striatum on frontal areas involved in reinforcement motor learning. These results show that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.

Corticostriatal activity related to performance during continuous de novo motor learning

Corticostriatal regions play a pivotal role in visuomotor learning. However, less research has been done on how fMRI activity in their subregions is related to task performance, which is provided as visual feedback during motor learning. To address this, we conducted an fMRI experiment in which participants acquired a complex de novo motor skill using continuous or binary visual feedback related to performance. We found a highly selective response related to performance in the entire striatum in both conditions and a relatively higher response in the caudate nucleus for the binary feedback condition. However, the ventromedial prefrontal cortex (vmPFC) response was significant only for the continuous feedback condition. Furthermore, we also found functional distinction of the striatal subregions in random versus goal-directed motor control. These findings underscore the substantial effects of the visual feedback indicating performance on distinct corticostriatal responses, thereby elucidating its significance in reinforcement-based motor learning.

Dorsomedial frontal cortex damage impairs error-based, but not reinforcement-based motor learning in humans

We adapt our movements to new and changing environments through multiple processes. Sensory error-based learning counteracts environmental perturbations that affect the sensory consequences of movements. Sensory errors also cause the upregulation of reflexes and muscle co-contraction. Reinforcement-based learning enhances the selection of movements that produce rewarding outcomes. Although some findings have identified dissociable neural substrates of sensory error- and reinforcement-based learning, correlative methods have implicated dorsomedial frontal cortex in both. Here, we tested the causal contributions of dorsomedial frontal to adaptive motor control, studying people with chronic damage to this region. Seven human participants with focal brain lesions affecting the dorsomedial frontal and 20 controls performed a battery of arm movement tasks. Three experiments tested: (i) the upregulation of visuomotor reflexes and muscle co-contraction in response to unpredictable mechanical perturbations, (ii) sensory error-based learning in which participants learned to compensate predictively for mechanical force-field perturbations, and (iii) reinforcement-based motor learning based on binary feedback in the absence of sensory error feedback. Participants with dorsomedial frontal damage were impaired in the early stages of force field adaptation, but performed similarly to controls in all other measures. These results provide evidence for a specific and selective causal role for the dorsomedial frontal in sensory error-based learning.

The Cerebellum and the Motor Cortex: Multiple Networks Controlling Multiple Aspects of Behavior

The cerebellum and its thalamic projections to the primary motor cortex (M1) are well known to play an essential role in executing daily actions. Anatomic investigations in animals and postmortem humans have established the reciprocal connections between these regions; however, how these pathways can shape cortical activity in behavioral contexts and help promote recovery in neuropathological conditions remains not well understood. The present review aims to provide a comprehensive description of these pathways in animals and humans and discuss how novel noninvasive brain stimulation (NIBS) methods can be used to gain a deeper understanding of the cerebellar-M1 connections. In the first section, we focus on recent animal literature that details how information sent from the cerebellum and thalamus is integrated into an broad network of cortical motor neurons. We then discuss how NIBS approaches in humans can be used to reliably assess the connectivity between the cerebellum and M1. Moreover, we provide the latest perspectives on using advanced NIBS approaches to investigate and modulate multiple cerebellar-cortical networks involved in movement behavior and plasticity. Finally, we discuss how these emerging methods have been used in translation research to produce long-lasting modifications of cerebellar-thalamic-M1 to restore cortical activity and motor function in neurologic patients.

Imaging the cerebellum in post-traumatic stress and anxiety disorders: a mini-review

Post-traumatic stress disorder (PTSD) and anxiety disorders are among the most prevalent psychiatric conditions worldwide sharing many clinical manifestations and, most likely, neural mechanisms as suggested by neuroimaging research. While the so-called fear circuitry and traditional limbic structures of the brain, particularly the amygdala, have been extensively studied in sufferers of these disorders, the cerebellum has been relatively underexplored. The aim of this paper was to present a mini-review of functional (task-activity or resting-state connectivity) and structural (gray matter volume) results on the cerebellum as reported in magnetic resonance imaging studies of patients with PTSD or anxiety disorders (49 selected studies in 1,494 patients). While mixed results were noted overall, e.g., regarding the direction of effects and anatomical localization, cerebellar structures like the vermis seem to be highly involved. Still, the neurofunctional and structural alterations reported for the cerebellum in excessive anxiety and trauma are complex, and in need of further evaluation.

Cerebellar Excitability Regulates Physical Fatigue Perception

Fatigue is the subjective sensation of weariness, increased sense of effort, or exhaustion and is pervasive in neurologic illnesses. Despite its prevalence, we have a limited understanding of the neurophysiological mechanisms underlying fatigue. The cerebellum, known for its role in motor control and learning, is also involved in perceptual processes. However, the role of the cerebellum in fatigue remains largely unexplored. We performed two experiments to examine whether cerebellar excitability is affected after a fatiguing task and its association with fatigue. Using a crossover design, we assessed cerebellar inhibition (CBI) and perception of fatigue in humans before and after "fatigue" and "control" tasks. Thirty-three participants (16 males, 17 females) performed five isometric pinch trials with their thumb and index finger at 80% maximum voluntary capacity (MVC) until failure (force <40% MVC; fatigue) or at 5% MVC for 30 s (control). We found that reduced CBI after the fatigue task correlated with a milder perception of fatigue. In a follow-up experiment, we investigated the behavioral consequences of reduced CBI after fatigue. We measured CBI, perception of fatigue, and performance during a ballistic goal-directed task before and after the same fatigue and control tasks. We replicated the observation that reduced CBI after the fatigue task correlated with a milder perception of fatigue and found that greater endpoint variability after the fatigue task correlated with reduced CBI. The proportional relation between cerebellar excitability and fatigue indicates a role of the cerebellum in the perception of fatigue, which might come at the expense of motor control.SIGNIFICANCE STATEMENT Fatigue is one of the most common and debilitating symptoms in neurologic, neuropsychiatric, and chronic illnesses. Despite its epidemiological importance, there is a limited understanding of the neurophysiological mechanisms underlying fatigue. In a series of experiments, we demonstrate that decreased cerebellar excitability relates to lesser physical fatigue perception and worse motor control. These results showcase the role of the cerebellum in fatigue regulation and suggest that fatigue- and performance-related processes might compete for cerebellar resources.

Cerebellar TMS Induces Motor Responses Mediating Modulation of Spinal Excitability: A Literature Review

Since individuals with cerebellar lesions often exhibit hypotonia, the cerebellum may contribute to the regulation of muscle tone and spinal motoneuron pool excitability. Neurophysiological methods using transcranial magnetic stimulation (TMS) of the cerebellum have been recently proposed for testing the role of the cerebellum in spinal excitability. Under specific conditions, single-pulse TMS administered to the cerebellar hemisphere or vermis elicits a long-latency motor response in the upper or lower limb muscles and facilitates the H-reflex of the soleus muscle, indicating increased excitability of the spinal motoneuron pool. This literature review examined the methods and mechanisms by which cerebellar TMS modulates spinal excitability.

Reinforcement Learning Is Impaired in the Sub-acute Post-stroke Period

Background

Neurorehabilitation approaches are frequently predicated on motor learning principles. However, much is left to be understood of how different kinds of motor learning are affected by stroke causing hemiparesis. Here we asked if two kinds of motor learning often employed in rehabilitation, (1) reinforcement learning and (2) error-based adaptation, are altered at different times after stroke.

Methods

In a cross-sectional design, we compared learning in two groups of patients with stroke, matched for their baseline motor execution deficit on the paretic side. The early group was tested within 3 months following stroke (N = 35) and the late group was tested more than 6 months after stroke (N = 30). Two types of task were studied: one based on reinforcement learning and the other on error-based learning.

Results

We found that reinforcement learning was impaired in the early but not the late group, whereas error-based learning was unaffected compared to controls. These findings could not be attributed to differences in baseline execution, cognitive impairment, gender, age, or lesion volume and location.

Conclusions

The presence of a specific impairment in reinforcement learning in the first 3 months after stroke has important implications for rehabilitation. It might be necessary to either increase the amount of reinforcement feedback given early or even delay onset of certain forms of rehabilitation training, e.g., like constraint-induced movement therapy, and instead emphasize others forms of motor learning in this early time period. A deeper understanding of stroke-related changes in motor learning capacity has the potential to facilitate the development of new, more precise treatment interventions.

Error-based and reinforcement learning in basketball free throw shooting

This study investigates the effects of error-based and reinforcement training on the acquisition and long-term retention of free throw accuracy in basketball. Sixty participants were divided into four groups (n = 15 per group): (i) the error-based group (sensory feedback), (ii) the reinforcement group (binary feedback including success or failure), (iii) the mixed group (sensory feedback followed by binary feedback), and (iv) the control group (without training). Free throws success was recorded before training (PreT), immediately after (Postd0), one day later (Postd1), and seven days later (Postd7). The error-based group, but not the reinforcement group, showed a significant immediate improvement in free throw accuracy (PreT vs Postd0). Interestingly, over time (Postd0 vs Postd1 vs Postd7), the reinforcement group significantly improved its accuracy, while the error-based group decreased it, returning to the PreT level (PreT vs Post7). The mixed group showed the advantage of both training methods, i.e., fast acquisition and retention on a long-term scale. Error-based learning leads to better acquisition, while reinforcement learning leads to better retention. Therefore, the combination of both types of learning is more efficient for both acquisition and retention processes. These findings provide new insight into the acquisition and retention of a fundamental basketball skill in free throw shooting.

Effects of cerebellar transcranial direct current stimulation on upper limb motor function after stroke: study protocol for the pilot of a randomized controlled trial

Background

Transcranial direct current stimulation (tDCS) is a technique that can noninvasively modulate neural states in a targeted brain region. As cerebellar activity levels are associated with upper limb motor improvement after stroke, the cerebellum is a plausible target of tDCS. However, the effect of tDCS remains unclear. Here, we designed a pilot study to assess: (1) the feasibility of a study that aims to examine the effects of cerebellar tDCS combined with an intensive rehabilitation approach based on the concept of constraint-induced movement therapy (CIMT) and (2) the preliminary outcome of the combined approach on upper limb motor function in patients with stroke in the chronic stage.

Methods

This pilot study has a double-blind randomized controlled design. Twenty-four chronic stroke patients with mild to moderate levels of upper limb motor impairment will be randomly assigned to an active or sham tDCS group. The participants will receive 20 min of active or sham tDCS to the contralesional cerebellum at the commencement of 4 h of daily intensive training, repeatedly for 5 days per week for 2 weeks. The primary outcomes are recruitment, enrollment, protocol adherence, and retention rates and measures to evaluate the feasibility of the study. The secondary outcome is upper limb motor function which will be evaluated using the Action Research Arm Test, Fugl-Meyer Assessment, for the upper extremity and the Motor Activity Log. Additionally, neurophysiological and neuroanatomical assessments of the cerebellum will be performed using transcranial magnetic stimulation and magnetic resonance imaging. These assessments will be conducted before, at the middle, and after the 2-week intervention, and finally, 1 month after the intervention. Any adverse events that occur during the study will be recorded.

Discussion

Cerebellar tDCS combined with intensive upper limb training may increase the gains of motor improvement when compared to the sham condition. The present study should provide valuable evidence regarding the feasibility of the design and the efficacy of cerebellar tDCS for upper limb motor function in patients with stroke before a future large trial is conducted.

Trial registration

This study has been registered at the Japan Registry of Clinical Trials (jRCTs042200078). Registered 17 December 2020

Memory decay and generalization following distinct motor learning mechanisms

Motor skill learning is considered to arise out of contributions from multiple learning mechanisms, including error-based learning (EBL), use-dependent learning (UDL), and reinforcement learning (RL). These learning mechanisms exhibit dissociable roles and engage different neural circuits during skill acquisition. However, it remains largely unknown how a newly formed motor memory acquired through each learning mechanism decays over time and whether distinct learning mechanisms produce different generalization patterns. Here, we used variants of reaching paradigms that dissociated these learning mechanisms to examine the time course of memory decay following each learning and the generalization patterns of each learning. We found that motor memories acquired through these learning mechanisms decayed as a function of time. Notably, 15 min, 6 h, and 24 h after acquisition, the memory of EBL decayed much greater than that of RL. The memory acquired through UDL faded away within a few minutes. Motor memories formed through EBL and RL for given movement directions generalized to untrained movement directions, with the generalization of EBL being greater than that of RL. In contrast, motor memory of UDL could not generalize to untrained movement directions. These results suggest that distinct learning mechanisms exhibit different patterns of memory decay and generalization.NEW & NOTEWORTHY Motor skill learning is likely to involve error-based learning, use-dependent plasticity, and operant reinforcement. Here, we showed that these dissociable learning mechanisms exhibited distinct patterns of memory decay and generalization. With a better understanding of the characteristics of these learning mechanisms, it becomes possible to regulate each learning process separately to improve neurological rehabilitation.

Cerebellar repetitive transcranial magnetic stimulation modulates the motor learning of visually guided voluntary postural control task

We investigated whether vermal cerebellar low-frequency repetitive transcranial magnetic stimulation (crTMS) affects motor learning of visually guided postural tracking training (VTT) using foot center of pressure (COP) as well as the stability and sensory contribution of upright standing. Twenty-one healthy volunteers participated (10 in the sham-crTMS group and 11 in the active-crTMS group). For VTT, participants stood on the force plate 1.5 m from the monitor on which the COP and target moved in a circle. Participants tracked the target with their own COP for 1 min, and 10 VTT sessions were conducted. The tracking error (TE) was compared between trials. Active- or sham-crTMS sessions were conducted prior to VTT. At baseline (before crTMS), pre-VTT (after crTMS), and post-VTT, the COP trajectory during upright static standing under four conditions (eyes, open/closed; surface, hard/rubber) was recorded. Comparison of the length of the COP trajectory or path and sensory-contribution-rate showed no significant difference between baseline and pre- and post-VTT. There was a significant decrease in TE in the sham-crTMS but not in the active-crTMS group. VTT and crTMS did not immediately affect the stability and sensory contribution of upright standing; however, crTMS immediately affected motor learning. The vermal cerebellum may contribute to motor learning of voluntary postural control.

Different Patterns of Neural Activity Characterize Motor Skill Performance During Acquisition and Retention

Motor performance and learning have distinct behavioral and neural signatures and can be uniquely modulated by various informational and motivational factors. Contemporary frameworks describe four different motor learning mechanisms mapped onto specific neural regions which are key for motor skill acquisition: error-based learning (cerebellum), reinforcement learning (basal ganglia), cognitive strategies (prefrontal cortex), and use-dependent learning (motor cortex). However, little is known about the neural circuits engaged during skill acquisition that are modulated specifically by practice-based performance improvement and those that predict recall performance. Based on previous work, we hypothesize that brain activity during practice in primary motor cortex and basal ganglia (1) is associated with trial-by-trial practice performance and (2) is predictive of immediate recall performance. Leveraging the contemporary framework, we use a well-known task paradigm that primarily relies upon cognitive strategy, reinforcement, and use-based learning mechanisms to test our hypotheses. Forty neurotypical young adults were asked to practice a pinch force tracking task. Participants received performance feedback after each trial during practice. We used whole brain analysis of functional magnetic resonance imaging (fMRI) and behavioral performance measures (i.e., time-on-target and self-efficacy) during the practice phase to determine which brain activation patterns are (1) associated with trial-by-trial tracking performance and (2) predictive of immediate no-feedback retention performance. We observed brain activations in the frontal orbital cortex, putamen, amygdala, and insula correlated with tracking performance improvement during practice. In contrast, a different set of performance-related activated regions were observed that were associated with immediate retention performance that included the primary motor cortex, superior frontal gyrus, somatosensory cortex, angular gyrus, and parietal gyrus. Our findings demonstrate that improved practice performance and recall of a sensorimotor skill are correlated with distinct neural activity patterns during acquisition, drawing on different motor learning mechanisms during encoding. While motor performance improvements depend on both cortical and subcortical regions, motor skill recall depends primarily on prefrontal and motor cortices. We discuss possible interpretations for why our hypothesis regarding basal ganglia activity and retention performance was not supported. Understanding the different neural mechanisms engaged in motor performance and learning may inform novel interventions to enhance motor skill learning.

Association between the Effects of Positive Social-Comparative Feedback and Learners' Competitiveness

This study examined the effects of positive social-comparative feedback on learners' intrinsic motivation on a motor learning task, as well as its association with learners' competitiveness. The participants, who performed a balance task in a positive social-comparative feedback and a control group, were assessed for performance outcome, perceived competence, and competitiveness. The positive social-comparative feedback group demonstrated more effective balance performance than the control group on the retention test. In addition, the participants in the positive social-comparative feedback group reported significantly higher perceived competence than the participants in the control group after practice. Further, a subscale of learners' competitiveness-instrumental competitiveness-predicted the performance on a retention test in the positive social-comparative feedback group, but not in the control group. Our results suggest that positive social-comparative feedback is not beneficial to all learners, and may even be less effective for learners with lower competitiveness.

Modulation of neural activity in frontopolar cortex drives reward-based motor learning

The frontopolar cortex (FPC) contributes to tracking the reward of alternative choices during decision making, as well as their reliability. Whether this FPC function extends to reward gradients associated with continuous movements during motor learning remains unknown. We used anodal transcranial direct current stimulation (tDCS) over the right FPC to investigate its role in reward-based motor learning. Nineteen healthy human participants practiced novel sequences of finger movements on a digital piano with corresponding auditory feedback. Their aim was to use trialwise reward feedback to discover a hidden performance goal along a continuous dimension: timing. We additionally modulated the contralateral motor cortex (left M1) activity, and included a control sham stimulation. Right FPC-tDCS led to faster learning compared to lM1-tDCS and sham through regulation of motor variability. Bayesian computational modelling revealed that in all stimulation protocols, an increase in the trialwise expectation of reward was followed by greater exploitation, as shown previously. Yet, this association was weaker in lM1-tDCS suggesting a less efficient learning strategy. The effects of frontopolar stimulation were dissociated from those induced by lM1-tDCS and sham, as motor exploration was more sensitive to inferred changes in the reward tendency (volatility). The findings suggest that rFPC-tDCS increases the sensitivity of motor exploration to updates in reward volatility, accelerating reward-based motor learning.

The reliability of cerebellar brain inhibition

OBJECTIVE

Connectivity between the cerebellum and primary motor cortex (M1) can be assessed by using transcranial magnetic stimulation to measure cerebellar brain inhibition (CBI). The aim of the present study was to determine the intra- and inter-day measurment error and relative reliability of CBI. The former informs the degree to which repeated measurements vary, whereas the latter informs how well the measure can distinguish individuals from one another within a sample.

METHODS

We obtained CBI data from 83 healthy young participants (n = 55 retrospective). Intra-day measurements were separated by ~ 30 min. Inter-day measurmenets were separated by a minimum of 24 h.

RESULTS

We show that CBI has low measurement error (~15%) within and between sessions. Using the measurment error, we demonstrate that change estimates which exceed measurment noise are large at an individual level, but can be detected with modest sample sizes. Finally, we demonstrate that the CBI measurement has fair to good relative reliability in healthy individuals, which may be deflated by low sample heterogeneity.

CONCLUSIONS

CBI has low measurement error supporting its use for tracking intra- and inter-day changes in cerebellar-M1 connectivity.

SIGNIFICANCE

Our findings provide clear reliability guidelines for future studies assessing modulation of cerebellar-M1 connectivity with intervention or disease progression.

Recognition capability of one's own skilled movement is dissociated from acquisition of motor skill memory

When we have rehearsed a movement using an object, we can reproduce the movement without holding the object. However, the reproduced movement sometimes differs from the movement holding a real object, likely because movement recognition is inaccurate. In the present study, we tested whether the recognition capability was dissociated from the acquisition of motor skill memory. Twelve novices were asked to rotate two balls with their right hand as quickly as possible; they practiced the task for 29 days. To evaluate recognition capability, we calculated the difference in coordination pattern of all five digits between the ball-rotation movement and the reproduced movement without holding balls. The recognition capability did not change within the first day, but improved after one week of practice. On the other hand, performance of the ball rotation significantly improved within the first day. Since improvement of performance is likely associated with acquisition of motor skill memory, we suggest that recognition capability, which reflects the capability to cognitively access motor skill memory, was dissociated from the acquisition of motor skill memory. Therefore, recognition of one’s own skilled movement would rely on a hierarchical structure of acquisition of motor skill memory and cognitive access to that memory.

Reward boosts reinforcement-based motor learning

Besides relying heavily on sensory and reinforcement feedback, motor skill learning may also depend on the level of motivation experienced during training. Yet, how motivation by reward modulates motor learning remains unclear. In 90 healthy subjects, we investigated the net effect of motivation by reward on motor learning while controlling for the sensory and reinforcement feedback received by the participants. Reward improved motor skill learning beyond performance-based reinforcement feedback. Importantly, the beneficial effect of reward involved a specific potentiation of reinforcement-related adjustments in motor commands, which concerned primarily the most relevant motor component for task success and persisted on the following day in the absence of reward. We propose that the long-lasting effects of motivation on motor learning may entail a form of associative learning resulting from the repetitive pairing of the reinforcement feedback and reward during training, a mechanism that may be exploited in future rehabilitation protocols.

Task-relevant and task-irrelevant variability causally shape error-based motor learning

Recent studies of motor learning show dissociable roles of reward- and sensory-prediction errors in updating motor commands by using typical adaptation paradigms where force or visual perturbations are imposed on hand movements. Such classic adaptation paradigms ignore a problem of redundancy inherently embedded in the motor pathways where the central nervous system has to find a unique solution in the high-dimensional motor command space. Computationally, a possible way of solving such a redundancy problem is exploring and updating motor commands based on the learned knowledge of the structures of both the motor pathways and the tasks. However, the effects of task-irrelevant motor command exploration in structure learning and its effects on reward-based and error-based learning have yet to be examined. Here, we used a redundant motor task where participants manipulated a cursor on a monitor screen with their hand gesture movements and then analyzed single-trial motor learning by fitting models consisting of reward-based and error-based learning contributions. We found that the error-based learning rate positively correlated with both task-relevant and task-irrelevant variability, likely reflecting the effect of motor exploration in structure learning. Further modeling results show that the effects of both task-relevant and task-irrelevant variability are simultaneous, and not mediated by one another. In contrast, the reward-based learning rate correlated with neither task-relevant nor task-irrelevant variability. Thus, although not having a direct influence on the task outcome, exploration in the task-irrelevant space late in training has a significant effect on the learning of a task structure used for error-based learning. This suggests that motor exploration, in both task-relevant and task-irrelevant spaces, has an essential role in error-based motor learning in a redundant motor mechanism.

Modulation of cerebellar brain inhibition during temporal adaptive learning in a coincident timing task

In the present study, we examined the role of the cerebellum in temporal adaptive learning during a coincident timing task, i.e., a baseball-like hitting task involving a moving ball presented on a computer monitor. The subjects were required to change the timing of their responses based on imposed temporal perturbations. Using paired-pulse transcranial magnetic stimulation, we measured cerebellar brain inhibition (CBI) before, during, and after the temporal adaptive learning. Reductions in CBI only occurred during and after the temporal adaptive learning, regardless of the direction of the temporal perturbations. In addition, the changes in CBI were correlated with the magnitude of the adaptation. Here, we showed that the cerebellum is essential for learning about and controlling the timing of movements during temporal adaptation. Furthermore, changes in cerebellar-primary motor cortex connectivity occurred during temporal adaptation, as has been previously reported for spatial adaptation.

Long-term motor skill training with individually adjusted progressive difficulty enhances learning and promotes corticospinal plasticity

Motor skill acquisition depends on central nervous plasticity. However, behavioural determinants leading to long lasting corticospinal plasticity and motor expertise remain unexplored. Here we investigate behavioural and electrophysiological effects of individually tailored progressive practice during long-term motor skill training. Two groups of participants practiced a visuomotor task requiring precise control of the right digiti minimi for 6 weeks. One group trained with constant task difficulty, while the other group trained with progressively increasing task difficulty, i.e. continuously adjusted to their individual skill level. Compared to constant practice, progressive practice resulted in a two-fold greater performance at an advanced task level and associated increases in corticospinal excitability. Differences were maintained 8 days later, whereas both groups demonstrated equal retention 14 months later. We demonstrate that progressive practice enhances motor skill learning and promotes corticospinal plasticity. These findings underline the importance of continuously challenging patients and athletes to promote neural plasticity, skilled performance, and recovery.

Brain Activity Reveals Multiple Motor-Learning Mechanisms in a Real-World Task

Many recent studies found signatures of motor learning in neural beta oscillations (13-30 Hz), and specifically in the post-movement beta rebound (PMBR). All these studies were in controlled laboratory-tasks in which the task designed to induce the studied learning mechanism. Interestingly, these studies reported opposing dynamics of the PMBR magnitude over learning for the error-based and reward-based tasks (increase vs. decrease, respectively). Here, we explored the PMBR dynamics during real-world motor-skill-learning in a billiards task using mobile-brain-imaging. Our EEG recordings highlight the opposing dynamics of PMBR magnitudes (increase vs. decrease) between different subjects performing the same task. The groups of subjects, defined by their neural dynamics, also showed behavioral differences expected for different learning mechanisms. Our results suggest that when faced with the complexity of the real-world different subjects might use different learning mechanisms for the same complex task. We speculate that all subjects combine multi-modal mechanisms of learning, but different subjects have different predominant learning mechanisms.

Repeated Concussions Impair Behavioral and Neurophysiological Changes in the Motor Learning System

Background. Concussions affect nearly 3 million people a year and are the leading cause of traumatic brain injury-related emergency department visits among youth. Evidence shows neuromotor regions are sensitive to concussive events and that motor symptoms may be the earliest clinical manifestations of neurodegenerative traumatic brain injuries. However, little is known about the effects repeated concussions play on motor learning. Namely, how does concussion acuity (time since injury) affect different behavioral and neurophysiological components of motor learning? Methods. Using a 3-pronged approach, we assessed (1) behavioral measures of motor learning, (2) neurophysiological measures underlying retention of motor learning known as occlusion, and (3) quantitative survey data capturing affective symptoms of each participant. Collegiate student athletes were stratified across 3 groups depending on their concussion history: (1) NonCon, no history of concussion; (2) Chronic, chronic-state of concussion (>1 year postinjury), or (3) Acute, acute state of concussion (<2 weeks postinjury). Results. We found that athletes in both the acute and chronic state of injury following a concussion had impaired retention and aberrant occlusion in motor skill learning as compared with athletes with no history of concussion. Moreover, higher numbers of previous concussions (regardless of the time since injury) correlated with more severe behavioral and neurophysiological motor impairments by specifically hindering neurophysiological mechanisms of learning to affect behavior. Conclusions. These results indicate the presence of motor learning impairment that is evident within 2 weeks postinjury. Our findings have significant implications for the prognosis of concussion and emphasize the need for prevention, but can also direct more relevant rehabilitation treatment targets.

Multiple Motor Learning Processes in Humans: Defining Their Neurophysiological Bases

Learning new motor behaviors or adjusting previously learned actions to account for dynamic changes in our environment requires the operation of multiple distinct motor learning processes, which rely on different neuronal substrates. For instance, humans are capable of acquiring new motor patterns via the formation of internal model representations of the movement dynamics and through positive reinforcement. In this review, we will discuss how changes in human physiological markers, assessed with noninvasive brain stimulation techniques from distinct brain regions, can be utilized to provide insights toward the distinct learning processes underlying motor learning. We will summarize the findings from several behavioral and neurophysiological studies that have made efforts to understand how distinct processes contribute to and interact when learning new motor behaviors. In particular, we will extensively review two types of behavioral processes described in human sensorimotor learning: (1) a recalibration process of a previously learned movement and (2) acquiring an entirely new motor control policy, such as learning to play an instrument. The selected studies will demonstrate in-detail how distinct physiological mechanisms contributions change depending on the time course of learning and the type of behaviors being learned.

Dissecting two distinct interneuronal networks in M1 with transcranial magnetic stimulation

Interactions from both inhibitory and excitatory interneurons are necessary components of cortical processing that contribute to the vast amount of motor actions executed by humans daily. As transcranial magnetic stimulation (TMS) over primary motor cortex is capable of activating corticospinal neurons trans-synaptically, studies over the past 30 years have provided how subtle changes in stimulation parameters (i.e., current direction, pulse width, and paired-pulse) can elucidate evidence for two distinct neuronal networks that can be probed with this technique. This article provides a brief review of some fundamental studies demonstrating how these networks have separable excitatory inputs to corticospinal neurons. Furthermore, the findings of recent investigations will be discussed in detail, illustrating how each network's sensitivity to different brain states (i.e., rest, movement preparation, and motor learning) is dissociable. Understanding the physiological characteristics of each network can help to explain why interindividual responses to TMS exist, while also providing insights into the role of these networks in various human motor behaviors.

Interactions between motor exploration and reinforcement learning

Motor exploration, a trial-and-error process in search for better motor outcomes, is known to serve a critical role in motor learning. This is particularly relevant during reinforcement learning, where actions leading to a successful outcome are reinforced while unsuccessful actions are avoided. Although early on motor exploration is beneficial to finding the correct solution, maintaining high levels of exploration later in the learning process might be deleterious. Whether and how the level of exploration changes over the course of reinforcement learning, however, remains poorly understood. Here we evaluated temporal changes in motor exploration while healthy participants learned a reinforcement-based motor task. We defined exploration as the magnitude of trial-to-trial change in movements as a function of whether the preceding trial resulted in success or failure. Participants were required to find the optimal finger-pointing direction using binary feedback of success or failure. We found that the magnitude of exploration gradually increased over time when participants were learning the task. Conversely, exploration remained low in participants who were unable to correctly adjust their pointing direction. Interestingly, exploration remained elevated when participants underwent a second training session, which was associated with faster relearning. These results indicate that the motor system may flexibly upregulate the extent of exploration during reinforcement learning as if acquiring a specific strategy to facilitate subsequent learning. Also, our findings showed that exploration affects reinforcement learning and vice versa, indicating an interactive relationship between them. Reinforcement-based tasks could be used as primers to increase exploratory behavior leading to more efficient subsequent learning.NEW & NOTEWORTHY Motor exploration, the ability to search for the correct actions, is critical to learning motor skills. Despite this, whether and how the level of exploration changes over the course of training remains poorly understood. We showed that exploration increased and remained high throughout training of a reinforcement-based motor task. Interestingly, elevated exploration persisted and facilitated subsequent learning. These results suggest that the motor system upregulates exploration as if learning a strategy to facilitate subsequent learning.

Combining reward and M1 transcranial direct current stimulation enhances the retention of newly learnt sensorimotor mappings

Background

Reward-based feedback given during motor learning has been shown to improve the retention of the behaviour being acquired. Interestingly, applying transcranial direct current stimulation (tDCS) during learning over the primary motor cortex (M1), an area associated with motor retention, also results in enhanced retention of the newly formed motor memories. However, it remains unknown whether combining these distinct interventions result in an additive benefit of motor retention.

Methods

We investigated whether combining both interventions while participants learned to account for a visuomotor transformation results in enhanced motor retention (total n = 56; each group n = 14). To determine whether these interventions share common physiological mechanisms underpinning learning, we assessed motor cortical excitability and inhibition (i.e. SICI) on a hand muscle before and after all participants learned the visuomotor rotation using their entire arm and hand.

Results

We found that both the Reward-Stim (i.e. reward + tDCS) and Reward-Sham (i.e. reward-only) groups had increased retention at the beginning of the retention phase, indicating an immediate effect of reward on behaviour. However, each intervention on their own did not enhance retention when compared to sham, but rather, only the combination of both reward and tDCS demonstrated prolonged retention. We also found that only the Reward-Stim group had a significant reduction in SICI after exposure to the perturbation.

Conclusions

We show that combining both interventions are additive in providing stronger retention of motor adaptation. These results indicate that the reliability and validity of using tDCS within a clinical context may depend on the type of feedback individuals receive when learning a new motor pattern.

•

Concurrently administering reward and M1 tDCS during learning results in enhanced motor retention.

•

The combination of these interventions also leads to a reduction in M1 inhibitory mechanisms.

•

No benefits of motor retention were found when reward or M1 tDCS were given alone.

Acquisition of motor memory determines the interindividual variability of learning-induced plasticity in the primary motor cortex

Acquisition of new motor skills induces plastic reorganization in the primary motor cortex (M1). Previous studies have demonstrated the increases in the M1 excitability through motor skill learning. However, this M1 reorganization is highly variable between individuals even though they improve their skill performance through the same training protocol. To reveal the source of this interindividual variability, we examined the relationship between an acquisition of memory-guided feedforward movements and the learning-induced increases in the M1 excitability. Twenty-eight subjects participated in experiment 1. We asked subjects to learn a visuomotor tracking task. The subjects controlled a cursor on a PC monitor to pursue a target line by performing ankle dorsiflexion and plantar flexion. In experiment 1, we removed the online visual feedback provided by the cursor movement once every six trials, which enabled us to assess whether the subjects could perform accurate memory-guided movements. Motor-evoked potentials (MEP) were elicited in the tibialis anterior muscle by transcranial magnetic stimulation of the relevant M1 before and after the learning of the visuomotor tracking task and after half the trials. We found that the MEP amplitude was increased along with the improvement in memory-guided movements. In experiment 2 ( n = 10), we confirmed this relationship by examining whether the improvement in memory-guided movements induces increases in MEP amplitude. The results of this study indicate that the plastic reorganization of the M1 induced by the learning of a visuomotor skill is associated with the acquisition of memory-guided movements. NEW & NOTEWORTHY Acquisition of novel motor skills increases excitability of the primary motor cortex (M1). We recently reported that the amount of increases in the M1 excitability is highly variable between individuals even though they learned the same skill to the similar extent, yet the sources of this interindividual variability still remain unclear. The present study revealed that this interindividual variability is associated with whether individuals acquire a motor memory, which enables them to produce accurate memory-guided movements.

Task-specificity in focal dystonia is shaped by aberrant diversity of a functional network kernel

OBJECTIVES

Task-specific focal dystonia selectively affects the motor control during skilled and highly learned behaviors. Recent data suggest the role of neural network abnormalities in the development of the pathophysiological dystonic cascade.

METHODS

We used resting-state functional MRI and analytic techniques rooted in network science and graph theory to examine the formation of abnormal subnetwork of highly influential brain regions, the functional network kernel, and its influence on aberrant dystonic connectivity specific to affected body region and skilled motor behavior.

RESULTS

We found abnormal embedding of sensorimotor cortex and prefrontal thalamus in dystonic network kernel as a hallmark of task-specific focal dystonia. Dependent on the affected body region, aberrant functional specialization of the network kernel included regions of motor control management in focal hand dystonia (writer's cramp, musician's focal hand dystonia) and sensorimotor processing in laryngeal dystonia (spasmodic dysphonia, singer's laryngeal dystonia). Dependent on skilled motor behavior, the network kernel featured altered connectivity between sensory and motor execution circuits in musician's dystonia (musician's focal hand dystonia, singer's laryngeal dystonia) and abnormal integration of sensory feedback into motor planning and executive circuits in non-musician's dystonia (writer's cramp, spasmodic dysphonia).

CONCLUSIONS

Our study identified specific traits in disorganization of large-scale neural connectivity that underlie the common pathophysiology of task-specific focal dystonia while reflecting distinct symptomatology of its different forms. Identification of specialized regions of information transfer that influence dystonic network activity is an important step for future delineation of targets for neuromodulation as a potential therapeutic option of task-specific focal dystonia. © 2018 International Parkinson and Movement Disorder Society.