The dynamics of motor learning through the formation of internal models

Corticostriatal activity related to performance during continuous de novo motor learning

Corticostriatal regions play a pivotal role in visuomotor learning. However, less research has been done on how fMRI activity in their subregions is related to task performance, which is provided as visual feedback during motor learning. To address this, we conducted an fMRI experiment in which participants acquired a complex de novo motor skill using continuous or binary visual feedback related to performance. We found a highly selective response related to performance in the entire striatum in both conditions and a relatively higher response in the caudate nucleus for the binary feedback condition. However, the ventromedial prefrontal cortex (vmPFC) response was significant only for the continuous feedback condition. Furthermore, we also found functional distinction of the striatal subregions in random versus goal-directed motor control. These findings underscore the substantial effects of the visual feedback indicating performance on distinct corticostriatal responses, thereby elucidating its significance in reinforcement-based motor learning.

Theoretical limits on the speed of learning inverse models explain the rate of adaptation in arm reaching tasks

An essential aspect of human motor learning is the formation of inverse models, which map desired actions to motor commands. Inverse models can be learned by adjusting parameters in neural circuits to minimize errors in the performance of motor tasks through gradient descent. However, the theory of gradient descent establishes limits on the learning speed. Specifically, the eigenvalues of the Hessian of the error surface around a minimum determine the maximum speed of learning in a task. Here, we use this theoretical framework to analyze the speed of learning in different inverse model learning architectures in a set of isometric arm-reaching tasks. We show theoretically that, in these tasks, the error surface and, thus the speed of learning, are determined by the shapes of the force manipulability ellipsoid of the arm and the distribution of targets in the task. In particular, rounder manipulability ellipsoids generate a rounder error surface, allowing for faster learning of the inverse model. Rounder target distributions have a similar effect. We tested these predictions experimentally in a quasi-isometric reaching task with a visuomotor transformation. The experimental results were consistent with our theoretical predictions. Furthermore, our analysis accounts for the speed of learning in previous experiments with incompatible and compatible virtual surgery tasks, and with visuomotor rotation tasks with different numbers of targets. By identifying aspects of a task that influence the speed of learning, our results provide theoretical principles for the design of motor tasks that allow for faster learning.

The Effect of Feedback Modality When Learning a Novel Wrist Sensorimotor Transformation Through a Body-Machine Interface

Body-Machine Interfaces (BoMIs) are promising assistive and rehabilitative tools for helping individuals with impaired motor abilities regain independence. When operating a BoMI, the user has to learn a novel sensorimotor transformation between the movement of certain body parts and the output of the device. In this study, we investigated how different feedback modalities impacted learning to operate a BoMI. Forty-seven able-bodied participants learned to control the velocity of a 1D cursor using the 3D rotation of their dominant wrist to reach as many targets as possible in a given amount of time. The map was designed to maximize cursor speed for movements around a predefined axis of wrist rotation. We compared the user's performance and control efficiency under three feedback modalities: i) visual feedback of the cursor position, ii) proprioceptive feedback of the cursor position delivered by a wrist manipulandum, iii) both i) and ii). We found that visual feedback led to a greater number of targets reached than proprioceptive feedback alone. Conversely, proprioceptive feedback yielded greater alignment between the axis of rotation of the wrist and the optimal axis represented by the map. These results suggest that proprioceptive feedback may be preferable over visual feedback when information about intrinsic task components, i.e. joint configurations, is of interest as in rehabilitative interventions aiming to promote more effective learning strategies.

Task space exploration improves adaptation after incompatible virtual surgeries

Humans have a remarkable capacity to learn new motor skills, a process that requires novel muscle activity patterns. Muscle synergies may simplify the generation of muscle patterns through the selection of a small number of synergy combinations. Learning new motor skills may then be achieved by acquiring novel muscle synergies. In a previous study, we used myoelectric control to construct virtual surgeries that altered the mapping from muscle activity to cursor movements. After compatible virtual surgeries, which could be compensated by recombining subject-specific muscle synergies, participants adapted quickly. In contrast, after incompatible virtual surgeries, which could not be compensated by recombining existing synergies, participants explored new muscle patterns, but failed to adapt. Here, we tested whether task space exploration can promote learning of novel muscle synergies, required to overcome an incompatible surgery. Participants performed the same reaching task as in our previous study, but with more time to complete each trial, thus allowing for exploration. We found an improvement in trial success after incompatible virtual surgeries. Remarkably, improvements in movement direction accuracy after incompatible surgeries occurred faster for corrective movements than for the initial movement, suggesting that learning of new synergies is more effective when used for feedback control. Moreover, reaction time was significantly higher after incompatible than after compatible virtual surgeries, suggesting an increased use of an explicit adaptive strategy to overcome incompatible surgeries. Taken together, these results indicate that exploration is important for skill learning and suggest that human participants, with sufficient time, can learn new muscle synergies.

Motor imagery helps updating internal models during microgravity exposure

Skilled movements result from a mixture of feedforward and feedback mechanisms conceptualized by internal models. These mechanisms subserve both motor execution and motor imagery. Current research suggests that imagery allows updating feedforward mechanisms, leading to better performance in familiar contexts. Does this still hold in radically new contexts? Here, we test this ability by asking participants to imagine swinging arm movements around shoulder in normal gravity condition and in microgravity in which studies showed that movements slow down. We timed several cycles of actual and imagined arm pendular movements in three groups of subjects during parabolic flight campaign. The first, control, group remained on the ground. The second group was exposed to microgravity but did not imagine movements inflight. The third group was exposed to microgravity and imagined movements inflight. All groups performed and imagined the movements before and after the flight. We predicted that a mere exposure to microgravity would induce changes in imagined movement duration. We found this held true for the group who imagined the movements, suggesting an update of internal representations of gravity. However, we did not find a similar effect in the group exposed to microgravity despite the fact participants lived the same gravitational variations as the first group. Overall, these results suggest that motor imagery contributes to update internal representations of movement in unfamiliar environments, while a mere exposure proved to be insufficient.

A precise and adaptive neural mechanism for predictive temporal processing in the frontal cortex

The theory of predictive processing posits that the brain computes expectations to process information predictively. Empirical evidence in support of this theory, however, is scarce and largely limited to sensory areas. Here, we report a precise and adaptive mechanism in the frontal cortex of non-human primates consistent with predictive processing of temporal events. We found that the speed of neural dynamics is precisely adjusted according to the average time of an expected stimulus. This speed adjustment, in turn, enables neurons to encode stimuli in terms of deviations from expectation. This lawful relationship was evident across multiple experiments and held true during learning: when temporal statistics underwent covert changes, neural responses underwent predictable changes that reflected the new mean. Together, these results highlight a precise mathematical relationship between temporal statistics in the environment and neural activity in the frontal cortex that may serve as a mechanism for predictive temporal processing.

De novo learning versus adaptation of continuous control in a manual tracking task

How do people learn to perform tasks that require continuous adjustments of motor output, like riding a bicycle? People rely heavily on cognitive strategies when learning discrete movement tasks, but such time-consuming strategies are infeasible in continuous control tasks that demand rapid responses to ongoing sensory feedback. To understand how people can learn to perform such tasks without the benefit of cognitive strategies, we imposed a rotation/mirror reversal of visual feedback while participants performed a continuous tracking task. We analyzed behavior using a system identification approach, which revealed two qualitatively different components of learning: adaptation of a baseline controller and formation of a new, task-specific continuous controller. These components exhibited different signatures in the frequency domain and were differentially engaged under the rotation/mirror reversal. Our results demonstrate that people can rapidly build a new continuous controller de novo and can simultaneously deploy this process with adaptation of an existing controller.

A Framework for Optimizing Co-adaptation in Body-Machine Interfaces

The operation of a human-machine interface is increasingly often referred to as a two-learners problem, where both the human and the interface independently adapt their behavior based on shared information to improve joint performance over a specific task. Drawing inspiration from the field of body-machine interfaces, we take a different perspective and propose a framework for studying co-adaptation in scenarios where the evolution of the interface is dependent on the users' behavior and that do not require task goals to be explicitly defined. Our mathematical description of co-adaptation is built upon the assumption that the interface and the user agents co-adapt toward maximizing the interaction efficiency rather than optimizing task performance. This work describes a mathematical framework for body-machine interfaces where a naïve user interacts with an adaptive interface. The interface, modeled as a linear map from a space with high dimension (the user input) to a lower dimensional feedback, acts as an adaptive “tool” whose goal is to minimize transmission loss following an unsupervised learning procedure and has no knowledge of the task being performed by the user. The user is modeled as a non-stationary multivariate Gaussian generative process that produces a sequence of actions that is either statistically independent or correlated. Dependent data is used to model the output of an action selection module concerned with achieving some unknown goal dictated by the task. The framework assumes that in parallel to this explicit objective, the user is implicitly learning a suitable but not necessarily optimal way to interact with the interface. Implicit learning is modeled as use-dependent learning modulated by a reward-based mechanism acting on the generative distribution. Through simulation, the work quantifies how the system evolves as a function of the learning time scales when a user learns to operate a static vs. an adaptive interface. We show that this novel framework can be directly exploited to readily simulate a variety of interaction scenarios, to facilitate the exploration of the parameters that lead to optimal learning dynamics of the joint system, and to provide an empirical proof for the superiority of human-machine co-adaptation over user adaptation.

A hybrid Body-Machine Interface integrating signals from muscles and motions

OBJECTIVE

Body-Machine Interfaces (BoMIs) establish a way to operate a variety of devices, allowing their users to extend the limits of their motor abilities by exploiting the redundancy of muscles and motions that remain available after spinal cord injury or stroke. Here, we considered the integration of two types of signals, motion signals derived from inertial measurement units (IMUs) and muscle activities recorded with electromyography (EMG), both contributing to the operation of the BoMI.

APPROACH

A direct combination of IMU and EMG signals might result in inefficient control due to the differences in their nature. Accordingly, we used a nonlinear-regression-based approach to predict IMU from EMG signals, after which the predicted and actual IMU signals were combined into a hybrid control signal. The goal of this approach was to provide users with the possibility to switch seamlessly between movement and EMG control, using the BoMI as a tool for promoting the engagement of selected muscles. We tested the interface in three control modalities, EMG-only, IMU-only and hybrid, in a cohort of 15 unimpaired participants. Participants practiced reaching movements by guiding a computer cursor over a set of targets.

MAIN RESULTS

We found that the proposed hybrid control led to comparable performance to IMU-based control and significantly outperformed the EMG-only control. Results also indicated that hybrid cursor control was predominantly influenced by EMG signals.

SIGNIFICANCE

We concluded that combining EMG with IMU signals could be an efficient way to target muscle activations while overcoming the limitations of an EMG-only control.