De novo learning versus adaptation of continuous control in a manual tracking task

Sensorimotor adaptation to destabilizing dynamics in weakly electric fish

Humans and other animals can readily learn to compensate for changes in the dynamics of movement. Such changes can result from an injury or changes in the weight of carried objects. These changes in dynamics can lead not only to reduced performance but also to dramatic instabilities. We evaluated the impacts of compensatory changes in control policies in relation to stability and robustness in Eigenmannia virescens, a species of weakly electric fish. We discovered that these fish retune their sensorimotor control system in response to experimentally generated destabilizing dynamics. Specifically, we used an augmented reality system to manipulate sensory feedback during an image stabilization task in which a fish maintained its position within a refuge. The augmented reality system measured the fish's movements in real time. These movements were passed through a high-pass filter and multiplied by a gain factor before being fed back to the refuge motion. We adjusted the gain factor to gradually destabilize the fish's sensorimotor loop. The fish retuned their sensorimotor control system to compensate for the experimentally induced destabilizing dynamics. This retuning was partially maintained when the augmented reality feedback was abruptly removed. The compensatory changes in sensorimotor control improved tracking performance as well as control-theoretic measures of robustness, including reduced sensitivity to disturbances and improved phase margins.

Motor learning in multijoint virtual arm movements with novel kinematics

Humans move their hands toward precise positions, a skill supported by the coordination of multiple joint movements, even in the presence of inherent redundancy. However, it remains unclear how the central nervous system learns the relationship between redundant joint movements and hand positions when starting from scratch. To address this question, a virtual-arm reaching task was performed in which participants were required to move a cursor corresponding to the hand of a virtual arm to a target. The joint angles of the virtual arm were determined by the heights of the participants' fingers. The results demonstrated that the participants moved the cursor to the target straighter and faster in the late phase than they did in the initial phase of learning. This improvement was accompanied by a reduction in the amount of angular changes in the virtual limb joint, predominantly characterized by an increased reliance on the virtual shoulder joint as opposed to the virtual wrist joint. These findings suggest that the central nervous system selects a combination of multijoint movements that minimize motor effort while learning novel upper-limb kinematics.

Neural Underpinnings of Learning in Dementia Populations: A Review of Motor Learning Studies Combined with Neuroimaging

The intent of this review article is to serve as an overview of current research regarding the neural characteristics of motor learning in Alzheimer disease (AD) as well as prodromal phases of AD: at-risk populations, and mild cognitive impairment. This review seeks to provide a cognitive framework to compare various motor tasks. We will highlight the neural characteristics related to cognitive domains that, through imaging, display functional or structural changes because of AD progression. In turn, this motivates the use of motor learning paradigms as possible screening techniques for AD and will build upon our current understanding of learning abilities in AD populations.

Distinct learning, retention, and generalization patterns in de novo learning versus motor adaptation

People correct for movement errors when acquiring new motor skills (de novo learning) or adapting well-known movements (motor adaptation). While de novo learning establishes new control policies, adaptation modifies existing ones, and previous work have distinguished behavioral and underlying brain mechanisms for each motor learning type. However, it is still unclear whether learning in each type interferes with the other. In study 1, we use a within-subjects design where participants train with both 30° visuomotor rotation and mirror reversal perturbations, to compare adaptation and de novo learning respectively. We find no perturbation order effects, and find no evidence for differences in learning rates and asymptotes for both perturbations. Explicit instructions also provide an advantage during early learning in both perturbations. However, mirror reversal learning shows larger inter-participant variability and slower movement initiation. Furthermore, we only observe reach aftereffects following rotation training. In study 2, we incorporate the mirror reversal in a browser-based task, to investigate under-studied de novo learning mechanisms like retention and generalization. Learning persists across three or more days, substantially transfers to the untrained hand, and to targets on both sides of the mirror axis. Our results extend insights for distinguishing motor skill acquisition from adapting well-known movements.

Eye-hand coordination all the way: from discrete to continuous hand movements

The differentiation between continuous and discrete actions is key for behavioral neuroscience. Although many studies have characterized eye-hand coordination during discrete (e.g., reaching) and continuous (e.g., pursuit tracking) actions, all these studies were conducted separately, using different setups and participants. In addition, how eye-hand coordination might operate at the frontier between discrete and continuous movements remains unexplored. Here we filled these gaps by means of a task that could elicit different movement dynamics. Twenty-eight participants were asked to simultaneously track with their eyes and a joystick a visual target that followed an unpredictable trajectory and whose position was updated at different rates (from 1.5 to 240 Hz). This procedure allowed us to examine actions ranging from discrete point-to-point movements (low refresh rate) to continuous pursuit (high refresh rate). For comparison, we also tested a manual tracking condition with the eyes fixed and a pure eye tracking condition (hand fixed). The results showed an abrupt transition between discrete and continuous hand movements around 3 Hz contrasting with a smooth trade-off between fixations and smooth pursuit. Nevertheless, hand and eye tracking accuracy remained strongly correlated, with each of these depending on whether the other effector was recruited. Moreover, gaze-cursor distance and lag were smaller when eye and hand performed the task conjointly than separately. Altogether, despite some dissimilarities in eye and hand dynamics when transitioning between discrete and continuous movements, our results emphasize that eye-hand coordination continues to smoothly operate and support the notion of synergies across eye movement types.NEW & NOTEWORTHY The differentiation between continuous and discrete actions is key for behavioral neuroscience. By using a visuomotor task in which we manipulate the target refresh rate to trigger different movement dynamics, we explored eye-hand coordination all the way from discrete to continuous actions. Despite abrupt changes in hand dynamics, eye-hand coordination continues to operate via a gradual trade-off between fixations and smooth pursuit, an observation confirming the notion of synergies across eye movement types.

Learning to stand with sensorimotor delays generalizes across directions and from hand to leg effectors

Humans receive sensory information from the past, requiring the brain to overcome delays to perform daily motor skills such as standing upright. Because delays vary throughout the body and change over a lifetime, it would be advantageous to generalize learned control policies of balancing with delays across contexts. However, not all forms of learning generalize. Here, we use a robotic simulator to impose delays into human balance. When delays are imposed in one direction of standing, participants are initially unstable but relearn to balance by reducing the variability of their motor actions and transfer balance improvements to untrained directions. Upon returning to normal standing, aftereffects from learning are observed as small oscillations in control, yet they do not destabilize balance. Remarkably, when participants train to balance with delays using their hand, learning transfers to standing with the legs. Our findings establish that humans use experience to broadly update their neural control to balance with delays.

Adaptation and learning as strategies to maximize reward in neurofeedback tasks

Introduction

Adaptation and learning have been observed to contribute to the acquisition of new motor skills and are used as strategies to cope with changing environments. However, it is hard to determine the relative contribution of each when executing goal directed motor tasks. This study explores the dynamics of neural activity during a center-out reaching task with continuous visual feedback under the influence of rotational perturbations.

Methods

Results for a brain-computer interface (BCI) task performed by two non-human primate (NHP) subjects are compared to simulations from a reinforcement learning agent performing an analogous task. We characterized baseline activity and compared it to the activity after rotational perturbations of different magnitudes were introduced. We employed principal component analysis (PCA) to analyze the spiking activity driving the cursor in the NHP BCI task as well as the activation of the neural network of the reinforcement learning agent.

Results and discussion

Our analyses reveal that both for the NHPs and the reinforcement learning agent, the task-relevant neural manifold is isomorphic with the task. However, for the NHPs the manifold is largely preserved for all rotational perturbations explored and adaptation of neural activity occurs within this manifold as rotations are compensated by reassignment of regions of the neural space in an angular pattern that cancels said rotations. In contrast, retraining the reinforcement learning agent to reach the targets after rotation results in substantial modifications of the underlying neural manifold. Our findings demonstrate that NHPs adapt their existing neural dynamic repertoire in a quantitatively precise manner to account for perturbations of different magnitudes and they do so in a way that obviates the need for extensive learning.

Generalization in de novo learning of virtual upper limb movements is influenced by motor exploration

The acquisition of new motor skills from scratch, also known as de novo learning, is an essential aspect of motor development. In de novo learning, the ability to generalize skills acquired under one condition to others is crucial because of the inherently limited range of motor experiences available for learning. However, the presence of generalization in de novo learning and its influencing factors remain unclear. This study aimed to elucidate the generalization of de novo motor learning by examining the motor exploration process, which is the accumulation of motor experiences. To this end, we manipulated the exploration process during practice by changing the target shape using either a small circular target or a bar-shaped target. Our findings demonstrated that the amount of learning during practice was generalized across different conditions. Furthermore, the extent of generalization is influenced by movement variability in the control space, which is irrelevant to the task, rather than the target shapes themselves. These results confirmed the occurrence of generalization in de novo learning and suggest that the exploration process within the control space plays a significant role in facilitating this generalization.

Right-left hand asymmetry in manual tracking: when poorer control is associated with better adaptation and interlimb transfer

To date, interlimb transfer following visuomotor adaptation has been mainly investigated through discrete reaching movements. Here we explored this issue in the context of continuous manual tracking, a task in which the contribution of online feedback mechanisms is crucial, and in which there is a well-established right (dominant) hand advantage under baseline conditions. We had two objectives (1) to determine whether this preexisting hand asymmetry would persist under visuomotor rotation, (2) to examine interlimb transfer by assessing whether prior experience with the rotation by one hand benefit to the other hand. To address these, 44 right-handed participants were asked to move a joystick and to track a visual target following a rather unpredictable trajectory. Visuomotor adaptation was elicited by introducing a 90° rotation between the joystick motion and the cursor motion. Half of the participants adapted to the rotation first with the right hand, and then with the left, while the other half performed the opposite protocol. As expected during baseline trials, the left hand was less accurate while also exhibiting more variable and exploratory behavior. However, participants exhibited a left hand advantage during first exposure to the rotation. Moreover, interlimb transfer was observed albeit more strongly from the left to the right hand. We suggest that the less effective and more variable/exploratory control strategy of the left hand promoted its adaptation, which incidentally favored transfer from left to right hand. Altogether, this study speaks for further attention to the dominant/non-dominant asymmetry during baseline before examining interlimb transfer of adaptation.

Initial development of skill with a reversed bicycle and a case series of experienced riders

Riding a bicycle is considered a durable skill that cannot be forgotten. Here, novice participants practiced riding a reversed bicycle, in which a reversing gear inverted the handlebar's rotation. Although learning to ride the reversed bicycle was possible, it was slow, highly variable, implicit, and followed an S-shape pattern. In the initial learning phase, failed attempts to ride the normal bicycle indicated strong interference between the two bicycle skills. While additional practice decreased this interference effect, a subset of learners could not ride either bicycle after eight sessions of practice. Experienced riders who performed extensive practice could switch bicycles without failed attempts and exhibited similar performance (i.e., similar handlebar oscillations) on both bicycles. However, their performance on the normal bicycle was worse than that of the novice bicycle riders at baseline. In conclusion, "unlearning" of the normal bicycle skill precedes the initial learning of the reversed bicycle skill, and a signature of such unlearning is still present following extensive practice.

Anticipatory postural control in adaptation of goal-directed lower extremity movements

Skilled football players can adapt their kicking movements depending on external environments. Predictive postural control movements, known as anticipatory postural adjustments (APAs), are needed preceding kicking movements to precisely control them while maintaining a standing posture only with the support leg. We aimed to clarify APAs of the support leg in the process of adaptation of goal-directed movements with the lower limb. Participants replicated ball-kicking movements such that they reached a cursor, representing a kicking-foot position towards a forward target while standing with the support leg. APAs were observed as the centre of pressure of the support leg shifted approximately 300 ms in advance of the onset of movement of the kicking foot. When the cursor trajectory of the kicking foot was visually rotated during the task, the kicking-foot movement was gradually modified to reach the target, indicating adaptation to the novel visuomotor environment. Interestingly, APAs in the mediolateral direction were also altered following the change in kicking-foot movements. Additionally, the APAs modified more slowly than the kicking-foot movements. These results suggest that flexible changes in predictive postural control might support the adaptation of goal-directed movements of the lower limb.

NSF DARE-transforming modeling in neurorehabilitation: a patient-in-the-loop framework

In 2023, the National Science Foundation (NSF) and the National Institute of Health (NIH) brought together engineers, scientists, and clinicians by sponsoring a conference on computational modelling in neurorehabiilitation. To facilitate multidisciplinary collaborations and improve patient care, in this perspective piece we identify where and how computational modelling can support neurorehabilitation. To address the where, we developed a patient-in-the-loop framework that uses multiple and/or continual measurements to update diagnostic and treatment model parameters, treatment type, and treatment prescription, with the goal of maximizing clinically-relevant functional outcomes. This patient-in-the-loop framework has several key features: (i) it includes diagnostic and treatment models, (ii) it is clinically-grounded with the International Classification of Functioning, Disability and Health (ICF) and patient involvement, (iii) it uses multiple or continual data measurements over time, and (iv) it is applicable to a range of neurological and neurodevelopmental conditions. To address the how, we identify state-of-the-art and highlight promising avenues of future research across the realms of sensorimotor adaptation, neuroplasticity, musculoskeletal, and sensory & pain computational modelling. We also discuss both the importance of and how to perform model validation, as well as challenges to overcome when implementing computational models within a clinical setting. The patient-in-the-loop approach offers a unifying framework to guide multidisciplinary collaboration between computational and clinical stakeholders in the field of neurorehabilitation.

The effects of prior exposure to prism lenses on de novo motor skill learning

Motor learning involves plasticity in a network of brain areas across the cortex and cerebellum. Such traces of learning have the potential to affect subsequent learning of other tasks. In some cases, prior learning can interfere with subsequent learning, but it may be possible to potentiate learning of one task with a prior task if they are sufficiently different. Because prism adaptation involves extensive neuroplasticity, we reasoned that the elevated excitability of neurons could increase their readiness to undergo structural changes, and in turn, create an optimal state for learning a subsequent task. We tested this idea, selecting two different forms of learning tasks, asking whether exposure to a sensorimotor adaptation task can improve subsequent de novo motor skill learning. Participants first learned a new visuomotor mapping induced by prism glasses in which prism strength varied trial-to-trial. Immediately after and the next day, we tested participants on a mirror tracing task, a form of de novo skill learning. Prism-trained and control participants both learned the mirror tracing task, with similar reductions in error and increases in distance traced. Both groups also showed evidence of offline performance gains between the end of day 1 and the start of day 2. However, we did not detect differences between groups. Overall, our results do not support the idea that prism adaptation learning can potentiate subsequent de novo learning. We discuss factors that may have contributed to this result.

Kinematic markers of skill in first-person shooter video games

Video games present a unique opportunity to study motor skill. First-person shooter (FPS) games have particular utility because they require visually guided hand movements that are similar to widely studied planar reaching tasks. However, there is a need to ensure the tasks are equivalent if FPS games are to yield their potential as a powerful scientific tool for investigating sensorimotor control. Specifically, research is needed to ensure that differences in visual feedback of a movement do not affect motor learning between the two contexts. In traditional tasks, a movement will translate a cursor across a static background, whereas FPS games use movements to pan and tilt the view of the environment. To this end, we designed an online experiment where participants used their mouse or trackpad to shoot targets in both visual contexts. Kinematic analysis showed player movements were nearly identical between contexts, with highly correlated spatial and temporal metrics. This similarity suggests a shared internal model based on comparing predicted and observed displacement vectors rather than primary sensory feedback. A second experiment, modeled on FPS-style aim-trainer games, found movements exhibited classic invariant features described within the sensorimotor literature. We found the spatial metrics tested were significant predictors of overall task performance. More broadly, these results show that FPS games offer a novel, engaging, and compelling environment to study sensorimotor skill, providing the same precise kinematic metrics as traditional planar reaching tasks.

Neural Plasticity in Sensorimotor Brain-Machine Interfaces

Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.

Asymmetrical transfer of adaptation between reaching and tracking: implications for feedforward and feedback processes

Reaching and manual tracking are two very common tasks for studying human sensorimotor processes. Although these motor tasks rely both on feedforward and feedback processes, emphasis is more on feedforward processes for reaching, and more on feedback processes for tracking. The extent to which feedforward and feedback processes are interrelated when being updated is not settled yet. Here, using reaching and tracking as proxies, we examined the bidirectional relationship between the update of feedforward and feedback processes. Forty right-handed participants were asked to move a joystick so as to either track a target moving rather unpredictably (pursuit tracking) or to make fast pointing movements toward a static target (center-out reaching task). Visuomotor adaptation was elicited by introducing a 45° rotation between the joystick motion and the cursor motion. Half of the participants adapted to rotation first via reaching movements, and then with pursuit tracking, while the other half performed both tasks in opposite order. Group comparisons revealed a strong asymmetrical transfer of adaptation between tasks. Namely, although nearly complete transfer of adaptation was observed from reaching to tracking, only modest transfer was found from tracking to reaching. A control experiment (N=10) revealed that making target motion fully predictable did not impact the latter finding. One possible interpretation is that the update of feedforward processes contributes directly to feedback processes, but the update of feedback processes engaged in tracking can be performed in isolation. These results suggest that reaching movements are supported by broader (i.e. more universal) mechanisms than tracking ones.

Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks

Humans learn movements naturally, but it takes a lot of time and training to achieve expert performance in motor skills. In this review, we show how modern technologies can support people in learning new motor skills. First, we introduce important concepts in motor control, motor learning and motor skill learning. We also give an overview about the rapid expansion of machine learning algorithms and sensor technologies for human motion analysis. The integration between motor learning principles, machine learning algorithms and recent sensor technologies has the potential to develop AI-guided assistance systems for motor skill training. We give our perspective on this integration of different fields to transition from motor learning research in laboratory settings to real world environments and real world motor tasks and propose a stepwise approach to facilitate this transition.

Understanding implicit sensorimotor adaptation as a process of proprioceptive re-alignment

Multiple learning processes contribute to successful goal-directed actions in the face of changing physiological states, biomechanical constraints, and environmental contexts. Amongst these processes, implicit sensorimotor adaptation is of primary importance, ensuring that movements remain well-calibrated and accurate. A large body of work on reaching movements has emphasized how adaptation centers on an iterative process designed to minimize visual errors. The role of proprioception has been largely neglected, thought to play a passive role in which proprioception is affected by the visual error but does not directly contribute to adaptation. Here, we present an alternative to this visuo-centric framework, outlining a model in which implicit adaptation acts to minimize a proprioceptive error, the distance between the perceived hand position and its intended goal. This proprioceptive re-alignment model (PReMo) is consistent with many phenomena that have previously been interpreted in terms of learning from visual errors, and offers a parsimonious account of numerous unexplained phenomena. Cognizant that the evidence for PReMo rests on correlational studies, we highlight core predictions to be tested in future experiments, as well as note potential challenges for a proprioceptive-based perspective on implicit adaptation.

Long-Term Motor Learning in the "Wild" With High Volume Video Game Data

Motor learning occurs over long periods of practice during which motor acuity, the ability to execute actions more accurately, precisely, and in less time, improves. Laboratory-based studies of motor learning are typically limited to a small number of participants and a time frame of minutes to several hours per participant. There is a need to assess the generalizability of theories and findings from lab-based motor learning studies on larger samples and time scales. In addition, laboratory-based studies of motor learning use relatively simple motor tasks which participants are unlikely to be intrinsically motivated to learn, limiting the interpretation of their findings in more ecologically valid settings ("in the wild"). We studied the acquisition and longitudinal refinement of a complex sensorimotor skill embodied in a first-person shooter video game scenario, with a large sample size (N = 7174, 682,564 repeats of the 60 s game) over a period of months. Participants voluntarily practiced the gaming scenario for up to several hours per day up to 100 days. We found improvement in performance accuracy (quantified as hit rate) was modest over time but motor acuity (quantified as hits per second) improved considerably, with 40-60% retention from 1 day to the next. We observed steady improvements in motor acuity across multiple days of video game practice, unlike most motor learning tasks studied in the lab that hit a performance ceiling rather quickly. Learning rate was a non-linear function of baseline performance level, amount of daily practice, and to a lesser extent, number of days between practice sessions. In addition, we found that the benefit of additional practice on any given day was non-monotonic; the greatest improvements in motor acuity were evident with about an hour of practice and 90% of the learning benefit was achieved by practicing 30 min per day. Taken together, these results provide a proof-of-concept in studying motor skill acquisition outside the confines of the traditional laboratory, in the presence of unmeasured confounds, and provide new insights into how a complex motor skill is acquired in an ecologically valid setting and refined across much longer time scales than typically explored.

Playing the piano with a robotic third thumb: assessing constraints of human augmentation

Contemporary robotics gives us mechatronic capabilities for augmenting human bodies with extra limbs. However, how our motor control capabilities pose limits on such augmentation is an open question. We developed a Supernumerary Robotic 3rd Thumbs (SR3T) with two degrees-of-freedom controlled by the user’s body to endow them with an extra contralateral thumb on the hand. We demonstrate that a pianist can learn to play the piano with 11 fingers within an hour. We then evaluate 6 naïve and 6 experienced piano players in their prior motor coordination and their capability in piano playing with the robotic augmentation. We show that individuals’ augmented performance with the SR3T could be explained by our new custom motor coordination assessment, the Human Augmentation Motor Coordination Assessment (HAMCA) performed pre-augmentation. Our work demonstrates how supernumerary robotics can augment humans in skilled tasks and that individual differences in their augmentation capability are explainable by their individual motor coordination abilities.