Learning New Feedforward Motor Commands Based on Feedback Responses

Cerebellar Purkinje cells in male macaques combine sensory and motor information to predict the sensory consequences of active self-motion

Accurate perception and behavior rely on distinguishing sensory signals arising from unexpected events from those originating from our own voluntary actions. In the vestibular system, sensory input that is the consequence of active self-motion is canceled early at the first central stage of processing to ensure postural and perceptual stability. However, the source of the required cancellation signal was unknown. Here, we show that the cerebellum combines sensory and motor-related information to predict the sensory consequences of active self-motion. Recordings during attempted but unrealized head movements in two male rhesus monkeys, revealed that the motor-related signals encoded by anterior vermis Purkinje cells explain their altered sensitivity to active versus passive self-motion. Further, a model combining responses from ~40 Purkinje cells accounted for the cancellation observed in early vestibular pathways. These findings establish how cerebellar Purkinje cells predict sensory outcomes of self-movements, resolving a long-standing issue of sensory signal suppression during self-motion.

Human reaching control in dynamic environments

Closed-loop models of movement control have attracted growing interest in how the nervous system transforms sensory information into motor commands, and several brain structures have been identified as neural substrates for these computational operations. Recently, several studies have focused on how these models need to be updated when environmental parameters change. Current evidence suggests that when the task changes, rapid control updates enable flexible modifications of current actions and online decisions. At the same time, when movement dynamics change, humans use different strategies based on a combination of adaptation and modulation of controller sensitivity to exogenous perturbations (robust control). This review proposes a unified framework to capture these results based on online estimation of model parameters with dynamic updates in control. The reviewed studies also identify the time scales of associated behavioral mechanisms to guide future research on the neural basis of movement control.

Error-independent effect of sensory uncertainty on motor learning when both feedforward and feedback control processes are engaged

Integrating sensory information during movement and adapting motor plans over successive movements are both essential for accurate, flexible motor behaviour. When an ongoing movement is off target, feedback control mechanisms update the descending motor commands to counter the sensed error. Over longer timescales, errors induce adaptation in feedforward planning so that future movements become more accurate and require less online adjustment from feedback control processes. Both the degree to which sensory feedback is integrated into an ongoing movement and the degree to which movement errors drive adaptive changes in feedforward motor plans have been shown to scale inversely with sensory uncertainty. However, since these processes have only been studied in isolation from one another, little is known about how they are influenced by sensory uncertainty in real-world movement contexts where they co-occur. Here, we show that sensory uncertainty may impact feedforward adaptation of reaching movements differently when feedback integration is present versus when it is absent. In particular, participants gradually adjust their movements from trial-to-trial in a manner that is well characterised by a slow and consistent envelope of error reduction. Riding on top of this slow envelope, participants exhibit large and abrupt changes in their initial movement vectors that are strongly correlated with the degree of sensory uncertainty present on the previous trial. However, these abrupt changes are insensitive to the magnitude and direction of the sensed movement error. These results prompt important questions for current models of sensorimotor learning under uncertainty and open up new avenues for future exploration in the field.

The effect of perturbation training with and without applying the dry needling on leg muscles in patients with chronic ankle sprain

INTRODUCTION

Ankle sprain is a common musculoskeletal injury that leads to recurrent instability. Repeated ankle sprain can be a mechanism for creating trigger point. Proper treatment of trigger points, in addition to preventing recurrence of sprains, may reduce pain and improve muscle function. This improvement can be the result of preserving the surrounding tissues from excessive pressure.

OBJECTIVE

Investigate the added value of dry needling into perturbation training protocol for chronic ankle sprain.

DESIGN

Randomized clinical trial; assessor-blind; before and after comparison.

SETTING

Treatment of patients referred to the institutional rehabilitation clinics.

MAIN OUTCOME MEASURE(S)

Functional assessment with FAAM questionnaire score, Pain with NPRS scale, ankle instability severity with Cumberland tool.

METHODS

Twenty-four patients with chronic ankle instability participated in this clinical trial and were randomly divided into two groups. Intervention was 12 sessions in which one group received only perturbation training and the other group received perturbation training along with dry needling. Repeated measures ANOVA was used to investigate the effect of treatment.

RESULTS

Data Analysis showed significant difference in NPRS and FAAM and Cumberland score before and after treatment in each group (P < 0.001). Comparison of the results between the groups did not show any significant difference (P > 0.05).

CONCLUSION

The findings showed that adding dry needling technique to the perturbation training does not have greater effects on the pain and function of patients with chronic ankle instability.

The nervous system tunes sensorimotor gains when reaching in variable mechanical environments

Humans often move in the presence of mechanical disturbances that can vary in direction and amplitude throughout movement. These disturbances can jeopardize the outcomes of our actions, such as when drinking from a glass of water on a turbulent flight or carrying a cup of coffee while walking on a busy sidewalk. Here, we examine control strategies that allow the nervous system to maintain performance when reaching in the presence of mechanical disturbances that vary randomly throughout movement. Healthy participants altered their control strategies to make movements more robust against disturbances. The change in control was associated with faster reaching movements and increased responses to proprioceptive and visual feedback that were tuned to the variability of the disturbances. Our findings highlight that the nervous system exploits a continuum of control strategies to increase its responsiveness to sensory feedback when reaching in the presence of increasingly variable physical disturbances.

Postural sway is not affected by estrogen fluctuations during the menstrual cycle

When people stand still, they exhibit a phenomenon called postural sway, or spontaneous movement of the body's center of pressure, which is related to balance control. In general females show less sway than males, but this difference only begins to appear around puberty, pointing to different levels of sex hormones as one potential mechanism for sway sex differences. In this study, we followed cohorts of young females using oral contraceptives (n = 32) and not using oral contraceptives (n = 19), to investigate associations between estrogen availability and postural sway. All participants visited the lab four times over the putative 28-day menstrual cycle. At each visit, we performed blood draws to measure plasma estrogen (estradiol) levels, and tests of postural sway using a force plate. During late follicular and mid-luteal phase, estradiol levels were lower in participants using oral contraceptives (mean differences [95% CI], respectively: -231.33; [-800.44, 337.87]; -613.26; [-1333.60, 107.07] pmol/L; main effect p < 0.001), reflecting expected consequences of oral contraceptive use. Despite these differences, postural sway was not significantly different between participants who were using oral contraceptives and participants who were not (mean difference: 2.09 cm; 95% CI = [-1.05, 5.22]; p = 0.132). Overall, we found no significant effects of the estimated menstrual cycle phase-or absolute levels of estradiol-on postural sway.

Separability of Human Motor Memories during reaching adaptation with force cues

Judging by the breadth of our motor repertoire during daily activities, it is clear that learning different tasks is a hallmark of the human motor system. However, for reaching adaptation to different force fields, the conditions under which this is possible in laboratory settings have remained a challenging question. Previous work has shown that independent movement representations or goals enabled dual adaptation. Considering the importance of force feedback during limb control, here we hypothesised that independent cues delivered by means of background loads could support simultaneous adaptation to various velocity-dependent force fields, for identical kinematic plan and movement goal. We demonstrate in a series of experiments that indeed healthy adults can adapt to opposite force fields, independently of the direction of the background force cue. However, when the cue and force field were in the same direction but differed by heir magnitude, the formation of different motor representations was still observed but the associated mechanism was subject to increased interference. Finally, we highlight that this paradigm allows dissociating trial-by-trial adaptation from online feedback adaptation, as these two mechanisms are associated with different time scales that can be identified reliably and reproduced in a computational model.

Asymmetrical transfer of adaptation between reaching and tracking: implications for feedforward and feedback processes

Reaching and manual tracking are two very common tasks for studying human sensorimotor processes. Although these motor tasks rely both on feedforward and feedback processes, emphasis is more on feedforward processes for reaching, and more on feedback processes for tracking. The extent to which feedforward and feedback processes are interrelated when being updated is not settled yet. Here, using reaching and tracking as proxies, we examined the bidirectional relationship between the update of feedforward and feedback processes. Forty right-handed participants were asked to move a joystick so as to either track a target moving rather unpredictably (pursuit tracking) or to make fast pointing movements toward a static target (center-out reaching task). Visuomotor adaptation was elicited by introducing a 45° rotation between the joystick motion and the cursor motion. Half of the participants adapted to rotation first via reaching movements, and then with pursuit tracking, while the other half performed both tasks in opposite order. Group comparisons revealed a strong asymmetrical transfer of adaptation between tasks. Namely, although nearly complete transfer of adaptation was observed from reaching to tracking, only modest transfer was found from tracking to reaching. A control experiment (N=10) revealed that making target motion fully predictable did not impact the latter finding. One possible interpretation is that the update of feedforward processes contributes directly to feedback processes, but the update of feedback processes engaged in tracking can be performed in isolation. These results suggest that reaching movements are supported by broader (i.e. more universal) mechanisms than tracking ones.

Contextual cues are not unique for motor learning: Task-dependant switching of feedback controllers

The separation of distinct motor memories by contextual cues is a well known and well studied phenomenon of feedforward human motor control. However, there is no clear evidence of such context-induced separation in feedback control. Here we test both experimentally and computationally if context-dependent switching of feedback controllers is possible in the human motor system. Specifically, we probe visuomotor feedback responses of our human participants in two different tasks—stop and hit—and under two different schedules. The first, blocked schedule, is used to measure the behaviour of stop and hit controllers in isolation, showing that it can only be described by two independent controllers with two different sets of control gains. The second, mixed schedule, is then used to compare how such behaviour evolves when participants regularly switch from one task to the other. Our results support our hypothesis that there is contextual switching of feedback controllers, further extending the accumulating evidence of shared features between feedforward and feedback control.

Extensive evidence has demonstrated that humans can learn distinct motor memories (i.e. independent feedforward controllers) using contextual cues. However, there is little evidence that such contextual cues produce similar separation of feedback controllers. As accumulating evidence highlights the connection between feedforward and feedback control, we propose that context may be used to separate feedback controllers as well. It has not been trivial to test experimentally whether a change in context also modulates the feedback control, as the controller output is affected by other non-contextual factors such as movement kinematics, time-to-target or the properties of the perturbation used to probe the control. Here we present a computational approach based on normative modelling where we separate the effects of the context from other non-contextual effects on the visuomotor feedback system. We then show experimentally that task context independently modulates the feedback control in a particular manner that can be reliably predicted using optimal feedback control.

Task space exploration improves adaptation after incompatible virtual surgeries

Humans have a remarkable capacity to learn new motor skills, a process that requires novel muscle activity patterns. Muscle synergies may simplify the generation of muscle patterns through the selection of a small number of synergy combinations. Learning new motor skills may then be achieved by acquiring novel muscle synergies. In a previous study, we used myoelectric control to construct virtual surgeries that altered the mapping from muscle activity to cursor movements. After compatible virtual surgeries, which could be compensated by recombining subject-specific muscle synergies, participants adapted quickly. In contrast, after incompatible virtual surgeries, which could not be compensated by recombining existing synergies, participants explored new muscle patterns, but failed to adapt. Here, we tested whether task space exploration can promote learning of novel muscle synergies, required to overcome an incompatible surgery. Participants performed the same reaching task as in our previous study, but with more time to complete each trial, thus allowing for exploration. We found an improvement in trial success after incompatible virtual surgeries. Remarkably, improvements in movement direction accuracy after incompatible surgeries occurred faster for corrective movements than for the initial movement, suggesting that learning of new synergies is more effective when used for feedback control. Moreover, reaction time was significantly higher after incompatible than after compatible virtual surgeries, suggesting an increased use of an explicit adaptive strategy to overcome incompatible surgeries. Taken together, these results indicate that exploration is important for skill learning and suggest that human participants, with sufficient time, can learn new muscle synergies.

Adaptive Feedback Control in Human Reaching Adaptation to Force Fields

Sensorimotor adaptation is a central function of the nervous system, as it allows humans and other animals to flexibly anticipate their interaction with the environment. In the context of human reaching adaptation to force fields, studies have traditionally separated feedforward (FF) and feedback (FB) processes involved in the improvement of behavior. Here, we review computational models of FF adaptation to force fields and discuss them in light of recent evidence highlighting a clear involvement of feedback control. Instead of a model in which FF and FB mechanisms adapt in parallel, we discuss how online adaptation in the feedback control system can explain both trial-by-trial adaptation and improvements in online motor corrections. Importantly, this computational model combines sensorimotor control and short-term adaptation in a single framework, offering novel perspectives for our understanding of human reaching adaptation and control.

Back to feedback: aberrant sensorimotor control in music performance under pressure

Precisely timed production of dexterous actions is often destabilized in anxiogenic situations. Previous studies demonstrated that cognitive functions such as attention and working memory as well as autonomic nervous functions are susceptible to psychological stress in skillful performance while playing sports or musical instruments. However, it is not known whether the degradation of sensorimotor functions underlies such a compromise of skillful performance due to psychophysiological distress. Here, we addressed this issue through a set of behavioral experiments. After artificially delaying the timing of tone production while playing the piano, the local tempo was abnormally disrupted only under pressure. The results suggest that psychological stress degraded the temporal stability of movement control due to an abnormal increase in feedback gain. A learning experiment further demonstrated that the temporal instability of auditory-motor control under pressure was alleviated after practicing piano while ignoring delayed auditory feedback but not after practicing while compensating for the delayed feedback. Together, these findings suggest an abnormal transition from feedforward to feedback control in expert piano performance with psychological stress, which can be mitigated through specialized sensorimotor training that involves piano practice while volitionally ignoring the artificially delayed provision of auditory feedback.

In order to establish if the degradation of sensorimotor functions underlies the stress-associated disruption of skilful performance, Furuya et al examined participants undergoing a piano playing task under stress. Their data suggests the occurrence of a stress-induced transition from feedforward to feedback control, which can be mitigated through sensorimotor training involving piano practice while volitionally ignoring the artificially delayed provision of auditory feedback.

Stretch reflexes

Many of us know about stretch reflexes from the doctor's office, when a physician taps the tendon near our kneecap to elicit a quick knee extension. This procedure is used as a diagnostic tool to determine the integrity of the spinal cord and the extension response it elicits may seem otherwise useless. In fact, the tendon tap taps into one aspect of a critical building block of mammalian motor control, the stretch reflexes. Stretch reflexes are often thought to quickly resist unexpected changes in muscle length via a very simple circuit in the spinal cord, and this is one circuit that the tendon tap engages. It turns out, however, that stretch reflexes support a myriad of functions and are highly flexible. Under naturalistic conditions, stretch reflexes are shaped by peripheral physiology and engage neural circuits spanning the spinal cord, brainstem and cerebral cortex. In this Primer, we outline what is currently known about stretch reflex function and its underlying mechanisms, with a specific focus on how the cascade of nested responses collectively known as stretch reflexes interact with and build off of one another to support real-world motor behavior.

Similar stretch reflexes and behavioral patterns are expressed by the dominant and nondominant arms during postural control

Limb dominance is evident in many daily activities, leading to the prominent idea that each hemisphere of the brain specializes in controlling different aspects of movement. Past studies suggest that the dominant arm is primarily controlled via an internal model of limb dynamics that enables the nervous system to produce efficient movements. In contrast, the nondominant arm may be primarily controlled via impedance mechanisms that rely on the strong modulation of sensory feedback from individual joints to control limb posture. We tested whether such differences are evident in behavioral responses and stretch reflexes following sudden displacement of the arm during posture control. Experiment 1 applied specific combinations of elbow-shoulder torque perturbations (the same for all participants). Peak joint displacements, return times, end point accuracy, and the directional tuning and amplitude of stretch reflexes in nearly all muscles were not statistically different between the two arms. Experiment 2 induced specific combinations of joint motion (the same for all participants). Again, peak joint displacements, return times, end point accuracy, and the directional tuning and amplitude of stretch reflexes in nearly all muscles did not differ statistically when countering the imposed loads with each arm. Moderate to strong correlations were found between stretch reflexes and behavioral responses to the perturbations with the two arms across both experiments. Collectively, the results do not support the idea that the dominant arm specializes in exploiting internal models and the nondominant arm in impedance control by increasing reflex gains to counter sudden loads imposed on the arms during posture control.NEW & NOTEWORTHY A prominent hypothesis is that the nervous system controls the dominant arm through predictive internal models and the nondominant arm through impedance mechanisms. We tested whether stretch reflexes of muscles in the two arms also display such specialization during posture control. Nearly all behavioral responses and stretch reflexes did not differ statistically but were strongly correlated between the arms. The results indicate individual signatures of feedback control that are common for the two arms.

Rapid Feedback Responses Parallel the Urgency of Voluntary Reaching Movements

Optimal feedback control is a prominent theory used to interpret human motor behaviour. The theory posits that skilled actions emerge from control policies that link voluntary motor control (feedforward) with flexible feedback corrections (feedback control). It is clear the nervous system can generate flexible motor corrections (reflexes) when performing actions with different goals. We know little, however, about shared features of voluntary actions and feedback control in human movement. Here we reveal a link between the timing demands of voluntary actions and flexible responses to mechanical perturbations. In two experiments, 40 human participants (21 females) made reaching movements with different timing demands. We disturbed the arm with mechanical perturbations at movement onset (Experiment 1) and at locations ranging from movement onset to completion (Experiment 2). We used the resulting muscle responses and limb displacements as a proxy for the control policies that support voluntary reaching movements. We observed an increase in the sensitivity of elbow and shoulder muscle responses and a reduction in limb motion when the task imposed greater urgency to respond to the same perturbations. The results reveal a relationship between voluntary actions and feedback control as the limb was displaced less when moving faster in perturbation trials. Muscle responses scaled with changes in the displacement of the limb in perturbation trials within each timing condition. Across both experiments, human behaviour was captured by simulations based on stochastic optimal feedback control. Taken together, the results highlight flexible control that links sensory processing with features of human reaching movements.

Online sensorimotor learning and adaptation for inverse dynamics control

We propose a micro-data (< 10 trials) sensorimotor learning and adaptation (SEED) model for human-like arm inverse dynamics control. The SEED model consists of a feedforward Gaussian motor primitive (GATE) neural network and an adaptive feedback impedance (AIM) mechanism. Sensorimotor weights over trials are learned in the GATE network, while the AIM mechanism is used to online tune impedance gains in a trial. The model was validated by periodic and non-periodic tracking tasks on a two-joint robot arm. As a result, the proposed model enables the arm to stably learn the tasks within 10 trials, compared to thousands of trials required by state-of-art deep learning. This model facilitates the exploration of unknown arm dynamics, in which the elbow joint requires much less active control compared to the shoulder. This control goes below 3% of the overall effort. This finding complies with a proximal-distal control gradient in human arm control. Taken together, the proposed SEED model paves a way for implementing data-efficient sensorimotor learning and adaptation of human-like arm movement.

Compensatory control between the legs in automatic postural responses to stance perturbations under single-leg fatigue

In response to sudden perturbations of stance stability, muscles of both legs are activated for balance recovery. In conditions that one of the legs has a reduced capacity to respond, the opposite leg is predicted to compensate by responding more powerfully to restore stable upright stance. In this investigation, we aimed to evaluate between-leg compensatory control in automatic postural responses to sudden perturbations in a situation in which plantar flexor muscles of a single leg were fatigued. Young participants were evaluated in response to a series of perturbations inducing forward body sway, with a focus on activation of plantar flexor muscles: lateral and medial gastrocnemii and soleus. Muscular responses were analyzed through activation magnitude and latency of muscular activation onset. For evaluation of balance and postural stability, we also analyzed the center of pressure and upper trunk displacement and weight-bearing asymmetry between the legs. Responses were assessed in three conditions: pre-fatigue, under single-leg fatigue, and following the recovery of muscular function. Results showed (a) compensation of the non-fatigued leg through the increased magnitude of muscular activation in the first perturbation under fatigue; (b) adaptation in the non-fatigued leg over repetitive perturbations, with a progressive decrement of muscular activation over trials; and (c) maintenance of increased muscular activation of the non-fatigued leg following fatigue dissipation. These findings suggest that the central nervous system is able to modulate the descending motor drive individually for each leg's muscles apparently based on their potential contribution for the achievement of the behavioral aim of recovering stable body balance following stance perturbations.

Rapid Changes in Movement Representations during Human Reaching Could Be Preserved in Memory for at Least 850 ms

Humans adapt to mechanical perturbations such as forcefields (FFs) during reaching within tens of trials. However, recent findings suggested that this adaptation may start within one single trial, i.e., online corrective movements can become tuned to the unanticipated perturbations within a trial. This was highlighted in previous works with a reaching experiment in which participants had to stop at a via-point (VP) located between the start and the goal. An FF was applied during the first and second parts of the movement and then occasionally unexpectedly switched off at the VP during catch trials. The results showed an after-effect during the second part of the movement when participants exited the VP. This behavioral result was interpreted as a standard after-effect, but it remained unclear how it was related to conventional trial-by-trial learning. The current study aimed to investigate how long do such changes in movement representations last in memory. For this, we have studied the same reaching task with VP in two situations: one with very short residing time in the VP and the second with an imposed minimum 500 ms dwell time in the VP. In both situations, during the unexpected absence of the FF after VP, after-effects were observed. This suggests that online corrections to the internal representation of reach dynamics can be preserved in memory for around 850 ms of resting time on average. Therefore, rapid changes occurring within movements can thus be preserved in memory long enough to influence trial-by-trial motor adaptation.

Shared internal models for feedforward and feedback control of arm dynamics in non-human primates

Previous work has shown that humans account for and learn novel properties or the arm's dynamics, and that such learning causes changes in both the predictive (i.e., feedforward) control of reaching and reflex (i.e., feedback) responses to mechanical perturbations. Here we show that similar observations hold in old-world monkeys (Macaca fascicularis). Two monkeys were trained to use an exoskeleton to perform a single-joint elbow reaching and to respond to mechanical perturbations that created pure elbow motion. Both of these tasks engaged robust shoulder muscle activity as required to account for the torques that typically arise at the shoulder when the forearm rotates around the elbow joint (i.e., intersegmental dynamics). We altered these intersegmental arm dynamics by having the monkeys generate the same elbow movements with the shoulder joint either free to rotate, as normal, or fixed by the robotic manipulandum, which eliminates the shoulder torques caused by forearm rotation. After fixing the shoulder joint, we found a systematic reduction in shoulder muscle activity. In addition, after releasing the shoulder joint again, we found evidence of kinematic aftereffects (i.e., reach errors) in the direction predicted if failing to compensate for normal arm dynamics. We also tested whether such learning transfers to feedback responses evoked by mechanical perturbations and found a reduction in shoulder feedback responses, as appropriate for these altered arm intersegmental dynamics. Demonstrating this learning and transfer in non-human primates will allow the investigation of the neural mechanisms involved in feedforward and feedback control of the arm's dynamics.