Robust Control in Human Reaching Movements: A Model-Free Strategy to Compensate for Unpredictable Disturbances

Sensorimotor adaptation to destabilizing dynamics in weakly electric fish

Humans and other animals can readily learn to compensate for changes in the dynamics of movement. Such changes can result from an injury or changes in the weight of carried objects. These changes in dynamics can lead not only to reduced performance but also to dramatic instabilities. We evaluated the impacts of compensatory changes in control policies in relation to stability and robustness in Eigenmannia virescens, a species of weakly electric fish. We discovered that these fish retune their sensorimotor control system in response to experimentally generated destabilizing dynamics. Specifically, we used an augmented reality system to manipulate sensory feedback during an image stabilization task in which a fish maintained its position within a refuge. The augmented reality system measured the fish's movements in real time. These movements were passed through a high-pass filter and multiplied by a gain factor before being fed back to the refuge motion. We adjusted the gain factor to gradually destabilize the fish's sensorimotor loop. The fish retuned their sensorimotor control system to compensate for the experimentally induced destabilizing dynamics. This retuning was partially maintained when the augmented reality feedback was abruptly removed. The compensatory changes in sensorimotor control improved tracking performance as well as control-theoretic measures of robustness, including reduced sensitivity to disturbances and improved phase margins.

Inferring control objectives in a virtual balancing task in humans and monkeys

Natural behaviors have redundancy, which implies that humans and animals can achieve their goals with different strategies. Given only observations of behavior, is it possible to infer the control objective that the subject is employing? This challenge is particularly acute in animal behavior because we cannot ask or instruct the subject to use a particular strategy. This study presents a three-pronged approach to infer an animal's control objective from behavior. First, both humans and monkeys performed a virtual balancing task for which different control strategies could be utilized. Under matched experimental conditions, corresponding behaviors were observed in humans and monkeys. Second, a generative model was developed that represented two main control objectives to achieve the task goal. Model simulations were used to identify aspects of behavior that could distinguish which control objective was being used. Third, these behavioral signatures allowed us to infer the control objective used by human subjects who had been instructed to use one control objective or the other. Based on this validation, we could then infer objectives from animal subjects. Being able to positively identify a subject's control objective from observed behavior can provide a powerful tool to neurophysiologists as they seek the neural mechanisms of sensorimotor coordination.

The control of the arm's equilibrium position

To generate a force, the brain activates muscles that act like springs to pull the arm toward a new equilibrium position. The equilibrium position (EP) is central to our understanding of the biological control of viscoelastic muscles. Although there is evidence of the EP during the control of limb posture, EPs have not been directly identified when the limb exerts a force against the environment. Here, we asked participants to apply a constant force in one of eight directions against a point-like constraint. This constraint was released abruptly to observe the final position to which the arm converged. Importantly, the same force magnitude was maintained while changing the arm's stiffness by modulating the strength of the hand's power grasp. The final position moved further away from the constraint as the arm became less stiff and was inversely proportional to the arm's stiffness, thereby confirming that the final position was the arm's EP. These results demonstrate how the EP changes with the arm's stiffness to produce a desired force in different directions.NEW & NOTEWORTHY According to numerous theories, the brain controls posture and movement by activating muscles that attract the limb toward a so-called equilibrium position, but the universality of this mechanism has not been shown for different motor behaviors. Here, we show that even when pushing or pulling against the environment, the brain achieves the desired force through an equilibrium position that lies beyond the physical constraint.

Human reaching control in dynamic environments

Closed-loop models of movement control have attracted growing interest in how the nervous system transforms sensory information into motor commands, and several brain structures have been identified as neural substrates for these computational operations. Recently, several studies have focused on how these models need to be updated when environmental parameters change. Current evidence suggests that when the task changes, rapid control updates enable flexible modifications of current actions and online decisions. At the same time, when movement dynamics change, humans use different strategies based on a combination of adaptation and modulation of controller sensitivity to exogenous perturbations (robust control). This review proposes a unified framework to capture these results based on online estimation of model parameters with dynamic updates in control. The reviewed studies also identify the time scales of associated behavioral mechanisms to guide future research on the neural basis of movement control.

Inferring control objectives in a virtual balancing task in humans and monkeys

Natural behaviors have redundancy, which implies that humans and animals can achieve their goals with different control objectives. Given only observations of behavior, is it possible to infer the control strategy that the subject is employing? This challenge is particularly acute in animal behavior because we cannot ask or instruct the subject to use a particular control strategy. This study presents a threepronged approach to infer an animal's control strategy from behavior. First, both humans and monkeys performed a virtual balancing task for which different control objectives could be utilized. Under matched experimental conditions, corresponding behaviors were observed in humans and monkeys. Second, a generative model was developed that represented two main control strategies to achieve the task goal. Model simulations were used to identify aspects of behavior that could distinguish which control objective was being used. Third, these behavioral signatures allowed us to infer the control objective used by human subjects who had been instructed to use one control objective or the other. Based on this validation, we could then infer strategies from animal subjects. Being able to positively identify a subject's control objective from behavior can provide a powerful tool to neurophysiologists as they seek the neural mechanisms of sensorimotor coordination.

Continuous evaluation of cost-to-go for flexible reaching control and online decisions

Humans consider the parameters linked to movement goal during reaching to adjust their control strategy online. Indeed, rapid changes in target structure or disturbances interfering with their initial plan elicit rapid changes in behavior. Here, we hypothesize that these changes could result from the continuous use of a decision variable combining motor and cognitive components. We combine an optimal feedback controller with a real-time evaluation of the expected cost-to-go, which considers target- and movement-related costs, in a common theoretical framework. This model reproduces human behaviors in presence of changes in the target structure occurring during movement and of online decisions to flexibly change target following external perturbations. It also predicts that the time taken to decide to select a novel goal after a perturbation depends on the amplitude of the disturbance and on the rewards of the different options, which is a direct result of the continuous monitoring of the cost-to-go. We show that this result was present in our previously collected dataset. Together our developments point towards a continuous evaluation of the cost-to-go during reaching to update control online and make efficient decisions about movement goal.

Task Instructions and the Need for Feedback Correction Influence the Contribution of Visual Errors to Reach Adaptation

Previous research has questioned whether motor adaptation is shaped by an optimal combination of multisensory error signals. Here, we expanded on this work by investigating how the use of visual and somatosensory error signals during online correction influences single-trial adaptation. To this end, we exposed participants to a random sequence of force-field perturbations and recorded their corrective responses as well as the after-effects exhibited during the subsequent unperturbed movement. In addition to the force perturbation, we artificially decreased or increased visual errors by multiplying hand deviations by a gain smaller or larger than one. Corrective responses to the force perturbation clearly scaled with the size of the visual error, but this scaling did not transfer one-to-one to motor adaptation and we observed no consistent interaction between limb and visual errors on adaptation. However, reducing visual errors during perturbation led to a small reduction of after-effects and this residual influence of visual feedback was eliminated when we instructed participants to control their hidden hand instead of the visual hand cursor. Taken together, our results demonstrate that task instructions and the need to correct for errors during perturbation are important factors to consider if we want to understand how the sensorimotor system uses and combines multimodal error signals to adapt movements.

The nervous system tunes sensorimotor gains when reaching in variable mechanical environments

Humans often move in the presence of mechanical disturbances that can vary in direction and amplitude throughout movement. These disturbances can jeopardize the outcomes of our actions, such as when drinking from a glass of water on a turbulent flight or carrying a cup of coffee while walking on a busy sidewalk. Here, we examine control strategies that allow the nervous system to maintain performance when reaching in the presence of mechanical disturbances that vary randomly throughout movement. Healthy participants altered their control strategies to make movements more robust against disturbances. The change in control was associated with faster reaching movements and increased responses to proprioceptive and visual feedback that were tuned to the variability of the disturbances. Our findings highlight that the nervous system exploits a continuum of control strategies to increase its responsiveness to sensory feedback when reaching in the presence of increasingly variable physical disturbances.

The sensorimotor system modulates muscular co-contraction relative to visuomotor feedback responses to regulate movement variability

The naturally occurring variability in our movements often poses a significant challenge when attempting to produce precise and accurate actions, which is readily evident when playing a game of darts. Two differing, yet potentially complementary, control strategies that the sensorimotor system may use to regulate movement variability are impedance control and feedback control. Greater muscular co-contraction leads to greater impedance that acts to stabilize the hand, while visuomotor feedback responses can be used to rapidly correct for unexpected deviations when reaching toward a target. Here, we examined the independent roles and potential interplay of impedance control and visuomotor feedback control when regulating movement variability. Participants were instructed to perform a precise reaching task by moving a cursor through a narrow visual channel. We manipulated cursor feedback by visually amplifying movement variability and/or delaying the visual feedback of the cursor. We found that participants decreased movement variability by increasing muscular co-contraction, aligned with an impedance control strategy. Participants displayed visuomotor feedback responses during the task but, unexpectedly, there was no modulation between conditions. However, we did find a relationship between muscular co-contraction and visuomotor feedback responses, suggesting that participants modulated impedance control relative to feedback control. Taken together, our results highlight that the sensorimotor system modulates muscular co-contraction, relative to visuomotor feedback responses, to regulate movement variability and produce accurate actions.NEW & NOTEWORTHY The sensorimotor system has the constant challenge of dealing with the naturally occurring variability in our movements. Here, we investigated the potential roles of muscular co-contraction and visuomotor feedback responses to regulate movement variability. When we visually amplified movements, we found that the sensorimotor system primarily uses muscular co-contraction to regulate movement variability. Interestingly, we found that muscular co-contraction was modulated relative to inherent visuomotor feedback responses, suggesting an interplay between impedance and feedback control.

Distinct adaptation processes underlie multidigit force coordination for dexterous manipulation

The human sensorimotor system can adapt to various changes in the environmental dynamics by updating motor commands to improve performance after repeated exposure to the same task. However, the characteristics and mechanisms of the adaptation process remain unknown for dexterous manipulation, a unique motor task in which the body physically interacts with the environment with multiple effectors, i.e., digits, in parallel. We addressed this gap by using robotic manipulanda to investigate the changes in the digit force coordination following mechanical perturbation of an object held by tripod grasps. As the participants gradually adapted to lifting the object under perturbations, we quantified two components of digit force coordination. One is the direction-specific manipulation moment that directly counteracts the perturbation, whereas the other one is the direction-independent internal moment that supports the stability and stiffness of the grasp. We found that trial-to-trial improvement of task performance was associated with increased manipulation moment and a gradual decrease of the internal moment. These two moments were characterized by different rates of adaptation. We also examined how these two force coordination components respond to changes in perturbation directions. Importantly, we found that the manipulation moment was sensitive to the extent of repetitive exposure to the previous context that has an opposite perturbation direction, whereas the internal moment did not. However, the internal moment was sensitive to whether the postchange perturbation direction was previously experienced. Our results reveal, for the first time, that two distinct processes underlie the adaptation of multidigit force coordination for dexterous manipulation.NEW & NOTEWORTHY Changes in digit force coordination in multidigit object manipulation were quantified with a novel experimental design in which human participants adapted to mechanical perturbations applied to the object. Our results show that the adaptation of digit force coordination can be characterized by two distinct components that operate at different timescales. We further show that these two components respond to changes in perturbation direction differently.

Proprioceptive and Visual Feedback Responses in Macaques Exploit Goal Redundancy

A common problem in motor control concerns how to generate patterns of muscle activity when there are redundant solutions to attain a behavioral goal. Optimal feedback control is a theory that has guided many behavioral studies exploring how the motor system incorporates task redundancy. This theory predicts that kinematic errors that deviate the limb should not be corrected if one can still attain the behavioral goal. Studies in humans demonstrate that the motor system can flexibly integrate visual and proprioceptive feedback of the limb with goal redundancy within 90 ms and 70 ms, respectively. Here, we show monkeys (Macaca mulatta) demonstrate similar abilities to exploit goal redundancy. We trained four male monkeys to reach for a goal that was either a narrow square or a wide, spatially redundant rectangle. Monkeys exhibited greater trial-by-trial variability when reaching to the wide goal consistent with exploiting goal redundancy. On random trials we jumped the visual feedback of the hand and found monkeys corrected for the jump when reaching to the narrow goal and largely ignored the jump when reaching for the wide goal. In a separate set of experiments, we applied mechanical loads to the arm of the monkey and found similar corrective responses based on goal shape. Muscle activity reflecting these different corrective responses were detected for the visual and mechanical perturbations starting at ∼90 and ∼70 ms, respectively. Thus, rapid motor responses in macaques can exploit goal redundancy similar to humans, creating a paradigm to study the neural basis of goal-directed motor action and motor redundancy.SIGNIFICANCE STATEMENT Moving in the world requires selecting from an infinite set of possible motor commands. Theories predict that motor commands are selected that exploit redundancies. Corrective responses in humans to either visual or proprioceptive disturbances of the limb can rapidly exploit redundant trajectories to a goal in <100 ms after a disturbance. However, uncovering the neural correlates generating these rapid motor corrections has been hampered by the absence of an animal model. We developed a behavioral paradigm in monkeys that incorporates redundancy in the form of the shape of the goal. Critically, monkeys exhibit corrective responses and timings similar to humans performing the same task. Our paradigm provides a model for investigating the neural correlates of sophisticated rapid motor corrections.

Learning vs. minding: How subjective costs can mask motor learning

When learning new movements some people make larger kinematic errors than others, interpreted as a reduction in motor-learning ability. Consider a learning task where error-cancelling strategies incur higher effort costs, specifically where subjects reach to targets in a force field. Concluding that those with greater error have learned less has a critical assumption: everyone uses the same error-canceling strategy. Alternatively, it could be that those with greater error may be choosing to sacrifice error reduction in favor of a lower effort movement. Here, we test this hypothesis in a dataset that includes both younger and older adults, where older adults exhibited greater kinematic errors. Utilizing the framework of optimal control theory, we infer subjective costs (i.e., strategies) and internal model accuracy (i.e., proportion of the novel dynamics learned) by fitting a model to each population's trajectory data. Our results demonstrate trajectories are defined by a combination of the amount learned and strategic differences represented by relative cost weights. Based on the model fits, younger adults could have learned between 65-90% of the novel dynamics. Critically, older adults could have learned between 60-85%. Each model fit produces trajectories that match the experimentally observed data, where a lower proportion learned in the model is compensated for by increasing costs on kinematic errors relative to effort. This suggests older and younger adults could be learning to the same extent, but older adults have a higher relative cost on effort compared to younger adults. These results call into question the proposition that older adults learn less than younger adults and provide a potential explanation for the equivocal findings in the literature. Importantly, our findings suggest that the metrics commonly used to probe motor learning paint an incomplete picture, and that to accurately quantify the learning process the subjective costs of movements should be considered.

Separability of Human Motor Memories during reaching adaptation with force cues

Judging by the breadth of our motor repertoire during daily activities, it is clear that learning different tasks is a hallmark of the human motor system. However, for reaching adaptation to different force fields, the conditions under which this is possible in laboratory settings have remained a challenging question. Previous work has shown that independent movement representations or goals enabled dual adaptation. Considering the importance of force feedback during limb control, here we hypothesised that independent cues delivered by means of background loads could support simultaneous adaptation to various velocity-dependent force fields, for identical kinematic plan and movement goal. We demonstrate in a series of experiments that indeed healthy adults can adapt to opposite force fields, independently of the direction of the background force cue. However, when the cue and force field were in the same direction but differed by heir magnitude, the formation of different motor representations was still observed but the associated mechanism was subject to increased interference. Finally, we highlight that this paradigm allows dissociating trial-by-trial adaptation from online feedback adaptation, as these two mechanisms are associated with different time scales that can be identified reliably and reproduced in a computational model.

Human motor learning is robust to control-dependent noise

Noises are ubiquitous in sensorimotor interactions and contaminate the information provided to the central nervous system (CNS) for motor learning. An interesting question is how the CNS manages motor learning with imprecise information. Integrating ideas from reinforcement learning and adaptive optimal control, this paper develops a novel computational mechanism to explain the robustness of human motor learning to the imprecise information, caused by control-dependent noise that exists inherently in the sensorimotor systems. Starting from an initial admissible control policy, in each learning trial the mechanism collects and uses the noisy sensory data (caused by the control-dependent noise) to form an imprecise evaluation of the performance of the current policy and then constructs an updated policy based on the imprecise evaluation. As the number of learning trials increases, the generated policies mathematically provably converge to a (potentially small) neighborhood of the optimal policy under mild conditions, despite the imprecise information in the learning process. The mechanism directly synthesizes the policies from the sensory data, without identifying an internal forward model. Our preliminary computational results on two classic arm reaching tasks are in line with experimental observations reported in the literature. The model-free control principle proposed in the paper sheds more lights into the inherent robustness of human sensorimotor systems to the imprecise information, especially control-dependent noise, in the CNS.

An approximate stochastic optimal control framework to simulate nonlinear neuro-musculoskeletal models in the presence of noise

Optimal control simulations have shown that both musculoskeletal dynamics and physiological noise are important determinants of movement. However, due to the limited efficiency of available computational tools, deterministic simulations of movement focus on accurately modelling the musculoskeletal system while neglecting physiological noise, and stochastic simulations account for noise while simplifying the dynamics. We took advantage of recent approaches where stochastic optimal control problems are approximated using deterministic optimal control problems, which can be solved efficiently using direct collocation. We were thus able to extend predictions of stochastic optimal control as a theory of motor coordination to include muscle coordination and movement patterns emerging from non-linear musculoskeletal dynamics. In stochastic optimal control simulations of human standing balance, we demonstrated that the inclusion of muscle dynamics can predict muscle co-contraction as minimal effort strategy that complements sensorimotor feedback control in the presence of sensory noise. In simulations of reaching, we demonstrated that nonlinear multi-segment musculoskeletal dynamics enables complex perturbed and unperturbed reach trajectories under a variety of task conditions to be predicted. In both behaviors, we demonstrated how interactions between task constraint, sensory noise, and the intrinsic properties of muscle influence optimal muscle coordination patterns, including muscle co-contraction, and the resulting movement trajectories. Our approach enables a true minimum effort solution to be identified as task constraints, such as movement accuracy, can be explicitly imposed, rather than being approximated using penalty terms in the cost function. Our approximate stochastic optimal control framework predicts complex features, not captured by previous simulation approaches, providing a generalizable and valuable tool to study how musculoskeletal dynamics and physiological noise may alter neural control of movement in both healthy and pathological movements.

Random Practice Enhances Retention and Spatial Transfer in Force Field Adaptation

The contextual-interference effect is a frequently examined phenomenon in motor skill learning but has not been extensively investigated in motor adaptation. Here, we first tested experimentally if the contextual-interference effect is detectable in force field adaptation regarding retention and spatial transfer, and then fitted state-space models to the data to relate the findings to the “forgetting-and-reconstruction hypothesis”. Thirty-two participants were divided into two groups with either a random or a blocked practice schedule. They practiced reaching to four targets and were tested 10 min and 24 h afterward for motor retention and spatial transfer on an interpolation and an extrapolation target, and on targets which were shifted 10 cm away. The adaptation progress was participant-specifically fitted with 4-slow-1-fast state-space models accounting for generalization and set breaks. The blocked group adapted faster (p = 0.007) but did not reach a better adaptation at practice end. We found better retention (10 min), interpolation transfer (10 min), and transfer to shifted targets (10 min and 24 h) for the random group (each p < 0.05). However, no differences were found for retention or for the interpolation target after 24 h. Neither group showed transfer to the extrapolation target. The extended state-space model could replicate the behavioral results with some exceptions. The study shows that the contextual-interference effect is partially detectable in practice, short-term retention, and spatial transfer in force field adaptation; and that state-space models provide explanatory descriptions for the contextual-interference effect in force field adaptation.

Task space exploration improves adaptation after incompatible virtual surgeries

Humans have a remarkable capacity to learn new motor skills, a process that requires novel muscle activity patterns. Muscle synergies may simplify the generation of muscle patterns through the selection of a small number of synergy combinations. Learning new motor skills may then be achieved by acquiring novel muscle synergies. In a previous study, we used myoelectric control to construct virtual surgeries that altered the mapping from muscle activity to cursor movements. After compatible virtual surgeries, which could be compensated by recombining subject-specific muscle synergies, participants adapted quickly. In contrast, after incompatible virtual surgeries, which could not be compensated by recombining existing synergies, participants explored new muscle patterns, but failed to adapt. Here, we tested whether task space exploration can promote learning of novel muscle synergies, required to overcome an incompatible surgery. Participants performed the same reaching task as in our previous study, but with more time to complete each trial, thus allowing for exploration. We found an improvement in trial success after incompatible virtual surgeries. Remarkably, improvements in movement direction accuracy after incompatible surgeries occurred faster for corrective movements than for the initial movement, suggesting that learning of new synergies is more effective when used for feedback control. Moreover, reaction time was significantly higher after incompatible than after compatible virtual surgeries, suggesting an increased use of an explicit adaptive strategy to overcome incompatible surgeries. Taken together, these results indicate that exploration is important for skill learning and suggest that human participants, with sufficient time, can learn new muscle synergies.

Reward-Dependent Selection of Feedback Gains Impacts Rapid Motor Decisions

Target reward influences motor planning strategies through modulation of movement vigor. Considering current theories of sensorimotor control suggesting that movement planning consists in selecting a goal-directed control strategy, we sought to investigate the influence of reward on feedback control. Here, we explored this question in three human reaching experiments. First, we altered the explicit reward associated with the goal target and found an overall increase in feedback gains for higher target rewards, highlighted by larger velocities, feedback responses to external loads, and background muscle activity. Then, we investigated whether the differences in target rewards across multiple goals impacted rapid motor decisions during movement. We observed idiosyncratic switching strategies dependent on both target rewards and, surprisingly, the feedback gains at perturbation onset: the more vigorous movements were less likely to switch to a new goal following perturbations. To gain further insight into a causal influence of the feedback gains on rapid motor decisions, we demonstrated that biasing the baseline activity and reflex gains by means of a background load evoked a larger proportion of target switches in the direction opposite to the background load associated with lower muscle activity. Together, our results demonstrate an impact of target reward on feedback control and highlight the competition between movement vigor and flexibility.

Adaptive Feedback Control in Human Reaching Adaptation to Force Fields

Sensorimotor adaptation is a central function of the nervous system, as it allows humans and other animals to flexibly anticipate their interaction with the environment. In the context of human reaching adaptation to force fields, studies have traditionally separated feedforward (FF) and feedback (FB) processes involved in the improvement of behavior. Here, we review computational models of FF adaptation to force fields and discuss them in light of recent evidence highlighting a clear involvement of feedback control. Instead of a model in which FF and FB mechanisms adapt in parallel, we discuss how online adaptation in the feedback control system can explain both trial-by-trial adaptation and improvements in online motor corrections. Importantly, this computational model combines sensorimotor control and short-term adaptation in a single framework, offering novel perspectives for our understanding of human reaching adaptation and control.

Integration of proprioceptive and visual feedback during online control of reaching

Visual and proprioceptive feedback both contribute to perceptual decisions, but it remains unknown how these feedback signals are integrated together or consider factors such as delays and variance during online control. We investigated this question by having participants reach to a target with randomly applied mechanical and/or visual disturbances. We observed that the presence of visual feedback during a mechanical disturbance did not increase the size of the muscle response significantly but did decrease variance, consistent with a dynamic Bayesian integration model. In a control experiment, we verified that vision had a potent influence when mechanical and visual disturbances were both present but opposite in sign. These results highlight a complex process for multisensory integration, where visual feedback has a relatively modest influence when the limb is mechanically disturbed, but a substantial influence when visual feedback becomes misaligned with the limb.

NEW & NOTEWORTHY Visual feedback is more accurate, but proprioceptive feedback is faster. How should you integrate these sources of feedback to guide limb movement? As predicted by dynamic Bayesian models, the size of the muscle response to a mechanical disturbance was essentially the same whether visual feedback was present or not. Only under artificial conditions, such as when shifting the position of a cursor representing hand position, can one observe a muscle response from visual feedback.

Savings in Human Force Field Learning Supported by Feedback Adaptation

Savings have been described as the ability of healthy humans to relearn a previously acquired motor skill faster than the first time, which in the context of motor adaptation suggests that the learning rate in the brain could be adjusted when a perturbation is recognized. Alternatively, it has been argued that apparent savings were the consequence of a distinct process that instead of reflecting a change in the learning rate, revealed an explicit re-aiming strategy. Based on recent evidence that feedback adaptation may be central to both planning and control, we hypothesized that this component could genuinely accelerate relearning in human adaptation to force fields (FFs) during reaching. Consistent with our hypothesis, we observed that on re-exposure to a previously learned FF, the very first movement performed by healthy volunteers in the relearning context was better adapted to the external disturbance, and this occurred without any anticipation or cognitive strategy because the relearning session was started unexpectedly. We conclude that feedback adaptation is a medium by which the nervous system can genuinely accelerate learning across movements.

Rapid Feedback Responses Parallel the Urgency of Voluntary Reaching Movements

Optimal feedback control is a prominent theory used to interpret human motor behaviour. The theory posits that skilled actions emerge from control policies that link voluntary motor control (feedforward) with flexible feedback corrections (feedback control). It is clear the nervous system can generate flexible motor corrections (reflexes) when performing actions with different goals. We know little, however, about shared features of voluntary actions and feedback control in human movement. Here we reveal a link between the timing demands of voluntary actions and flexible responses to mechanical perturbations. In two experiments, 40 human participants (21 females) made reaching movements with different timing demands. We disturbed the arm with mechanical perturbations at movement onset (Experiment 1) and at locations ranging from movement onset to completion (Experiment 2). We used the resulting muscle responses and limb displacements as a proxy for the control policies that support voluntary reaching movements. We observed an increase in the sensitivity of elbow and shoulder muscle responses and a reduction in limb motion when the task imposed greater urgency to respond to the same perturbations. The results reveal a relationship between voluntary actions and feedback control as the limb was displaced less when moving faster in perturbation trials. Muscle responses scaled with changes in the displacement of the limb in perturbation trials within each timing condition. Across both experiments, human behaviour was captured by simulations based on stochastic optimal feedback control. Taken together, the results highlight flexible control that links sensory processing with features of human reaching movements.

Online modification of goal-directed control in human reaching movements

Humans are able to perform very sophisticated reaching movements in a myriad of contexts based on flexible control strategies influenced by the task goal and environmental constraints such as obstacles. However, it remains unknown whether these control strategies can be adjusted online. The objective of this study was to determine whether the factors that determine control strategies during planning also modify the execution of an ongoing movement following sudden changes in task demand. More precisely, we investigated whether, and at which latency, feedback responses to perturbation loads followed the change in the structure of the goal target or environment. We changed the target width (square or rectangle) to alter the task redundancy, or the presence of obstacles to induce different constraints on the reach path, and assessed based on surface electromyography (EMG) recordings when the change in visual display altered the feedback response to mechanical perturbations. Task-related EMG responses were detected within 150 ms of a change in target shape. Considering visuomotor delays of ∼ 100 ms, these results suggest that it takes 50 ms to change control policy within a trial. An additional 30-ms delay was observed when the change in context involved sudden appearance or disappearance of obstacles. Overall, our results demonstrate that the control policy within a reaching movement is not static: contextual factors that influence movement planning also influence movement execution at surprisingly short latencies. Moreover, the additional 30 ms associated with obstacles suggests that these two types of changes may be mediated via distinct processes.NEW & NOTEWORTHY The present work demonstrates that the control strategies used to perform reaching movements are adjusted online when the structure of the target or the presence of obstacles are altered during movements. Thus, the properties of goal-directed reaching control are not simply selected during the planning stage of a movement prior to execution. Rather, they are dynamically and rapidly adjusted online, within ∼150 ms, according to changes in environment.

Transient deactivation of dorsal premotor cortex or parietal area 5 impairs feedback control of the limb in macaques

We can generate goal-directed motor corrections with surprising speed, but their neural basis is poorly understood. Here, we show that temporary cooling of dorsal premotor cortex (PMd) impaired both spatial accuracy and the speed of corrective responses, whereas cooling parietal area 5 (A5) impaired only spatial accuracy. Simulations based on optimal feedback control (OFC) models demonstrated that "deactivation" of the control policy (reduction in feedback gain) and state estimation (reduction in Kalman gain) caused impairments similar to that observed for PMd and A5 cooling, respectively. Furthermore, combined deactivation of both cortical regions led to additive impairments of individual deactivations, whereas reducing the amount of cooling to PMd led to impairments in response speed but not spatial accuracy, both also predicted by OFC models. These results provide causal support that frontoparietal circuits beyond primary somatosensory and motor cortices are involved in generating goal-directed motor corrections.

Somatosensory deafferentation reveals lateralized roles of proprioception in feedback and adaptive feedforward control of movement and posture

Proprioception provides crucial information necessary for determining limb position and movement, and plausibly also for updating internal models that might underlie the control of movement and posture. Seminal studies of upper-limb movements in individuals living with chronic, large fiber deafferentation have provided evidence for the role of proprioceptive information in the hypothetical formation and maintenance of internal models to produce accurate motor commands. Vision also contributes to sensorimotor functions but cannot fully compensate for proprioceptive deficits. More recent work has shown that posture and movement control processes are lateralized in the brain, and that proprioception plays a fundamental role in coordinating the contributions of these processes to the control of goal-directed actions. In fact, the behavior of each limb in a deafferented individual resembles the action of a controller in isolation. Proprioception, thus, provides state estimates necessary for the nervous system to efficiently coordinate multiple motor control processes.

Rapid Changes in Movement Representations during Human Reaching Could Be Preserved in Memory for at Least 850 ms

Humans adapt to mechanical perturbations such as forcefields (FFs) during reaching within tens of trials. However, recent findings suggested that this adaptation may start within one single trial, i.e., online corrective movements can become tuned to the unanticipated perturbations within a trial. This was highlighted in previous works with a reaching experiment in which participants had to stop at a via-point (VP) located between the start and the goal. An FF was applied during the first and second parts of the movement and then occasionally unexpectedly switched off at the VP during catch trials. The results showed an after-effect during the second part of the movement when participants exited the VP. This behavioral result was interpreted as a standard after-effect, but it remained unclear how it was related to conventional trial-by-trial learning. The current study aimed to investigate how long do such changes in movement representations last in memory. For this, we have studied the same reaching task with VP in two situations: one with very short residing time in the VP and the second with an imposed minimum 500 ms dwell time in the VP. In both situations, during the unexpected absence of the FF after VP, after-effects were observed. This suggests that online corrections to the internal representation of reach dynamics can be preserved in memory for around 850 ms of resting time on average. Therefore, rapid changes occurring within movements can thus be preserved in memory long enough to influence trial-by-trial motor adaptation.

Movement Kinematics and Interjoint Coordination Are Influenced by Target Location and Arm in 6-Year-Old Children

Rapid aiming movements are typically used to study upper limb motor control and development. Despite the large corpus of work in this area, few studies have examined kinematic manual asymmetries in children who have just started formal schooling and until now, none have characterized how children coordinate their joints to complete these movements (i.e., interjoint coordination). In the present study, manual asymmetries in kinematics and interjoint coordination in strongly right-handed 6-year-old children were investigated when reaching for ipsilateral and contralateral targets with their dominant right arm and the non-dominant left arm. Overall, manual asymmetries in interjoint coordination are apparent for both 6-year-old children and young adults, although young children completed the task by adopting a different strategy than adults. Also, control strategies employed by 6-year-old children were influenced by both the location of the target as well as the arm used to perform the task. Specifically, compared to all other conditions, children’s trajectories were more curved when performing contralateral movements with the non-dominant left arm, which were driven by smaller shoulder excursions combined with larger elbow excursions for this condition. Based on these results, we argue that the differences in interjoint coordination reflect the stage of development of 6-year-old children, the origin of which derives from maturational (e.g., hand dominance) and environmental factors (e.g., school-based experience).

Double-Step Paradigm in Microgravity: Preservation of Sensorimotor Flexibility in Altered Gravitational Force Field

The way we can correct our ongoing movements to sudden and unforeseen perturbations is key to our ability to rapidly adjust our behavior to novel environmental demands. Referred to as sensorimotor flexibility, this ability can be assessed by the double-step paradigm in which participants must correct their ongoing arm movements to reach targets that unexpectedly change location (i.e., target jump). While this type of corrections has been demonstrated in normogravity in the extent of reasonable spatiotemporal constraints underpinning the target jumps, less is known about sensorimotor flexibility in altered gravitational force fields. We thus aimed to assess sensorimotor flexibility by comparing online arm pointing corrections observed during microgravity episodes of parabolic flights with normogravity standards. Seven participants were asked to point as fast and as accurately as possible toward one of two visual targets with their right index finger. The targets were aligned vertically in the mid-sagittal plane and were separated by 10 cm. In 20% of the trials, the initially illuminated lower target was switched off at movement onset while the upper target was concomitantly switched on prompting participants to change the trajectory of their ongoing movements. Results showed that, both in normogravity and microgravity, participants successfully performed the pointing task including when the target jumped unexpectedly (i.e., comparable success rate). Most importantly, no significant difference was found in target jump trials regarding arm kinematics between both gravitational environments, neither in terms of peak velocity, relative deceleration duration, peak acceleration or time to peak acceleration. Using inverse dynamics based on experimental and anthropometrical data, we demonstrated that the shoulder torques for accelerating and decelerating the vertical arm movements substantially differed between microgravity and normogravity. Our data therefore highlight the capacity of the central nervous system to perform very fast neuromuscular adjustments that are adapted to the gravitational constraints. We discuss our findings by considering the contribution of feedforward and feedback mechanisms in the online control of arm pointing movements.

Feedback Adaptation to Unpredictable Force Fields in 250 ms

Motor learning and adaptation are important functions of the nervous system. Classical studies have characterized how humans adapt to changes in the environment during tasks such as reaching, and have documented improvements in behavior across movements. However, little is known about how quickly the nervous system adapts to such disturbances. In particular, recent work has suggested that adaptation could be sufficiently fast to alter the control strategies of an ongoing movement. To further address the possibility that learning occurred within a single movement, we designed a series of human reaching experiments to extract from muscles recordings the latency of feedback adaptation. Our results confirmed that participants adapted their feedback responses to unanticipated force fields applied randomly. In addition, our analyses revealed that the feedback response was specifically and finely tuned to the ongoing perturbation not only across trials with the same force field, but also across different kinds of force fields. Finally, changes in muscle activity consistent with feedback adaptation occurred in ∼250 ms following reach onset. The adaptation that we observed across trials presented in a random context was similar to the one observed when the force fields could be anticipated, suggesting that these two adaptive processes may be closely linked to each other. In such case, our measurement of 250 ms may correspond to the latency of motor adaptation in the nervous system.

Reaching around obstacles accounts for uncertainty in coordinate transformations

When reaching to a visual target, humans need to transform the spatial target representation into the coordinate system of their moving arm. It has been shown that increased uncertainty in such coordinate transformations, for instance, when the head is rolled toward one shoulder, leads to higher movement variability and influence movement decisions. However, it is unknown whether the brain incorporates such added variability in planning and executing movements. We designed an obstacle avoidance task in which participants had to reach with or without visual feedback of the hand to a visual target while avoiding collisions with an obstacle. We varied coordinate transformation uncertainty by varying head roll (straight, 30° clockwise, and 30° counterclockwise). In agreement with previous studies, we observed that the reaching variability increased when the head was tilted. Indeed, head roll did not influence the number of collisions during reaching compared with the head-straight condition, but it did systematically change the obstacle avoidance behavior. Participants changed the preferred direction of passing the obstacle and increased the safety margins indicated by stronger movement curvature. These results suggest that the brain takes the added movement variability during head roll into account and compensates for it by adjusting the reaching trajectories.NEW & NOTEWORTHY We show that changing body geometry such as head roll results in compensatory reaching behaviors around obstacles. Specifically, we observed head roll causes changed preferred movement direction and increased trajectory curvature. As has been shown before, head roll increases movement variability due to stochastic coordinate transformations. Thus these results provide evidence that the brain must consider the added movement variability caused by coordinate transformations for accurate reach movements.

Robustness in Human Manipulation of Dynamically Complex Objects through Control Contraction Metrics

Control and manipulation of objects with underactuated dynamics remains a challenge for robots. Due to their typically nonlinear dynamics, it is computationally taxing to implement model-based planning and control techniques. Yet humans can skillfully manipulate such objects, seemingly with ease. More insight into human control strategies may inform how to enhance control strategies in robots. This study examined human control of objects that exhibit complex - underactuated and nonlinear - dynamics. We hypothesized that humans seek to make their trajectories exponentially stable to achieve robustness in the face of external perturbations. A stable trajectory is also robust to the high levels of noise in the human neuromotor system. Motivated by the task of carrying a cup of coffee, a virtual implementation of transporting a cart-pendulum system was developed. Subjects interacted with the virtual system via a robotic manipulandum that provided a haptic and visual interface. Human subjects were instructed to transport this simplified system to a target position as fast as possible without 'spilling coffee', while accommodating different visible perturbations that could be anticipated. To test the hypothesis of exponential convergence, tools from the framework of control contraction metrics were leveraged to analyze human trajectories. Results showed that with practice the trajectories indeed became exponentially stable, selectively around the perturbation. While these findings are agnostic about the involvement of feedback and feedforward control, they do support the hypothesis that humans learn to make trajectories stable, consistent with achieving predictability.

Investigating the Muscular and Kinematic Responses to Sudden Wrist Perturbations During a Dynamic Tracking Task

Sudden disturbances (perturbations) to the hand and wrist are commonplace in daily activities and workplaces when interacting with tools and the environment. It is important to understand how perturbations influence forearm musculature and task performance when identifying injury mechanisms. The purpose of this work was to evaluate changes in forearm muscle activity and co-contraction caused by wrist perturbations during a dynamic wrist tracking task. Surface electromyography was recorded from eight muscles of the upper-limb. Participants performed trials consisting of 17 repetitions of ±40° of wrist flexion/extension using a robotic device. During trials, participants received radial or ulnar perturbations that were delivered during flexion or extension, and with known or unknown timing. Co-contraction ratios for all muscle pairs showed significantly greater extensor activity across all experimental conditions. Of all antagonistic muscle pairs, the flexor carpi radialis (FCR)-extensor carpi radialis (ECR) muscle pair had the greatest change in co-contraction, producing 1602% greater co-contraction during flexion trials than during extensions trials. Expected perturbations produced greater anticipatory (immediately prior to the perturbation) muscle activity than unexpected, resulting in a 30% decrease in wrist displacement. While improving performance, this increase in anticipatory muscle activity may leave muscles susceptible to early-onset fatigue, which could lead to chronic overuse injuries in the workplace.

A Very Fast Time Scale of Human Motor Adaptation: Within Movement Adjustments of Internal Representations during Reaching

Humans and other animals adapt motor commands to predictable disturbances within tens of trials in laboratory conditions. A central question is how does the nervous system adapt to disturbances in natural conditions when exactly the same movements cannot be practiced several times. Because motor commands and sensory feedback together carry continuous information about limb dynamics, we hypothesized that the nervous system could adapt to unexpected disturbances online.

Humans and other animals adapt motor commands to predictable disturbances within tens of trials in laboratory conditions. A central question is how does the nervous system adapt to disturbances in natural conditions when exactly the same movements cannot be practiced several times. Because motor commands and sensory feedback together carry continuous information about limb dynamics, we hypothesized that the nervous system could adapt to unexpected disturbances online. We tested this hypothesis in two reaching experiments during which velocity-dependent force fields (FFs) were randomly applied. We found that within-movement feedback corrections gradually improved, despite the fact that the perturbations were unexpected. Moreover, when participants were instructed to stop at a via-point, the application of a FF prior to the via-point induced mirror-image after-effects after the via-point, consistent with within-trial adaptation to the unexpected dynamics. These findings suggest a fast time-scale of motor learning, which complements feedback control and supports adaptation of an ongoing movement.

Model-Free Robust Optimal Feedback Mechanisms of Biological Motor Control

Sensorimotor tasks that humans perform are often affected by different sources of uncertainty. Nevertheless, the central nervous system (CNS) can gracefully coordinate our movements. Most learning frameworks rely on the internal model principle, which requires a precise internal representation in the CNS to predict the outcomes of our motor commands. However, learning a perfect internal model in a complex environment over a short period of time is a nontrivial problem. Indeed, achieving proficient motor skills may require years of training for some difficult tasks. Internal models alone may not be adequate to explain the motor adaptation behavior during the early phase of learning. Recent studies investigating the active regulation of motor variability, the presence of suboptimal inference, and model-free learning have challenged some of the traditional viewpoints on the sensorimotor learning mechanism. As a result, it may be necessary to develop a computational framework that can account for these new phenomena. Here, we develop a novel theory of motor learning, based on model-free adaptive optimal control, which can bypass some of the difficulties in existing theories. This new theory is based on our recently developed adaptive dynamic programming (ADP) and robust ADP (RADP) methods and is especially useful for accounting for motor learning behavior when an internal model is inaccurate or unavailable. Our preliminary computational results are in line with experimental observations reported in the literature and can account for some phenomena that are inexplicable using existing models.