Properties of synergies arising from a theory of optimal motor behavior

Motor synergy generalization framework for new targets in multi-planar and multi-directional reaching task

Humans can rapidly adapt to new situations, even though they have redundant degrees of freedom (d.f.). Previous studies in neuroscience revealed that human movements could be accounted for by low-dimensional control signals, known as motor synergies. Many studies have suggested that humans use the same repertories of motor synergies among similar tasks. However, it has not yet been confirmed whether the combinations of motor synergy repertories can be re-used for new targets in a systematic way. Here we show that the combination of motor synergies can be generalized to new targets that each repertory cannot handle. We use the multi-directional reaching task as an example. We first trained multiple policies with limited ranges of targets by reinforcement learning and extracted sets of motor synergies. Finally, we optimized the activation patterns of sets of motor synergies and demonstrated that combined motor synergy repertories were able to reach new targets that were not achieved with either original policies or single repertories of motor synergies. We believe this is the first study that has succeeded in motor synergy generalization for new targets in new planes, using a full 7-d.f. arm model, which is a realistic mechanical environment for general reaching tasks.

How does the CNS control arm reaching movements? Introducing a hierarchical nonlinear predictive control organization based on the idea of muscle synergies

In this study, we introduce a hierarchical and modular computational model to explain how the CNS (Central Nervous System) controls arm reaching movement (ARM) in the frontal plane and under different conditions. The proposed hierarchical organization was established at three levels: 1) motor planning, 2) command production, and 3) motor execution. Since in this work we are not discussing motion learning, no learning procedure was considered in the model. Previous models mainly assume that the motor planning level produces the desired trajectories of the joints and feeds it to the next level to be tracked. In the proposed model, the motion control is described based on a regulatory control policy, that is, the output of the motor planning level is a step function defining the initial and final desired position of the hand. For the command production level, a nonlinear predictive model was developed to explain how the time-invariant muscle synergies (MSs) are recruited. We used the same computational model to explain the arm reaching motion for a combined ARM task. The combined ARM is defined as two successive ARM such that it starts from point A and reaches to point C via point B. To develop the model, kinematic and kinetic data from six subjects were recorded and analyzed during ARM task performance. The subjects used a robotic manipulator while moving their hand in the frontal plane. The EMG data of 15 muscles were also recorded. The MSs used in the model were extracted from the recorded EMG data. The proposed model explains two aspects of the motor control system by a novel computational approach: 1) the CNS reduces the dimension of the control space using the notion of MSs and thereby, avoids immense computational loads; 2) at the level of motor planning, the CNS generates the desired position of the hand at the starting, via and the final points, and this amounts to a regulatory and non-tracking structure.

Designing Human-like Behaviors for Anthropomorphic Arm in Humanoid Robot NAO

Human-like motion of robots can improve human–robot interaction and increase the efficiency. In this paper, a novel human-like motion planning strategy is proposed to help anthropomorphic arms generate human-like movements accurately. The strategy consists of three parts: movement primitives, Bayesian network (BN), and a novel coupling neural network (CPNN). The movement primitives are used to decouple the human arm movements. The classification of arm movements improves the accuracy of human-like movements. The motion-decision algorithm based on BN is able to predict occurrence probabilities of the motions and choose appropriate mode of motion. Then, a novel CPNN is proposed to solve the inverse kinematics problems of anthropomorphic arms. The CPNN integrates different models into a single network and reflects the features of these models by changing the network structure. Through the strategy, the anthropomorphic arms can generate various human-like movements with satisfactory accuracy. Finally, the availability of the proposed strategy is verified by simulations for the general motion of humanoid NAO.

On the Relationship Between Human Motor Control Performance and Kinematic Synergies in Upper Limb Prosthetics

Current prosthesis command interfaces only allow for a single degree of freedom to be commanded at a time, making coordinated motion difficult to achieve. Thus it becomes crucial to develop methods that complement these interfaces to allow for intuitive coordinated arm motion. Kinematic synergies have been shown as an alternate method where the motion of the prosthetic device is coordinated with that of the residual limb. In this paper, the mapping between the parameters of a kinematic synergy model and a measure of task performance is established experimentally in order to test the applicability of online optimization methods for the identification of synergies. To achieve this, a cost function that captures the objective of the reaching task and a linear kinematic synergy model were chosen. A human experiment was developed in a Virtual Reality (VR) platform in order to determine the synergy-performance relationship. The experiments were performed on 10 able-bodied subjects. The relationship observed between the synergy parameter and the reaching task cost function suggests existing online optimization methods may be applicable.

Modularity speeds up motor learning by overcoming mechanical bias in musculoskeletal geometry

We can easily learn and perform a variety of movements that fundamentally require complex neuromuscular control. Many empirical findings have demonstrated that a wide range of complex muscle activation patterns could be well captured by the combination of a few functional modules, the so-called muscle synergies. Modularity represented by muscle synergies would simplify the control of a redundant neuromuscular system. However, how the reduction of neuromuscular redundancy through a modular controller contributes to sensorimotor learning remains unclear. To clarify such roles, we constructed a simple neural network model of the motor control system that included three intermediate layers representing neurons in the primary motor cortex, spinal interneurons organized into modules and motoneurons controlling upper-arm muscles. After a model learning period to generate the desired shoulder and/or elbow joint torques, we compared the adaptation to a novel rotational perturbation between modular and non-modular models. A series of simulations demonstrated that the modules reduced the effect of the bias in the distribution of muscle pulling directions, as well as in the distribution of torques associated with individual cortical neurons, which led to a more rapid adaptation to multi-directional force generation. These results suggest that modularity is crucial not only for reducing musculoskeletal redundancy but also for overcoming mechanical bias due to the musculoskeletal geometry allowing for faster adaptation to certain external environments.

Synergies in coordination: a comprehensive overview of neural, computational, and behavioral approaches

At face value, the term "synergy" provides a unifying concept within a fractured field that encompasses complementary neural, computational, and behavioral approaches. However, the term is not used synonymously by different researchers but has substantially different meanings depending on the research approach. With so many operational definitions for the one term, it becomes difficult to use as either a descriptive or explanatory concept, yet it remains pervasive and apparently indispensable. Here we provide a summary of different approaches that invoke synergies in a descriptive or explanatory context, summarizing progress, not within the one approach, but across the theoretical landscape. Bernstein's framework of flexible hierarchical control may provide a unifying framework here, since it can incorporate divergent ideas about synergies. In the current motor control literature, synergy may refer to conceptually different processes that could potentially operate in parallel, across different levels within the same hierarchical control scheme. There is evidence for the concurrent existence of synergies with different features, both "hard-wired" and "soft-wired," and task independent and task dependent. By providing a comprehensive overview of the multifaceted ideas about synergies, our goal is to move away from the compartmentalization and narrow the focus on one level and promote a broader perspective on the control and coordination of movement.

Deciphering the functional role of spatial and temporal muscle synergies in whole-body movements

Voluntary movement is hypothesized to rely on a limited number of muscle synergies, the recruitment of which translates task goals into effective muscle activity. In this study, we investigated how to analytically characterize the functional role of different types of muscle synergies in task performance. To this end, we recorded a comprehensive dataset of muscle activity during a variety of whole-body pointing movements. We decomposed the electromyographic (EMG) signals using a space-by-time modularity model which encompasses the main types of synergies. We then used a task decoding and information theoretic analysis to probe the role of each synergy by mapping it to specific task features. We found that the temporal and spatial aspects of the movements were encoded by different temporal and spatial muscle synergies, respectively, consistent with the intuition that there should a correspondence between major attributes of movement and major features of synergies. This approach led to the development of a novel computational method for comparing muscle synergies from different participants according to their functional role. This functional similarity analysis yielded a small set of temporal and spatial synergies that describes the main features of whole-body reaching movements.

Space-by-Time Modular Decomposition Effectively Describes Whole-Body Muscle Activity During Upright Reaching in Various Directions

The modular control hypothesis suggests that motor commands are built from precoded modules whose specific combined recruitment can allow the performance of virtually any motor task. Despite considerable experimental support, this hypothesis remains tentative as classical findings of reduced dimensionality in muscle activity may also result from other constraints (biomechanical couplings, data averaging or low dimensionality of motor tasks). Here we assessed the effectiveness of modularity in describing muscle activity in a comprehensive experiment comprising 72 distinct point-to-point whole-body movements during which the activity of 30 muscles was recorded. To identify invariant modules of a temporal and spatial nature, we used a space-by-time decomposition of muscle activity that has been shown to encompass classical modularity models. To examine the decompositions, we focused not only on the amount of variance they explained but also on whether the task performed on each trial could be decoded from the single-trial activations of modules. For the sake of comparison, we confronted these scores to the scores obtained from alternative non-modular descriptions of the muscle data. We found that the space-by-time decomposition was effective in terms of data approximation and task discrimination at comparable reduction of dimensionality. These findings show that few spatial and temporal modules give a compact yet approximate representation of muscle patterns carrying nearly all task-relevant information for a variety of whole-body reaching movements.

Gravitational and Dynamic Components of Muscle Torque Underlie Tonic and Phasic Muscle Activity during Goal-Directed Reaching

Human reaching movements require complex muscle activations to produce the forces necessary to move the limb in a controlled manner. How gravity and the complex kinetic properties of the limb contribute to the generation of the muscle activation pattern by the central nervous system (CNS) is a long-standing and controversial question in neuroscience. To tackle this issue, muscle activity is often subdivided into static and phasic components. The former corresponds to posture maintenance and transitions between postures. The latter corresponds to active movement production and the compensation for the kinetic properties of the limb. In the present study, we improved the methodology for this subdivision of muscle activity into static and phasic components by relating them to joint torques. Ten healthy subjects pointed in virtual reality to visual targets arranged to create a standard center-out reaching task in three dimensions. Muscle activity and motion capture data were synchronously collected during the movements. The motion capture data were used to calculate postural and dynamic components of active muscle torques using a dynamic model of the arm with 5 degrees of freedom. Principal Component Analysis (PCA) was then applied to muscle activity and the torque components, separately, to reduce the dimensionality of the data. Muscle activity was also reconstructed from gravitational and dynamic torque components. Results show that the postural and dynamic components of muscle torque represent a significant amount of variance in muscle activity. This method could be used to define static and phasic components of muscle activity using muscle torques.

Muscle recruitment and coordination with an ankle exoskeleton

Exoskeletons have the potential to assist and augment human performance. Understanding how users adapt their movement and neuromuscular control in response to external assistance is important to inform the design of these devices. The aim of this research was to evaluate changes in muscle recruitment and coordination for ten unimpaired individuals walking with an ankle exoskeleton. We evaluated changes in the activity of individual muscles, cocontraction levels, and synergistic patterns of muscle coordination with increasing exoskeleton work and torque. Participants were able to selectively reduce activity of the ankle plantarflexors with increasing exoskeleton assistance. Increasing exoskeleton net work resulted in greater reductions in muscle activity than increasing exoskeleton torque. Patterns of muscle coordination were not restricted or constrained to synergistic patterns observed during unassisted walking. While three synergies could describe nearly 95% of the variance in electromyography data during unassisted walking, these same synergies could describe only 85-90% of the variance in muscle activity while walking with the exoskeleton. Synergies calculated with the exoskeleton demonstrated greater changes in synergy weights with increasing exoskeleton work versus greater changes in synergy activations with increasing exoskeleton torque. These results support the theory that unimpaired individuals do not exclusively use central pattern generators or other low-level building blocks to coordinate muscle activity, especially when learning a new task or adapting to external assistance, and demonstrate the potential for using exoskeletons to modulate muscle recruitment and coordination patterns for rehabilitation or performance.

Modular control during incline and level walking in humans

The neuromuscular control of human movement can be described by a set of muscle synergies factorized from myoelectric signals. There is some evidence that the selection, activation and flexible combination of these basic activation patterns are of a neural origin. We investigated the muscle synergies during incline and level walking to evaluate changes in the modular organization of neuromuscular control related to changes in the mechanical demands. Our results revealed five fundamental (not further factorizable) synergies for both walking conditions but with different frequencies of appearance of the respective synergies during incline compared with level walking. Low similarities across conditions were observed in the timing of the activation patterns (motor primitives) and the weightings of the muscles within the respective elements (motor modules) for the synergies associated with the touchdown, mid-stance and early push-off phase. The changes in neuromuscular control could be attributed to changes in the mechanical demands in support, propulsion and medio-lateral stabilization of the body during incline compared with level walking. Our findings provide further evidence that the central nervous system flexibly uses a consistent set of neural control elements with a flexible temporal recruitment and modifications of the relative muscle weightings within each element to provide stable locomotion under varying mechanical demands during walking.

The Movement- and Load-Dependent Differences in the EMG Patterns of the Human Arm Muscles during Two-Joint Movements (A Preliminary Study)

Slow circular movements of the hand with a fixed wrist joint that were produced in a horizontal plane under visual guidance during conditions of action of the elastic load directed tangentially to the movement trajectory were studied. The positional dependencies of the averaged surface EMGs in the muscles of the elbow and shoulder joints were compared for four possible combinations in the directions of load and movements. The EMG intensities were largely correlated with the waves of the force moment computed for a corresponding joint in the framework of a simple geometrical model of the system: arm - experimental setup. At the same time, in some cases the averaged EMGs exit from the segments of the trajectory restricted by the force moment singular points (FMSPs), in which the moments exhibited altered signs. The EMG activities display clear differences for the eccentric and concentric zones of contraction that are separated by the joint angle singular points (JASPs), which present extreme at the joint angle traces. We assumed that the modeled patterns of FMSPs and JASPs may be applied for an analysis of the synergic interaction between the motor commands arriving at different muscles in arbitrary two-joint movements.

How Variability and Effort Determine Coordination at Large Forces

Motor control is a challenging task for the central nervous system, since it involves redundant degrees of freedom, nonlinear dynamics of actuators and limbs, as well as noise. When an action is carried out, which factors does your nervous system consider to determine the appropriate set of muscle forces between redundant degrees-of-freedom? Important factors determining motor output likely encompass effort and the resulting motor noise. However, the tasks used in many previous motor control studies could not identify these two factors uniquely, as signal-dependent noise monotonically increases as a function of the effort. To address this, a recent paper introduced a force control paradigm involving one finger in each hand that can disambiguate these two factors. It showed that the central nervous system considers both force noise and amplitude, with a larger weight on the absolute force and lower weights on both noise and normalized force. While these results are valid for the relatively low force range considered in that paper, the magnitude of the force shared between the fingers for large forces is not known. This paper investigates this question experimentally, and develops an appropriate Markov chain Monte Carlo method in order to estimate the weightings given to these factors. Our results demonstrate that the force sharing strongly depends on the force level required, so that for higher force levels the normalized force is considered as much as the absolute force, whereas the role of noise minimization becomes negligible.

Extracting motor synergies from random movements for low-dimensional task-space control of musculoskeletal robots

In the field of human motor control, the motor synergy hypothesis explains how humans simplify body control dimensionality by coordinating groups of muscles, called motor synergies, instead of controlling muscles independently. In most applications of motor synergies to low-dimensional control in robotics, motor synergies are extracted from given optimal control signals. In this paper, we address the problems of how to extract motor synergies without optimal data given, and how to apply motor synergies to achieve low-dimensional task-space tracking control of a human-like robotic arm actuated by redundant muscles, without prior knowledge of the robot. We propose to extract motor synergies from a subset of randomly generated reaching-like movement data. The essence is to first approximate the corresponding optimal control signals, using estimations of the robot's forward dynamics, and to extract the motor synergies subsequently. In order to avoid modeling difficulties, a learning-based control approach is adopted such that control is accomplished via estimations of the robot's inverse dynamics. We present a kernel-based regression formulation to estimate the forward and the inverse dynamics, and a sliding controller in order to cope with estimation error. Numerical evaluations show that the proposed method enables extraction of motor synergies for low-dimensional task-space control.

Neuromechanical principles underlying movement modularity and their implications for rehabilitation

Neuromechanical principles define the properties and problems that shape neural solutions for movement. Although the theoretical and experimental evidence is debated, we present arguments for consistent structures in motor patterns, i.e., motor modules, that are neuromechanical solutions for movement particular to an individual and shaped by evolutionary, developmental, and learning processes. As a consequence, motor modules may be useful in assessing sensorimotor deficits specific to an individual and define targets for the rational development of novel rehabilitation therapies that enhance neural plasticity and sculpt motor recovery. We propose that motor module organization is disrupted and may be improved by therapy in spinal cord injury, stroke, and Parkinson's disease. Recent studies provide insights into the yet-unknown underlying neural mechanisms of motor modules, motor impairment, and motor learning and may lead to better understanding of the causal nature of modularity and its underlying neural substrates.

Real-Time Task Discrimination for Myoelectric Control Employing Task-Specific Muscle Synergies

We present a novel formulation that employs task-specific muscle synergies and state-space representation of neural signals to tackle the challenging myoelectric control problem for lower arm prostheses. The proposed framework incorporates information about muscle configurations, e.g., muscles acting synergistically or in agonist/antagonist pairs, using the hypothesis of muscle synergies. The synergy activation coefficients are modeled as the latent system state and are estimated using a constrained Kalman filter. These task-dependent synergy activation coefficients are estimated in real-time from the electromyogram (EMG) data and are used to discriminate between various tasks. The task discrimination is helped by a post-processing algorithm that uses posterior probabilities. The proposed algorithm is robust as well as computationally efficient, yielding a decision with > 90% discrimination accuracy in approximately 3 ms . The real-time performance and controllability of the algorithm were evaluated using the targeted achievement control (TAC) test. The proposed algorithm outperformed common machine learning algorithms for single- as well as multi-degree-of-freedom (DOF) tasks in both off-line discrimination accuracy and real-time controllability (p < 0.01).

Stroke parameters identification algorithm in handwriting movements analysis by synthesis

This paper presents a new approach to identify the stroke parameters in handwriting movement data understanding. A two-step analysis by synthesis paradigm is employed to facilitate the coarse-to-fine parameter identification for all strokes. One is the stroke data extraction, the other is the coarse-to-fine stroke parameter identification. The new consideration of using this two-step paradigm is that the nonnegative primitive factorization technique is incorporated to decouple the overlapped strokes from the measurement data. In comparison to the existing paradigms of using the heuristic stroke data decoupling techniques, our paradigm presented here contributes to alleviating the difficulty of local optimum traps with the well-shaped initializations in the global optimization for jointly identifying stroke parameters. Moreover, our paradigm excludes the iteration between two steps, which contributes to the enhancement of computational efficiency. Experimental results are reported to validate the proposed approach.

Dimensionality of joint torques and muscle patterns for reaching

Muscle activities underlying many motor behaviors can be generated by a small number of basic activation patterns with specific features shared across movement conditions. Such low-dimensionality suggests that the central nervous system (CNS) relies on a modular organization to simplify control. However, the relationship between the dimensionality of muscle patterns and that of joint torques is not fixed, because of redundancy and non-linearity in mapping the former into the latter, and needs to be investigated. We compared the torques acting at four arm joints during fast reaching movements in different directions in the frontal and sagittal planes and the underlying muscle patterns. The dimensionality of the non-gravitational components of torques and muscle patterns in the spatial, temporal, and spatiotemporal domains was estimated by multidimensional decomposition techniques. The spatial organization of torques was captured by two or three generators, indicating that not all the available coordination patterns are employed by the CNS. A single temporal generator with a biphasic profile was identified, generalizing previous observations on a single plane. The number of spatiotemporal generators was equal to the product of the spatial and temporal dimensionalities and their organization was essentially synchronous. Muscle pattern dimensionalities were higher than torques dimensionalities but also higher than the minimum imposed by the inherent non-negativity of muscle activations. The spatiotemporal dimensionality of the muscle patterns was lower than the product of their spatial and temporal dimensionality, indicating the existence of specific asynchronous coordination patterns. Thus, the larger dimensionalities of the muscle patterns may be required for CNS to overcome the non-linearities of the musculoskeletal system and to flexibly generate endpoint trajectories with simple kinematic features using a limited number of building blocks.

Can modular strategies simplify neural control of multidirectional human locomotion?

Each human lower limb contains over 50 muscles that are coordinated during locomotion. It has been hypothesized that the nervous system simplifies muscle control through modularity, using neural patterns to activate muscles in groups called synergies. Here we investigate how simple modular controllers based on invariant neural primitives (synergies or patterns) might generate muscle activity observed during multidirectional locomotion. We extracted neural primitives from unilateral electromyographic recordings of 25 lower limb muscles during five locomotor tasks: walking forward, backward, leftward and rightward, and stepping in place. A subset of subjects also performed five variations of forward (unidirectional) walking: self-selected cadence, fast cadence, slow cadence, tiptoe, and uphill (20% incline). We assessed the results in the context of dimensionality reduction, defined here as the number of neural signals needing to be controlled. For an individual task, we found that modular architectures could theoretically reduce dimensionality compared with independent muscle control, but we also found that modular strategies relying on neural primitives shared across different tasks were limited in their ability to account for muscle activations during multi- and unidirectional locomotion. The utility of shared primitives may thus depend on whether they can be adapted for specific task demands, for instance, by means of sensory feedback or by being embedded within a more complex sensorimotor controller. Our findings indicate the need for more sophisticated formulations of modular control or alternative motor control hypotheses in order to understand muscle coordination during locomotion.

A Simplified Spinal-Like Controller Facilitates Muscle Synergies and Robust Reaching Motions

We develop an adaptive controller for multi-joint, multi-muscle arm movements based on simplified spinal-like circuits found in the periphery, muscle synergies, and interpretations of gain-field projections from reach related neurons in the Superior Colliculus. The resulting innovation provides a highly robust sensory based controller that can be adapted to systems which require multi-muscle co-ordination. It provides human-like responses during perturbations elicited either internally or by the environment and for simple point-to-point reaching. We simulate limb motion and EMGs in Simulink using Virtual Muscle models and a variety of paradigms, including motion with external perturbations, and varying levels of antagonist muscle co-contractions. The results show that the system can exhibit smooth coordinated motions, without explicit kinematic or dynamic planning even in the presence of perturbations. In addition, we show by varying the level of muscle co-contractions from 0% to 40%, that the effects of external perturbations on joint trajectories can be reduced by up to 42%. The improved controller design is novel providing robust behavior during dynamic events and an automatic adaptive response from sensory-integration.

Motor primitives and synergies in the spinal cord and after injury--the current state of play

Modular pattern generator elements, also known as burst synergies or motor primitives, have become a useful and important way of describing motor behavior, albeit controversial. It is suggested that these synergy elements may constitute part of the pattern-shaping layers of a McCrea/Rybak two-layer pattern generator, as well as being used in other ways in the spinal cord. The data supporting modular synergies range across species including humans and encompass motor pattern analyses and neural recordings. Recently, synergy persistence and changes following clinical trauma have been presented. These new data underscore the importance of understanding the modular structure of motor behaviors and the underlying circuitry to best provide principled therapies and to understand phenomena reported in the clinic. We discuss the evidence and different viewpoints on modularity, the neural underpinnings identified thus far, and possible critical issues for the future of this area.

Are muscle synergies useful for neural control?

The observation that the activity of multiple muscles can be well approximated by a few linear synergies is viewed by some as a sign that such low-dimensional modules constitute a key component of the neural control system. Here, we argue that the usefulness of muscle synergies as a control principle should be evaluated in terms of errors produced not only in muscle space, but also in task space. We used data from a force-aiming task in two dimensions at the wrist, using an electromyograms (EMG)-driven virtual biomechanics technique that overcomes typical errors in predicting force from recorded EMG, to illustrate through simulation how synergy decomposition inevitably introduces substantial task space errors. Then, we computed the optimal pattern of muscle activation that minimizes summed-squared muscle activities, and demonstrated that synergy decomposition produced similar results on real and simulated data. We further assessed the influence of synergy decomposition on aiming errors (AEs) in a more redundant system, using the optimal muscle pattern computed for the elbow-joint complex (i.e., 13 muscles acting in two dimensions). Because EMG records are typically not available from all contributing muscles, we also explored reconstructions from incomplete sets of muscles. The redundancy of a given set of muscles had opposite effects on the goodness of muscle reconstruction and on task achievement; higher redundancy is associated with better EMG approximation (lower residuals), but with higher AEs. Finally, we showed that the number of synergies required to approximate the optimal muscle pattern for an arbitrary biomechanical system increases with task-space dimensionality, which indicates that the capacity of synergy decomposition to explain behavior depends critically on the scope of the original database. These results have implications regarding the viability of muscle synergy as a putative neural control mechanism, and also as a control algorithm to restore movements.

Learned graphical models for probabilistic planning provide a new class of movement primitives

Biological movement generation combines three interesting aspects: its modular organization in movement primitives (MPs), its characteristics of stochastic optimality under perturbations, and its efficiency in terms of learning. A common approach to motor skill learning is to endow the primitives with dynamical systems. Here, the parameters of the primitive indirectly define the shape of a reference trajectory. We propose an alternative MP representation based on probabilistic inference in learned graphical models with new and interesting properties that complies with salient features of biological movement control. Instead of endowing the primitives with dynamical systems, we propose to endow MPs with an intrinsic probabilistic planning system, integrating the power of stochastic optimal control (SOC) methods within a MP. The parameterization of the primitive is a graphical model that represents the dynamics and intrinsic cost function such that inference in this graphical model yields the control policy. We parameterize the intrinsic cost function using task-relevant features, such as the importance of passing through certain via-points. The system dynamics as well as intrinsic cost function parameters are learned in a reinforcement learning (RL) setting. We evaluate our approach on a complex 4-link balancing task. Our experiments show that our movement representation facilitates learning significantly and leads to better generalization to new task settings without re-learning.

Flexible cortical control of task-specific muscle synergies

Correlation structure in the activity of muscles across movements is often interpreted as evidence for low-level, hardwired constraints on upper-limb function. However, muscle synergies may also emerge from optimal strategies to achieve high-level task goals within a redundant control space. To distinguish these contrasting interpretations, we examined the structure of muscle variability during operation of a myoelectric interface in which task constraints were dissociated from natural limb biomechanics. We found that, with practice, human subjects learned to shape patterns of covariation between arbitrary pairs of hand and forearm muscles appropriately for elliptical targets whose orientation varied on a trial-by-trial basis. Thus, despite arriving at the same average location in the effector space, performance was improved by buffering variability into those dimensions that least impacted task success. Task modulation of beta-frequency intermuscular coherence indicated that differential recruitment of divergent corticospinal pathways contributed to positive correlations among muscles. However, this feedforward mechanism could not account for negative correlations observed in the presence of visual feedback. A second experiment revealed the development of fast, target-dependent visual responses consistent with "minimum intervention" control correcting predominantly task-relevant errors. Together, these mechanisms contribute to the dynamic emergence of task-specific muscle synergies appropriate for a wide range of abstract task goals.

Review and perspective: neuromechanical considerations for predicting muscle activation patterns for movement

Muscle coordination may be difficult or impossible to predict accurately based on biomechanical considerations alone because of redundancy in the musculoskeletal system. Because many solutions exist for any given movement, the role of the nervous system in further constraining muscle coordination patterns for movement must be considered in both healthy and impaired motor control. On the basis of computational neuromechanical analyses of experimental data combined with modeling techniques, we have demonstrated several such neural constraints on the temporal and spatial patterns of muscle activity during both locomotion and postural responses to balance perturbations. We hypothesize that subject-specific and trial-by-trial differences in muscle activation can be parameterized and understood by a hierarchical and low-dimensional framework that reflects the neural control of task-level goals. In postural control, we demonstrate that temporal patterns of muscle activity may be governed by feedback control of task-level variables that represent the overall goal-directed motion of the body. These temporal patterns then recruit spatially-fixed patterns of muscle activity called muscle synergies that produce the desired task-level biomechanical functions that require multijoint coordination. Moreover, these principles apply more generally to movement, and in particular to locomotor tasks in both healthy and impaired individuals. Overall, understanding the goals and organization of the neural control of movement may provide useful reduced dimension parameter sets to address the degrees-of-freedom problem in musculoskeletal movement control. More importantly, however, neuromechanical analyses may lend insight and provide a framework for understanding subject-specific and trial-by-trial differences in movement across both healthy and motor-impaired populations.

Optimization of muscle activity for task-level goals predicts complex changes in limb forces across biomechanical contexts

Optimality principles have been proposed as a general framework for understanding motor control in animals and humans largely based on their ability to predict general features movement in idealized motor tasks. However, generalizing these concepts past proof-of-principle to understand the neuromechanical transformation from task-level control to detailed execution-level muscle activity and forces during behaviorally-relevant motor tasks has proved difficult. In an unrestrained balance task in cats, we demonstrate that achieving task-level constraints center of mass forces and moments while minimizing control effort predicts detailed patterns of muscle activity and ground reaction forces in an anatomically-realistic musculoskeletal model. Whereas optimization is typically used to resolve redundancy at a single level of the motor hierarchy, we simultaneously resolved redundancy across both muscles and limbs and directly compared predictions to experimental measures across multiple perturbation directions that elicit different intra- and interlimb coordination patterns. Further, although some candidate task-level variables and cost functions generated indistinguishable predictions in a single biomechanical context, we identified a common optimization framework that could predict up to 48 experimental conditions per animal (n = 3) across both perturbation directions and different biomechanical contexts created by altering animals' postural configuration. Predictions were further improved by imposing experimentally-derived muscle synergy constraints, suggesting additional task variables or costs that may be relevant to the neural control of balance. These results suggested that reduced-dimension neural control mechanisms such as muscle synergies can achieve similar kinetics to the optimal solution, but with increased control effort (≈2×) compared to individual muscle control. Our results are consistent with the idea that hierarchical, task-level neural control mechanisms previously associated with voluntary tasks may also be used in automatic brainstem-mediated pathways for balance.

Passive motion paradigm: an alternative to optimal control

IN THE LAST YEARS, OPTIMAL CONTROL THEORY (OCT) HAS EMERGED AS THE LEADING APPROACH FOR INVESTIGATING NEURAL CONTROL OF MOVEMENT AND MOTOR COGNITION FOR TWO COMPLEMENTARY RESEARCH LINES: behavioral neuroscience and humanoid robotics. In both cases, there are general problems that need to be addressed, such as the "degrees of freedom (DoFs) problem," the common core of production, observation, reasoning, and learning of "actions." OCT, directly derived from engineering design techniques of control systems quantifies task goals as "cost functions" and uses the sophisticated formal tools of optimal control to obtain desired behavior (and predictions). We propose an alternative "softer" approach passive motion paradigm (PMP) that we believe is closer to the biomechanics and cybernetics of action. The basic idea is that actions (overt as well as covert) are the consequences of an internal simulation process that "animates" the body schema with the attractor dynamics of force fields induced by the goal and task-specific constraints. This internal simulation offers the brain a way to dynamically link motor redundancy with task-oriented constraints "at runtime," hence solving the "DoFs problem" without explicit kinematic inversion and cost function computation. We argue that the function of such computational machinery is not only restricted to shaping motor output during action execution but also to provide the self with information on the feasibility, consequence, understanding and meaning of "potential actions." In this sense, taking into account recent developments in neuroscience (motor imagery, simulation theory of covert actions, mirror neuron system) and in embodied robotics, PMP offers a novel framework for understanding motor cognition that goes beyond the engineering control paradigm provided by OCT. Therefore, the paper is at the same time a review of the PMP rationale, as a computational theory, and a perspective presentation of how to develop it for designing better cognitive architectures.

What is the biological basis of sensorimotor integration?

This Prospects presents the problems that must be solved by the vertebrate nervous system in the process of sensorimotor integration and motor control. The concepts of efference copy and inverse model are defined, and multiple biological mechanisms are described, including those that form the basis of integration, extrapolation, and comparison/cancellation operations. Open questions for future research include the biological basis of continuous and distributed versus modular control, and somatosensory-motor coordination.

A neural basis for motor primitives in the spinal cord

Motor primitives and modularity may be important in biological movement control. However, their neural basis is not understood. To investigate this, we recorded 302 neurons, making multielectrode recordings in the spinal cord gray of spinalized frogs, at 400, 800, and 1200 mum depth, at the L2/L3 segment border. Simultaneous muscle activity recordings were used with independent components analysis to infer premotor drive patterns. Neurons were divided into groups based on motor pattern modulation and sensory responses, depth recorded, and behavior. The 187 motor pattern modulated neurons recorded comprised 14 cutaneous neurons and 28 proprioceptive neurons at 400 mum in the dorsal horn, 131 intermediate zone interneurons from approximately 800 microm depth without sensory responses, and 14 motoneuron-like neurons at approximately 1200 microm. We examined all such neurons during spinal behaviors. Mutual information measures showed that cutaneous neurons and intermediate zone neurons were related better to premotor drives than to individual muscle activity. In contrast, proprioceptive-related neurons and ventral horn neurons divided evenly. For 46 of the intermediate zone interneurons, we found significant postspike facilitation effects on muscle responses using spike-triggered averages representing short-latency postspike facilitations to multiple motor pools. Furthermore, these postspike facilitations matched significantly in both their patterns and strengths with the weighting parameters of individual primitives extracted statistically, although both were initially obtained without reference to one another. Our data show that sets of dedicated interneurons may organize individual spinal primitives. These may be a key to understanding motor development, motor learning, recovery after CNS injury, and evolution of motor behaviors.

Subject-specific muscle synergies in human balance control are consistent across different biomechanical contexts

The musculoskeletal redundancy of the body provides multiple solutions for performing motor tasks. We have proposed that the nervous system solves this unconstrained problem through the recruitment of motor modules or functional muscle synergies that map motor intention to action. Consistent with this hypothesis, we showed that trial-by-trial variations in muscle activation for multidirectional balance control in humans were constrained by a small set of muscle synergies. However, apparent muscle synergy structures could arise from characteristic patterns of sensory input resulting from perturbations or from low-dimensional optimal motor solutions. Here we studied electromyographic (EMG) responses for balance control across a range of biomechanical contexts, which alter not only the sensory inflow generated by postural perturbations, but also the muscle activation patterns used to restore balance. Support-surface translations in 12 directions were delivered to subjects standing in six different postural configurations: one-leg, narrow, wide, very wide, crouched, and normal stance. Muscle synergies were extracted from each condition using nonnegative matrix factorization. In addition, muscle synergies from the normal stance condition were used to reconstruct muscle activation patterns across all stance conditions. A consistent set of muscle synergies were recruited by each subject across conditions. When balance demands were extremely different from the normal stance (e.g., one-legged or crouched stance), task-specific muscle synergies were recruited in addition to the preexisting ones, rather generating de novo muscle synergies. Taken together, our results suggest that muscle synergies represent consistent motor modules that map intention to action, regardless of the biomechanical context of the task.

The coordination of movement: optimal feedback control and beyond

Optimal control theory and its more recent extension, optimal feedback control theory, provide valuable insights into the flexible and task-dependent control of movements. Here, we focus on the problem of coordination, defined as movements that involve multiple effectors (muscles, joints or limbs). Optimal control theory makes quantitative predictions concerning the distribution of work across multiple effectors. Optimal feedback control theory further predicts variation in feedback control with changes in task demands and the correlation structure between different effectors. We highlight two crucial areas of research, hierarchical control and the problem of movement initiation, that need to be developed for an optimal feedback control theory framework to characterise movement coordination more fully and to serve as a basis for studying the neural mechanisms involved in voluntary motor control.

The case for and against muscle synergies

A long standing goal in motor control is to determine the fundamental output controlled by the CNS: does the CNS control the activation of individual motor units, individual muscles, groups of muscles, kinematic or dynamic features of movement, or does it simply care about accomplishing a task? Of course, the output controlled by the CNS might not be exclusive but instead multiple outputs might be controlled in parallel or hierarchically. In this review we examine one particular hypothesized level of control: that the CNS produces movement through the flexible combination of groups of muscles, or muscle synergies. Several recent studies have examined this hypothesis, providing evidence both in support and in opposition to it. We discuss these results and the current state of the muscle synergy hypothesis.

A feedback model explains the differential scaling of human postural responses to perturbation acceleration and velocity

Although the neural basis of balance control remains unknown, recent studies suggest that a feedback law on center-of-mass (CoM) kinematics determines the temporal patterning of muscle activity during human postural responses. We hypothesized that the same feedback law would also explain variations in muscle activity to support-surface translation as perturbation characteristics vary. Subject CoM motion was experimentally modulated using 34 different anterior-posterior support-surface translations of varying peak acceleration and velocity but the same total displacement. Electromyographic (EMG) recordings from several muscles of the lower limbs and trunk were compared to predicted EMG patterns from an inverted pendulum model under delayed feedback control. In both recorded and predicted EMG patterns, the initial burst of muscle activity scaled linearly with peak acceleration, whereas the tonic "plateau" region scaled with peak velocity. The relatively invariant duration of the initial burst was modeled by incorporating a transient, time-limited encoding of CoM acceleration inspired by muscle spindle primary afferent dynamic responses. The entire time course of recorded and predicted muscle activity compared favorably across all conditions, suggesting that the initial burst of muscle activity is not generated by feedforward neural mechanisms. Perturbation conditions were presented randomly and subjects maintained relatively constant feedback gains across all conditions. In contrast, an optimal feedback solution based on a trade-off between CoM stabilization and energy expenditure predicted that feedback gains should change with perturbation characteristics. These results suggest that an invariant feedback law was used to generate the entire time course of muscle activity across a variety of postural disturbances.

Neuromechanics of muscle synergies for posture and movement

Recent research suggests that the nervous system controls muscles by activating flexible combinations of muscle synergies to produce a wide repertoire of movements. Muscle synergies are like building blocks, defining characteristic patterns of activation across multiple muscles that may be unique to each individual, but perform similar functions. The identification of muscle synergies has strong implications for the organization and structure of the nervous system, providing a mechanism by which task-level motor intentions are translated into detailed, low-level muscle activation patterns. Understanding the complex interplay between neural circuits and biomechanics that give rise to muscle synergies will be crucial to advancing our understanding of neural control mechanisms for movement.

Optimal sensorimotor transformations for balance

Here we have identified a sensorimotor transformation that is used by a mammalian nervous system to produce a multijoint motor behavior. Using a simple biomechanical model, a delayed-feedback rule based on an optimal tradeoff between postural error and neural effort explained patterns of muscle activation in response to a sudden loss of balance in cats. Following the loss of large sensory afferents, changes in these muscle-activation patterns reflected an optimal reweighting of sensory feedback gains to minimize postural instability. Specifically, a loss of center-of-mass-acceleration information, which allowed for a rapid initial rise in the muscle activity in intact animals, was absent after large-fiber sensory neuropathy. Our results demonstrate that a simple and flexible neural feedback control strategy coordinates multiple muscles over time via a small set of extrinsic, task-level variables during complex multijoint natural movements.

Optimality, stochasticity, and variability in motor behavior

Recent theories of motor control have proposed that the nervous system acts as a stochastically optimal controller, i.e. it plans and executes motor behaviors taking into account the nature and statistics of noise. Detrimental effects of noise are converted into a principled way of controlling movements. Attractive aspects of such theories are their ability to explain not only characteristic features of single motor acts, but also statistical properties of repeated actions. Here, we present a critical analysis of stochastic optimality in motor control which reveals several difficulties with this hypothesis. We show that stochastic control may not be necessary to explain the stochastic nature of motor behavior, and we propose an alternative framework, based on the action of a deterministic controller coupled with an optimal state estimator, which relieves drawbacks of stochastic optimality and appropriately explains movement variability.