New insights into action–perception coupling

Referent Control of Side-to-Side Body-Weight Transfer During Standing and Stepping in Adults

Research suggests that locomotion may be primarily caused by shifting stable body balance from one location in the environment to another with subsequent rhythmical muscle activation by the central pattern generator (CPG), constituting a multi-level control system. All levels interact with environmental forces affected by proprioceptive and vestibular reflexes as well as vision. A similar multi-level control schema is likely used to shift body balance laterally when the body weight is rhythmically transferred from side-to-side. In order to do so, the system shifts a specific body posture in space. This body posture is referred to as the threshold or referent body posture, R, at which all muscles involved can be at rest but are activated depending on the deflection of the actual body posture, Q, from R. This concept has previously been investigated for forward and backward locomotion. The purpose of the present study was to verify if it was also applicable to locomotor tasks in other directions such as sidestepping. We predicted that during sidestepping, the actual and referent posture can transiently match each other bringing the activity of multiple muscles to a minimum. The existence of such minima was demonstrated in healthy adults performing three locomotor tasks involving shifts of the body weight from side-to-side thus further supporting the validity of the multi-level control scheme of locomotion.

The Role of Imitation, Primitives, and Spatial Referent Coordinates in Motor Control: Implications for Writing and Reading

We review a body of literature related to the drawing and recognition of geometrical two-dimensional linear drawings including letters. Handwritten letters are viewed not as two-dimensional geometrical objects but as one-dimensional trajectories of the tip of the implement. Handwritten letters are viewed as composed of a small set of kinematic primitives. Recognition of objects is mediated by processes of their creation (actual or imagined)-the imitation principle, a particular example of action-perception coupling. The concept of spatial directional field guiding the trajectories is introduced and linked to neuronal population vectors. Further, we link the kinematic description to the theory of control with spatial referent coordinates. This framework allows interpreting a number of experimental observations and clinical cases of agnosia. It also allows formulating predictions for new experimental studies of writing.

Comparing Taste Perception Across Modalities in Healthy Adults: Liquids Versus Dissolvable Taste Strips

Taste stimulation has rehabilitative value in dysphagia management, as it activates salient underlying afferent pathways to swallowing which may evoke feedforward effects on swallow biomechanics. Despite its potential beneficial effects on swallow physiology, taste stimulation's clinical application is limited for persons unsafe to orally consume food/liquid. This study aimed to create edible, dissolvable taste strips matched to flavor profiles previously used in research assessing taste's effects on swallowing physiology and brain activity, and to evaluate how similar their perceived intensity and hedonic, or palatability, ratings were between their liquid counterparts. Plain, sour, sweet-sour, lemon, and orange flavor profiles were custom-made in taste strips and liquid modalities. The generalized Labeled Magnitude Scale and hedonic generalized Labeled Magnitude Scale were used to assess intensity and palatability ratings for flavor profiles in each modality. Healthy participants were recruited and stratified across age and sex. Liquids were rated as more intense than taste strips; however, there was no difference in palatability ratings between the modalities. There were significant differences across flavor profiles in intensity and palatability ratings. Collapsed across liquid and taste strip modalities, pairwise comparisons revealed all flavored stimuli were rated as more intense than the plain profile, sour was perceived as more intense and less palatable than all other profiles, and orange was rated as more palatable than sour, lemon, and plain tastants. Taste strips have useful implications for dysphagia management, as they could offer safe and patient-preferred flavor profiles to potentially provide advantageous swallowing and neural hemodynamic responses.

Nested Selves: Self-Organization and Shared Markov Blankets in Prenatal Development in Humans

The immune system is a central component of organismic function in humans. This paper addresses self-organization of biological systems in relation to-and nested within-other biological systems in pregnancy. Pregnancy constitutes a fundamental state for human embodiment and a key step in the evolution and conservation of our species. While not all humans can be pregnant, our initial state of emerging and growing within another person's body is universal. Hence, the pregnant state does not concern some individuals but all individuals. Indeed, the hierarchical relationship in pregnancy reflects an even earlier autopoietic process in the embryo by which the number of individuals in a single blastoderm is dynamically determined by cell- interactions. The relationship and the interactions between the two self-organizing systems during pregnancy may play a pivotal role in understanding the nature of biological self-organization per se in humans. Specifically, we consider the role of the immune system in biological self-organization in addition to neural/brain systems that furnish us with a sense of self. We examine the complex case of pregnancy, whereby two immune systems need to negotiate the exchange of resources and information in order to maintain viable self-regulation of nested systems. We conclude with a proposal for the mechanisms-that scaffold the complex relationship between two self-organising systems in pregnancy-through the lens of the Active Inference, with a focus on shared Markov blankets.

Dual-stream cortical pathways mediate sensory prediction

Predictions are constantly generated from diverse sources to optimize cognitive functions in the ever-changing environment. However, the neural origin and generation process of top-down induced prediction remain elusive. We hypothesized that motor-based and memory-based predictions are mediated by distinct descending networks from motor and memory systems to the sensory cortices. Using functional magnetic resonance imaging (fMRI) and a dual imagery paradigm, we found that motor and memory upstream systems activated the auditory cortex in a content-specific manner. Moreover, the inferior and posterior parts of the parietal lobe differentially relayed predictive signals in motor-to-sensory and memory-to-sensory networks. Dynamic causal modeling of directed connectivity revealed selective enabling and modulation of connections that mediate top-down sensory prediction and ground the distinctive neurocognitive basis of predictive processing.

How the conception of control influences our understanding of actions

Wilful movement requires neural control. Commonly, neural computations are thought to generate motor commands that bring the musculoskeletal system - that is, the plant - from its current physical state into a desired physical state. The current state can be estimated from past motor commands and from sensory information. Modelling movement on the basis of this concept of plant control strives to explain behaviour by identifying the computational principles for control signals that can reproduce the observed features of movements. From an alternative perspective, movements emerge in a dynamically coupled agent-environment system from the pursuit of subjective perceptual goals. Modelling movement on the basis of this concept of perceptual control aims to identify the controlled percepts and their coupling rules that can give rise to the observed characteristics of behaviour. In this Perspective, we discuss a broad spectrum of approaches to modelling human motor control and their notions of control signals, internal models, handling of sensory feedback delays and learning. We focus on the influence that the plant control and the perceptual control perspective may have on decisions when modelling empirical data, which may in turn shape our understanding of actions.

Unintentional drifts in performance during one-hand and two-hand finger force production

We explored the phenomena of force drifts and unintentional finger force production (enslaving), and their dependence on visual feedback. Predictions have been drawn based on the theory of control with spatial referent coordinates for condition with feedback on instructed (master) finger force, enslaved finger force, and total force for one-hand and two-hand tasks. Subjects produced force under the different feedback conditions without their knowledge. No feedback condition was also used for the one-hand tasks. Overall, feedback of master finger force led to an increase in the enslaved force, feedback on the slave finger force led to a drop in the master force, feedback on the total force led to balanced drifts in the master force down and enslaved force up, and under the no-feedback condition, master and total force drifted down with large variability in the enslaved force drifts. The patterns were the same in both hands in the two-hand tasks when feedback was provided on the forces of one hand only (without subject's knowledge). The index of enslaving always drifted toward higher values. We interpret the findings as reflecting three main factors: drifts in the referent coordinates toward actual finger coordinates, spread of cortical excitation over representations of the fingers, and robust sharing of referent coordinates between the two hands in bimanual tasks. The large consistent drifts in enslaving toward higher values have to be considered in studies of multi-finger synergies.

Turn-Taking Mechanisms in Imitative Interaction: Robotic Social Interaction Based on the Free Energy Principle

This study explains how the leader-follower relationship and turn-taking could develop in a dyadic imitative interaction by conducting robotic simulation experiments based on the free energy principle. Our prior study showed that introducing a parameter during the model training phase can determine leader and follower roles for subsequent imitative interactions. The parameter is defined as w, the so-called meta-prior, and is a weighting factor used to regulate the complexity term versus the accuracy term when minimizing the free energy. This can be read as sensory attenuation, in which the robot's prior beliefs about action are less sensitive to sensory evidence. The current extended study examines the possibility that the leader-follower relationship shifts depending on changes in w during the interaction phase. We identified a phase space structure with three distinct types of behavioral coordination using comprehensive simulation experiments with sweeps of w of both robots during the interaction. Ignoring behavior in which the robots follow their own intention was observed in the region in which both ws were set to large values. One robot leading, followed by the other robot was observed when one w was set larger and the other was set smaller. Spontaneous, random turn-taking between the leader and the follower was observed when both ws were set at smaller or intermediate values. Finally, we examined a case of slowly oscillating w in anti-phase between the two agents during the interaction. The simulation experiment resulted in turn-taking in which the leader-follower relationship switched during determined sequences, accompanied by periodic shifts of ws. An analysis using transfer entropy found that the direction of information flow between the two agents also shifted along with turn-taking. Herein, we discuss qualitative differences between random/spontaneous turn-taking and agreed-upon sequential turn-taking by reviewing both synthetic and empirical studies.

The evolution of brain architectures for predictive coding and active inference

This article considers the evolution of brain architectures for predictive processing. We argue that brain mechanisms for predictive perception and action are not late evolutionary additions of advanced creatures like us. Rather, they emerged gradually from simpler predictive loops (e.g. autonomic and motor reflexes) that were a legacy from our earlier evolutionary ancestors-and were key to solving their fundamental problems of adaptive regulation. We characterize simpler-to-more-complex brains formally, in terms of generative models that include predictive loops of increasing hierarchical breadth and depth. These may start from a simple homeostatic motif and be elaborated during evolution in four main ways: these include the multimodal expansion of predictive control into an allostatic loop; its duplication to form multiple sensorimotor loops that expand an animal's behavioural repertoire; and the gradual endowment of generative models with hierarchical depth (to deal with aspects of the world that unfold at different spatial scales) and temporal depth (to select plans in a future-oriented manner). In turn, these elaborations underwrite the solution to biological regulation problems faced by increasingly sophisticated animals. Our proposal aligns neuroscientific theorising-about predictive processing-with evolutionary and comparative data on brain architectures in different animal species. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Role of Post-Trial Visual Feedback on Unintentional Force Drift During Isometric Finger Force Production Tasks

A reduction in fingertip forces during a visually occluded isometric task is called unintentional drift. In this study, unintentional drift was studied for two conditions, with and without "epilogue." We define epilogue as the posttrial visual feedback in which the outcome of the just-concluded trial is shown before the start of the next trial. For this study, 14 healthy participants were recruited and were instructed to produce fingertip forces to match a target line at 15% maximum voluntary contraction. The results showed a significant reduction in unintentional drift in the epilogue condition. This reduction is probably due to the difference in the shift in λ, the threshold of the tonic stretch reflex, the hypothetical control variable that the central controller can set.

Understanding and Synergy: A Single Concept at Different Levels of Analysis?

Biological systems differ from the inanimate world in their behaviors ranging from simple movements to coordinated purposeful actions by large groups of muscles, to perception of the world based on signals of different modalities, to cognitive acts, and to the role of self-imposed constraints such as laws of ethics. Respectively, depending on the behavior of interest, studies of biological objects based on laws of nature (physics) have to deal with different salient sets of variables and parameters. Understanding is a high-level concept, and its analysis has been linked to other high-level concepts such as "mental model" and "meaning". Attempts to analyze understanding based on laws of nature are an example of the top-down approach. Studies of the neural control of movements represent an opposite, bottom-up approach, which starts at the interface with classical physics of the inanimate world and operates with traditional concepts such as forces, coordinates, etc. There are common features shared by the two approaches. In particular, both assume organizations of large groups of elements into task-specific groups, which can be described with only a handful of salient variables. Both assume optimality criteria that allow the emergence of families of solutions to typical tasks. Both assume predictive processes reflected in anticipatory adjustments to actions (motor and non-motor). Both recognize the importance of generating dynamically stable solutions. The recent progress in studies of the neural control of movements has led to a theory of hierarchical control with spatial referent coordinates for the effectors. This theory, in combination with the uncontrolled manifold hypothesis, allows quantifying the stability of actions with respect to salient variables. This approach has been used in the analysis of motor learning, changes in movements with typical and atypical development and with aging, and impaired actions by patients with various neurological disorders. It has been developed to address issues of kinesthetic perception. There seems to be hope that the two counter-directional approaches will meet and result in a single theoretical scheme encompassing biological phenomena from figuring out the best next move in a chess position to activating motor units appropriate for implementing that move on the chessboard.

One more time about motor (and non-motor) synergies

We revisit the concept of synergy based on the recently translated classical book by Nikolai Bernstein (On the construction of movements, Medgiz, Moscow 1947; Latash, Bernstein's Construction of Movements, Routledge, Abingdon 2020b) and progress in understanding the physics and neurophysiology of biological action. Two aspects of synergies are described: organizing elements into stable groups (modes) and ensuring dynamical stability of salient performance variables. The ability of the central nervous system to attenuate synergies in preparation for a quick action-anticipatory synergy adjustments-is emphasized. Recent studies have demonstrated synergies at the level of hypothetical control variables associated with spatial referent coordinates for effectors. Overall, the concept of synergies fits naturally the hierarchical scheme of control with referent coordinates with an important role played by back-coupling loops within the central nervous system and from peripheral sensory endings. Further, we review studies showing non-trivial changes in synergies with development, aging, fatigue, practice, and a variety of neurological disorders. Two aspects of impaired synergic control-impaired stability and impaired agility-are introduced. The recent generalization of the concept of synergies for non-motor domains, including perception, is discussed. We end the review with a list of unresolved and troubling issues.

Attention separates sensory and motor signals in the mouse visual cortex

Visually guided behaviors depend on the activity of cortical networks receiving visual inputs and transforming these signals to guide appropriate actions. However, non-retinal inputs, carrying motor signals as well as cognitive and attentional modulatory signals, also activate these cortical regions. How these networks integrate coincident signals ensuring reliable visual behaviors is poorly understood. In this study, we observe neural responses in the dorsal-parietal cortex of mice during a visual discrimination task driven by visual stimuli and movements. We find that visual and motor signals interact according to two mechanisms: divisive normalization and separation of responses. Interactions are contextually modulated by the animal's state of sustained attention, which amplifies visual and motor signals and increases their discriminability in a low-dimensional space of neural activations. These findings reveal computational principles operating in dorsal-parietal networks that enable separation of incoming signals for reliable visually guided behaviors during interactions with the environment.

Production and Perception of Intentional and Unintentional Actions

Physical approach to biological movement is based on the idea of control with referent spatial coordinates for effectors, from the whole body to single muscles. Within this framework, neural control signals induce changes in parameters of corresponding biology-specific laws of nature, and motor performance emerges as a result of interaction with the external force field. This approach is naturally compatible with the principle of abundance and the uncontrolled manifold hypothesis, which offer the framework for analysis of movement stability. The presence of two basic commands, reciprocal and co-activation, makes even single-effector tasks abundant and allows stabilizing their performance at the control level. Kinesthetic perception can be viewed as the process of estimating afferent signals within a reference system provided by the efferent process. Percepts are reflections of stable iso-perceptual manifolds in the combined afferent-efferent multi-dimensional space. This approach offers new, logical and based on laws of nature, interpretations for such phenomena as muscle co-activation, unintentional drifts in performance, and vibration-induced kinesthetic illusions. It also allows predicting new phenomena such as counter-intuitive effects of muscle co-activation of force production and perception, vibration-induced force illusions, performance drifts at two different speeds, and high variability in matching the contribution of individual elements in multi-element tasks. This approach can be developed for various subfields of movement studies including studies of athletics, movement disorders, and movement rehabilitation.

Stability of Action and Kinesthetic Perception in Parkinson's Disease

We present a review of action and perception stability within the theoretical framework based on the idea of control with spatial referent coordinates for the effectors at a number of hierarchical levels. Stability of salient variables is ensured by synergies, neurophysiological structures that act in multi-dimensional spaces of elemental variables and limit variance to the uncontrolled manifold during action and iso-perceptual manifold during perception. Patients with Parkinson’s disease show impaired synergic control reflected in poor stability (low synergy indices) and poor agility (low indices of anticipatory synergy adjustments prior to planned quick actions). They also show impaired perception across modalities, including kinesthetic perception. We suggest that poor stability at the level of referent coordinates can be the dominant factor leading to poor stability of percepts.

Efference copy in kinesthetic perception: a copy of what is it?

A number of notions in the fields of motor control and kinesthetic perception have been used without clear definitions. In this review, we consider definitions for efference copy, percept, and sense of effort based on recent studies within the physical approach, which assumes that the neural control of movement is based on principles of parametric control and involves defining time-varying profiles of spatial referent coordinates for the effectors. The apparent redundancy in both motor and perceptual processes is reconsidered based on the principle of abundance. Abundance of efferent and afferent signals is viewed as the means of stabilizing both salient action characteristics and salient percepts formalized as stable manifolds in high-dimensional spaces of relevant elemental variables. This theoretical scheme has led recently to a number of novel predictions and findings. These include, in particular, lower accuracy in perception of variables produced by elements involved in a multielement task compared with the same elements in single-element tasks, dissociation between motor and perceptual effects of muscle coactivation, force illusions induced by muscle vibration, and errors in perception of unintentional drifts in performance. Taken together, these results suggest that participation of efferent signals in perception frequently involves distorted copies of actual neural commands, particularly those to antagonist muscles. Sense of effort is associated with such distorted efferent signals. Distortions in efference copy happen spontaneously and can also be caused by changes in sensory signals, e.g., those produced by muscle vibration.

Distortions of the Efferent Copy during Force Perception: A Study of Force Drifts and Effects of Muscle Vibration

We used a finger force matching task to explore the role of efferent signals in force perception. Healthy, young participants performed accurate force production tasks at different force levels with the index and middle fingers of one hand (task-hand). They received visual feedback during an early part of each trial only. After the feedback was turned off, the force drifted toward lower magnitudes. After 5 s of the drift, the participants matched the force with the same finger pair of the other hand (match-hand). The match-hand consistently overshot the task-hand force by a magnitude invariant over the initial force levels. During force matching, both hands were lifted and lowered smoothly to estimate their referent coordinate (RC) and apparent stiffness values. These trials were performed without muscle vibration and under vibration applied to the finger/hand flexors or extensors of the task-hand or match-hand. Effects of vibration were seen in the match-hand only; they were the same during vibration of flexors and extensors. We interpret the vibration-induced effects as consequences of using distorted copies of the central commands to the task-hand during force matching. In particular, using distorted copies of the RC for the antagonist muscle group could account for the differences between the task-hand and match-hand. We conclude that efferent signals may be distorted before their participation in the perceptual process. Such distortions emerge spontaneously and may be amplified by the response of sensory endings to muscle vibration combined over both agonist and antagonist muscle groups.

Embodied skillful performance: where the action is

When someone masters a skill, their performance looks to us like second nature: it looks as if their actions are smoothly performed without explicit, knowledge-driven, online monitoring of their performance. Contemporary computational models in motor control theory, however, are instructionist: that is, they cast skillful performance as a knowledge-driven process. Optimal motor control theory (OMCT), as representative par excellence of such approaches, casts skillful performance as an instruction, instantiated in the brain, that needs to be executed-a motor command. This paper aims to show the limitations of such instructionist approaches to skillful performance. We specifically address the question of whether the assumption of control-theoretic models is warranted. The first section of this paper examines the instructionist assumption, according to which skillful performance consists of the execution of theoretical instructions harnessed in motor representations. The second and third sections characterize the implementation of motor representations as motor commands, with a special focus on formulations from OMCT. The final sections of this paper examine predictive coding and active inference-behavioral modeling frameworks that descend, but are distinct, from OMCT-and argue that the instructionist, control-theoretic assumptions are ill-motivated in light of new developments in active inference.

Minds and Brains, Sleep and Psychiatry

Objective

This article offers a philosophical thesis for psychiatric disorders that rests upon some simple truths about the mind and brain. Specifically, it asks whether the dual aspect monism—that emerges from sleep research and theoretical neurobiology—can be applied to pathophysiology and psychopathology in psychiatry.

Methods

Our starting point is that the mind and brain are emergent aspects of the same (neuronal) dynamics; namely, the brain–mind. Our endpoint is that synaptic dysconnection syndromes inherit the same dual aspect; namely, aberrant inference or belief updating on the one hand, and a failure of neuromodulatory synaptic gain control on the other. We start with some basic considerations from sleep research that integrate the phenomenology of dreaming with the neurophysiology of sleep.

Results

We then leverage this treatment by treating the brain as an organ of inference. Our particular focus is on the role of precision (i.e., the representation of uncertainty) in belief updating and the accompanying synaptic mechanisms.

Conclusions

Finally, we suggest a dual aspect approach—based upon belief updating (i.e., mind processes) and its neurophysiological implementation (i.e., brain processes)—has a wide explanatory compass for psychiatry and various movement disorders. This approach identifies the kind of pathophysiology that underwrites psychopathology—and points to certain psychotherapeutic and psychopharmacological targets, which may stand in mechanistic relation to each other.

The ‘mind’ emerges from Bayesian belief updating in the ‘brain’

Psychopathology can be read as aberrant belief updating.

Aberrant belief updating follows from any neuromodulatory synaptopathy

Finger Force Matching and Verbal Reports: Testing Predictions of the Iso-Perceptual Manifold Concept

We used force matching and verbal reports of finger force to explore a prediction of the iso-perceptual manifold concept, which assumes that stable percepts are associated with a manifold in the afferent-efferent space. Young subjects produced various force magnitudes with the index finger, middle finger, or both fingers together. Further, they reported the force level using a verbal scale and by matching the force with fingers of the contralateral hand. Force matching, but not verbal reports, showed larger variable errors for individual fingers in the two-finger task compared to the single-finger tasks. We discuss possible differences in afferent and efferent contributions to force perception at low and high forces based on the idea of motor control with referent coordinates for the effectors. The differences between the force matching and verbal reports are possibly related to neural circuitry differences between perceiving without action and perceiving-to-act.

What do people match when they try to match force? Analysis at the level of hypothetical control variables

We used the theory of control with spatial referent coordinates (RC) to explore how young, healthy persons modify finger pressing force and match forces between the two hands. Three specific hypotheses were tested related to patterns of RC and apparent stiffness (defined as the slope of force-coordinate relation) used in the presence of visual feedback on the force and in its absence. The subjects used the right hand to produce accurate force under visual feedback; further the force could be increased or decreased, intentionally or unintentionally (induced by controlled lifting or lowering of the fingertips). The left hand was used to match force without visual feedback before and after the force change; the match hand consistently underestimated the actual force change in the task hand. The "inverse piano" device was used to compute RC and apparent stiffness. We found very high coefficients of determination for the inter-trial hyperbolic regressions between RC and apparent stiffness in the presence of visual feedback; the coefficients of determination dropped significantly without visual feedback. There were consistent preferred sharing patterns in the space of RC and apparent stiffness between the task and match hands across subjects. In contrast, there was much less consistency between the task and match hands in the magnitudes of RC and apparent stiffness observed in individual trials. Compared to the task hand, the match hand showed consistently lower magnitudes of apparent stiffness and, correspondingly, larger absolute magnitudes of RC. Involuntary force changes produced by lifting and lowering the force sensors led to significantly lower force changes compared to what could be expected based on the computed values of apparent stiffness and sensor movement amplitude. The results confirm the importance of visual feedback for stabilization of force in the space of hypothetical control variables. They suggest the existence of personal traits reflected in preferred ranges of RC and apparent stiffness across the two hands. They also show that subjects react to external perturbations, even when instructed "not to interfere": Such perturbations cause unintentional and unperceived drifts in both RC and apparent stiffness.

An Investigation of the Free Energy Principle for Emotion Recognition

This paper offers a prospectus of what might be achievable in the development of emotional recognition devices. It provides a conceptual overview of the free energy principle; including Markov blankets, active inference, and-in particular-a discussion of selfhood and theory of mind, followed by a brief explanation of how these concepts can explain both neural and cultural models of emotional inference. The underlying hypothesis is that emotion recognition and inference devices will evolve from state-of-the-art deep learning models into active inference schemes that go beyond marketing applications and become adjunct to psychiatric practice. Specifically, this paper proposes that a second wave of emotion recognition devices will be equipped with an emotional lexicon (or the ability to epistemically search for one), allowing the device to resolve uncertainty about emotional states by actively eliciting responses from the user and learning from these responses. Following this, a third wave of emotional devices will converge upon the user's generative model, resulting in the machine and human engaging in a reciprocal, prosocial emotional interaction, i.e., sharing a generative model of emotional states.

Exploring the Concept of Iso-perceptual Manifold (IPM): A Study of Finger Force-Matching Tasks

We used force-matching tasks between the two hands to test predictions of the recently introduced scheme of perception based on the concept of iso-perceptual manifold (IPM) in the combined afferent-efferent space of neural signals. The main hypothesis was that accuracy and variability of individual finger force matching would be worse in a four-finger task compared to one-finger tasks. The subjects produced accurate force levels under visual feedback by pressing with either all four fingers or by one of the fingers of a hand (task-hand). They tried to match the total four-finger force or individual finger forces by pressing with the other hand (match-hand, no visual feedback). The match-hand consistently overshot the task-hand force during single-finger matching episodes. It showed higher inter-trial force variability during single-finger matching when the task-hand performed the four-finger task compared to trials when the task-hand performed single-finger tasks. These findings confirm our main hypothesis by showing that perception of individual finger forces can vary in multi-finger tasks within a space (IPM) corresponding to veridical perception of total force. Matching hypothetical commands to fingers, rather than finger forces, could be responsible for the consistent force overshoots. Indices of inter-trial variance affecting and unaffecting total force showed strong stabilization of total force in the task-hand but not in the match-hand in support of an earlier hypothesis on the importance of visual feedback for force stabilization. No differences were seen between the right and left hands suggesting that the dynamic dominance hypothesis may not be generalizable to perceptual phenomena.

The Discrete and Continuous Brain: From Decisions to Movement-And Back Again

To act upon the world, creatures must change continuous variables such as muscle length or chemical concentration. In contrast, decision making is an inherently discrete process, involving the selection among alternative courses of action. In this article, we consider the interface between the discrete and continuous processes that translate our decisions into movement in a Newtonian world—and how movement informs our decisions. We do so by appealing to active inference, with a special focus on the oculomotor system. Within this exemplar system, we argue that the superior colliculus is well placed to act as a discrete-continuous interface. Interestingly, when the neuronal computations within the superior colliculus are formulated in terms of active inference, we find that many aspects of its neuroanatomy emerge from the computations it must perform in this role.

Force illusions and drifts observed during muscle vibration

We explored predictions of a scheme that views position and force perception as a result of measuring proprioceptive signals within a reference frame set by ongoing efferent process. In particular, this hypothesis predicts force illusions caused by muscle vibration and mediated via changes in both afferent and efferent components of kinesthesia. Healthy subjects performed accurate steady force production tasks by pressing with the four fingers of one hand (the task hand) on individual force sensors with and without visual feedback. At various times during the trials, subjects matched the perceived force using the other hand. High-frequency vibration was applied to one or both of the forearms (over the hand and finger extensors). Without visual feedback, subjects showed a drop in the task hand force, which was significantly smaller under the vibration of that forearm. Force production by the matching hand was consistently higher than that of the task hand. Vibrating one of the forearms affected the matching hand in a manner consistent with the perception of higher magnitude of force produced by the vibrated hand. The findings were consistent between the dominant and nondominant hands. The effects of vibration on both force drift and force mismatching suggest that vibration led to shifts in both signals from proprioceptors and the efferent component of perception, the referent coordinate and/or coactivation command. The observations fit the hypothesis on combined perception of kinematic-kinetic variables with little specificity of different groups of peripheral receptors that all contribute to perception of forces and coordinates. NEW & NOTEWORTHY We show that vibration of hand/finger extensors produces consistent errors in finger force perception. Without visual feedback, finger force drifted to lower values without a drift in the matching force produced by the other hand; hand extensor vibration led to smaller finger force drift. The findings fit the scheme with combined perception of kinematic-kinetic variables and suggest that vibration leads to consistent shifts of the referent coordinate and, possibly, of coactivation command to the effector.

Stability of Kinesthetic Perception in Efferent-Afferent Spaces: The Concept of Iso-perceptual Manifold

The main goal of this paper is to introduce the concept of iso-perceptual manifold for perception of body configuration and related variables (kinesthetic perception) and to discuss its relation to the equilibrium-point hypothesis and the concepts of reference coordinate and uncontrolled manifold. Hierarchical control of action is postulated with abundant transformations between sets of spatial reference coordinates for salient effectors at different levels. Iso-perceptual manifold is defined in the combined space of afferent and efferent variables as the subspace corresponding to a stable percept. Examples of motion along an iso-perceptual manifold (perceptually equivalent motion) are considered during various natural actions. Some combinations of afferent and efferent signals, in particular those implying a violation of body's integrity, give rise to variable percepts by artificial projection onto iso-perceptual manifolds. This framework is used to interpret unusual features of vibration-induced kinesthetic illusions and to predict new illusions not yet reported in the literature.

Active Inference: A Process Theory

This article describes a process theory based on active inference and belief propagation. Starting from the premise that all neuronal processing (and action selection) can be explained by maximizing Bayesian model evidence-or minimizing variational free energy-we ask whether neuronal responses can be described as a gradient descent on variational free energy. Using a standard (Markov decision process) generative model, we derive the neuronal dynamics implicit in this description and reproduce a remarkable range of well-characterized neuronal phenomena. These include repetition suppression, mismatch negativity, violation responses, place-cell activity, phase precession, theta sequences, theta-gamma coupling, evidence accumulation, race-to-bound dynamics, and transfer of dopamine responses. Furthermore, the (approximately Bayes' optimal) behavior prescribed by these dynamics has a degree of face validity, providing a formal explanation for reward seeking, context learning, and epistemic foraging. Technically, the fact that a gradient descent appears to be a valid description of neuronal activity means that variational free energy is a Lyapunov function for neuronal dynamics, which therefore conform to Hamilton's principle of least action.

Active sensing without efference copy: referent control of perception

Although action and perception are different behaviors, they are likely to be interrelated, as implied by the notions of perception-action coupling and active sensing. Traditionally, it has been assumed that the nervous system directly preprograms motor commands required for actions and uses a copy of them called efference copy (EC) to also influence our senses. This review offers a critical analysis of the EC concept by identifying its limitations. An alternative to the EC concept is based on the experimentally confirmed notion that sensory signals from receptors are perceived relative to referent signals specified by the brain. These referents also underlie the control of motor actions by predetermining where, in the spatial domain, muscles can work without preprogramming how they should work in terms of motor commands or EC. This approach helps solve several problems of action and explain several sensory experiences, including position sense and the sense that the world remains stationary despite changes in its retinal image during eye or body motion (visual space constancy). The phantom limb phenomenon and other kinesthetic illusions are also explained within this framework.

Towards physics of neural processes and behavior

Behavior of biological systems is based on basic physical laws, common across inanimate and living systems, and currently unknown physical laws that are specific for living systems. Living systems are able to unite basic laws of physics into chains and clusters leading to new stable and pervasive relations among variables (new physical laws) involving new parameters and to modify these parameters in a purposeful way. Examples of such laws are presented starting from the tonic stretch reflex. Further, the idea of control with referent coordinates is formulated and merged with the idea of hierarchical control and the principle of abundance. The notion of controlled stability of behaviors is linked to the idea of structured variability, which is a common feature across living systems and actions. The explanatory and predictive power of this approach is illustrated with respect to the control of both intentional and unintentional movements, the phenomena of equifinality and its violations, preparation to quick actions, development of motor skills, changes with aging and neurological disorders, and perception.

A Riemannian geometry theory of human movement: The geodesic synergy hypothesis

Mass-inertia loads on muscles change with posture and with changing mechanical interactions between the body and the environment. The nervous system must anticipate changing mass-inertia loads, especially during fast multi-joint coordinated movements. Riemannian geometry provides a mathematical framework for movement planning that takes these inertial interactions into account. To demonstrate this we introduce the controlled (vs. biomechanical) degrees of freedom of the body as the coordinate system for a configuration space with movements represented as trajectories. This space is not Euclidean. It is endowed at each point with a metric equal to the mass-inertia matrix of the body in that configuration. This warps the space to become Riemannian with curvature at each point determined by the differentials of the mass-inertia at that point. This curvature takes nonlinear mass-inertia interactions into account with lengths, velocities, accelerations and directions of movement trajectories all differing from those in Euclidean space. For newcomers to Riemannian geometry we develop the intuitive groundwork for a Riemannian field theory of human movement encompassing the entire body moving in gravity and in mechanical interaction with the environment. In particular we present a geodesic synergy hypothesis concerning planning of multi-joint coordinated movements to achieve goals with minimal muscular effort.

Lateralization of spatial information processing in response monitoring

The current study aims at identifying how lateralized multisensory spatial information processing affects response monitoring and action control. In a previous study, we investigated multimodal sensory integration in response monitoring processes using a Simon task. Behavioral and neurophysiologic results suggested that different aspects of response monitoring are asymmetrically and independently allocated to the hemispheres: while efference-copy-based information on the motor execution of the task is further processed in the hemisphere that originally generated the motor command, proprioception-based spatial information is processed in the hemisphere contralateral to the effector. Hence, crossing hands (entering a “foreign” spatial hemifield) yielded an augmented bilateral activation during response monitoring since these two kinds of information were processed in opposing hemispheres. Because the traditional Simon task does not provide the possibility to investigate which aspect of the spatial configuration leads to the observed hemispheric allocation, we introduced a new “double crossed” condition that allows for the dissociation of internal/physiological and external/physical influences on response monitoring processes. Comparing behavioral and neurophysiologic measures of this new condition to those of the traditional Simon task setup, we could demonstrate that the egocentric representation of the physiological effector's spatial location accounts for the observed lateralization of spatial information in action control. The finding that the location of the physical effector had a very small influence on response monitoring measures suggests that this aspect is either less important and/or processed in different brain areas than egocentric physiological information.

Heritability of motor control and motor learning

The aim of this study was to elucidate the relative contribution of genes and environment on individual differences in motor control and acquisition of a force control task, in view of recent association studies showing that several candidate polymorphisms may have an effect on them. Forty‐four healthy female twins performed brisk isometric abductions with their right thumb. Force was recorded by a transducer and fed back to the subject on a computer screen. The task was to place the tracing of the peak force in a force window defined between 30% and 40% of the subject's maximum force, as determined beforehand. The initial level of proficiency was defined as the number of attempts reaching the force window criterion within the first 100 trials. The difference between the number of successful trials within the last and the first 100 trials was taken as a measure of motor learning. For motor control, defined by the initial level of proficiency, the intrapair differences in monozygotic (MZ) and dizygotic (DZ) twins were 6.8 ± 7.8 and 13.8 ± 8.4, and the intrapair correlations 0.77 and 0.39, respectively. Heritability was estimated at 0.68. Likewise for motor learning intrapair differences in the increment of the number of successful trials in MZ and DZ twins were 5.4 ± 5.2 and 12.8 ± 7, and the intrapair correlations 0.58 and 0.19. Heritability reached 0.70. The present findings suggest that heredity accounts for a major part of existing differences in motor control and motor learning, but uncertainty remains which gene polymorphisms may be responsible.

Individual differences in motor control and learning are attributed to a great extent to genetic predisposition. However, uncertainty remains which gene polymorphism may be responsible.

Active prospective control is required for effective sensorimotor learning

Passive modeling of movements is often used in movement therapy to overcome disabilities caused by stroke or other disorders (e.g. Developmental Coordination Disorder or Cerebral Palsy). Either a therapist or, recently, a specially designed robot moves or guides the limb passively through the movement to be trained. In contrast, action theory has long suggested that effective skill acquisition requires movements to be actively generated. Is this true? In view of the former, we explicitly tested the latter. Previously, a method was developed that allows children with Developmental Coordination Disorder to produce effective movements actively, so as to improve manual performance to match that of typically developing children. In the current study, we tested practice using such active movements as compared to practice using passive movement. The passive movement employed, namely haptic tracking, provided a strong test of the comparison, one that showed that the mere inaction of the muscles is not the problem. Instead, lack of prospective control was. The result was no effective learning with passive movement while active practice with prospective control yielded significant improvements in performance.

Action-perception coupling in kinesthesia: a new approach

According to recent findings, intentional motor actions are controlled by resetting the referent position, R, at which neuromuscular elements, including reflexes, begin to act. It is suggested that somatosensory afferents inform the brain about the deviation (P) of body segments from the centrally set referent position. To perceive the actual position (Q) of body segments and form the position sense (PS), the central and afferent signals are combined: Q=R+P. In previous studies, the R has been shown to remain invariant during involuntary changes in the wrist position elicited by sudden unloading of muscles, suggesting that only the afferent component is responsible for the PS during this reflex. In contrast, the central PS component, R, is predominantly responsible for PS during intentional motion in isotonic conditions. We tested the hypothesis that the R and P are interchangeable PS components such that involuntary changes in wrist position elicited by the unloading reflex can easily be reproduced by making intentional changes in wrist position in isotonic conditions, in the absence of vision. The PS rule also suggests that PS is independent of sense of effort, which was tested by asking subjects to reproduce elbow joint angles under different constant loads. We also tested the hypothesis that tendon vibration may elicit motion that may not be perceived by subjects (no-motion illusion). These hypotheses were confirmed in three experiments. It is concluded that the R and P are additive components of PS and that, contrary to the conventional view, PS is independent of the sense of effort or efference copy. The PS rule also explains kinesthetic illusions and the phantom limb phenomenon. This study advances the understanding of action-perception coupling in kinesthesia.

Predictions not commands: active inference in the motor system

The descending projections from motor cortex share many features with top-down or backward connections in visual cortex; for example, corticospinal projections originate in infragranular layers, are highly divergent and (along with descending cortico-cortical projections) target cells expressing NMDA receptors. This is somewhat paradoxical because backward modulatory characteristics would not be expected of driving motor command signals. We resolve this apparent paradox using a functional characterisation of the motor system based on Helmholtz's ideas about perception; namely, that perception is inference on the causes of visual sensations. We explain behaviour in terms of inference on the causes of proprioceptive sensations. This explanation appeals to active inference, in which higher cortical levels send descending proprioceptive predictions, rather than motor commands. This process mirrors perceptual inference in sensory cortex, where descending connections convey predictions, while ascending connections convey prediction errors. The anatomical substrate of this recurrent message passing is a hierarchical system consisting of functionally asymmetric driving (ascending) and modulatory (descending) connections: an arrangement that we show is almost exactly recapitulated in the motor system, in terms of its laminar, topographic and physiological characteristics. This perspective casts classical motor reflexes as minimising prediction errors and may provide a principled explanation for why motor cortex is agranular.

Differential effects of motor efference copies and proprioceptive information on response evaluation processes

It is well-kown that sensory information influences the way we execute motor responses. However, less is known about if and how sensory and motor information are integrated in the subsequent process of response evaluation. We used a modified Simon Task to investigate how these streams of information are integrated in response evaluation processes, applying an in-depth neurophysiological analysis of event-related potentials (ERPs), time-frequency decomposition and sLORETA. The results show that response evaluation processes are differentially modulated by afferent proprioceptive information and efference copies. While the influence of proprioceptive information is mediated via oscillations in different frequency bands, efference copy based information about the motor execution is specifically mediated via oscillations in the theta frequency band. Stages of visual perception and attention were not modulated by the interaction of proprioception and motor efference copies. Brain areas modulated by the interactive effects of proprioceptive and efference copy based information included the middle frontal gyrus and the supplementary motor area (SMA), suggesting that these areas integrate sensory information for the purpose of response evaluation. The results show how motor response evaluation processes are modulated by information about both the execution and the location of a response.

Control of finger force vectors with changes in fingertip referent coordinates

The central hypothesis explored in the experiment is that adjustments of fingertip force vectors during object manipulation could result from a simple scaling rule applied to commands to individual digits. The commands have been associated with referent coordinates of the digit tips. The subjects performed quick lifting movements (over 20 cm in under 0.5 s) of a horizontally oriented handle with different combinations of the external load and torque. The prismatic grasp was used with the 4 fingers pressing on the bottom of the handle and the thumb acting on its top. Principal component and correlation analyses applied to the normal and tangential force vector components confirmed that the force direction of each digit was kept nearly constant in the object-centered referent frame across the loading conditions and movement phases. The middle and ring fingers showed weaker correlations between the force components as compared to the index and little fingers. The differences were likely related to the different roles of the normal force components in the moment of force production. The neural control of the hand, within the studied task, may be adequately described as a simple rule applied to a handful of parameters, such as the referent digit-tip coordinates.

Adaptations to fatigue of a single digit violate the principle of superposition in a multi-finger static prehension task

We investigated the effects of exercise-induced fatigue of a digit on the biomechanics of a static prehension task. The participants were divided into two groups. One group performed the fatiguing exercise using the thumb (group-thumb) and the second group performed the exercise using the index finger (group-index). We analyzed the prehensile action as being based on a two-level hierarchy. Our first hypothesis was that fatigue of the thumb would have stronger effects at the upper level (action shared between the thumb and all four fingers combined-virtual finger) and fatigue of the index finger would have stronger effects at the lower level of the hierarchy (action of the virtual finger shared among actual fingers). We also hypothesized that fatigue would cause a decrease in the normal force applied by the exercised digit and correspondingly lead to a decrease in the normal force applied by the opposing digit(s). Our third hypothesis was that fatigue would leave the tangential forces unaffected. Fatigue led to a significant drop in the normal force of both exercised and non-exercised (opposing) digits. The tangential forces of the exercised digits increased after fatigue. This led to a drop in the safety margin in the group-thumb, but not group-index. As such, the results supported the first two hypotheses but not the third hypothesis. Overall, the results suggested that fatigue triggered a chain reaction that involved both forces and moments of force produced by individual digits leading to a violation of the principle of superposition. The findings are interpreted within the framework of the referent configuration hypothesis.

Motor learning and its sensory effects: time course of perceptual change and its presence with gradual introduction of load

A complex interplay has been demonstrated between motor and sensory systems. We showed recently that motor learning leads to changes in the sensed position of the limb (Ostry DJ, Darainy M, Mattar AA, Wong J, Gribble PL. J Neurosci 30: 5384-5393, 2010). Here, we document further the links between motor learning and changes in somatosensory perception. To study motor learning, we used a force field paradigm in which subjects learn to compensate for forces applied to the hand by a robotic device. We used a task in which subjects judge lateral displacements of the hand to study somatosensory perception. In a first experiment, we divided the motor learning task into incremental phases and tracked sensory perception throughout. We found that changes in perception occurred at a slower rate than changes in motor performance. A second experiment tested whether awareness of the motor learning process is necessary for perceptual change. In this experiment, subjects were exposed to a force field that grew gradually in strength. We found that the shift in sensory perception occurred even when awareness of motor learning was reduced. These experiments argue for a link between motor learning and changes in somatosensory perception, and they are consistent with the idea that motor learning drives sensory change.

Corticospinal control strategies underlying voluntary and involuntary wrist movements

The difference between voluntary and involuntary motor actions has been recognized since ancient times, but the nature of this difference remains unclear. We compared corticospinal influences at wrist positions established before and after voluntary motion with those established before and after involuntary motion elicited by sudden removal of a load (the unloading reflex). To minimize the effect of motoneuronal excitability on the evaluation of corticospinal influences, motor potentials from transcranial magnetic stimulation of the wrist motor cortex area were evoked during an EMG silent period produced by brief muscle shortening. The motoneuronal excitability was thus equalized at different wrist positions. Results showed that the unloading reflex was generated in the presence of a corticospinal drive, rather than autonomously by the spinal cord. Although the tonic EMG levels were substantially different, the corticospinal influences remained the same at the pre- and post-unloading wrist positions. These influences however changed when subjects voluntarily moved the wrist to another position. Previous studies showed that the corticospinal system sets the referent position (R) at which neuromuscular posture-stabilizing mechanisms begin to act. In self-initiated actions, the corticospinal system shifts the R to relay these mechanisms to a new posture, thus converting them from mechanisms resisting to those assisting motion. This solves the classical posture-movement problem. In contrast, by maintaining the R value constant, the corticospinal system relies on these posture-stabilizing mechanisms to allow involuntary responses to occur after unloading. Thus, central control strategies underlying the two types of motor actions are fundamentally different.

Functional synergies underlying control of upright posture during changes in head orientation

Background

Studies of human upright posture typically have stressed the need to control ankle and hip joints to achieve postural stability. Recent studies, however, suggest that postural stability involves multi degree-of-freedom (DOF) coordination, especially when performing supra-postural tasks. This study investigated kinematic synergies related to control of the body’s position in space (two, four and six DOF models) and changes in the head’s orientation (six DOF model).

Methodology/Principal Findings

Subjects either tracked a vertically moving target with a head-mounted laser pointer or fixated a stationary point during 4-min trials. Uncontrolled manifold (UCM) analysis was performed across tracking cycles at each point in time to determine the structure of joint configuration variance related to postural stability or tracking consistency. The effect of simulated removal of covariance among joints on that structure was investigated to further determine the role of multijoint coordination. Results indicated that cervical joint motion was poorly coordinated with other joints to stabilize the position of the body center of mass (CM). However, cervical joints were coordinated in a flexible manner with more caudal joints to achieve consistent changes in head orientation.

Conclusions/Significance

An understanding of multijoint coordination requires reference to the stability/control of important performance variables. The nature of that coordination differs depending on the reference variable. Stability of upright posture primarily involved multijoint coordination of lower extremity and lower trunk joints. Consistent changes in the orientation of the head, however, required flexible coordination of those joints with motion of the cervical spine. A two-segment model of postural control was unable to account for the observed stability of the CM position during the tracking task, further supporting the need to consider multijoint coordination to understand postural stability.

Visual Experiences during Paralysis

Rationale: Paralyzed human volunteers (n = 6) participated in several studies the primary one of which required full neuromuscular paralysis while awake. After the primary experiment, while still paralyzed and awake, subjects undertook studies of humor and of attempted eye-movement. The attempted eye-movements tested a central, intentional component to one’s internal visual model and are the subject of this report. Methods: Subjects reclined in a supportive chair and were ventilated after paralysis (cisatracurium, 20 mg intravenously). In illumination, subjects were requested to focus alternately on the faces of investigators standing on the left and the right within peripheral vision. In darkness, subjects were instructed to look away from a point source of light. Subjects were to report their experiences after reversal of paralysis. Results: During attempted eye-movement in illumination, one subject had an illusion of environmental movement but four subjects perceived faces as clearly as if they were in central vision. In darkness, four subjects reported movement of the target light in the direction of attempted eye-movements and three could control the movement of the light at will. Conclusion: The hypothesis that internal visual models receive intended ocular-movement-information directly from oculomotor centers is strengthened by this evidence.