Rapid Visuomotor Corrective Responses during Transport of Hand-Held Objects Incorporate Novel Object Dynamics

Adaptation of the gain of the corrective lifting response in object manipulation transfers across the hand

Our ability to skillfully manipulate objects is supported by rapid corrective responses that are initiated when we experience perturbations that interfere with movement goals. For example, the corrective lifting response is triggered when an object is heavier than expected and fails to lift off the surface. In this situation, the absence of expected sensory feedback signalling lift off initiates, within ~ 90 ms, an increase in lifting force. Importantly, when people repeatedly lift an object that, on occasional catch trials, is heavier than expected, the gain of the corrective response, defined as the rate of force increase, adapts to the 'catch' weight. In the present study, we investigated whether this response adaption transfers intermanually. In the training phase, participants used either their left or right hand (counterbalanced) to repeatedly lift a 3 N object that unexpectedly increased to 9 N on catch trials, leading to an increase in the gain of the lifting response across catch trials. Participants then lifted the object with their other hand. On the first catch trial, the gain remained elevated and thus transferred across the hands. This finding suggests that the history of lifts performed by one hand updates the corrective responses for both hands.

Fast Feedback Responses to Categorical Sensorimotor Errors That Do Not Indicate Error Magnitude Are Optimized Based on Short- and Long-Term Memory

Skilled motor performance depends critically on rapid corrective responses that act to preserve the goal of the movement in the face of perturbations. Although it is well established that the gain of corrective responses elicited while reaching toward objects adapts to different contexts, little is known about the adaptability of corrective responses supporting the manipulation of objects after they are grasped. Here, we investigated the adaptability of the corrective response elicited when an object being lifted is heavier than expected and fails to lift off when predicted. This response involves a monotonic increase in vertical load force triggered, within ∼90 ms, by the absence of expected sensory feedback signaling lift off and terminated when actual lift off occurs. Critically, because the actual weight of the object cannot be directly sensed at the moment the object fails to lift off, any adaptation of the corrective response would have to be based on memory from previous lifts. We show that when humans, including men and women, repeatedly lift an object that on occasional catch trials increases from a baseline weight to a fixed heavier weight, they scale the gain of the response (i.e., the rate of force increase) to the heavier weight within two to three catch trials. We also show that the gain of the response scales, on the first catch trial, with the baseline weight of the object. Thus, the gain of the lifting response can be adapted by both short- and long-term experience. Finally, we demonstrate that this adaptation preserves the efficacy of the response across contexts.SIGNIFICANCE STATEMENT Here, we present the first investigation of the adaptability of the corrective lifting response elicited when an object is heavier than expected and fails to lift off when predicted. A striking feature of the response, which is driven by a sensory prediction error arising from the absence of expected sensory feedback, is that the magnitude of the error is unknown. That is, the motor system only receives a categorical error indicating that the object is heavier than expected but not its actual weight. Although the error magnitude is not known at the moment the response is elicited, we show that the response can be scaled to predictions of error magnitude based on both recent and long-term memories.

Reach-to-grasp kinematics and kinetics with and without visual feedback in early-stage Alzheimer’s disease

This study aimed to investigate the effects of early-stage Alzheimer’s disease (AD) on the reach-to-grasp kinematics and kinetics with and without visual supervision of the grasping arm and hand. Seventeen patients who had been diagnosed with early-stage AD and 17 age- and gender-matched, cognitive normal (CN) adults participated in the experiment. A mirror operating system was designed to block the visual feedback of their grasping hand and forearms but to virtually show grasped targets. The target for reach-to-grasp kinematics was a reflective marker installed on a base; and the target for reach-to-grasp kinetics was a custom-made apparatus installed with two six-component force/torque transducers. Kinematics and kinetic parameters were used to quantify the reach-to-grasp performances. Results showed that the early-stage AD remarkably decreased the reaching speed, reduced the grasping accuracy and increased the transportation variability for reach-to-grasp kinematics. For kinetic analysis, early-stage AD extended the preload duration, disturbed the grip and lift forces coordination, and increased the feedforward proportion in the grasping force control. The AD-related changes in the reach-to-grasp kinematic and kinetic parameters depended on visual feedback and were associated with nervous system function according to correlation analyses with the neuropsychological testing. These results suggest that the abnormal kinematic and kinetic characteristics may correlate with the neuropsychological status of early-stage AD, and that the reach-to-grasp kinematic and kinetic maneuver could potentially be used as a novel tool for non-invasive screening or evaluation of early-stage AD.

Asymmetrical transfer of adaptation between reaching and tracking: implications for feedforward and feedback processes

Reaching and manual tracking are two very common tasks for studying human sensorimotor processes. Although these motor tasks rely both on feedforward and feedback processes, emphasis is more on feedforward processes for reaching, and more on feedback processes for tracking. The extent to which feedforward and feedback processes are interrelated when being updated is not settled yet. Here, using reaching and tracking as proxies, we examined the bidirectional relationship between the update of feedforward and feedback processes. Forty right-handed participants were asked to move a joystick so as to either track a target moving rather unpredictably (pursuit tracking) or to make fast pointing movements toward a static target (center-out reaching task). Visuomotor adaptation was elicited by introducing a 45° rotation between the joystick motion and the cursor motion. Half of the participants adapted to rotation first via reaching movements, and then with pursuit tracking, while the other half performed both tasks in opposite order. Group comparisons revealed a strong asymmetrical transfer of adaptation between tasks. Namely, although nearly complete transfer of adaptation was observed from reaching to tracking, only modest transfer was found from tracking to reaching. A control experiment (N=10) revealed that making target motion fully predictable did not impact the latter finding. One possible interpretation is that the update of feedforward processes contributes directly to feedback processes, but the update of feedback processes engaged in tracking can be performed in isolation. These results suggest that reaching movements are supported by broader (i.e. more universal) mechanisms than tracking ones.

Bayes, predictive processing, and the cognitive architecture of motor control

Despite their popularity, relatively scant attention has been paid to the upshot of Bayesian and predictive processing models of cognition for views of overall cognitive architecture. Many of these models are hierarchical; they posit generative models at multiple distinct "levels," whose job is to predict the consequences of sensory input at lower levels. I articulate one possible position that could be implied by these models, namely, that there is a continuous hierarchy of perception, cognition, and action control comprising levels of generative models. I argue that this view is not entailed by a general Bayesian/predictive processing outlook. Bayesian approaches are compatible with distinct formats of mental representation. Focusing on Bayesian approaches to motor control, I argue that the junctures between different types of mental representation are places where the transitivity of hierarchical prediction may be broken, and I consider the upshot of this conclusion for broader discussions of cognitive architecture.

Distinct adaptation patterns between grip dynamics and arm kinematics when the body is upside-down

In humans, practically all movements are learnt and performed in a constant gravitational field. Yet, studies on arm movements and object manipulation in parabolic flight have highlighted very fast sensorimotor adaptations to altered gravity environments. Here, we wondered if the motor adjustments observed in those altered gravity environments could also be observed on Earth in a situation where the body is upside-down. To address this question, we asked participants to perform rhythmic arm movements in two different body postures (right-side-up and upside-down) while holding an object in precision grip. Analyses of grip-load force coordination and of movement kinematics revealed distinct adaptation patterns between grip and arm control. Grip force and load force were tightly synchronized from the first movements performed in upside-down posture, reflecting a malleable allocentric grip control. In contrast, velocity profiles showed a more progressive adaptation to the upside-down posture and reflected an egocentric planning of arm kinematics. In addition to suggesting distinct mechanisms between grip dynamics and arm kinematics for adaptation to novel contexts, these results also suggest the existence of general mechanisms underlying gravity-dependent motor adaptation that can be used for fast sensorimotor coordination across different postures on Earth and, incidentally, across different gravitational conditions in parabolic flights, in human centrifuges, or in Space.NEW & NOTEWORTHY During rhythmic arm movements performed in an upside-down posture, grip control adapted very quickly, but kinematics adaptation was more progressive. Our results suggest that grip control and movement kinematics planning might operate in different reference frames. Moreover, by comparing our results with previous results from parabolic flight studies, we propose that a common mechanism underlies adaptation to unfamiliar body postures and adaptation to altered gravity.

Online corrective responses following target jump in altered gravitoinertial force field point to nested feedforward and feedback control

Studies on goal-directed arm movements have shown a close link between feedforward and feedback control in protocols where both planning and online control processes faced a similar type of perturbation, either mechanical or visual. This particular context might have facilitated the use of an adapted internal model by feedforward and feedback control. Here, we considered this link in a context where, after feedforward control was adapted through proprioception-based processes, feedback control was tested under visual perturbation. We analyzed the response of the reaching hand to target displacements following adaptation to an altered force field induced by rotating participants at constant velocity. Reaching corrections were assessed through variables related to the accuracy (lateral and longitudinal end point errors) and kinematics (movement time, peak velocity) of the corrective movements. The electromyographic activity of different arm muscles (pectoralis, posterior deltoid, biceps brachii, and triceps brachii) was analyzed. Statistical analyses revealed that accuracy and kinematics of corrective movements were strikingly alike between normal and altered gravitoinertial force fields. However, pectoralis and biceps muscle activities recorded during corrective movements were significantly modified to counteract the effect of rotation-induced Coriolis and centrifugal forces on the arm. Remarkably, feedback control was functional from the very first time participants encountered a target jump in the altered force field. Overall, the present results demonstrate that feedforward control enables immediate functional feedback control even when applied to distinct sensorimotor processes.NEW & NOTEWORTHY We investigated the link between feedforward and feedback control when applying a double-step perturbation (visual target jump) during reaching movements performed in modified gravitoinertial environments. Altogether, kinematics and EMG analyses showed that movement corrections were highly effective in the different force fields, suggesting that, although feedforward and feedback control were driven by different sensory inputs, feedback control was remarkably functional from the very first time participants encountered a target jump in the altered force field.

Visually-updated hand state estimates modulate the proprioceptive reflex independently of motor task requirements

Fast signaling from vision and proprioception to muscle activation plays essential roles in quickly correcting movement. Though many studies have demonstrated modulation of the quick sensorimotor responses as depending on context in each modality, the contribution of multimodal information has not been established. Here, we examined whether state estimates contributing to stretch reflexes are represented solely by proprioceptive information or by multimodal information. Unlike previous studies, we newly found a significant stretch-reflex attenuation by the distortion and elimination of visual-feedback without any change in motor tasks. Furthermore, the stretch-reflex amplitude reduced with increasing elimination durations which would degrade state estimates. By contrast, even though a distortion was introduced in the target-motor-mapping, the stretch reflex was not simultaneously attenuated with visuomotor reflex. Our results therefore indicate that the observed stretch-reflex attenuation is specifically ascribed to uncertainty increase in estimating hand states, suggesting multimodal contributions to the generation of stretch reflexes.

Sensory information from a slipping object elicits a rapid and automatic shoulder response

Humans have the remarkable ability to hold, grasp, and manipulate objects. Previous work has reported rapid and coordinated reactions in hand and shoulder muscles in response to external perturbations to the arm during object manipulation; however, little is known about how somatosensory feedback of an object slipping in the hand influences responses of the arm. We built a handheld device to stimulate the sensation of slipping at all five fingertips. The device was integrated into an exoskeleton robot that supported it against gravity. The setup allowed us to decouple somatosensory stimulation in the fingers from forces applied to the arm, two variables that are highly interdependent in real-world scenarios. Fourteen participants performed three experiments in which we measured their arm feedback responses during slip stimulation. Slip stimulations were applied horizontally in one of two directions, and participants were instructed to either follow the slip direction or move the arm in the opposite direction. Participants showed shoulder muscle responses within ∼67 ms of slip onset when following the direction of slip but significantly slower responses when instructed to move in the opposite direction. Shoulder responses were modulated by the speed but not the distance of the slip. Finally, when slip stimulation was combined with mechanical perturbations to the arm, we found that sensory information from the fingertips significantly modulated the shoulder feedback responses. Overall, the results demonstrate the existence of a rapid feedback system that stabilizes handheld objects.NEW & NOTEWORTHY We tested whether the sensation of an object slipping from the fingers modulates shoulder feedback responses. We found rapid shoulder feedback responses when participants were instructed to follow the slip direction with the arm. Shoulder responses following mechanical joint perturbations were also potentiated when combined with slipping. These results demonstrate the existence of fast and automatic feedback responses in the arm in reaction to sensory input to the fingertips that maintain grip on handheld objects.

Decision-making in sensorimotor control

Skilled sensorimotor interactions with the world result from a series of decision-making processes that determine, on the basis of information extracted during the unfolding sequence of events, which movements to make and when and how to make them. Despite this inherent link between decision-making and sensorimotor control, research into each of these two areas has largely evolved in isolation, and it is only fairly recently that researchers have begun investigating how they interact and, together, influence behaviour. Here, we review recent behavioural, neurophysiological and computational research that highlights the role of decision-making processes in the selection, planning and control of goal-directed movements in humans and nonhuman primates.

Sensorimotor Learning during a Marksmanship Task in Immersive Virtual Reality

Sensorimotor learning refers to improvements that occur through practice in the performance of sensory-guided motor behaviors. Leveraging novel technical capabilities of an immersive virtual environment, we probed the component kinematic processes that mediate sensorimotor learning. Twenty naïve subjects performed a simulated marksmanship task modeled after Olympic Trap Shooting standards. We measured movement kinematics and shooting performance as participants practiced 350 trials while receiving trial-by-trial feedback about shooting success. Spatiotemporal analysis of motion tracking elucidated the ballistic and refinement phases of hand movements. We found systematic changes in movement kinematics that accompanied improvements in shot accuracy during training, though reaction and response times did not change over blocks. In particular, we observed longer, slower, and more precise ballistic movements that replaced effort spent on corrections and refinement. Collectively, these results leverage developments in immersive virtual reality technology to quantify and compare the kinematics of movement during early learning of full-body sensorimotor orienting.

Grip force when reaching with target uncertainty provides evidence for motor optimization over averaging

When presented with competing potential reach targets and required to launch a movement before knowing which one will be cued as the target, people initially reach in the average target direction. Although this spatial averaging could arise from executing a weighted average of motor plans for the potential targets, it could also arise from planning a single, optimal movement. To test between these alternatives we used a task in which participants were required to reach to either a single target or towards two potential targets while grasping an object. A robotic device applied a lateral elastic load to the object requiring large grip forces for reaches to targets either side of midline and a minimal grip force for midline movements. As expected, in trials with two targets located either side of midline, participants initially reached straight ahead. Critically, on these trials the initial grip force was minimal, appropriate for the midline movement, and not the average of the large grip forces required for movements to the individual targets. These results indicate that under conditions of target uncertainty, people do not execute an average of planned actions but rather a single movement that optimizes motor costs.

Rapid visuomotor feedback gains are tuned to the task dynamics

Here, we test whether rapid visuomotor feedback responses are selectively tuned to the task dynamics. The responses do not exhibit gain scaling, but they do vary with the level and stability of task dynamics. Moreover, these feedback gains are independently tuned to perturbations to the left and right, depending on these dynamics. Our results demonstrate that the sensorimotor control system regulates the feedback gain as part of the adaptation process, tuning them appropriately to the environment.

Adaptation to novel dynamics requires learning a motor memory, or a new pattern of predictive feedforward motor commands. Recently, we demonstrated the upregulation of rapid visuomotor feedback gains early in curl force field learning, which decrease once a predictive motor memory is learned. However, even after learning is complete, these feedback gains are higher than those observed in the null field trials. Interestingly, these upregulated feedback gains in the curl field were not observed in a constant force field. Therefore, we suggest that adaptation also involves selectively tuning the feedback sensitivity of the sensorimotor control system to the environment. Here, we test this hypothesis by measuring the rapid visuomotor feedback gains after subjects adapt to a variety of novel dynamics generated by a robotic manipulandum in three experiments. To probe the feedback gains, we measured the magnitude of the motor response to rapid shifts in the visual location of the hand during reaching. While the feedback gain magnitude remained similar over a larger than a fourfold increase in constant background load, the feedback gains scaled with increasing lateral resistance and increasing instability. The third experiment demonstrated that the feedback gains could also be independently tuned to perturbations to the left and right, depending on the lateral resistance, demonstrating the fractionation of feedback gains to environmental dynamics. Our results show that the sensorimotor control system regulates the gain of the feedback system as part of the adaptation process to novel dynamics, appropriately tuning them to the environment.

NEW & NOTEWORTHY Here, we test whether rapid visuomotor feedback responses are selectively tuned to the task dynamics. The responses do not exhibit gain scaling, but they do vary with the level and stability of task dynamics. Moreover, these feedback gains are independently tuned to perturbations to the left and right, depending on these dynamics. Our results demonstrate that the sensorimotor control system regulates the feedback gain as part of the adaptation process, tuning them appropriately to the environment.

Compensating for intersegmental dynamics across the shoulder, elbow, and wrist joints during feedforward and feedback control

Moving the arm is complicated by mechanical interactions that arise between limb segments. Such intersegmental dynamics cause torques applied at one joint to produce movement at multiple joints, and in turn, the only way to create single joint movement is by applying torques at multiple joints. We investigated whether the nervous system accounts for intersegmental limb dynamics across the shoulder, elbow, and wrist joints during self-initiated planar reaching and when countering external mechanical perturbations. Our first experiment tested whether the timing and amplitude of shoulder muscle activity account for interaction torques produced during single-joint elbow movements from different elbow initial orientations and over a range of movement speeds. We found that shoulder muscle activity reliably preceded movement onset and elbow agonist activity, and was scaled to compensate for the magnitude of interaction torques arising because of forearm rotation. Our second experiment tested whether elbow muscles compensate for interaction torques introduced by single-joint wrist movements. We found that elbow muscle activity preceded movement onset and wrist agonist muscle activity, and thus the nervous system predicted interaction torques arising because of hand rotation. Our third and fourth experiments tested whether shoulder muscles compensate for interaction torques introduced by different hand orientations during self-initiated elbow movements and to counter mechanical perturbations that caused pure elbow motion. We found that the nervous system predicted the amplitude and direction of interaction torques, appropriately scaling the amplitude of shoulder muscle activity during self-initiated elbow movements and rapid feedback control. Taken together, our results demonstrate that the nervous system robustly accounts for intersegmental dynamics and that the process is similar across the proximal to distal musculature of the arm as well as between feedforward (i.e., self-initiated) and feedback (i.e., reflexive) control.NEW & NOTEWORTHY Intersegmental dynamics complicate the mapping between applied joint torques and the resulting joint motions. We provide evidence that the nervous system robustly predicts these intersegmental limb dynamics across the shoulder, elbow, and wrist joints during reaching and when countering external perturbations.

Parallel Specification of Visuomotor Feedback Gains during Bimanual Reaching to Independent Goals

During goal-directed reaching, rapid visuomotor feedback processes enable the human motor system to quickly correct for errors in the trajectory of the hand that arise from motor noise and, in some cases, external perturbations. To date, these visuomotor responses, the gain of which is sensitive to features of the task and environment, have primarily been examined in the context of unimanual reaching movements toward a single target. However, many natural tasks involve moving both hands together, often to separate targets, such that errors can occur in parallel and at different spatial locations. Here, we examined the resource capacity of automatic visuomotor corrective mechanisms by comparing feedback gains during bimanual reaches, toward two targets, to feedback gains during unimanual reaches toward single targets. To investigate the sensitivity of the feedback gains and their relation to visual-spatial processing, we manipulated the widths of the targets and participants’ gaze location. We found that the gain of corrective responses to cursor displacements, while strongly modulated by target width and gaze position, were only slightly reduced during bimanual control. Our results show that automatic visuomotor corrective mechanisms can efficiently operate in parallel across multiple spatial locations.

Processes of believing: Where do they come from? What are they good for?

Despite the long scholarly discourse in Western theology and philosophy on religion, spirituality, and faith, explanations of what a belief and what believing is are still lacking. Recently, cognitive neuroscience research addressed the human capacity of believing. We present evidence suggesting that believing is a human brain function which results in probabilistic representations with attributes of personal meaning and value and thereby guides individuals’ behavior. We propose that the same mental processes operating on narratives and rituals constitute belief systems in individuals and social groups. Our theoretical model of believing is suited to account for secular and non-secular belief formation.

Processes of believing: Where do they come from? What are they good for?

Despite the long scholarly discourse in Western theology and philosophy on religion, spirituality, and faith, explanations of what a belief and what believing is are still lacking. Recently, cognitive neuroscience research addressed the human capacity of believing. We present evidence suggesting that believing is a human brain function which results in probabilistic representations with attributes of personal meaning and value and thereby guides individuals' behavior. We propose that the same mental processes operating on narratives and rituals constitute belief systems in individuals and social groups. Our theoretical model of believing is suited to account for secular and non-secular belief formation.

Long-Latency Feedback Coordinates Upper-Limb and Hand Muscles during Object Manipulation Tasks

Suppose that someone bumps into your arm at a party while you are holding a glass of wine. Motion of the disturbed arm will engage rapid and goal-directed feedback responses in the upper-limb. Although such responses can rapidly counter the perturbation, it is also clearly desirable not to destabilize your grasp and/or spill the wine. Here we investigated how healthy humans maintain a stable grasp following perturbations by using a paradigm that requires spatial tuning of the motor response dependent on the location of a virtual target. Our results highlight a synchronized expression of target-directed feedback in shoulder and hand muscles occurring at ∼60 ms. Considering that conduction delays are longer for the more distal hand muscles, these results suggest that target-directed responses in hand muscles were initiated before those for the shoulder muscles. These results show that long-latency feedback can coordinate upper limb and hand muscles during object manipulation tasks.

Computations underlying sensorimotor learning

The study of sensorimotor learning has a long history. With the advent of innovative techniques for studying learning at the behavioral and computational levels new insights have been gained in recent years into how the sensorimotor system acquires, retains, represents, retrieves and forgets sensorimotor tasks. In this review we highlight recent advances in the field of sensorimotor learning from a computational perspective. We focus on studies in which computational models are used to elucidate basic mechanisms underlying adaptation and skill acquisition in human behavior.