Better grip force control by attending to the controlled object: Evidence for direct force estimation from visual motion

Motion state-dependent motor learning based on explicit visual feedback has limited spatiotemporal properties compared with adaptation to physical perturbations

We recently showed that subjects can learn motion state-dependent changes to motor output (temporal force patterns) based on explicit visual feedback of the equivalent force field (i.e., without the physical perturbation). Here, we examined the spatiotemporal properties of this learning compared with learning based on physical perturbations. There were two human subject groups and two experimental paradigms. One group (n = 40) experienced physical perturbations (i.e., a velocity-dependent force field, vFF), whereas the second (n = 40) was given explicit visual feedback (EVF) of the force-velocity relationship. In the latter, subjects moved in force channels and we provided visual feedback of the lateral force exerted during the movement, as well as the required force pattern based on movement velocity. In the first paradigm (spatial generalization), following vFF or EVF training, generalization of learning was tested by requiring subjects to move to 14 untrained target locations (0° to ±135° around the trained location). In the second paradigm (temporal stability), following training, we examined the decay of learning over eight delay periods (0 to 90 s). Results showed that learning based on EVF did not generalize to untrained directions, whereas the generalization for the vFF was significant for targets ≤ 45° away. In addition, the decay of learning for the EVF group was significantly faster than the FF group (a time constant of 2.72 ± 1.74 s vs. 12.53 ± 11.83 s). Collectively, our results suggest that recalibrating motor output based on explicit motion state information, in contrast to physical disturbances, uses learning mechanisms with limited spatiotemporal properties.NEW & NOTEWORTHY Adjustment of motor output based on limb motion state information can be achieved based on explicit information or from physical perturbations. Here, we investigated the spatiotemporal characteristics of short-term motor learning to determine the properties of the respective learning mechanisms. Our results suggest that adjustments based on physical perturbations are more temporally stable and applied over a greater spatial range than the learning based on explicit visual feedback, suggesting largely separate learning mechanisms.

Vision-Based Human-Machine Interface for an Assistive Robotic Exoskeleton Glove

This paper presents a vision-based Human-Machine Interface (HMI) for an assistive exoskeleton glove, designed to incorporate force planning capabilities. While Electroencephalogram (EEG) and Electromyography (EMG)-based HMIs allow direct grasp force planning via user signals, voice and vision-based HMIs face limitations. In particular, two primary force planning methods encounter issues in these HMIs. First, traditional force optimization struggles with unfamiliar objects due to lack of object information. Second, the slip-grasp method faces a high failure rate due to inadequate initial grasp force. To address these challenges, this paper introduces a vision-based HMI to estimate the initial grasp forces of the target object. The initial grasp force estimation is performed based on the size and surface material of the target object. The experimental results demonstrate a grasp success rate of 87. 5%, marking significant improvements over the slip-grasp method (71.9%).

Motion state-dependent motor learning based on explicit visual feedback is quickly recalled, but is less stable than adaptation to physical perturbations

Recent studies have shown that adaptation to visual feedback perturbations during arm reaching movements involves implicit and explicit learning components. Evidence also suggests that explicit, intentional learning mechanisms are largely responsible for savings—a faster recalibration compared with initial training. However, the extent explicit learning mechanisms facilitate learning and early savings (i.e., the rapid recall of previous performance) for motion state-dependent learning is generally unknown. To address this question, we compared the early savings/recall achieved by two groups of human subjects. One experienced physical perturbations (a velocity-dependent force-field, vFF) to promote adaptation that is thought to be a largely implicit process. The second was only given visual feedback of the required force-velocity relationship; subjects moved in force channels and we provided visual feedback of the lateral force exerted during the movement, as well as the required force pattern based on the movement velocity. Thus, subjects were shown explicit information on the extent the applied temporal pattern of force matched the required velocity-dependent force profile if the force-field perturbation had been applied. After training, both groups experienced a decay and washout period, which was followed by a reexposure block to assess early savings/recall. Although decay was faster for the explicit visual feedback group, the single-trial recall was similar to the physical perturbation group. Thus, compared with visual feedback perturbations, conscious modification of motor output based on motion state-dependent feedback demonstrates rapid recall, but this adjustment is less stable than adaptation based on experiencing the multisensory errors that accompany physical perturbations.

NEW & NOTEWORTHY The extent explicit feedback facilitates motion state-dependent changes to motor output is largely unknown. Here, we examined motor adaptation for subjects that experienced physical perturbations and another that made adjustments based on explicit visual feedback information of the required force-velocity relationship. Our results suggest that adjustment of motor output can be based on explicit motion state-dependent information and demonstrates rapid recall, but this learning is less stable than adaptation based on physical perturbations to movement.

Evidence for an internal model of friction when controlling kinetic energy at impact to slide an object along a surface toward a target

Although the role of an internal model of gravity for the predictive control of the upper limbs is quite well established, evidence is lacking regarding an internal model of friction. In this study, 33 male and female human participants performed a striking movement (with the index finger) to slide a plastic cube-like object to a given target distance. The surface material (aluminum or balsa wood) on which the object slides, the surface slope (-10°, 0, or +10°) and the target distance (25 cm or 50 cm) varied across conditions, with ten successive trials in each condition. Analysis of the object speed at impact and spatial error suggests that: 1) the participants chose to impart a similar speed to the object in the first trial regardless of the surface material to facilitate the estimation of the coefficient of friction; 2) the movement is parameterized across repetitions to reduce spatial error; 3) an internal model of friction can be generalized when the slope changes. Biomechanical analysis showed interindividual variability in the recruitment of the upper limb segments and in the adjustment of finger speed at impact in order to transmit the kinetic energy required to slide the object to the target distance. In short, we provide evidence that the brain builds an internal model of friction that makes it possible to parametrically control a striking movement in order to regulate the amount of kinetic energy required to impart the appropriate initial speed to the object.

Anticipation and compensation for somatosensory deficits in object handling: evidence from a patient with large fiber sensory neuropathy

The purpose of this study was to determine the contributions of feedforward and feedback processes on grip force regulation and object orientation during functional manipulation tasks. One patient with massive somatosensory loss resulting from large fiber sensory neuropathy and 10 control participants were recruited. Three experiments were conducted: 1) perturbation to static holding; 2) discrete vertical movement; and 3) functional grasp and place. The availability of visual feedback was also manipulated to assess the nature of compensatory mechanisms. Results from experiment 1 indicated that both the deafferented patient and controls used anticipatory grip force adjustments before self-induced perturbation to static holding. The patient exhibited increased grip response time, but the magnitude of grip force adjustments remained correlated with perturbation forces in the self-induced and external perturbation conditions. In experiment 2, the patient applied peak grip force substantially in advance of maximum load force. Unlike controls, the patient's ability to regulate object orientation was impaired without visual feedback. In experiment 3, the duration of unloading, transport, and release phases were longer for the patient, with increased deviation of object orientation at phase transitions. These findings show that the deafferented patient uses distinct modes of anticipatory control according to task constraints and that responses to perturbations are mediated by alternative afferent information. The loss of somatosensory feedback thus appears to impair control of object orientation, whereas variation in the temporal organization of functional tasks may reflect strategies to mitigate object instability associated with changes in movement dynamics.NEW & NOTEWORTHY This study evaluates the effects of sensory neuropathy on the scaling and timing of grip force adjustments across different object handling tasks (i.e., holding, vertical movement, grasping, and placement). In particular, these results illustrate how novel anticipatory and online control processes emerge to compensate for the loss of somatosensory feedback. In addition, we provide new evidence on the role of somatosensory feedback for regulating object orientation during functional prehensile movement.

Distinct adaptation patterns between grip dynamics and arm kinematics when the body is upside-down

In humans, practically all movements are learnt and performed in a constant gravitational field. Yet, studies on arm movements and object manipulation in parabolic flight have highlighted very fast sensorimotor adaptations to altered gravity environments. Here, we wondered if the motor adjustments observed in those altered gravity environments could also be observed on Earth in a situation where the body is upside-down. To address this question, we asked participants to perform rhythmic arm movements in two different body postures (right-side-up and upside-down) while holding an object in precision grip. Analyses of grip-load force coordination and of movement kinematics revealed distinct adaptation patterns between grip and arm control. Grip force and load force were tightly synchronized from the first movements performed in upside-down posture, reflecting a malleable allocentric grip control. In contrast, velocity profiles showed a more progressive adaptation to the upside-down posture and reflected an egocentric planning of arm kinematics. In addition to suggesting distinct mechanisms between grip dynamics and arm kinematics for adaptation to novel contexts, these results also suggest the existence of general mechanisms underlying gravity-dependent motor adaptation that can be used for fast sensorimotor coordination across different postures on Earth and, incidentally, across different gravitational conditions in parabolic flights, in human centrifuges, or in Space.NEW & NOTEWORTHY During rhythmic arm movements performed in an upside-down posture, grip control adapted very quickly, but kinematics adaptation was more progressive. Our results suggest that grip control and movement kinematics planning might operate in different reference frames. Moreover, by comparing our results with previous results from parabolic flight studies, we propose that a common mechanism underlies adaptation to unfamiliar body postures and adaptation to altered gravity.

Seeing motion of controlled object improves grip timing in adults with autism spectrum condition: evidence for use of inverse dynamics in motor control

Previous studies (Haswell et al. in Nat Neurosci 12:970-972, 2009; Marko et al. in Brain J Neurol 138:784-797, 2015) reported that people with autism rely less on vision for learning to reach in a force field. This suggested a possibility that they have difficulties in extracting force information from visual motion signals, a process called inverse dynamics computation. Our recent study (Takamuku et al. in J Int Soc Autism Res 11:1062-1075, 2018) examined the ability of inverse computation with two perceptual tasks and found similar performances in typical and autistic adults. However, this tested the computation only in the context of sensory perception while it was possible that the suspected disability is specific to the motor domain. Here, in order to address the concern, we tested the use of inverse dynamics computation in the context of motor control by measuring changes in grip timing caused by seeing/not seeing a controlled object. The motion of the object was informative of its inertial force and typical participants improved their grip timing based on the visual feedback. Our interest was on whether the autism participants show the same improvement. While some autism participants showed atypical hand slowing when seeing the controlled object, we found no evidence of abnormalities in the inverse computation in our grip timing task or in a replication of the perceptual task. This suggests that the ability of inverse dynamics computation is preserved not only for sensory perception but also for motor control in adults with autism.

A review of the neurobiomechanical processes underlying secure gripping in object manipulation

O'SHEA, H. and S. J. Redmond. A review of the neurobiomechanical processes underlying secure gripping in object manipulation. NEUROSCI BIOBEHAV REV 286-300, 2021. Humans display skilful control over the objects they manipulate, so much so that biomimetic systems have yet to emulate this remarkable behaviour. Two key control processes are assumed to facilitate such dexterity: predictive cognitive-motor processes that guide manipulation procedures by anticipating action outcomes; and reactive sensorimotor processes that provide important error-based information for movement adaptation. Notwithstanding increased interdisciplinary research interest in object manipulation behaviour, the complexity of the perceptual-sensorimotor-cognitive processes involved and the theoretical divide regarding the fundamentality of control mean that the essential mechanisms underlying manipulative action remain undetermined. In this paper, following a detailed discussion of the theoretical and empirical bases for understanding human dexterous movement, we emphasise the role of tactile-related sensory events in secure object handling, and consider the contribution of certain biophysical and biomechanical phenomena. We aim to provide an integrated account of the current state-of-art in skilled human-object interaction that bridges the literature in neuroscience, cognitive psychology, and biophysics. We also propose novel directions for future research exploration in this area.

A Computational Internal Model to Quantify the Effect of Sensorimotor Augmentation on Motor Output

The aging process, as well as neurological disorders, causes a decline in sensorimotor functions, which can often bring degraded motor output. As a means of compensation for such sensorimotor deficiencies, sensorimotor augmentation has been actively investigated. Consequently, exoskeleton devices or functional electrical stimulation could augment the muscle activity, while textured surfaces or electrical nerve stimulations could augment the sensory feedback. However, it is not easy to precisely anticipate the effects of specific augmentation because sensory feedback and motor output interact with each other as a closed-loop operation via the central and peripheral nervous systems. A computational internal model can play a crucial role in anticipating such an effect of augmentation therapy on the motor outcome. Still, no existing internal sensorimotor loop model has been represented in a complete computational form facilitating the anticipation. This paper presents such a computational internal model, including numerical values representing the effect of sensorimotor augmentation. With the existing experimental results, the model performance was evaluated indirectly. The change of sensory gain affects motor output inversely, while the change of motor gain did not change or minimally affects the motor output.Clinical Relevance- The presented computational internal model will provide a simple and easy tool for clinicians to design therapeutic intervention using sensorimotor augmentation.