Control strategies in object manipulation tasks

Exploring motor skill acquisition in bimanual coordination: insights from navigating a novel maze task

In this study, we introduce a novel maze task designed to investigate naturalistic motor learning in bimanual coordination. We developed and validated an extended set of movement primitives tailored to capture the full spectrum of scenarios encountered in a maze game. Over a 3-day training period, we evaluated participants' performance using these primitives and a custom-developed software, enabling precise quantification of performance. Our methodology integrated the primitives with in-depth kinematic analyses and thorough thumb pressure assessments, charting the trajectory of participants' progression from novice to proficient stages. Results demonstrated consistent improvement in maze performance and significant adaptive changes in joint behaviors and strategic recalibrations in thumb pressure distribution. These findings highlight the central nervous system's adaptability in orchestrating sophisticated motor strategies and the crucial role of tactile feedback in precision tasks. The maze platform and setup emerge as a valuable foundation for future experiments, providing a tool for the exploration of motor learning and coordination dynamics. This research underscores the complexity of bimanual motor learning in naturalistic environments, enhancing our understanding of skill acquisition and task efficiency while emphasizing the necessity for further exploration and deeper investigation into these adaptive mechanisms.

Real-time vibrotactile pattern generation and identification using discrete event-driven feedback

This study assesses human identification of vibrotactile patterns by using real-time discrete event-driven feedback. Previously acquired force and bend sensor data from a robotic hand were used to predict movement-type (stationary, flexion, contact, extension, release) and object-type (no object, hard object, soft object) states by using decision tree (DT) algorithms implemented in a field-programmable gate array (FPGA). Six able-bodied humans performed a 2- and 3-step sequential pattern recognition task in which state transitions were signaled as vibrotactile feedback. The stimuli were generated according to predicted classes represented by two frequencies (F1: 80 Hz, F2: 180 Hz) and two magnitudes (M1: low, M2: high) calibrated psychophysically for each participant; and they were applied by two actuators (Haptuators) placed on upper arms. A soft/hard object was mapped to F1/F2; and manipulating it with low/high force was assigned to M1/M2 in the left actuator. On the other hand, flexion/extension movement was mapped to F1/F2 in the right actuator, with movement in air as M1 and during object manipulation as M2. DT algorithm performed better for the object-type (97%) than the movement-type (88%) classification in real time. Participants could recognize feedback associated with 14 discrete-event sequences with low-to-medium accuracy. The performance was higher (76 ± 9% recall, 76 ± 17% precision, 78 ± 4% accuracy) for recognizing any one event in the sequences. The results show that FPGA implementation of classification for discrete event-driven vibrotactile feedback can be feasible in haptic devices with additional cues in the physical context.

Unveiling the invisible: receivers use object weight cues for grip force planning in handover actions

Handover actions are part of our daily lives. Whether it is the milk carton at the breakfast table or tickets at the box office, we usually perform these joint actions without much conscious attention. The individual actions involved in handovers, that have already been studied intensively at the level of individual actions, are grasping, lifting, and transporting objects. Depending on the object's properties, actors must plan their execution in order to ensure smooth and efficient object transfer. Therefore, anticipatory grip force scaling is crucial. Grip forces are planned in anticipation using weight estimates based on experience or visual cues. This study aimed to investigate whether receivers are able to correctly estimate object weight by observing the giver's kinematics. For this purpose, handover actions were performed with 20 dyads, manipulating the participant role (giver/receiver) and varying the size and weight of the object. Due to the random presentation of the object weight and the absence of visual cues, the participants were unaware of the object weight from trial to trial. Kinematics were recorded with a motion tracking system and grip forces were recorded with customized test objects. Peak grip force rates were used as a measure of anticipated object weight. Results showed that receiver kinematics are significantly affected by object weight. The peak grip force rates showed that receivers anticipate object weight, but givers not. This supports the hypothesis that receivers obtain information about the object weight by observing giver's kinematics and integrating this information into their own action execution.

Simple visual stimuli are sufficient to drive responses in action observation and execution neurons in macaque ventral premotor cortex

Neurons responding during action execution and action observation were discovered in the ventral premotor cortex 3 decades ago. However, the visual features that drive the responses of action observation/execution neurons (AOENs) have not been revealed at present. We investigated the neural responses of AOENs in ventral premotor area F5c of 4 macaques during the observation of action videos and crucial control stimuli. The large majority of AOENs showed highly phasic responses during the action videos, with a preference for the moment that the hand made contact with the object. They also responded to an abstract shape moving towards but not interacting with an object, even when the shape moved on a scrambled background, implying that most AOENs in F5c do not require the perception of causality or a meaningful action. Additionally, the majority of AOENs responded to static frames of the videos. Our findings show that very elementary stimuli, even without a grasping context, are sufficient to drive responses in F5c AOENs.

Fast Feedback Responses to Categorical Sensorimotor Errors That Do Not Indicate Error Magnitude Are Optimized Based on Short- and Long-Term Memory

Skilled motor performance depends critically on rapid corrective responses that act to preserve the goal of the movement in the face of perturbations. Although it is well established that the gain of corrective responses elicited while reaching toward objects adapts to different contexts, little is known about the adaptability of corrective responses supporting the manipulation of objects after they are grasped. Here, we investigated the adaptability of the corrective response elicited when an object being lifted is heavier than expected and fails to lift off when predicted. This response involves a monotonic increase in vertical load force triggered, within ∼90 ms, by the absence of expected sensory feedback signaling lift off and terminated when actual lift off occurs. Critically, because the actual weight of the object cannot be directly sensed at the moment the object fails to lift off, any adaptation of the corrective response would have to be based on memory from previous lifts. We show that when humans, including men and women, repeatedly lift an object that on occasional catch trials increases from a baseline weight to a fixed heavier weight, they scale the gain of the response (i.e., the rate of force increase) to the heavier weight within two to three catch trials. We also show that the gain of the response scales, on the first catch trial, with the baseline weight of the object. Thus, the gain of the lifting response can be adapted by both short- and long-term experience. Finally, we demonstrate that this adaptation preserves the efficacy of the response across contexts.SIGNIFICANCE STATEMENT Here, we present the first investigation of the adaptability of the corrective lifting response elicited when an object is heavier than expected and fails to lift off when predicted. A striking feature of the response, which is driven by a sensory prediction error arising from the absence of expected sensory feedback, is that the magnitude of the error is unknown. That is, the motor system only receives a categorical error indicating that the object is heavier than expected but not its actual weight. Although the error magnitude is not known at the moment the response is elicited, we show that the response can be scaled to predictions of error magnitude based on both recent and long-term memories.

Individual Muscle Force Estimation in the Human Forearm Using Multi-Muscle MR Elastography (MM-MRE)

OBJECTIVE

To establish the sensitivity of magnetic resonance elastography (MRE) to active muscle contraction in multiple muscles of the forearm.

METHODS

We combined MRE of forearm muscles with an MRI-compatible device, the MREbot, to simultaneously measure the mechanical properties of tissues in the forearm and the torque applied by the wrist joint during isometric tasks. We measured shear wave speed of thirteen forearm muscles via MRE in a series of contractile states and wrist postures and fit these outputs to a force estimation algorithm based on a musculoskeletal model.

RESULTS

Shear wave speed changed significantly upon several factors, including whether the muscle was recruited as an agonist or antagonist (p = 0.0019), torque amplitude (p = <0.0001), and wrist posture (p = 0.0002). Shear wave speed increased significantly during both agonist (p = <0.0001) and antagonist (p = 0.0448) contraction. Additionally, there was a greater increase in shear wave speed at greater levels of loading. The variations due to these factors indicate the sensitivity to functional loading of muscle. Under the assumption of a quadratic relationship between shear wave speed and muscle force, MRE measurements accounted for an average of 70% of the variance in the measured joint torque.

CONCLUSION

This study shows the ability of MM-MRE to capture variations in individual muscle shear wave speed due to muscle activation and presents a method to estimate individual muscle force through MM-MRE derived measurements of shear wave speed.

SIGNIFICANCE

MM-MRE could be used to establish normal and abnormal muscle co-contraction patterns in muscles of the forearm controlling hand and wrist function.

Enhancing Force Control of Prosthetic Controller for Hand Prosthesis by Mimicking Biological Properties

Prosthetic hands are frequently rejected due to frustrations in daily uses. By adopting principles of human neuromuscular control, it could potentially achieve human-like compliance in hand functions, thereby improving functionality in prosthetic hand. Previous studies have confirmed the feasibility of real-time emulation of neuromuscular reflex for prosthetic control. This study further to explore the effect of feedforward electromyograph (EMG) decoding and proprioception on the biomimetic controller. The biomimetic controller included a feedforward Bayesian model for decoding alpha motor commands from stump EMG, a muscle model, and a closed-loop component with a model of muscle spindle modified with spiking afferents. Real-time control was enabled by neuromorphic hardware to accelerate evaluation of biologically inspired models. This allows us to investigate which aspects in the controller could benefit from biological properties for improvements on force control performance. 3 non-disabled and 3 amputee subjects were recruited to conduct a "press-without-break" task, subjects were required to press a transducer till the pressure stabilized in an expected range without breaking the virtual object. We tested whether introducing more complex but biomimetic models could enhance the task performance. Data showed that when replacing proportional feedback with the neuromorphic spindle, success rates of amputees increased by 12.2% and failures due to breakage decreased by 26.3%. More prominently, success rates increased by 55.5% and failures decreased by 79.3% when replacing a linear model of EMG with the Bayesian model in the feedforward EMG processing. Results suggest that mimicking biological properties in feedback and feedforward control may improve the manipulation of objects by amputees using prosthetic hands. Clinical and Translational Impact Statement: This control approach may eventually assist amputees to perform fine force control when using prosthetic hands, thereby improving the motor performance of amputees. It highlights the promising potential of the biomimetic controller integrating biological properties implemented on neuromorphic models as a viable approach for clinical application in prosthetic hands.

Memory, perceptual, and motor costs affect the strength of categorical encoding during motor learning of object properties

Nearly all tasks of daily life involve skilled object manipulation, and successful manipulation requires knowledge of object dynamics. We recently developed a motor learning paradigm that reveals the categorical organization of motor memories of object dynamics. When participants repeatedly lift a constant-density "family" of cylindrical objects that vary in size, and then an outlier object with a greater density is interleaved into the sequence of lifts, they often fail to learn the weight of the outlier, persistently treating it as a family member despite repeated errors. Here we examine eight factors (Similarity, Cardinality, Frequency, History, Structure, Stochasticity, Persistence, and Time Pressure) that could influence the formation and retrieval of category representations in the outlier paradigm. In our web-based task, participants (N = 240) anticipated object weights by stretching a virtual spring attached to the top of each object. Using Bayesian t-tests, we analyze the relative impact of each manipulated factor on categorical encoding (strengthen, weaken, or no effect). Our results suggest that category representations of object weight are automatic, rigid, and linear and, as a consequence, the key determinant of whether an outlier is encoded as a member of the family is its discriminability from the family members.

Sensorimotor control of object manipulation following middle cerebral artery (MCA) stroke

Methods for assessing the loss of hand function post-stroke examine limited aspects of motor performance and are not sensitive to subtle changes that can cause deficits in everyday object manipulation tasks. Efficiently lifting an object entails a prediction of required forces based on intrinsic features of the object (sensorimotor integration), short-term updates in the forces required to lift objects that are poorly predicted (sensorimotor memory), as well as the ability to modulate distal fingertip forces, which are not measured by existing assessment tools used in clinics for both diagnostic and rehabilitative purposes. The presented research examined these three components of skilled object manipulation in 60 chronic, unilateral middle cerebral artery stroke participants. Performance was compared to age-matched control participants, and linear regressions were used to predict performance based on clinical scores. Most post-stroke participants performed below control levels in at least one of the tasks. Post-stroke participants presented with combinations of deficits in each of the tasks performed, regardless of the hemisphere damaged by the stroke. Surprisingly, the ability to modulate distal forces was impaired in those patients with damage ipsilateral (right hemisphere) to the hand being used. Sensorimotor integration was also impaired in patients with right hemisphere damage, though they performed at control levels in later lifts, whereas left-hemisphere-damaged patients did not. Lastly, during a task requiring sensorimotor memory, neither patient group performed outside of control ranges on initial lifts, with patients with right hemisphere damage showing impaired performance in later lifts suggesting they were unable to learn the mapping novel mapping of color and mass of the objects. The presented research demonstrates unilateral MCA stroke patients can have deficits in one or more components required for the successful manipulation of hand-held objects and that skillful object lifting requires intact bilateral systems. Further, this information may be used in future studies to aid efforts that target rehabilitation regimens to a stroke survivor's specific pattern of deficits.

Spatiotemporal Modeling of Grip Forces Captures Proficiency in Manual Robot Control

New technologies for monitoring grip forces during hand and finger movements in non-standard task contexts have provided unprecedented functional insights into somatosensory cognition. Somatosensory cognition is the basis of our ability to manipulate and transform objects of the physical world and to grasp them with the right amount of force. In previous work, the wireless tracking of grip-force signals recorded from biosensors in the palm of the human hand has permitted us to unravel some of the functional synergies that underlie perceptual and motor learning under conditions of non-standard and essentially unreliable sensory input. This paper builds on this previous work and discusses further, functionally motivated, analyses of individual grip-force data in manual robot control. Grip forces were recorded from various loci in the dominant and non-dominant hands of individuals with wearable wireless sensor technology. Statistical analyses bring to the fore skill-specific temporal variations in thousands of grip forces of a complete novice and a highly proficient expert in manual robot control. A brain-inspired neural network model that uses the output metric of a self-organizing pap with unsupervised winner-take-all learning was run on the sensor output from both hands of each user. The neural network metric expresses the difference between an input representation and its model representation at any given moment in time and reliably captures the differences between novice and expert performance in terms of grip-force variability.Functionally motivated spatiotemporal analysis of individual average grip forces, computed for time windows of constant size in the output of a restricted amount of task-relevant sensors in the dominant (preferred) hand, reveal finger-specific synergies reflecting robotic task skill. The analyses lead the way towards grip-force monitoring in real time. This will permit tracking task skill evolution in trainees, or identify individual proficiency levels in human robot-interaction, which represents unprecedented challenges for perceptual and motor adaptation in environmental contexts of high sensory uncertainty. Cross-disciplinary insights from systems neuroscience and cognitive behavioral science, and the predictive modeling of operator skills using parsimonious Artificial Intelligence (AI), will contribute towards improving the outcome of new types of surgery, in particular the single-port approaches such as NOTES (Natural Orifice Transluminal Endoscopic Surgery) and SILS (Single-Incision Laparoscopic Surgery).

Grip force as a functional window to somatosensory cognition

Analysis of grip force signals tailored to hand and finger movement evolution and changes in grip force control during task execution provide unprecedented functional insight into somatosensory cognition. Somatosensory cognition is the basis of our ability to act upon and to transform the physical world around us, to recognize objects on the basis of touch alone, and to grasp them with the right amount of force for lifting and manipulating them. Recent technology has permitted the wireless monitoring of grip force signals recorded from biosensors in the palm of the human hand to track and trace human grip forces deployed in cognitive tasks executed under conditions of variable sensory (visual, auditory) input. Non-invasive multi-finger grip force sensor technology can be exploited to explore functional interactions between somatosensory brain mechanisms and motor control, in particular during learning a cognitive task where the planning and strategic execution of hand movements is essential. Sensorial and cognitive processes underlying manual skills and/or hand-specific (dominant versus non-dominant hand) behaviors can be studied in a variety of contexts by probing selected measurement loci in the fingers and palm of the human hand. Thousands of sensor data recorded from multiple spatial locations can be approached statistically to breathe functional sense into the forces measured under specific task constraints. Grip force patterns in individual performance profiling may reveal the evolution of grip force control as a direct result of cognitive changes during task learning. Grip forces can be functionally mapped to from-global-to-local coding principles in brain networks governing somatosensory processes for motor control in cognitive tasks leading to a specific task expertise or skill. Under the light of a comprehensive overview of recent discoveries into the functional significance of human grip force variations, perspectives for future studies in cognition, in particular the cognitive control of strategic and task relevant hand movements in complex real-world precision task, are pointed out.

Masking Vibrations and Contact Force Affect the Discrimination of Slip Motion Speed in Touch

Multiple cues contribute to the discrimination of slip motion speed by touch. In our previous article, we demonstrated that masking vibrations at various frequencies impaired the discrimination of speed. In this article, we extended the previous results to evaluate this phenomenon on a smooth glass surface, and for different values of contact force and duration of the masking stimulus. Speed discrimination was significantly impaired by masking vibrations at high but not at low contact force. Furthermore, a short pulse of masking vibrations at motion onset produced a similar effect as the long masking stimulus, delivered throughout slip motion duration. This last result suggests that mechanical events at motion onset provide important cues to the discrimination of speed.

Sequential contact-based adaptive grasping for robotic hands

This paper proposes a novel type of grasping strategy that draws inspiration from the role of touch and the importance of wrist motions in human grasping. The proposed algorithm, which we call Sequential Contact-based Adaptive Grasping, can be used to reactively modify a given grasp plan according to contacts arising between the hand and the object. This technique, based on a systematic constraint categorization and an iterative task inversion procedure, is shown to lead to synchronized motions of the fingers and the wrist, as it can be observed in humans, and to increase grasp success rate by substantially mitigating the relevant problems of object slippage during hand closure and of uncertainties caused by the environment and by the perception system. After describing the grasping problem in its quasi-static aspects, the algorithm is derived and discussed with some simple simulations. The proposed method is general as it can be applied to different kinds of robotic hands. It refines a priori defined grasp plans and significantly reduces their accuracy requirements by relying only on a forward kinematic model and elementary contact information. The efficacy of our approach is confirmed by experimental results of tests performed on a collaborative robot manipulator equipped with a state-of-the-art underactuated soft hand.

Spatial Instability during Precision Grip–Lift in Children with Poor Manual Dexterity

Although children with developmental coordination disorder (DCD) show impaired precision grip control due to a sensory-motor integration deficit, their spatial instability (such as changes in force direction and object roll during a precision grip task) is unclear. Herein, we investigated the spatial instability in the precision grip force control of children with poor manual dexterity. We divided 66 school-aged children who performed a precision gripping and lifting of heavy- or lightweight objects into those with low manual dexterity (n = 11) and those with high manual dexterity (n = 55) as revealed by the Movement Assessment Battery for Children (2nd edition). The group and weight effects were then determined. The results revealed that the total trajectory lengths of the center of pressure (COP) were longer in the lightweight object data of the children in the low-manual-dexterity group and were related to the children’s grip force. The low-manual-dexterity group also showed a shifted COP position from the center of the object in the medial–lateral direction and in the object roll regardless of the object’s weight; these were closely related in both weights’ tests. These results demonstrated that children with poor manual dexterity show spatial instability and different adaptations to the weight of objects during a precision grip task. Further studies are needed to determine whether these findings would be replicated in children with a diagnosis of DCD.

Effect of Apical Microsurgery on Force Regulation of Incisor Teeth during Unpredictable Force Control Task

BACKGROUND

Apical microsurgery (AMS) involves removal of the root-end which can affect the force regulation of teeth.

OBJECTIVE

To investigate the force regulation of incisor teeth treated with AMS during the unpredictable force control task in comparison to their contralateral teeth with complete root apices, in humans.

METHODS

Fifteen eligible participants (8 women and 7 men; mean age 52.9 ± SD 4.4 years) performed a standardized unpredictable force control task which involved pulling and holding a force transducer with AMS-treated incisors and its contralateral control teeth (n =30 teeth). A series of four load masses: 100, 200, 50, and 300 gm were attached to the force transducer through a string in an unpredictable manner. The force profile obtained was divided into initial and later time-segments. The peak force and peak force rate during the initial time-segment, and the holding force and coefficient of variability during the later time-segments were calculated and compared by the repeated measures analysis of variance.

RESULTS

During the initial time-segment, the peak force and peak force rate were significantly lower in the AMS-treated teeth than in the controls (P = 0.001, P = 0.013, respectively). However, during the later time-segment, no significant differences in the holding force nor the coefficient of variability were observed between the AMS-treated teeth and their controls (P = 0.755, P = 0.213, respectively).

CONCLUSION

In contrast to incisors with complete normal root apices, AMS-treated incisors do not show robust changes in force regulation.

Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks

Humans learn movements naturally, but it takes a lot of time and training to achieve expert performance in motor skills. In this review, we show how modern technologies can support people in learning new motor skills. First, we introduce important concepts in motor control, motor learning and motor skill learning. We also give an overview about the rapid expansion of machine learning algorithms and sensor technologies for human motion analysis. The integration between motor learning principles, machine learning algorithms and recent sensor technologies has the potential to develop AI-guided assistance systems for motor skill training. We give our perspective on this integration of different fields to transition from motor learning research in laboratory settings to real world environments and real world motor tasks and propose a stepwise approach to facilitate this transition.

Alpha-band activity in parietofrontal cortex predicts future availability of vibrotactile feedback in prosthesis use

Prosthesis disuse and abandonment is an ongoing issue in upper-limb amputation. In addition to lost structural and motor function, amputation also results in decreased task-specific sensory information. One proposed remedy is augmenting somatosensory information using vibrotactile feedback to provide tactile feedback of grasping objects. While the role of frontal and parietal areas in motor tasks is well established, the neural and kinematic effects of this augmented vibrotactile feedback remain in question. In this study, we sought to understand the neurobehavioral effects of providing augmented feedback during a reach-grasp-transport task. Ten persons with sound limbs performed a motor task while wearing a prosthesis simulator with and without vibrotactile feedback. We hypothesized that providing vibrotactile feedback during prosthesis use would increase activity in frontal and parietal areas and improve grasp-related behavior. Results show that anticipation of upcoming vibrotactile feedback may be encoded in motor and parietal areas during the reach-to-grasp phase of the task. While grasp aperture is unaffected by vibrotactile feedback, the availability of vibrotactile feedback does lead to a reduction in velocity during object transport. These results help shed light on how engineered feedback is utilized by prostheses users and provide methodologies for further assessment in advanced prosthetics research.

Marine Robotics for Deep-Sea Specimen Collection: A Taxonomy of Underwater Manipulative Actions

In order to develop a gripping system or control strategy that improves scientific sampling procedures, knowledge of the process and the consequent definition of requirements is fundamental. Nevertheless, factors influencing sampling procedures have not been extensively described, and selected strategies mostly depend on pilots' and researchers' experience. We interviewed 17 researchers and remotely operated vehicle (ROV) technical operators, through a formal questionnaire or in-person interviews, to collect evidence of sampling procedures based on their direct field experience. We methodologically analyzed sampling procedures to extract single basic actions (called atomic manipulations). Available equipment, environment and species-specific features strongly influenced the manipulative choices. We identified a list of functional and technical requirements for the development of novel end-effectors for marine sampling. Our results indicate that the unstructured and highly variable deep-sea environment requires a versatile system, capable of robust interactions with hard surfaces such as pushing or scraping, precise tuning of gripping force for tasks such as pulling delicate organisms away from hard and soft substrates, and rigid holding, as well as a mechanism for rapidly switching among external tools.

Motor memories of object dynamics are categorically organized

The ability to predict the dynamics of objects, linking applied force to motion, underlies our capacity to perform many of the tasks we carry out on a daily basis. Thus, a fundamental question is how the dynamics of the myriad objects we interact with are organized in memory. Using a custom-built three-dimensional robotic interface that allowed us to simulate objects of varying appearance and weight, we examined how participants learned the weights of sets of objects that they repeatedly lifted. We find strong support for the novel hypothesis that motor memories of object dynamics are organized categorically, in terms of families, based on covariation in their visual and mechanical properties. A striking prediction of this hypothesis, supported by our findings and not predicted by standard associative map models, is that outlier objects with weights that deviate from the family-predicted weight will never be learned despite causing repeated lifting errors.

Multivariate Analysis of Electrophysiological Signals Reveals the Time Course of Precision Grasps Programs: Evidence for Nonhierarchical Evolution of Grasp Control

Current understanding of the neural processes underlying human grasping suggests that grasp computations involve gradients of higher to lower level representations and, relatedly, visual to motor processes. However, it is unclear whether these processes evolve in a strictly canonical manner from higher to intermediate and to lower levels given that this knowledge importantly relies on functional imaging, which lacks temporal resolution. To examine grasping in fine temporal detail here we used multivariate EEG analysis. We asked participants to grasp objects while controlling the time at which crucial elements of grasp programs were specified. We first specified the orientation with which participants should grasp objects, and only after a delay we instructed participants about which effector to use to grasp, either the right or the left hand. We also asked participants to grasp with both hands because bimanual and left-hand grasping share intermediate-level grasp representations. We observed that grasp programs evolved in a canonical manner from visual representations, which were independent of effectors to motor representations that distinguished between effectors. However, we found that intermediate representations of effectors that partially distinguished between effectors arose after representations that distinguished among all effector types. Our results show that grasp computations do not proceed in a strictly hierarchically canonical fashion, highlighting the importance of the fine temporal resolution of EEG for a comprehensive understanding of human grasp control.SIGNIFICANCE STATEMENT A long-standing assumption of the grasp computations is that grasp representations progress from higher to lower level control in a regular, or canonical, fashion. Here, we combined EEG and multivariate pattern analysis to characterize the temporal dynamics of grasp representations while participants viewed objects and were subsequently cued to execute an unimanual or bimanual grasp. Interrogation of the temporal dynamics revealed that lower level effector representations emerged before intermediate levels of grasp representations, thereby suggesting a partially noncanonical progression from higher to lower and then to intermediate level grasp control.

The interaction between motion and texture in the sense of touch

Besides providing information on elementary properties of objects, like texture, roughness, and softness, the sense of touch is also important in building a representation of object movement and the movement of our hands. Neural and behavioral studies shed light on the mechanisms and limits of our sense of touch in the perception of texture and motion, and of its role in the control of movement of our hands. The interplay between the geometrical and mechanical properties of the touched objects, such as shape and texture, the movement of the hand exploring the object, and the motion felt by touch, will be discussed in this article. Interestingly, the interaction between motion and textures can generate perceptual illusions in touch. For example, the orientation and the spacing of the texture elements on a static surface induces the illusion of surface motion when we move our hand on it or can elicit the perception of a curved trajectory during sliding, straight hand movements. In this work we present a multiperspective view that encompasses both the perceptual and the motor aspects, as well as the response of peripheral and central nerve structures, to analyze and better understand the complex mechanisms underpinning the tactile representation of texture and motion. Such a better understanding of the spatiotemporal features of the tactile stimulus can reveal novel transdisciplinary applications in neuroscience and haptics.

Flexible constraint hierarchy during the visual encoding of tool-object interactions

Tools and objects are associated with numerous action possibilities that are reduced depending on the task-related internal and external constraints presented to the observer. Action hierarchies propose that goals represent higher levels of the hierarchy while kinematic patterns represent lower levels of the hierarchy. Prior work suggests that tool-object perception is heavily influenced by grasp and action context. The current study sought to evaluate whether the presence of action hierarchy can be perceptually identified using eye tracking during tool-object observation. We hypothesize that gaze patterns will reveal a perceptual hierarchy based on the observed task context and grasp constraints. Participants viewed tool-objects scenes with two types of constraints: task-context and grasp constraints. Task-context constraints consisted of correct (e.g., frying pan-spatula) and incorrect tool-object pairings (e.g., stapler-spatula). Grasp constraints involved modified tool orientations, which requires participants to understand how initially awkward grasp postures can help achieve the task. The visual scene contained three areas of interests (AOIs): the object, the functional tool-end (e.g., spoon handle) and the manipulative tool-end (e.g., spoon bowl). Results revealed two distinct processes based on stimuli constraints. Goal-oriented encoding, the attentional bias towards the object and manipulative tool-end, was demonstrated when grasp did not lead to meaningful tool-use. In images where grasp postures were critical to action performance, attentional bias was primarily between the object and functional tool-end, which suggests means-related encoding of the graspable properties of the object. This study expands from previous work and demonstrates a flexible constraint hierarchy depending on the observed task constraints.

Development of eye-hand coordination in typically developing children and adolescents assessed using a reach-to-grasp sequencing task

Eye-hand coordination is required to accurately perform daily activities that involve reaching, grasping and manipulating objects. Studies using aiming, grasping or sequencing tasks have shown a stereotypical temporal coupling pattern where the eyes are directed to the object in advance of the hand movement, which may facilitate the planning and execution required for reaching. While the temporal coordination between the ocular and manual systems has been extensively investigated in adults, relatively little is known about the typical development of eye-hand coordination. Therefore, the current study addressed an important knowledge gap by characterizing the profile of eye-hand coupling in typically developing school-age children (n = 57) and in a cohort of adults (n = 30). Eye and hand movements were recorded concurrently during the performance of a bead threading task which consists of four distinct movements: reach to bead, grasp, reach to needle, and thread. Results showed a moderate to high correlation between eye and hand latencies in children and adults, supporting that both movements were planned in parallel. Eye and reach latencies, latency differences, and dwell time during grasping and threading, showed significant age-related differences, suggesting eye-hand coupling becomes more efficient in adolescence. Furthermore, visual acuity, stereoacuity and accommodative facility were also found to be associated with the efficiency of eye-hand coordination in children. Results from this study can serve as reference values when examining eye and hand movement during the performance of fine motor skills in children with neurodevelopmental disorders.

End in view: Joint end-state comfort depends on gaze and extraversion

This study investigated how humans adapt to a partner's movement in a joint pick-and-place task and examined the role of gaze behavior and personality traits in adapting to a partner. Two participants sitting side-by-side transported a cup from one end of a table to the other. The participant sitting on the left (the agent) moved the cup to an intermediate position from where the participant sitting on the right (the partner) transported it to a goal position with varying orientations. Hand, finger, cup movements and gaze behavior were recorded synchronously via motion tracking and portable eye tracking devices. Results showed interindividual differences in the extent of the agents' motor adaptation to the joint action goal, which were accompanied by differences in gaze patterns. The longer agents directed their gaze to a cue indicating the goal orientation, the more they adapted the rotation of the cup's handle when placing it at the intermediate position. Personality trait assessment showed that higher extraverted tendencies to strive for social potency went along with more adaptation to the joint goal. These results indicate that agents who consider their partner's end-state comfort use their gaze to gather more information about the joint action goal compared to agents who do not. Moreover, the disposition to enjoy leadership and make decisions in interpersonal situations seems to play a role in determining who adapts to a partner's task in joint action.

Action Selection and Execution in Everyday Activities: A Cognitive Robotics and Situation Model Perspective

We examine the mechanisms required to handle everyday activities from the standpoint of cognitive robotics, distinguishing activities on the basis of complexity and transparency. Task complexity (simple or complex) reflects the intrinsic nature of a task, while task transparency (easy or difficult) reflects an agent's ability to identify a solution strategy in a given task. We show how the CRAM cognitive architecture allows a robot to carry out simple and complex activities such as laying a table for a meal and loading a dishwasher afterward. It achieves this by using generalized action plans that exploit reasoning with modular, composable knowledge chunks representing general knowledge to transform underdetermined everyday action requests into motion plans that successfully accomplish the required task. Noting that CRAM does not yet have the ability to deal with difficult activities, we leverage insights from the situation model perspective on the cognitive mechanisms underlying flexible context-sensitive behavior with a view to extending CRAM to overcome this deficit.

Human Somatosensory Cortex Is Modulated during Motor Planning

Recent data and motor control theory argues that movement planning involves preparing the neural state of primary motor cortex (M1) for forthcoming action execution. Theories related to internal models, feedback control, and predictive coding also emphasize the importance of sensory prediction (and processing) before (and during) the movement itself, explaining why motor-related deficits can arise from damage to primary somatosensory cortex (S1). Motivated by this work, here we examined whether motor planning, in addition to changing the neural state of M1, changes the neural state of S1, preparing it for the sensory feedback that arises during action. We tested this idea in two human functional MRI studies (N = 31, 16 females) involving delayed object manipulation tasks, focusing our analysis on premovement activity patterns in M1 and S1. We found that the motor effector to be used in the upcoming action could be decoded, well before movement, from neural activity in M1 in both studies. Critically, we found that this effector information was also present, well before movement, in S1. In particular, we found that the encoding of effector information in area 3b (S1 proper) was linked to the contralateral hand, similarly to that found in M1, whereas in areas 1 and 2 this encoding was present in both the contralateral and ipsilateral hemispheres. Together, these findings suggest that motor planning not only prepares the motor system for movement but also changes the neural state of the somatosensory system, presumably allowing it to anticipate the sensory information received during movement.SIGNIFICANCE STATEMENT Whereas recent work on motor cortex has emphasized the critical role of movement planning in preparing neural activity for movement generation, it has not investigated the extent to which planning also modulates the activity in the adjacent primary somatosensory cortex. This reflects a key gap in knowledge, given that recent motor control theories emphasize the importance of sensory feedback processing in effective movement generation. Here, we find through a convergence of experiments and analyses, that the planning of object manipulation tasks, in addition to modulating the activity in the motor cortex, changes the state of neural activity in different subfields of the human S1. We suggest that this modulation prepares the S1 for the sensory information it will receive during action execution.

Anticipation and compensation for somatosensory deficits in object handling: evidence from a patient with large fiber sensory neuropathy

The purpose of this study was to determine the contributions of feedforward and feedback processes on grip force regulation and object orientation during functional manipulation tasks. One patient with massive somatosensory loss resulting from large fiber sensory neuropathy and 10 control participants were recruited. Three experiments were conducted: 1) perturbation to static holding; 2) discrete vertical movement; and 3) functional grasp and place. The availability of visual feedback was also manipulated to assess the nature of compensatory mechanisms. Results from experiment 1 indicated that both the deafferented patient and controls used anticipatory grip force adjustments before self-induced perturbation to static holding. The patient exhibited increased grip response time, but the magnitude of grip force adjustments remained correlated with perturbation forces in the self-induced and external perturbation conditions. In experiment 2, the patient applied peak grip force substantially in advance of maximum load force. Unlike controls, the patient's ability to regulate object orientation was impaired without visual feedback. In experiment 3, the duration of unloading, transport, and release phases were longer for the patient, with increased deviation of object orientation at phase transitions. These findings show that the deafferented patient uses distinct modes of anticipatory control according to task constraints and that responses to perturbations are mediated by alternative afferent information. The loss of somatosensory feedback thus appears to impair control of object orientation, whereas variation in the temporal organization of functional tasks may reflect strategies to mitigate object instability associated with changes in movement dynamics.NEW & NOTEWORTHY This study evaluates the effects of sensory neuropathy on the scaling and timing of grip force adjustments across different object handling tasks (i.e., holding, vertical movement, grasping, and placement). In particular, these results illustrate how novel anticipatory and online control processes emerge to compensate for the loss of somatosensory feedback. In addition, we provide new evidence on the role of somatosensory feedback for regulating object orientation during functional prehensile movement.

Grip Force Control During Virtual Interaction With Deformable and Rigid Objects Via a Haptic Gripper

Most studies of grip force control focus on the manipulation of rigid objects. However, in virtual and teleoperation applications, objects are often elastic in the pinch degree-of-freedom, and are manipulated using a handle that presents haptic feedback to the user. When designing controllers for haptic grippers, it is crucial to understand how grip force is controlled when manipulating rigid and deformable objects. Here, we used a virtual teleoperation setup with a haptic gripper interface to investigate grip force control in virtual environments. Ten participants lifted virtual objects and performed vertical, cyclic motions using a haptic gripper. We manipulated the control signal to the virtual interface, the haptic properties of the gripper, and the visual properties of the virtual objects to test their effect on the grip force control. We found that participants modulated their grip force as a function of the anticipated load force in all of the experimental conditions. The control signal and properties of the haptic gripper, but not the visual properties of the objects, affected the baseline and the extent of the grip force modulation. These results can provide design guidelines for haptic grippers that elicit natural grasping in virtual and bilateral teleoperation applications.

Learning Transferable Push Manipulation Skills in Novel Contexts

This paper is concerned with learning transferable forward models for push manipulation that can be applying to novel contexts and how to improve the quality of prediction when critical information is available. We propose to learn a parametric internal model for push interactions that, similar for humans, enables a robot to predict the outcome of a physical interaction even in novel contexts. Given a desired push action, humans are capable to identify where to place their finger on a new object so to produce a predictable motion of the object. We achieve the same behaviour by factorising the learning into two parts. First, we learn a set of local contact models to represent the geometrical relations between the robot pusher, the object, and the environment. Then we learn a set of parametric local motion models to predict how these contacts change throughout a push. The set of contact and motion models represent our internal model. By adjusting the shapes of the distributions over the physical parameters, we modify the internal model's response. Uniform distributions yield to coarse estimates when no information is available about the novel context. We call this an unbiased predictor. A more accurate predictor can be learned for a specific environment/object pair (e.g., low friction/high mass), called a biased predictor. The effectiveness of our approach is demonstrated in a simulated environment in which a Pioneer 3-DX robot equipped with a bumper needs to predict a push outcome for an object in a novel context, and we support those results with a proof of concept on a real robot. We train on two objects (a cube and a cylinder) for a total of 24,000 pushes in various conditions, and test on six objects encompassing a variety of shapes, sizes, and physical parameters for a total of 14,400 predicted push outcomes. Our experimental results show that both biased and unbiased predictors can reliably produce predictions in line with the outcomes of a carefully tuned physics simulator.

Motor Planning Modulates Neural Activity Patterns in Early Human Auditory Cortex

It is well established that movement planning recruits motor-related cortical brain areas in preparation for the forthcoming action. Given that an integral component to the control of action is the processing of sensory information throughout movement, we predicted that movement planning might also modulate early sensory cortical areas, readying them for sensory processing during the unfolding action. To test this hypothesis, we performed 2 human functional magnetic resonance imaging studies involving separate delayed movement tasks and focused on premovement neural activity in early auditory cortex, given the area's direct connections to the motor system and evidence that it is modulated by motor cortex during movement in rodents. We show that effector-specific information (i.e., movements of the left vs. right hand in Experiment 1 and movements of the hand vs. eye in Experiment 2) can be decoded, well before movement, from neural activity in early auditory cortex. We find that this motor-related information is encoded in a separate subregion of auditory cortex than sensory-related information and is present even when movements are cued visually instead of auditorily. These findings suggest that action planning, in addition to preparing the motor system for movement, involves selectively modulating primary sensory areas based on the intended action.

Temporal dynamics of neural activity in macaque frontal cortex assessed with large-scale recordings

The cortical network controlling the arm and hand when grasping objects consists of several areas in parietal and frontal cortex. Recently, more anterior prefrontal areas have also been implicated in object grasping, but their exact role is currently unclear. To investigate the neuronal encoding of objects during grasping in these prefrontal regions and their relation with other cortical areas of the grasping network, we performed large-scale recordings (more than 2000 responsive sites) in frontal cortex of monkeys during a saccade-reach-grasp task. When an object appeared in peripheral vision, the first burst of activity emerged in prearcuate areas (the FEF and area 45B), followed by dorsal and ventral premotor cortex, and a buildup of activity in primary motor cortex. After the saccade, prearcuate activity remained elevated while primary motor and premotor activity rose in anticipation of the upcoming arm and hand movement. Remarkably, a large number of premotor and prearcuate sites responded when the object appeared in peripheral vision and remained active when the object came into foveal vision. Thus, prearcuate and premotor areas continuously encode object information when directing gaze and grasping objects.

Stereopsis contributes to the predictive control of grip forces during prehension

Binocular viewing is associated with a superior prehensile performance, which is particularly evident in the latter part of the reach as the hand approaches and makes contact with the target object. However, the visuomotor mechanisms through which binocular vision serves prehension are not fully understood. This study assessed the role of stereopsis in the predictive control of grasping by measuring grip force. Twenty participants performed a precision reach-to-grasp task in four viewing conditions: binocular, monocular, and with reduced stereoacuity (200 arc sec, > 400 arc sec). Monocular, compared to binocular viewing, was associated with a fourfold increase in grasp errors, a 56% increase in grasp duration, 22% decrease in grip force at 50 ms following grasp initiation, and the time of peak force occurred 40% later after grasp initiation (all p < 0.05). Grasp performance was also disrupted when viewing with reduced stereoacuity. Notably, grip force at the time of object lift-off was comparable between all viewing conditions. These results demonstrate that binocular stereopsis contributes to the efficient programming of grip forces. Specifically, stereopsis may provide important sensory information that enables the central nervous system to engage in predictive control of grasping.

A Human-like Upper-limb Motion Planner: Generating naturalistic movements for humanoid robots

As robots are starting to become part of our daily lives, they must be able to cooperate in a natural and efficient manner with humans to be socially accepted. Human-like morphology and motion are often considered key features for intuitive human–robot interactions because they allow human peers to easily predict the final intention of a robotic movement. Here, we present a novel motion planning algorithm, the Human-like Upper-limb Motion Planner, for the upper limb of anthropomorphic robots, that generates collision-free trajectories with human-like characteristics. Mainly inspired from established theories of human motor control, the planning process takes into account a task-dependent hierarchy of spatial and postural constraints modelled as cost functions. For experimental validation, we generate arm-hand trajectories in a series of tasks including simple point-to-point reaching movements and sequential object-manipulation paradigms. Being a major contribution to the current literature, specific focus is on the kinematics of naturalistic arm movements during the avoidance of obstacles. To evaluate human-likeness, we observe kinematic regularities and adopt smoothness measures that are applied in human motor control studies to distinguish between well-coordinated and impaired movements. The results of this study show that the proposed algorithm is capable of planning arm-hand movements with human-like kinematic features at a computational cost that allows fluent and efficient human–robot interactions.

Hierarchical Tactile-Based Control Decomposition of Dexterous In-Hand Manipulation Tasks

In-hand manipulation and grasp adjustment with dexterous robotic hands is a complex problem that not only requires highly coordinated finger movements but also deals with interaction variability. The control problem becomes even more complex when introducing tactile information into the feedback loop. Traditional approaches do not consider tactile feedback and attempt to solve the problem either by relying on complex models that are not always readily available or by constraining the problem in order to make it more tractable. In this paper, we propose a hierarchical control approach where a higher level policy is learned through reinforcement learning, while low level controllers ensure grip stability throughout the manipulation action. The low level controllers are independent grip stabilization controllers based on tactile feedback. The independent controllers allow reinforcement learning approaches to explore the manipulation tasks state-action space in a more structured manner. We show that this structure allows learning the unconstrained task with RL methods that cannot learn it in a non-hierarchical setting. The low level controllers also provide an abstraction to the tactile sensors input, allowing transfer to real robot platforms. We show preliminary results of the transfer of policies trained in simulation to the real robot hand.

Selective Modulation of Early Visual Cortical Activity by Movement Intention

The primate visual system contains myriad feedback projections from higher- to lower-order cortical areas, an architecture that has been implicated in the top-down modulation of early visual areas during working memory and attention. Here we tested the hypothesis that these feedback projections also modulate early visual cortical activity during the planning of visually guided actions. We show, across three separate human functional magnetic resonance imaging (fMRI) studies involving object-directed movements, that information related to the motor effector to be used (i.e., limb, eye) and action goal to be performed (i.e., grasp, reach) can be selectively decoded-prior to movement-from the retinotopic representation of the target object(s) in early visual cortex. We also find that during the planning of sequential actions involving objects in two different spatial locations, that motor-related information can be decoded from both locations in retinotopic cortex. Together, these findings indicate that movement planning selectively modulates early visual cortical activity patterns in an effector-specific, target-centric, and task-dependent manner. These findings offer a neural account of how motor-relevant target features are enhanced during action planning and suggest a possible role for early visual cortex in instituting a sensorimotor estimate of the visual consequences of movement.

The Grasp Strategy of a Robot Passer Influences Performance and Quality of the Robot-Human Object Handover

Task-aware robotic grasping is critical if robots are to successfully cooperate with humans. The choice of a grasp is multi-faceted; however, the task to perform primes this choice in terms of hand shaping and placement on the object. This grasping strategy is particularly important for a robot companion, as it can potentially hinder the success of the collaboration with humans. In this work, we investigate how different grasping strategies of a robot passer influence the performance and the perceptions of the interaction of a human receiver. Our findings suggest that a grasping strategy that accounts for the subsequent task of the receiver improves substantially the performance of the human receiver in executing the subsequent task. The time to complete the task is reduced by eliminating the need of a post-handover re-adjustment of the object. Furthermore, the human perceptions of the interaction improve when a task-oriented grasping strategy is adopted. The influence of the robotic grasp strategy increases as the constraints induced by the object's affordances become more restrictive. The results of this work can benefit the wider robotics community, with application ranging from industrial to household human-robot interaction for cooperative and collaborative object manipulation.

Motor memories in manipulation tasks are linked to contact goals between objects

Skillful manipulation requires forming memories of object dynamics, linking applied force to motion. Although it has been assumed that such memories are linked to objects, a recent study showed that people can form separate memories when these are linked to different controlled points on an object (Heald JB, Ingram JN, Flanagan JR, Wolpert DM. Nat Hum Behav 2: 300-311, 2018). In that study, participants controlled the handle of a robotic device to move a virtual bar with circles (control points) on the left and right sides. Participants were instructed to move either the left or right control point to a target on the left or right, respectively, such that the required movement was constant. When these control points were paired with opposing force fields, adaptation was observed. In this previous study, both the controlled point and the target changed between contexts. To assess which of these factors is critical for learning, here, we used a similar paradigm but with a bar that automatically rotated as it was moved. In the first experiment, the bar rotated, such that the left and right control points moved to a common target. In the second experiment, the bar rotated such that a single control point moved to a target located on either the left or right. In both experiments, participants were able to learn opposing force fields applied in the two contexts. We conclude that separate memories of dynamics can be formed for different "contact goals," involving a unique combination of the controlled point on an object and the target location this point "contacts."NEW & NOTEWORTHY Skilled manipulation requires forming memories of object dynamics, previously assumed to be associated with entire objects. However, we recently demonstrated that people can form multiple motor memories when explicitly instructed to move different locations on an object to different targets. Here, we show that separate motor memories can be learned for different contact goals, which involve a unique combination of a control point and target.

Motor control strategies during unpredictable force control tasks in humans

BACKGROUND

There are fundamental similarities and differences between the jaw and hand motor systems. However, it is unclear how the two systems respond to unpredictable task demands.

OBJECTIVE

To investigate and compare the force control of the jaw motor system (OMS) and the hand motor system (HMS) during unpredictable load changes.

METHODS

Seventeen healthy adults (24.0 ± 4.3 years) performed two standardised force control tasks (OMS and HMS). During the OMS, the participants asked to bite and pull a force transducer with the front teeth. While during HMS they pinched and pulled the same force transducer with their index and thumb fingers. Series of loads were added to a string attached to the transducer in an unpredictable (sequential and non-sequential) manner. The entire force profile during the task was divided into "initial" and "latter" segments. The force control was analysed and compared between the OMS and HMS in terms of peak force during the initial segment and holding force and force variability during the latter segment.

RESULTS

The peak force, holding force and force variability were higher for the OMS than the HMS (P < .001). However, there were no differences in the peak force, holding force or force variability between the sequential and non-sequential load changes (P > .05).

CONCLUSIONS

The results showed that unpredictable load changes did not affect the force control during the motor control task. This study suggests that both the motor systems are optimised in performing simple motor control tasks and are rather resilient to motor unpredictability.

Modulation of finger muscle activation patterns across postures is coordinated across all muscle groups

Successful grasp requires that grip forces be properly directed between the fingertips and the held object. Changes in digit posture significantly affect the mapping between muscle force and fingertip force. Joint torques must subsequently be altered to maintain the desired force direction at the fingertips. Our current understanding of the roles of hand muscles in force production remains incomplete, as past studies focused on a limited set of postures or force directions. To thoroughly examine how hand muscles adapt to changing external (force direction) and internal (posture) conditions, activation patterns of six index finger muscles were examined with intramuscular electrodes in 10 healthy subjects. Participants produced submaximal isometric forces in each of six orthogonal directions at nine different finger postures. Across force directions, participants significantly altered activation patterns to accommodate postural changes in the interphalangeal joint angles but not changes in the metacarpophalangeal joint angles. Modulation of activation levels of the extrinsic hand muscles, particularly the extensors, were as great as those of intrinsic muscles, suggesting that both extrinsic and intrinsic muscles were involved in creating the desired forces. Despite considerable between-subject variation in the absolute activation patterns, principal component analysis revealed that participants used similar strategies to accommodate the postural changes. The changes in muscle coordination also helped increase joint impedance in order to stabilize the end-point force direction. This effect counteracts the increased signal-dependent motor noise that arises with greater magnitude of muscle activation as interphalangeal joint flexion is increased. These results highlight the role of the extrinsic muscles in controlling fingertip force direction across finger postures.NEW & NOTEWORTHY We examined how hand muscles adapt to changing external (force direction) and internal (posture) conditions. Muscle activations, particularly of the extrinsic extensors, were significantly affected by postural changes of the interphalangeal, but not metacarpophalangeal, joints. Joint impedance was modulated so that the effects of the signal-dependent motor noise on the force output were reduced. Comparisons with theoretical solutions showed that the chosen activation patterns occupied a small portion of the possible solution space, minimizing the maximum activation of any one muscle.

A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.

Learning Sequential Force Interaction Skills

Learning skills from kinesthetic demonstrations is a promising way of minimizing the gap between human manipulation abilities and those of robots. We propose an approach to learn sequential force interaction skills from such demonstrations. The demonstrations are decomposed into a set of movement primitives by inferring the underlying sequential structure of the task. The decomposition is based on a novel probability distribution which we call Directional Normal Distribution. The distribution allows infering the movement primitive’s composition, i.e., its coordinate frames, control variables and target coordinates from the demonstrations. In addition, it permits determining an appropriate number of movement primitives for a task via model selection. After finding the task’s composition, the system learns to sequence the resulting movement primitives in order to be able to reproduce the task on a real robot. We evaluate the approach on three different tasks, unscrewing a light bulb, box stacking and box flipping. All tasks are kinesthetically demonstrated and then reproduced on a Barrett WAM robot.

Psychophysical principles of discrete event-driven vibrotactile feedback for prostheses

Purpose/aim of the study: We aimed to establish psychophysical principles for non-invasive vibrotactile feedback signalling discrete transition events (e.g., extension to flexion) during use of prostheses, especially for the upper limbs.Materials and methods: Two vibrotactile actuators were used on both upper arms of 10 able-bodied human participants. Absolute thresholds, psychometric functions, and magnitude estimates were measured to equalize the sensation magnitudes for the tested vibrotactile frequencies and skin sites. Then, same-different and pattern recognition tasks were run to evaluate, respectfully, the discrimination and closed-set identification of stimuli with varying parameters (2 frequencies, 2 magnitudes, 2 sites). Finally, parameters of the left/right stimuli were mapped to hypothetical prosthesis events representing object/force and movement type. The stimuli were applied sequentially in accordance with the discrete event-driven feedback paradigm.Results: Reliable psychophysical models could be established for individual participants as verified by repetitive threshold measurements and relative adjustment of stimulus levels based on sensation magnitudes. Discrimination accuracy was higher for magnitude versus frequency comparisons; and magnitude discrimination accuracy was correlated with magnitude estimate differences. Pattern recognition recall/precision rates decreased from ∼0.7 to ∼0.5 for sequential delivery of two stimulus patterns to one arm versus to two arms. Using the patterns as two and three consecutive prosthesis events yielded statistically similar performance rates not correlated with magnitude estimate differences.Conclusions: By careful calibration of stimuli based on psychophysical principles, discrete event-driven vibrotactile feedback can be used to signal manipulated object and movement information with moderate identification rates as shown by confusion matrices.

Sensory retraining improves light touch threshold of the paretic hand in chronic stroke survivors: a single-subject A-B design

Background: Light touch, one of the primary and basic sensations, is often neglected in sensory retraining programmes for stroke survivors.Objective: This study aimed to investigate the effects of sensory retraining on the light touch threshold of the hand, dexterity and upper limb motor function of chronic stroke survivors.Methods: Five chronic stroke survivors with sensory impairment participated in this single-subject A-B design study. In baseline (A) phase, they only received standard rehabilitation. In the treatment (B) phase, they received a 6-week sensory retraining intervention in addition to standard rehabilitation. In both phases, they were evaluated every 3 days. Light touch threshold, manual dexterity and upper limb motor function were assessed using Semmes-Weinstein Monofilaments, Box-Block Test and Fugl-Meyer Assessment, respectively. Visual analysis, nonparametric Mann-Whitney U test and, c-statistic were used for assessing the changes between phases.Results: All participants indicated changes in trend or slope of the total score of light touch or both between the two phases. The results of the c-statistic also showed the statistical difference in the total score of light touch between baseline and treatment in all participants (p < 0.001). Also, the results of the c-statistic and Mann-Whitney U test supported the difference of manual dexterity and motor function of the upper limb between baseline and treatment in all participants (p < 0.001).Conclusion: Current findings showed that sensory retraining may be an effective adjunctive intervention for improving the light touch threshold of the hand, dexterity and upper limb motor function in chronic stroke survivors.

The Visual Perception of Material Properties Affects Motor Planning in Prehension: An Analysis of Temporal and Spatial Components of Lifting Cups

The current study examined the role of visually perceived material properties in motor planning, where we analyzed the temporal and spatial components of motor movements during a seated reaching task. We recorded hand movements of 14 participants in three dimensions while they lifted and transported paper cups that differed in weight and glossiness. Kinematic- and spatial analysis revealed speed-accuracy trade-offs to depend on visual material properties of the objects, in which participants reached slower and grabbed closer to the center of mass for stimuli that required to be handled with greater precision. We found grasp-preparation during the first encounters with the cups was not only governed by the anticipated weight of the cups, but also by their visual material properties, namely glossiness. After a series of object lifting, the execution of reaching, the grip position, and the transportation of the cups from one location to another were preeminently guided by the object weight. We also found the planning phase in reaching to be guided by the expectation of hardness and surface gloss. The findings promote the role of general knowledge of material properties in reach-to-grasp movements, in which visual material properties are incorporated in the spatio-temporal components.