Effects of object shape and visual feedback on hand configuration during grasping

Are reaching and grasping effector-independent? Similarities and differences in reaching and grasping kinematics between the hand and foot

While reaching and grasping are highly prevalent manual actions, neuroimaging studies provide evidence that their neural representations may be shared between different body parts, i.e., effectors. If these actions are guided by effector-independent mechanisms, similar kinematics should be observed when the action is performed by the hand or by a cortically remote and less experienced effector, such as the foot. We tested this hypothesis with two characteristic components of action: the initial ballistic stage of reaching, and the preshaping of the digits during grasping based on object size. We examined if these kinematic features reflect effector-independent mechanisms by asking participants to reach toward and to grasp objects of different widths with their hand and foot. First, during both reaching and grasping, the velocity profile up to peak velocity matched between the hand and the foot, indicating a shared ballistic acceleration phase. Second, maximum grip aperture and time of maximum grip aperture of grasping increased with object size for both effectors, indicating encoding of object size during transport. Differences between the hand and foot were found in the deceleration phase and time of maximum grip aperture, likely due to biomechanical differences and the participants’ inexperience with foot actions. These findings provide evidence for effector-independent visuomotor mechanisms of reaching and grasping that generalize across body parts.

Hand dominance in the performance and perceptions of virtual reach control

PURPOSE

Efforts to optimize human-computer interactions are becoming increasingly prevalent, especially with virtual reality (VR) rehabilitation paradigms that utilize engaging interfaces. We hypothesized that motor and perceptional behaviors within a virtual environment are modulated uniquely through different modes of control of a hand avatar depending on limb dominance. This study investigated the effects of limb dominance on performance and concurrent changes in perceptions, such as time-based measures for intentional binding, during virtual reach-to-grasp.

METHODS

Participants (n = 16, healthy) controlled a virtual hand through their own hand motions with control adaptations in speed, noise, and automation.

RESULTS

A significant (p < 0.01) positive relationship between performance (reaching pathlength) and binding (time-interval estimation of beep-sound after grasp contact) was observed for the dominant hand. Unique changes in performance (p < 0.0001) and binding (p < 0.0001) were observed depending on handedness and which control mode was applied.

CONCLUSIONS

Developers of VR paradigms should consider limb dominance to optimize settings that facilitate better performance and perceptional engagement. Adapting VR rehabilitation for handedness may particularly benefit unilateral impairments, like hemiparesis or single-limb amputation.

Hand-Object Interaction: From Human Demonstrations to Robot Manipulation

Human-object interaction is of great relevance for robots to operate in human environments. However, state-of-the-art robotic hands are far from replicating humans skills. It is, therefore, essential to study how humans use their hands to develop similar robotic capabilities. This article presents a deep dive into hand-object interaction and human demonstrations, highlighting the main challenges in this research area and suggesting desirable future developments. To this extent, the article presents a general definition of the hand-object interaction problem together with a concise review for each of the main subproblems involved, namely: sensing, perception, and learning. Furthermore, the article discusses the interplay between these subproblems and describes how their interaction in learning from demonstration contributes to the success of robot manipulation. In this way, the article provides a broad overview of the interdisciplinary approaches necessary for a robotic system to learn new manipulation skills by observing human behavior in the real world.

An image-computable model of human visual shape similarity

Shape is a defining feature of objects, and human observers can effortlessly compare shapes to determine how similar they are. Yet, to date, no image-computable model can predict how visually similar or different shapes appear. Such a model would be an invaluable tool for neuroscientists and could provide insights into computations underlying human shape perception. To address this need, we developed a model (‘ShapeComp’), based on over 100 shape features (e.g., area, compactness, Fourier descriptors). When trained to capture the variance in a database of >25,000 animal silhouettes, ShapeComp accurately predicts human shape similarity judgments between pairs of shapes without fitting any parameters to human data. To test the model, we created carefully selected arrays of complex novel shapes using a Generative Adversarial Network trained on the animal silhouettes, which we presented to observers in a wide range of tasks. Our findings show that incorporating multiple ShapeComp dimensions facilitates the prediction of human shape similarity across a small number of shapes, and also captures much of the variance in the multiple arrangements of many shapes. ShapeComp outperforms both conventional pixel-based metrics and state-of-the-art convolutional neural networks, and can also be used to generate perceptually uniform stimulus sets, making it a powerful tool for investigating shape and object representations in the human brain.

The ability to describe and compare shapes is crucial in many scientific domains from visual object recognition to computational morphology and computer graphics. Across disciplines, considerable effort has been devoted to the study of shape and its influence on object recognition, yet an important stumbling block is the quantitative characterization of shape similarity. Here we develop a psychophysically validated model that takes as input an object’s shape boundary and provides a high-dimensional output that can be used for predicting visual shape similarity. With this precise control of shape similarity, the model’s description of shape is a powerful tool that can be used across the neurosciences and artificial intelligence to test role of shape in perception and the brain.

Force Anticipation and Its Potential Implications on Feedforward and Feedback Human Motor Control

OBJECTIVE

To investigate the effects of human force anticipation, we conducted an experimental load-pushing task with diverse combinations of informed and actual loading weights.

BACKGROUND

Human motor control tends to rely upon the anticipated workload to plan the force to exert, particularly in fast tasks such as pushing objects in less than 1 s. The motion and force responses in such tasks may depend on the anticipated resistive forces, based on a learning process.

METHOD

Pushing performances of 135 trials were obtained from 9 participants. We varied the workload by changing the masses from 0.2 to 5 kg. To influence anticipation, participants were shown a display of the workload that was either correct or incorrect. We collected the motion and force data, as well as electromyography (EMG) signals from the actively used muscle groups.

RESULTS

Overanticipation produced overshoot performances in more than 80% of trials. Lighter actual workloads were also associated with overshoot. Pushing behaviors with heavier workloads could be classified into feedforward-dominant and feedback-dominant responses based on the timing of force, motion, and EMG responses. In addition, we found that the preceding trial condition affected the performance of the subsequent trial.

CONCLUSION

Our results show that the first peak of the pushing force increases consistently with anticipatory workload.

APPLICATION

This study improves our understanding of human motion control and can be applied to situations such as simulating interactions between drivers and assistive systems in intelligent vehicles.

Manipulation of physical 3-D and virtual 2-D stimuli: comparing digit placement and fixation position

The visuomotor processes involved in grasping a 2-D target are known to be fundamentally different than those involved in grasping a 3-D object, and this has led to concerns regarding the generalizability of 2-D grasping research. This study directly compared participants' fixation positions and digit placement during interaction with either physical square objects or 2-D virtual versions of these objects. Participants were instructed to either simply grasp the stimulus or grasp and slide it to another location. Participants' digit placement and fixation positions did not significantly differ as a function of stimulus type when grasping in the center of the display. However, gaze and grasp positions shifted toward the near side of non-central virtual stimuli, while consistently remaining close to the horizontal midline of the physical stimulus. Participants placed their digits at less stable locations when grasping the virtual stimulus in comparison to the physical stimulus on the right side of the display, but this difference disappeared when grasping in the center and on the left. Similar outward shifts in digit placement and lowered fixations were observed when sliding both stimulus types, suggesting participants incorporated similar adjustments in grasp selection in anticipation of manipulation in both Physical and Virtual stimulus conditions. These results suggest that while fixation position and grasp point selection differed between stimulus type as a function of stimulus position, certain eye-hand coordinated behaviours were maintained when grasping both physical and virtual stimuli.

Grasping performance depends upon the richness of hand feedback

Although visual feedback of the hand allows fast and accurate grasping actions, little is known about whether the nature of feedback of the hand affects performance. We investigated kinematics during precision grasping (with the index finger and thumb) when participants received different levels of hand feedback, with or without visual feedback of the target. Specifically, we compared performance when participants saw (1) no hand feedback; (2) only the two critical points on the index finger and thumb tips; (3) 21 points on all digit tips and hand joints; (4) 21 points connected by a "skeleton", or (5) full feedback of the hand wearing a glove. When less hand feedback was available, participants took longer to execute the movement because they allowed more time to slow the reach and close the hand. When target feedback was unavailable, participants took longer to plan the movement and reached with higher velocity. We were particularly interested in investigating maximum grip aperture (MGA), which can reflect the margin of error that participants allow to compensate for uncertainty. A trend suggested that MGA was smallest when ample feedback was available (skeleton and full hand feedback, regardless of target feedback) and when only essential information about hand and target was provided (2-point hand feedback + target feedback) but increased when non-essential points were included (21-point feedback). These results suggest that visual feedback of the hand affects grasping performance and that, while more feedback is usually beneficial, this is not necessarily always the case.

Integration of haptics and vision in human multisensory grasping

Grasping actions are directed not only toward objects we see but also toward objects we both see and touch (multisensory grasping). In this latter case, the integration of visual and haptic inputs improves movement performance compared to each sense alone. This performance advantage could be due to the integration of all the redundant positional and size cues or to the integration of only a subset of these cues. Here we selectively provided specific cues to tease apart how these different sensory sources contribute to visuo-haptic multisensory grasping. We demonstrate that the availability of the haptic positional cue together with the visual cues is sufficient to achieve the same grasping performance as when all cues are available. These findings provide strong evidence that the human sensorimotor system relies on non-visual sensory inputs and open new perspectives on their role in supporting vision during both development and adulthood.

Predicting precision grip grasp locations on three-dimensional objects

We rarely experience difficulty picking up objects, yet of all potential contact points on the surface, only a small proportion yield effective grasps. Here, we present extensive behavioral data alongside a normative model that correctly predicts human precision grasping of unfamiliar 3D objects. We tracked participants' forefinger and thumb as they picked up objects of 10 wood and brass cubes configured to tease apart effects of shape, weight, orientation, and mass distribution. Grasps were highly systematic and consistent across repetitions and participants. We employed these data to construct a model which combines five cost functions related to force closure, torque, natural grasp axis, grasp aperture, and visibility. Even without free parameters, the model predicts individual grasps almost as well as different individuals predict one another's, but fitting weights reveals the relative importance of the different constraints. The model also accurately predicts human grasps on novel 3D-printed objects with more naturalistic geometries and is robust to perturbations in its key parameters. Together, the findings provide a unified account of how we successfully grasp objects of different 3D shape, orientation, mass, and mass distribution.

Virtual image of a hand displaced in space influences action performance of the real hand

The rubber hand illusion (RHI) demonstrates that under some circumstances a fake hand can be regarded as part of one’s body; the RHI and related phenomena have been used to explore the flexibility of the body schema. Recent work has shown that a sense of embodiment may be generated by virtual reality (VR). In a series of experiments, we used VR to assess the effects of the displacement of the virtual image of subjects’ hands on action. Specifically, we tested whether spatial and temporal parameters of action change when participants perform a reaching movement towards the location of their virtual hand, the position of which was distorted on some trials. In different experiments, participants were sometimes provided with incorrect visual feedback regarding the position of the to-be-touched hand (Experiment 1), were deprived of visual feedback regarding the position of the reaching hand when acting (Experiment 2) or reached with the hand, the apparent position of which had been manipulated (Experiment 3). The effect was greatest when participants reached towards (Experiment 1) or with (Experiment 3) the displaced hand when the hand was visible during the reaching, but not when the vision of the hand was removed during the action (Experiment 2). Taken together, these data suggest that visual images of one’s hand presented in VR influence the body schema and action performance.

The combined effects of motor and social goals on the kinematics of object-directed motor action

Voluntary actions towards manipulable objects are usually performed with a particular motor goal (i.e., a task-specific object-target-effector interaction) and in a particular social context (i.e., who would benefit from these actions), but the mutual influence of these two constraints has not yet been properly studied. For this purpose, we asked participants to grasp an object and place it on either a small or large target in relation to Fitts’ law (motor goal). This first action prepared them for a second grasp-to-place action which was performed under temporal constraints, either by the participants themselves or by a confederate (social goal). Kinematic analysis of the first preparatory grasp-to-place action showed that, while deceleration time was impacted by the motor goal, peak velocity was influenced by the social goal. Movement duration and trajectory height were modulated by both goals, the effect of the social goal being attenuated by the effect of the motor goal. Overall, these results suggest that both motor and social constraints influence the characteristics of object-oriented actions, with effects that combine in a hierarchical way.

Grasping movements toward seen and handheld objects

Grasping movements are typically performed toward visually sensed objects. However, planning and execution of grasping movements can be supported also by haptic information when we grasp objects held in the other hand. In the present study we investigated this sensorimotor integration process by comparing grasping movements towards objects sensed through visual, haptic or visuo-haptic signals. When movements were based on haptic information only, hand preshaping was initiated earlier, the digits closed on the object more slowly, and the final phase was more cautious compared to movements based on only visual information. Importantly, the simultaneous availability of vision and haptics led to faster movements and to an overall decrease of the grip aperture. Our findings also show that each modality contributes to a different extent in different phases of the movement, with haptics being more crucial in the initial phases and vision being more important for the final on-line control. Thus, vision and haptics can be flexibly combined to optimize the execution of grasping movement.

Magnifying vision improves motor performance in individuals with stroke

Increasing perceived hand size using magnifying lenses improves tactile discrimination and motor performance in neurologically-intact individuals. We tested whether magnification of the hand can improve motor function in individuals with chronic stroke. Twenty-five individuals with a history of stroke more than 6 months prior to testing underwent a series of tasks exploring different aspects of motor performance (grip force, finger tapping, reaching and grasping, and finger matching) under two visual conditions: magnified or normal vision. Performance was also assessed shortly after visual manipulation to test if these effects persisted. Twenty-eight percent of individuals showed an immediate significant improvement averaged across all tasks with magnification; similar beneficial responses were also observed in 32% of individuals after a short delay. These results suggest that magnification of the image of the hand may be of utility in rehabilitation of individuals with stroke.

Evidence for the representation of movement kinematics in the discharge of F5 mirror neurons during the observation of transitive and intransitive actions

Mirror neurons (MirNs) are sensorimotor neurons that fire both when an animal performs a goal-directed action and when the same animal observes another agent performing the same or a similar transitive action. It has been claimed that the observation of intransitive actions does not activate MirNs in a monkey’s brain. Prompted by recent evidence indicating that the discharge of MirNs is modulated also by non-object-directed actions, we investigated thoroughly the efficacy of intransitive actions to trigger MirNs’ discharge. Using representational similarity analysis, we also studied whether the elements constituting the visual scene presented to the monkey during the observation of actions (both transitive and intransitive) are represented in the discharge of MirNs. For this purpose, the moving hand was modeled by its kinematics and the object by features of its geometry. We found that MirNs respond to the observation of both transitive and intransitive actions and that the discharge differences evoked by the observation of object- and non-object-directed actions are correlated more with the kinematic differences of these actions than with the differences of the objects’ features. These findings support the view that observed action kinematics contribute to action mirroring.

NEW & NOTEWORTHY Mirror neurons in the monkey brain are thought to respond exclusively to the observation of object-directed actions. Here, we show that mirror neurons also respond to the observation of intransitive actions and that the kinematics of the observed movements are represented in their discharge. This finding supports the view that mirror neurons provide also a kinematics-based representation of actions.

When perception trips action! The increase in the perceived size of both hand and target matters in reaching and grasping movements

Reaching and grasping movements rely on visual information regarding the target characteristics (e.g. size) and the hand position during the action execution. Changes in the visual representation of the body (e.g. increase in the perceived size of the hand) can modify action performance, but it is still unclear how these modifications interact with changes in the external environment. We investigated this topic by manipulating the perceived size of both hand and target objects and the degree of visual feedback available during the movement execution. Ten young adults were asked to reach and grasp geometrical objects in four different conditions: (i) with normal vision with the light on, (ii) with normal vision in the dark, (iii) using magnifying lenses in the light and (iv) using magnifying lenses in the dark. In contrast with previous works, our results show that movement execution is longer in magnified vision compared to normal when the action is executed in the light, but the grasping component was not affected by changes in size in this condition. On the contrary, when the visual feedback of the hand was removed and participants performed the action in the dark, movements were faster and the distances across fingers larger in the magnified than normal vision. This pattern of data suggests that grasping movements adapt rapidly and compensate for changes in vision when this process depends on the degree of visual feedback and/or environmental cues available. In the debate regarding the dissociation between action and perception, our data suggest that action may overcome changes in perception when visual feedback is available, but perception may trick action in situations of reduced visual information.

Visual information about object size and object position are retained differently in the visual brain: Evidence from grasping studies

Many experiments have examined how the visual information used for action control is represented in our brain, and whether or not visually-guided and memory-guided hand movements rely on dissociable visual representations that are processed in different brain areas (dorsal vs. ventral). However, little is known about how these representations decay over longer time periods and whether or not different visual properties are retained in a similar fashion. In three experiments we investigated how information about object size and object position affect grasping as visual memory demands increase. We found that position information decayed rapidly with increasing delays between viewing the object and initiating subsequent actions - impacting both the accuracy of the transport component (lower end-point accuracy) and the grasp component (larger grip apertures) of the movement. In contrast, grip apertures and fingertip forces remained well-adjusted to target size in conditions in which positional information was either irrelevant or provided, regardless of delay, indicating that object size is encoded in a more stable manner than object position. The findings provide evidence that different grasp-relevant properties are encoded differently by the visual system. Furthermore, we argue that caution is required when making inferences about object size representations based on alterations in the grip component as these variations are confounded with the accuracy with which object position is represented. Instead fingertip forces seem to provide a reliable and confound-free measure to assess internal size estimations in conditions of increased visual uncertainty.

Recovery of Grasp versus Reach in People with Hemiparesis Poststroke

Background and Objective. The authors recently found that grasping was not relatively more disrupted than reaching in people with acute hemiparesis. They now extend this work to the recovery of reach versus grasp.

Methods. Hemiparetic subjects were tested acutely, after 90 days, and then after 1 year poststroke, and a control group was evaluated once. Using kinematic techniques, subjects were studied performing reach and reach-to-grasp movements. The authors quantified 3 characteristics of performance for each movement: speed, accuracy, and efficiency, where an efficient movement was defined as a movement directly to the target without extraneous or abnormally circuitous movements. To evaluate the relative deficits and recovery in reach versus grasp, performance measures were converted to z scores using control group means and standard deviations.

Results. The authors’ results showed that, starting with small deficits in speed acutely, both reach speed and grasp speed improved over time. Deficits in accuracy were greater in the reach than the grasp acutely, and these deficits lessened such that by the 90-day time point, the relative accuracy of the 2 movements was the same. In contrast, deficits in efficiency were greater in the grasp than the reach acutely, and grasp efficiency did not recover. The majority of recovery in reaching and grasping occurred by the 90-day time point, with little change occurring between the 90-day and 1-year time points.

Conclusions. The authors hypothesize that, in chronic hemiparesis, purposeful movements requiring distal control may be more impaired than purposeful movements requiring proximal control, not because of the initial lesion, but because, over the course of recovery, spared components of the descending motor systems may be able to compensate for the accuracy deficits in reaching (proximal control) but not the efficiency deficits in grasping (distal muscular control).

Sensorimotor Training in a Virtual Reality Environment: Does It Improve Functional Recovery Poststroke?

Objective. To investigate the effectiveness of computerized virtual reality (VR) training of the hemiparetic hand of patients poststroke using a system that provides repetitive motor reeducation and skill reacquisition. Methods. Eight subjects in the chronic phase poststroke participated in a 3-week program using their hemiparetic hand in a series of interactive computer games for 13 days of training, weekend breaks, and pretests and posttests. Each subject trained for about 2 to 2.5 h per day. Outcome measures consisted of changes in the computerized measures of thumb and finger range of motion, thumb and finger velocity, fractionation (the ability to move fingers independently), thumb and finger strength, the Jebsen Test of Hand Function, and a Kinematic reach to grasp test. Results. Subjects as a group improved in fractionation of the fingers, thumb and finger range of motion, and thumb and finger speed, retaining those gains at the 1-week retention test. Transfer of these improvements was demonstrated through changes in the Jebsen Test of Hand Function and a decrease after the therapy in the overall time from hand peak velocity to the moment when an object was lifted from the table. Conclusions. It is difficult in current service delivery models to provide the intensity of practice that appears to be needed to effect neural reorganization and functional changes poststroke. Computerized exercise systems may be a way to maximize both the patients’ and the clinicians’ time. The data in this study add support to the proposal to explore novel technologies for incorporation into current practice.

Learning to grasp and extract affordances: the Integrated Learning of Grasps and Affordances (ILGA) model

The activity of certain parietal neurons has been interpreted as encoding affordances (directly perceivable opportunities) for grasping. Separate computational models have been developed for infant grasp learning and affordance learning, but no single model has yet combined these processes in a neurobiologically plausible way. We present the Integrated Learning of Grasps and Affordances (ILGA) model that simultaneously learns grasp affordances from visual object features and motor parameters for planning grasps using trial-and-error reinforcement learning. As in the Infant Learning to Grasp Model, we model a stage of infant development prior to the onset of sophisticated visual processing of hand-object relations, but we assume that certain premotor neurons activate neural populations in primary motor cortex that synergistically control different combinations of fingers. The ILGA model is able to extract affordance representations from visual object features, learn motor parameters for generating stable grasps, and generalize its learned representations to novel objects.

The contributions of vision and haptics to reaching and grasping

This review aims to provide a comprehensive outlook on the sensory (visual and haptic) contributions to reaching and grasping. The focus is on studies in developing children, normal, and neuropsychological populations, and in sensory-deprived individuals. Studies have suggested a right-hand/left-hemisphere specialization for visually guided grasping and a left-hand/right-hemisphere specialization for haptically guided object recognition. This poses the interesting possibility that when vision is not available and grasping relies heavily on the haptic system, there is an advantage to use the left hand. We review the evidence for this possibility and dissect the unique contributions of the visual and haptic systems to grasping. We ultimately discuss how the integration of these two sensory modalities shape hand preference.

Manual preferences for visually- and haptically-guided grasping

Studies have shown that individuals exhibit a right-hand preference for grasping during visually-guided tasks. Recently, we have found that when vision is occluded right-hand preference decreases dramatically. It remains unknown however, if this decrease is a result of visual occlusion or the effects of relying only on haptic feedback. Therefore, in the present study, we sought to explore the contributions of vision and haptics (separately and in conjunction) to hand preference for grasping. Right- and left-handed individuals were tested on a block building task under four different visual and haptic conditions: 1) vision/normal haptic feedback (V/H), 2) no vision/normal haptic feedback (NV/H), 3) vision/constrained haptic feedback (V/Constrained-H), and 4) no vision/constrained haptic feedback (NV/Constrained-H). Vision was occluded using a blindfold and haptic feedback was constrained by asking participants to wear textured gloves. Right-handed individuals displayed a right-hand preference when vision was available (V/H and V/Constrained-H groups), but this preference was much greater when haptic feedback was constrained (V/Constrained-H group). When vision was occluded and haptic feedback was used to complete the task (NV/H) no hand preference was found. Finally hand preference was similar between the V/H and the NV/Constrained-H groups. For left-handed individuals, no differences in hand use were found between the different sensory groups, but the NV/H group showed a clear left-hand preference for haptically-guided grasping. The results suggest that haptics plays an important role in hand preference for grasping. Furthermore, they support a left-hand/right-hemisphere specialization for haptically-guided grasping (regardless of handedness) and a right-hand/left-hemisphere specialization for visually-guided grasping (at least in right-handed individuals).

Effects of a Single Hand-Exercise Session on Manual Dexterity and Strength in Persons with Parkinson Disease: A Randomized Controlled Trial

OBJECTIVE

To evaluate the effects on manual dexterity, hand grip, and pinch strength of a single intervention focused on hand exercises.

DESIGN

Randomized, controlled, blinded study.

PATIENTS

Sixty people with Parkinson disease (PD) were recruited; 30 participants were allocated to a brief exercise session and 30 to a control group.

INTERVENTIONS

Participants randomized to the experimental group received a 15-minute exercise session focused on hand training using therapeutic putty. Participants allocated to the control group performed active upper limb exercises.

MAIN OUTCOME MEASUREMENTS

Measures of manual dexterity (assessed by the Purdue Pegboard Test and the Chessington Occupational Therapy Neurologic Assessment Battery dexterity task) and strength (hand grip and pinch strength) were recorded at baseline and after the intervention.

RESULTS

Participants had significantly improved manual dexterity values (P < .05) after the intervention. They also had increased hand grip (P < .001) and pinch strength (P < .05).

CONCLUSIONS

A single hand-exercise session showed an improvement in manual dexterity and strength in persons with PD.

Use of early phase online vision for grip configuration is modulated according to movement duration in prehension

Our previous study (Hum Mov Sci 25:349-371, 2006) investigated whether and how online vision in the early phase of movement influences the control of reach-to-grasp movements (movement duration: approximately 1000 ms). We used liquid-crystal shutter goggles to manipulate the duration of available online vision during the movement and specified that online vision during the early phase influences grasping movements. The current study examined the effect of online early phase vision on the grip configuration according to the movement duration and compared it between two different movement durations (approximately 500 and 1000 ms). We found that non-availability of early phase online vision affected the grip configuration (i.e., inducing a larger peak grip aperture) even in the shorter movement duration. The influential period for online vision for grasping control shifts to an earlier time when movement time is shorter (i.e., from approximately 214 to 106 ms after movement onset), indicating a flexible mechanism for grip configuration according to the movement duration and the available online vision.

Disruption of activity in the ventral premotor but not the anterior intraparietal area interferes with on-line correction to a haptic perturbation during grasping

Replanning ongoing movements following perturbations requires the accurate and immediate estimation of the motor response based on sensory input. Previous studies have used transcranial magnetic stimulation (TMS) in humans to demonstrate the participation of the anterior intraparietal sulcus (aIPS) and ventral premotor cortex (PMv) in visually mediated state estimation for grasping. Here, we test the role of parietofrontal circuits in processing the corrective responses to haptic perturbations of the finger during prehension. Subjects reached to grasp an object while having to compensate for a novel and unpredictable haptic perturbation of finger extension. TMS-based transient disruptions to the PMv and aIPS were delivered 0, 50, or 100 ms after the perturbation. TMS to the PMv delivered 50 ms after the perturbation (but not 0 or 100 ms, or in unperturbed trials) led to an overestimation of grasp aperture. No effects on grasp aperture were noted for the aIPS. Our results indicate that the PMv (but not aIPS) is involved in the deployment of the compensatory response in the presence of haptic perturbations during prehension. Our data also identify the time window of neural processing in the PMv when reprogramming occurs to be 50-100 ms following the perturbation onset.

Functional Properties of Parietal Hand Manipulation–related Neurons and Mirror Neurons Responding to Vision of Own Hand Action

Parietofrontal pathways play an important role in visually guided motor control. In this pathway, hand manipulation-related neurons in the inferior parietal lobule represent 3-D properties of an object and motor patterns to grasp it. Furthermore, mirror neurons show visual responses that are concerned with the actions of others and motor-related activity during execution of the same grasping action. Because both of these categories of neurons integrate visual and motor signals, these neurons may play a role in motor control based on visual feedback signals. The aim of this study was to investigate whether these neurons in inferior parietal lobule including the anterior intraparietal area and PFG of macaques represent visual images of the monkey's own hand during a self-generated grasping action. We recorded 235 neurons related to hand manipulation tasks. Of these, 54 responded to video clips of the monkey's own hand action, the same as visual feedback during that action or clips of the experimenter's hand action in a lateral view. Of these 54 neurons, 25 responded to video clips of the monkey's own hand, even without an image of the target object. We designated these 25 neurons as “hand-type.” Thirty-three of 54 neurons that were defined as mirror neurons showed visual responses to the experimenter's action and motor responses. Thirteen of these mirror neurons were classified as hand-type. These results suggest that activity of hand manipulation-related and mirror neurons in anterior intraparietal/PFG plays a fundamental role in monitoring one's own body state based on visual feedback.

Digit mechanics in relation to endpoint compliance during precision pinch

This study investigates the mechanics of the thumb and index finger in relation to compliant endpoint forces during precision pinch. The objective was to gain insight into how individuals modulate motor output at the digit endpoints and joints according to compliance-related sensory feedback across the digits. Thirteen able-bodied subjects performed precision pinch upon elastic resistance bands of a customized apparatus instrumented with six degree-of-freedom load-cells. Compliance levels were discretely adjusted according to the number of bands connected. Subjects were provided visual feedback to control the rate of force application. Fifteen repetitions of low-to-moderate force (<20N) pinches were analyzed at each of five compliance levels, during which force and motion data were collected. Joint angles and moments normalized by pinch force magnitude were computed. Second-order polynomials were used to characterize joint mechanics as a function of compliance. The joint degrees-of-freedom (DOFs) at the finger showed greater dependence on compliance for angular position while the thumb joint DOFs demonstrated greater dependence for normalized joint moment. The digits also adjusted coordination of their endpoint forces according to compliance. Overall, the finger may be altering its position to increase load to the joints of the thumb with changing compliance. These findings describe naturally emergent changes in digit mechanics for compliant precision pinch, which involves motor execution in response to endpoint sensory feedback. Identifying and understanding these motor patterns may provide theoretical basis for restoring and rehabilitating sensorimotor pathologies of the hand.

Non-obstructing 3D depth cues influence reach-to-grasp kinematics

It has been demonstrated that both visual feedback and the presence of certain types of non-target objects in the workspace can affect kinematic measures and the trajectory path of the moving hand during reach-to-grasp movements. Yet no study to date has examined the possible effect of providing non-obstructing three-dimensional (3D) depth cues within the workspace and with consistent retinal inputs and whether or not these alter manual prehension movements. Participants performed a series of reach-to-grasp movements in both open- (without visual feedback) and closed-loop (with visual feedback) conditions in the presence of one of three possible 3D depth cues. Here, it is reported that preventing online visual feedback (or not) and the presence of a particular depth cue had a profound effect on kinematic measures for both the reaching and grasping components of manual prehension-despite the fact that the 3D depth cues did not act as a physical obstruction at any point. The depth cues modulated the trajectory of the reaching hand when the target block was located on the left side of the workspace but not on the right. These results are discussed in relation to previous reports and implications for brain-computer interface decoding algorithms are provided.

Grasping without sight: insights from the congenitally blind

We reach for and grasp different sized objects numerous times per day. Most of these movements are visually-guided, but some are guided by the sense of touch (i.e. haptically-guided), such as reaching for your keys in a bag, or for an object in a dark room. A marked right-hand preference has been reported during visually-guided grasping, particularly for small objects. However, little is known about hand preference for haptically-guided grasping. Recently, a study has shown a reduction in right-hand use in blindfolded individuals, and an absence of hand preference if grasping was preceded by a short haptic experience. These results suggest that vision plays a major role in hand preference for grasping. If this were the case, then one might expect congenitally blind (CB) individuals, who have never had a visual experience, to exhibit no hand preference. Two novel findings emerge from the current study: first, the results showed that contrary to our expectation, CB individuals used their right hand during haptically-guided grasping to the same extent as visually-unimpaired (VU) individuals did during visually-guided grasping. And second, object size affected hand use in an opposite manner for haptically- versus visually-guided grasping. Big objects were more often picked up with the right hand during haptically-guided, but less often during visually-guided grasping. This result highlights the different demands that object features pose on the two sensory systems. Overall the results demonstrate that hand preference for grasping is independent of visual experience, and they suggest a left-hemisphere specialization for the control of grasping that goes beyond sensory modality.

Evidence for right-hand feeding biases in a left-handed population

We have recently shown that actions with similar kinematic requirements, but different end-state goals may be supported by distinct neural networks. Specifically, we demonstrated that when right-handed individuals reach-to-grasp food items with intent to eat, they produce smaller maximum grip apertures (MGAs) than when they grasp the same item with intent to place it in a location near the mouth. This effect was restricted to right-handed movements; left-handed movements showed no difference between tasks. The current study investigates whether (and to which side) the effect may be lateralized in left-handed individuals. Twenty-one self-identified left-handed participants grasped food items of three different sizes while grasp kinematics were captured via an Optotrak Certus motion capture array. A main effect of task was identified wherein the grasp-to-eat action generated significantly smaller MGAs than did the grasp-to-place action. Further analysis revealed that similar to the findings in right-handed individuals, this effect was significant only during right-handed movements. Upon further inspection however, we found individual differences in the magnitude and direction of the observed lateralization. These results underscore the evolutionary significance of the grasp-to-eat movement in producing population-level right-handedness in humans as well as highlighting the heterogeneity of the left-handed population.

A method to study precision grip control in viscoelastic force fields using a robotic gripper

Instrumented objects and multipurpose haptic displays have commonly been used to investigate sensorimotor control of grasping and manipulation. A major limitation of these devices, however, is the extent to which the experimenter can vary the interaction dynamics to fully probe sensorimotor control mechanisms. We propose a novel method to study precision grip control using a grounded robotic gripper with two moving, mechanically coupled finger pads instrumented with force sensors. The device is capable of stably rendering virtual mechanical properties with a wide dynamic range of achievable impedances. Eight viscoelastic force fields with different combinations of stiffness and damping parameters were implemented, and tested on eight healthy subjects performing 30 consecutive repetitions of a grasp, hold, and release task with time and position constraints. Rates of thumb and finger force were found to be highly correlated (r>0.9) during grasping, revealing that, despite the mechanical coupling of the two finger pads, subjects performed grasping movements in a physiological fashion. Subjects quickly adapted to the virtual dynamics (within seven trials), but, depending on the presented force field condition, used different control strategies to correctly perform the task. The proof of principle presented in this paper underscores the potential of such a one-degree-of-freedom robotic gripper to study neural control of grasping, and to provide novel insights on sensorimotor control mechanisms.

Development of a biomimetic hand exotendon device (BiomHED) for restoration of functional hand movement post-stroke

Significant functional impairment of the hand is common among stroke survivors and restoration of hand function should be prioritized during post-stroke rehabilitation. The goal of this study was to develop a novel biomimetic device to assist patients in producing complex hand movements with a limited number of actuators. The Biomimetic Hand Exoskeleton Device (BiomHED) is actuated by exotendons that mimic the geometry of the major tendons of the hand. Ten unimpaired subjects and four chronic stroke survivors participated in experiments that tested the efficacy of the system. The exotendons reproduced distinct spatial joint coordination patterns similar to their target muscle-tendon units for both subject groups. In stroke survivors, the exotendon-produced joint angular displacements were smaller, but not significantly different, than those of unimpaired subjects [Formula: see text]. Even with limited use of the BiomHED, the kinematic workspace of the index finger increased by 63%-1014% in stroke survivors. The device improved the kinematics of the tip-pinch task in stroke survivors and resulted in a significant reduction in the fingertip-thumb tip distance ( 17.9 ±15.3 mm). This device is expected to enable effective "task-oriented" training of the hand post-stroke.

On the evolution of handedness: evidence for feeding biases

Many theories have been put forward to explain the origins of right-handedness in humans. Here we present evidence that this preference may stem in part from a right hand advantage in grasping for feeding. Thirteen participants were asked to reach-to-grasp food items of 3 different sizes: SMALL (Cheerios®), MEDIUM (Froot Loops®), and LARGE (Oatmeal Squares®). Participants used both their right- and left-hands in separate blocks (50 trials each, starting order counterbalanced) to grasp the items. After each grasp, participants either a) ate the food item, or b) placed it inside a bib worn beneath his/her chin (25 trials each, blocked design, counterbalanced). The conditions were designed such that the outward and inward movement trajectories were similar, differing only in the final step of placing it in the mouth or bib. Participants wore Plato liquid crystal goggles that blocked vision between trials. All trials were conducted in closed-loop with 5000 ms of vision. Hand kinematics were recorded by an Optotrak Certus, which tracked the position of three infrared diodes attached separately to the index finger, thumb, and wrist. We found a task (EAT/PLACE) by hand (LEFT/RIGHT) interaction on maximum grip aperture (MGA; the maximum distance between the index finger and thumb achieved during grasp pre-shaping). MGAs were smaller during right-handed movements, but only when grasping with intent to eat. Follow-up tests show that the RIGHT-HAND/EAT MGA was significantly smaller than all other hand/task conditions. Because smaller grip apertures are typically associated with greater precision, our results demonstrate a right-hand advantage for the grasp-to-eat movement. From an evolutionary perspective, early humans may have preferred the hand that could grasp food with more precision, thereby maximizing the likelihood of retrieval, consumption, and consequently, survival.

Parkinson's disease patients show impaired corrective grasp control and eye-hand coupling when reaching to grasp virtual objects

The effect of Parkinson's disease (PD) on hand-eye coordination and corrective response control during reach-to-grasp tasks remains unclear. Moderately impaired PD patients (n=9) and age-matched controls (n=12) reached to and grasped a virtual rectangular object, with haptic feedback provided to the thumb and index fingertip by two 3-degree of freedom manipulanda. The object rotated unexpectedly on a minority of trials, requiring subjects to adjust their grasp aperture. On half the trials, visual feedback of finger positions disappeared during the initial phase of the reach, when feedforward mechanisms are known to guide movement. PD patients were tested without (OFF) and with (ON) medication to investigate the effects of dopamine depletion and repletion on eye-hand coordination online corrective response control. We quantified eye-hand coordination by monitoring hand kinematics and eye position during the reach. We hypothesized that if the basal ganglia are important for eye-hand coordination and online corrections to object perturbations, then PD patients tested OFF medication would show reduced eye-hand spans and impoverished arm-hand coordination responses to the perturbation, which would be further exasperated when visual feedback of the hand was removed. Strikingly, PD patients tracked their hands with their gaze, and their movements became destabilized when having to make online corrective responses to object perturbations exhibiting pauses and changes in movement direction. These impairments largely remained even when tested in the ON state, despite significant improvement on the Unified Parkinson's Disease Rating Scale. Our findings suggest that basal ganglia-cortical loops are essential for mediating eye-hand coordination and adaptive online responses for reach-to-grasp movements, and that restoration of tonic levels of dopamine may not be adequate to remediate this coordinative nature of basal ganglia-modulated function.

Spatial-orientation priming impedes rather than facilitates the spontaneous control of hand-retraction speeds in patients with Parkinson's disease

Background

Often in Parkinson’s disease (PD) motor-related problems overshadow latent non-motor deficits as it is difficult to dissociate one from the other with commonly used observational inventories. Here we ask if the variability patterns of hand speed and acceleration would be revealing of deficits in spatial-orientation related decisions as patients performed a familiar reach-to-grasp task. To this end we use spatial-orientation priming which normally facilitates motor-program selection and asked whether in PD spatial-orientation priming helps or hinders performance.

Methods

To dissociate spatial-orientation- and motor-related deficits participants performed two versions of the task. The biomechanical version (DEFAULT) required the same postural- and hand-paths as the orientation-priming version (primed-UP). Any differences in the patients here could not be due to motor issues as the tasks were biomechanically identical. The other priming version (primed-DOWN) however required additional spatial and postural processing. We assessed in all three cases both the forward segment deliberately aimed towards the spatial-target and the retracting segment, spontaneously bringing the hand to rest without an instructed goal.

Results and Conclusions

We found that forward and retracting segments belonged in two different statistical classes according to the fluctuations of speed and acceleration maxima. Further inspection revealed conservation of the forward (voluntary) control of speed but in PD a discontinuity of this control emerged during the uninstructed retractions which was absent in NC. Two PD groups self-emerged: one group in which priming always affected the retractions and the other in which only the more challenging primed-DOWN condition was affected. These PD-groups self-formed according to the speed variability patterns, which systematically changed along a gradient that depended on the priming, thus dissociating motor from spatial-orientation issues. Priming did not facilitate the motor task in PD but it did reveal a breakdown in the spatial-orientation decision that was independent of the motor-postural path.

Activity in ventral premotor cortex is modulated by vision of own hand in action

Parietal and premotor cortices of the macaque monkey contain distinct populations of neurons which, in addition to their motor discharge, are also activated by visual stimulation. Among these visuomotor neurons, a population of grasping neurons located in the anterior intraparietal area (AIP) shows discharge modulation when the own hand is visible during object grasping. Given the dense connections between AIP and inferior frontal regions, we aimed at investigating whether two hand-related frontal areas, ventral premotor area F5 and primary motor cortex (area F1), contain neurons with similar properties. Two macaques were involved in a grasping task executed in various light/dark conditions in which the to-be-grasped object was kept visible by a dim retro-illumination. Approximately 62% of F5 and 55% of F1 motor neurons showed light/dark modulations. To better isolate the effect of hand-related visual input, we introduced two further conditions characterized by kinematic features similar to the dark condition. The scene was briefly illuminated (i) during hand preshaping (pre-touch flash, PT-flash) and (ii) at hand-object contact (touch flash, T-flash). Approximately 48% of F5 and 44% of F1 motor neurons showed a flash-related modulation. Considering flash-modulated neurons in the two flash conditions, ∼40% from F5 and ∼52% from F1 showed stronger activity in PT- than T-flash (PT-flash-dominant), whereas ∼60% from F5 and ∼48% from F1 showed stronger activity in T- than PT-flash (T-flash-dominant). Furthermore, F5, but not F1, flash-dominant neurons were characterized by a higher peak and mean discharge in the preferred flash condition as compared to light and dark conditions. Still considering F5, the distribution of the time of peak discharge was similar in light and preferred flash conditions. This study shows that the frontal cortex contains neurons, previously classified as motor neurons, which are sensitive to the observation of meaningful phases of the own grasping action. We conclude by discussing the possible functional role of these populations.

The organization of digit contact timing during grasping

While the process of hand preshaping during grasping has been studied for over a decade, there is relatively little information regarding the organization of digit contact timing (DCT). This dearth of information may be due to the assumption that DCT while grasping exhibits few regularities or to the difficulty in obtaining information through traditional movement recording techniques. In this study, we employed a novel technique to determine the time of digit contacts with the target object at a high precision rate in normal healthy participants. Our results indicate that, under our task conditions, subjects tend to employ a radial to ulnar pattern of DCT which may be modulated by the shape of the target object. Moreover, a number of parameters, such as the total contact time, the frequency of first contacts by the thumb and index fingers and the number of simultaneous contacts, are affected by the relative complexity of the target object. Our data support the notion that a great deal of information about the object's physical features is obtained during the early moments of the grasp.

Neural bases of hand synergies

The human hand has so many degrees of freedom that it may seem impossible to control. A potential solution to this problem is “synergy control” which combines dimensionality reduction with great flexibility. With applicability to a wide range of tasks, this has become a very popular concept. In this review, we describe the evolution of the modern concept using studies of kinematic and force synergies in human hand control, neurophysiology of cortical and spinal neurons, and electromyographic (EMG) activity of hand muscles. We go beyond the often purely descriptive usage of synergy by reviewing the organization of the underlying neuronal circuitry in order to propose mechanistic explanations for various observed synergy phenomena. Finally, we propose a theoretical framework to reconcile important and still debated concepts such as the definitions of “fixed” vs. “flexible” synergies and mechanisms underlying the combination of synergies for hand control.

Gaze strategies during visually-guided versus memory-guided grasping

Vision plays a crucial role in guiding motor actions. But sometimes we cannot use vision and must rely on our memory to guide action-e.g. remembering where we placed our eyeglasses on the bedside table when reaching for them with the lights off. Recent studies show subjects look towards the index finger grasp position during visually-guided precision grasping. But, where do people look during memory-guided grasping? Here, we explored the gaze behaviour of subjects as they grasped a centrally placed symmetrical block under open- and closed-loop conditions. In Experiment 1, subjects performed grasps in either a visually-guided task or memory-guided task. The results show that during visually-guided grasping, gaze was first directed towards the index finger's grasp point on the block, suggesting gaze targets future grasp points during the planning of the grasp. Gaze during memory-guided grasping was aimed closer to the blocks' centre of mass from block presentation to the completion of the grasp. In Experiment 2, subjects performed an 'immediate grasping' task in which vision of the block was removed immediately at the onset of the reach. Similar to the visually-guided results from Experiment 1, gaze was primarily directed towards the index finger location. These results support the 2-stream theory of vision in that motor planning with visual feedback at the onset of the movement is driven primarily by real-time visuomotor computations of the dorsal stream, whereas grasping remembered objects without visual feedback is driven primarily by the perceptual memory representations mediated by the ventral stream.

Relation between object properties and EMG during reaching to grasp

In order to stably grasp an object with an artificial hand, a priori knowledge of the object's properties is a major advantage, especially to ensure subsequent manipulation of the object held by the hand. This is also true for hand prostheses: pre-shaping of the hand while approaching the object, similar to able-bodied, allows the wearer for a much faster and more intuitive way of handling and grasping an object. For hand prostheses, it would be advantageous to obtain this information about object properties from a surface electromyography (sEMG) signal, which is already present and used to control the active prosthetic hand. We describe experiments in which human subjects grasp different objects at different positions while their muscular activity is recorded through eight sEMG electrodes placed on the forearm. Results show that sEMG data, gathered before the hand is in contact with the object, can be used to obtain relevant information on object properties such as size and weight.

Classification of hand preshaping in persons with stroke using Linear Discriminant Analysis

OBJECTIVE

This study describes the analysis of hand preshaping using Linear Discriminant Analysis (LDA) to predict hand formation during reaching and grasping tasks of the hemiparetic hand, following a series of upper extremity motor intervention treatments. The purpose of this study is to use classification of hand posture as an additional tool for evaluating the effectiveness of therapies for upper extremity rehabilitation such as virtual reality (VR) therapy and conventional physical therapy. Classification error for discriminating between two objects during hand preshaping is obtained for the hemiparetic and unimpaired hands pre and post training.

METHODS

Eight subjects post stroke participated in a two-week training session consisting of upper extremity motor training. Four subjects trained with interactive VR computer games and four subjects trained with clinical physical therapy procedures of similar intensity. Subjects' finger joint angles were measured during a kinematic reach to grasp test using CyberGlove® and arm joint angles were measured using the trackSTAR™ system prior to training and after training.

RESULTS

The unimpaired hand of subjects preshape into the target object with greater accuracy than the hemiparetic hand as indicated by lower classification errors. Hemiparetic hand improved in preshaping accuracy and time to reach minimum error.

CONCLUSION

Classification of hand preshaping may provide insight into improvements in motor performance elicited by robotically facilitated virtually simulated training sessions or conventional physical therapy.

Eye movements during action preparation

Looking at actions of others activates representations of similar own actions, that is, the action resonates. This may facilitate or interfere with the actions that one intends to make. We asked whether people promote or block those effects by making eye movements to or away from the actions of others. We investigated gaze behavior with a cup-clinking task: An actor shown on a video grabbed a cup and moved it toward the participant who next grabbed his own cup in the 'same' or in a different, 'complementary', way. In the 'same' condition, participants mostly looked at the place where the actor held the cup. In the 'complementary' condition, gaze behavior was similar at the start of the actor's action. To our surprise, as the action reached completion, participants started to look at the cup's site that corresponded to the grabbing instruction for their own action. A second experiment showed that this effect grew with delay of the go-signal. This indicates that a reason for the effect may be to support memorizing the instructed action. The bottom line of the study is that passively viewed scenes (passive in the sense that nothing in the observed scene is manipulated by the viewer) are scanned to support preparation of actions that one intends to make. We discuss how this finding relates to action resonance and how it relates to links between representations of actions and objects.