Grasping kinematics from the perspective of the individual digits: a modelling study

Wing-shaped walls: A directional effect of obstacles on manual avoidance

Visual information can be used to plan, start, and coordinate manual movements in obstacle avoidance. An intriguing example of visuomotor coordination is the effect of wing-shaped walls, in which walls are oriented away from or toward a moving agent. A historical story from medieval Japan recounts that wing-shaped walls disrupted the agent's movement more when oriented toward the agent than when oriented away from the agent. This study aimed at examining whether the disruptive effect of wing-shaped walls occurs in a schematic situation represented on a 2D plane. In this study, we conducted psychophysical experiments in which participants were asked to move a stylus from a start point to a goal while avoiding multiple line obstacles that were arranged alternately at a course. In the two experiments, we manipulated the orientation and the size of the visible parts of the obstacles systematically. We found that the obstacles oriented toward the agent produced frequent contacts with the agent and attracted manual movements to the endpoints of obstacles. We discussed possible interpretations of the results in the context of attentional guidance.

A direct comparison of sound and vibration as sources of stimulation for a sensory substitution glove

Sensory substitution devices (SSDs) facilitate the detection of environmental information through enhancement of touch and/or hearing capabilities. Research has demonstrated that several tasks can be successfully completed using acoustic, vibrotactile, and multimodal devices. The suitability of a substituting modality is also mediated by the type of information required to perform the specific task. The present study tested the adequacy of touch and hearing in a grasping task by utilizing a sensory substitution glove. The substituting modalities inform, through increases in stimulation intensity, about the distance between the fingers and the objects. A psychophysical experiment of magnitude estimation was conducted. Forty blindfolded sighted participants discriminated equivalently the intensity of both vibrotactile and acoustic stimulation, although they experienced some difficulty with the more intense stimuli. Additionally, a grasping task involving cylindrical objects of varying diameters, distances and orientations was performed. Thirty blindfolded sighted participants were divided into vibration, sound, or multimodal groups. High performance was achieved (84% correct grasps) with equivalent success rate between groups. Movement variables showed more precision and confidence in the multimodal condition. Through a questionnaire, the multimodal group indicated their preference for using a multimodal SSD in daily life and identified vibration as their primary source of stimulation. These results demonstrate that there is an improvement in performance with specific-purpose SSDs, when the necessary information for a task is identified and coupled with the delivered stimulation. Furthermore, the results suggest that it is possible to achieve functional equivalence between substituting modalities when these previous steps are met.

Not only perception but also grasping actions can obey Weber's law

Weber's law, the principle that the uncertainty of perceptual estimates increases proportionally with object size, is regularly violated when considering the uncertainty of the grip aperture during grasping movements. The origins of this perception-action dissociation are debated and are attributed to various reasons, including different coding of visual size information for perception and action, biomechanical factors, the use of positional information to guide grasping, or, sensorimotor calibration. Here, we contrasted these accounts and compared perceptual and grasping uncertainties by asking people to indicate the visually perceived center of differently sized objects (Perception condition) or to grasp and lift the same objects with the requirement to achieve a balanced lift (Action condition). We found that the variability (uncertainty) of contact positions increased as a function of object size in both perception and action. The adherence of the Action condition to Weber's law and the consequent absence of a perception-action dissociation contradict the predictions based on different coding of visual size information and sensorimotor calibration. These findings provide clear evidence that human perceptual and visuomotor systems rely on the same visual information and suggest that the previously reported violations of Weber's law in grasping movements should be attributed to other factors.

Visual uncertainty unveils the distinct role of haptic cues in multisensory grasping

Human multisensory grasping movements (i.e., seeing and feeling a handheld object while grasping it with the contralateral hand) are superior to movements guided by each separate modality. This multisensory advantage might be driven by the integration of vision with either the haptic position only or with both position and size cues. To contrast these two hypotheses, we manipulated visual uncertainty (central vs. peripheral vision) and the availability of haptic cues during multisensory grasping. We showed a multisensory benefit irrespective of the degree of visual uncertainty suggesting that the integration process involved in multisensory grasping can be flexibly modulated by the contribution of each modality. Increasing visual uncertainty revealed the role of the distinct haptic cues. The haptic position cue was sufficient to promote multisensory benefits evidenced by faster actions with smaller grip apertures, whereas the haptic size was fundamental in fine-tuning the grip aperture scaling. These results support the hypothesis that, in multisensory grasping, vision is integrated with all haptic cues, with the haptic position cue playing the key part. Our findings highlight the important role of non-visual sensory inputs in sensorimotor control and hint at the potential contributions of the haptic modality in developing and maintaining visuomotor functions.Significance statementThe longstanding view that vision is considered the primary sense we rely on to guide grasping movements relegates the equally important haptic inputs, such as touch and proprioception, to a secondary role. Here we show that by increasing visual uncertainty during visuo-haptic grasping, the central nervous system exploits distinct haptic inputs about the object position and size to optimize grasping performance. Specifically, we demonstrate that haptic inputs about the object position are fundamental to support vision in enhancing grasping performance, whereas haptic size inputs can further refine hand shaping. Our results provide strong evidence that non-visual inputs serve an important, previously under-appreciated, functional role in grasping.

A kinematic and EMG dataset of online adjustment of reach-to-grasp movements to visual perturbations

Control of reach-to-grasp movements for deft and robust interactions with objects requires rapid sensorimotor updating that enables online adjustments to changing external goals (e.g., perturbations or instability of objects we interact with). Rarely do we appreciate the remarkable coordination in reach-to-grasp, until control becomes impaired by neurological injuries such as stroke, neurodegenerative diseases, or even aging. Modeling online control of human reach-to-grasp movements is a challenging problem but fundamental to several domains, including behavioral and computational neuroscience, neurorehabilitation, neural prostheses, and robotics. Currently, there are no publicly available datasets that include online adjustment of reach-to-grasp movements to object perturbations. This work aims to advance modeling efforts of reach-to-grasp movements by making publicly available a large kinematic and EMG dataset of online adjustment of reach-to-grasp movements to instantaneous perturbations of object size and distance performed in immersive haptic-free virtual environment (hf-VE). The presented dataset is composed of a large number of perturbation types (10 for both object size and distance) applied at three different latencies after the start of the movement.

Toward natural grasping with a tool: effects of practice and required accuracy on the kinematics of tool-use grasping

Studies have suggested that the proficiency of an end effector is the primary factor that defines kinematics of reach-to-grasp movements across the types of effectors, such as the hand or a tool. In particular, the duration of the plateau, or the time of static open aperture (i.e., the distance between tips of effectors), is typically longer for tool use compared with natural grasping with a hand. This study investigated how improvement in the proficiency of tool use modifies the kinematics of reach-to-grasp movements. To clarify the effects of required accuracy on the kinematics in tool-use grasping, movement speed and difficulty of grasping were manipulated. The results showed that plateau duration, the length of which indicates that reaching and grasping components are temporally dissociated, shortened as tool-use practice proceeded. These results indirectly support the idea that shortened plateau duration was induced by improvement in the proficiency of tool use. Moreover, plateau duration was shortened at faster movement speeds or under conditions not requiring accurate grasping, even without any practice of tool-use grasping. Additional analyses found that plateau duration did not scale with movement time. These results suggest that the kinematic features supposed to be characteristic of tool-use grasping are not inevitable but are greatly influenced by a strategy that is not intentionally but rather automatically implemented to compensate for the lack of proficiency of end effectors, in agreement with the idea that the brain focuses on the tips of an end effector regardless of its effector type in reach-to-grasp movements.NEW & NOTEWORTHY This study is the first reporting the relation between characteristic aperture time profile, called plateau duration, and movement time of tool-use grasping. The results suggest that improved coordination between reaching and grasping components was induced by improvement in the proficiency of tool use but not by just shortened movement time. The results also indicate the possibility that the constraints for calculations in motor planning are essentially the same between hand-use grasping and tool-use grasping.

Why some size illusions affect grip aperture

There is extensive literature debating whether perceived size is used to guide grasping. A possible reason for not using judged size is that using judged positions might lead to more precise movements. As this argument does not hold for small objects and all studies showing an effect of the Ebbinghaus illusion on grasping used small objects, we hypothesized that size information is used for small objects but not for large ones. Using a modified diagonal illusion, we obtained an effect of about 10% on perceptual judgements, without an effect on grasping, irrespective of object size. We therefore reject our precision hypothesis. We discuss the results in the framework of grasping as moving digits to positions on an object. We conclude that the reported disagreement on the effect of illusions is because the Ebbinghaus illusion not only affects size, but—unlike most size illusions—also affects perceived positions.

A review of grasping as the movements of digits in space

It is tempting to describe human reach-to-grasp movements in terms of two, more or less independent visuomotor channels, one relating hand transport to the object’s location and the other relating grip aperture to the object’s size. Our review of experimental work questions this framework for reasons that go beyond noting the dependence between the two channels. Both the lack of effect of size illusions on grip aperture and the finding that the variability in grip aperture does not depend on the object’s size indicate that size information is not used to control grip aperture. An alternative is to describe grip formation as emerging from controlling the movements of the digits in space. Each digit’s trajectory when grasping an object is remarkably similar to its trajectory when moving to tap the same position on its own. The similarity is also evident in the fast responses when the object is displaced. This review develops a new description of the speed-accuracy trade-off for multiple effectors that is applied to grasping. The most direct support for the digit-in-space framework is that prism-induced adaptation of each digit’s tapping movements transfers to that digit’s movements when grasping, leading to changes in grip aperture for adaptation in opposite directions for the two digits. We conclude that although grip aperture and hand transport are convenient variables to describe grasping, treating grasping as movements of the digits in space is a more suitable basis for understanding the neural control of grasping.

The endless visuomotor calibration of reach-to-grasp actions

It is reasonable to assume that when we grasp an object we carry out the movement based only on the currently available sensory information. Unfortunately, our senses are often prone to err. Here, we show that the visuomotor system exploits the mismatch between the predicted and sensory outcomes of the immediately preceding action (sensory prediction error) to attain a degree of robustness against the fallibility of our perceptual processes. Participants performed reach-to-grasp movements toward objects presented at eye level at various distances. Grip aperture was affected by the object distance, even though both visual feedback of the hand and haptic feedback were provided. Crucially, grip aperture as well as the trajectory of the hand were systematically influenced also by the immediately preceding action. These results are well predicted by a model that modifies an internal state of the visuomotor system by adjusting the visuomotor mapping based on the sensory prediction errors. In sum, the visuomotor system appears to be in a constant fine-tuning process which makes the generation and control of grasping movements more resistant to interferences caused by our perceptual errors.

Multiple distance cues do not prevent systematic biases in reach to grasp movements

The perceived distance of objects is biased depending on the distance from the observer at which objects are presented, such that the egocentric distance tends to be overestimated for closer objects, but underestimated for objects further away. This leads to the perceived depth of an object (i.e., the perceived distance from the front to the back of the object) also being biased, decreasing with object distance. Several studies have found the same pattern of biases in grasping tasks. However, in most of those studies, object distance and depth were solely specified by ocular vergence and binocular disparities. Here we asked whether grasping objects viewed from above would eliminate distance-dependent depth biases, since this vantage point introduces additional information about the object's distance, given by the vertical gaze angle, and its depth, given by contour information. Participants grasped objects presented at different distances (1) at eye-height and (2) 130 mm below eye-height, along their depth axes. In both cases, grip aperture was systematically biased by the object distance along most of the trajectory. The same bias was found whether the objects were seen in isolation or above a ground plane to provide additional depth cues. In two additional experiments, we verified that a consistent bias occurs in a perceptual task. These findings suggest that grasping actions are not immune to biases typically found in perceptual tasks, even when additional cues are available. However, online visual control can counteract these biases when direct vision of both digits and final contact points is available.

The target as an obstacle: Grasping an object at different heights

Humans use a stereotypical movement pattern to grasp a target object. What is the cause of this stereotypical pattern? One of the possible factors is that the target object is considered an obstacle at positions other than the envisioned goal positions for the digits: while each digit aims for a goal position on the target object, they avoid other positions on the target object even if these positions do not obstruct the movement. According to this hypothesis, the maximum grip aperture will be higher if the risk of colliding with the target object is larger. Based on this hypothesis, we made a set of two unique predictions for grasping a vertically oriented cuboid at its sides at different heights. For cuboids of the same height, the maximum grip aperture will be smaller when grasped higher. For cuboids whose height varies with grip height, the maximum grip aperture will be larger when grasped higher. Both predicted relations were experimentally confirmed. This result supports the idea that considering the target object as an obstacle at positions other than the envisioned goal positions for the digits is underlying the stereotypical movement patterns in grasping. The goal positions of the digits thus influence the maximum grip aperture even if the distance between the goal positions on the target object does not change.

How removing visual information affects grasping movements

Our interaction with objects is facilitated by the availability of visual feedback. Here, we investigate how and when visual feedback affects the way we grasp an object. Based on the main views on grasping (reach-and-grasp and double-pointing views), we designed four experiments to test: (1) whether the availability of visual feedback influences the digits independently, and (2) whether the absence of visual feedback affects the initial part of the movement. Our results show that occluding (part of) the hand's movement path influences the movement trajectory from the beginning. Thus, people consider the available feedback when planning their movements. The influence of the visual feedback depends on which digit is occluded, but its effect is not restricted to the occluded digit. Our findings indicate that the control mechanisms are more complex than those suggested by current views on grasping.

Unusual prism adaptation reveals how grasping is controlled

There are three main theories on how human grasping movements are controlled. Two of them state that grip aperture and the movement of the hand are controlled. They differ in whether the wrist or the thumb of the hand is controlled. We have proposed a third theory, which states that grasping is a combination of two goal-directed single-digit movements, each directed at a specific position on the object. In this study, we test predictions based on each of the theories by examining the transfer of prism adaptation during single-digit pointing movements to grasping movements. We show that adaptation acquired during single-digit movements transfers to the hand opening when subsequently grasping objects, leaving the movement of the hand unaffected. Our results provide strong evidence for our theory that grasping with the thumb and index finger is based on a combination of two goal-directed single-digit movements.

Hitting a target is fundamentally different from avoiding obstacles

To successfully move our hand to a target, it is important not only to consider the target of our movements but also to consider other objects in the environment that may act as obstacles. We previously found that the time needed to respond to a change in position was considerably longer for a displacement of an obstacle than for a displacement of the target (Aivar, Brenner, & Smeets, 2008. Experimental Brain Research 190, 251-264). In that study, the movement constraints imposed by the obstacles differed from those imposed by the target. To examine whether the latency is really different for targets and obstacles, irrespective of any constraints they impose, we modified the design of the previous experiment to make sure that the constraints were matched. In each trial, two aligned 'objects' of the same size were presented at different distances to the left of the initial position of the hand. Each of these objects could either be a target or a gap (opening between two obstacles). Participants were instructed to pass through both objects. All possible combinations of these two objects were tested: gap-target, target-gap, gap-gap, target-target. On some trials one of the objects changed position after movement onset. Participants systematically responded faster to the displacement of a target than to the displacement of a gap at the same location. We conclude that targets are prioritized over obstacles in movement control.

Movements of individual digits in bimanual prehension are coupled into a grasping component

The classic understanding of prehension is that of coordinated reaching and grasping. An alternative view is that the grasping in prehension emerges from independently controlled individual digit movements (the double-pointing model). The current study tested this latter model in bimanual prehension: participants had to grasp an object between their two index fingers. Right after the start of the movement, the future end position of one of the digits was perturbed. The perturbations resulted in expected changes in the kinematics of the perturbed digit but also in adjusted kinematics in the unperturbed digit. The latter effects showed up when the end position of the right index finger was perturbed, but not when the end position of the left index finger was perturbed. Because the absence of a coupling between the digits is the core assumption of the double-pointing model, finding any perturbation effects challenges this account of prehension; the double-pointing model predicts that the unperturbed digit would be unaffected by the perturbation. The authors conclude that the movement of the digits in prehension is coupled into a grasping component.

Ultra-fast selection of grasping points

To grasp an object one needs to determine suitable positions on its surface for placing the digits and to move the digits to those positions. If the object is displaced during a reach-to-grasp movement, the digit movements are quickly adjusted. Do these fast adjustments only guide the digits to previously chosen positions on the surface of the object, or is the choice of contact points also constantly reconsidered? Subjects grasped a ball or a cube that sometimes rotated briefly when the digits started moving. The digits followed the rotation within 115 ms. When the object was a ball, subjects quickly counteracted the initial following response by reconsidering their choice of grasping points so that the digits ended at different positions on the rotated surface of the ball, and the ball was grasped with the preferred orientation of the hand. When the object was a cube, subjects sometimes counteracted the initial following response to grasp the cube by a different pair of sides. This altered choice of grasping points was evident within ∼160 ms of rotation onset, which is shorter than regular reaction times.

The recognition of everyday objects changes grasp scaling

Current concepts of action and perception emphasise a dissociation between conscious visual recognition and visual action control. These models do not expect an effect of the recognisable identity of an object on the kinematic parameterisation of grasping movements under binocular viewing conditions without pre-test learning periods. We performed two experiments presenting participants with familiar everyday objects or neutral geometrical objects. The participants grasped either with full vision or without visual feedback after movement onset without an explicit training phase before the experiment. In general, the familiarity of objects increased the sensitivity to physical object size changes measured by the slope of the maximal grip aperture relative to object size. We conclude that associations between object identity and a particular size, presumably encoded in long-term memory, are integrated in the parameterisation of grasping movements upon the identification of individual objects.