Utilization of visual feedback of the hand according to target view availability in the online control of prehension movements

Eye-hand coordination: memory-guided grasping during obstacle avoidance

When reaching to grasp previously seen, now out-of-view objects, we rely on stored perceptual representations to guide our actions, likely encoded by the ventral visual stream. So-called memory-guided actions are numerous in daily life, for instance, as we reach to grasp a coffee cup hidden behind our morning newspaper. Little research has examined obstacle avoidance during memory-guided grasping, though it is possible obstacles with increased perceptual salience will provoke exacerbated avoidance maneuvers, like exaggerated deviations in eye and hand position away from obtrusive obstacles. We examined the obstacle avoidance strategies adopted as subjects reached to grasp a 3D target object under visually-guided (closed loop or open loop with full vision prior to movement onset) and memory-guided (short- or long-delay) conditions. On any given trial, subjects reached between a pair of flanker obstacles to grasp a target object. The positions and widths of the obstacles were manipulated, though their inner edges remained a constant distance apart. While reach and grasp behavior was consistent with the obstacle avoidance literature, in that reach, grasp, and gaze positions were biased away from obstacles most obtrusive to the reaching hand, our results reveal distinctive avoidance approaches undertaken depend on the availability of visual feedback. Contrary to expectation, we found subjects reaching to grasp after a long delay in the absence of visual feedback failed to modify their final fixation and grasp positions to accommodate the different positions of obstacles, demonstrating a more moderate, rather than exaggerative, obstacle avoidance strategy.

Some binocular advantages for planning reach, but not grasp, components of prehension

Proficient (fast, accurate, precise) hand actions for reaching-to-grasp 3D objects are known to benefit significantly from the use of binocular vision compared to one eye alone. We examined whether these binocular advantages derive from increased reliability in encoding the goal object’s properties for feedforward planning of prehension movements or from enhanced feedback mediating their online control. Adult participants reached for, precision grasped and lifted cylindrical table-top objects (two sizes, 2 distances) using binocular vision or only their dominant/sighting eye or their non-dominant eye to program and fully execute their movements or using each of the three viewing conditions only to plan their reach-to-grasp during a 1 s preview, with vision occluded just before movement onset. Various kinematic measures of reaching and grasping proficiency, including corrective error rates, were quantified and compared by view, feedback and object type. Some significant benefits of binocular over monocular vision when they were just available for pre-movement planning were retained for the reach regardless of target distance, including higher peak velocities, straighter paths and shorter low velocity approach times, although these latter were contaminated by more velocity corrections and by poorer coordination with object contact. By contrast, virtually all binocular advantages for grasping, including improvements in peak grip aperture scaling, the accuracy and precision of digit placements at object contact and shorter grip application times preceding the lift, were eliminated with no feedback available, outcomes that were influenced by the object’s size. We argue that vergence cues can improve the reliability of binocular internal representations of object distance for the feedforward programming of hand transport, whereas the major benefits of binocular vision for enhancing grasping performance derive exclusively from its continuous presence online.

How removing visual information affects grasping movements

Our interaction with objects is facilitated by the availability of visual feedback. Here, we investigate how and when visual feedback affects the way we grasp an object. Based on the main views on grasping (reach-and-grasp and double-pointing views), we designed four experiments to test: (1) whether the availability of visual feedback influences the digits independently, and (2) whether the absence of visual feedback affects the initial part of the movement. Our results show that occluding (part of) the hand's movement path influences the movement trajectory from the beginning. Thus, people consider the available feedback when planning their movements. The influence of the visual feedback depends on which digit is occluded, but its effect is not restricted to the occluded digit. Our findings indicate that the control mechanisms are more complex than those suggested by current views on grasping.

Grasping in absence of feedback: systematic biases endure extensive training

Reach-to-grasp movements performed without visual and haptic feedback of the hand are subject to systematic inaccuracies. Grasps directed at an object specified by binocular information usually end at the wrong distance with an incorrect final grip aperture. More specifically, moving the target object away from the observer leads to increasingly larger undershoots and smaller grip apertures. These systematic biases suggest that the visuomotor mapping is based on inaccurate estimates of an object's egocentric distance and 3D structure that compress the visual space. Here we ask whether the appropriate visuomotor mapping can be learned through an extensive exposure to trials where haptic and visual feedback of the hand is provided. By intermixing feedback trials with test trials without feedback, we aimed at maximizing the likelihood that the motor execution of test trials is positively influenced by that of preceding feedback trials. We found that the intermittent presence of feedback trials both (1) largely reduced the positioning error of the hand with respect to the object and (2) affected the shaping of the hand before the final grasp, leading to an overall more accurate performance. While this demonstrates an effective transfer of information from feedback trials to test trials, the remaining biases indicate that a compression of visual space is still taking place. The correct visuomotor mapping, therefore, could not be learned. We speculate that an accurate reconstruction of the scene at movement onset may not actually be needed. Instead, the online monitoring of the hand position relative to the object and the final contact with the object are sufficient for a successful execution of a grasp.

Use of early phase online vision for grip configuration is modulated according to movement duration in prehension

Our previous study (Hum Mov Sci 25:349-371, 2006) investigated whether and how online vision in the early phase of movement influences the control of reach-to-grasp movements (movement duration: approximately 1000 ms). We used liquid-crystal shutter goggles to manipulate the duration of available online vision during the movement and specified that online vision during the early phase influences grasping movements. The current study examined the effect of online early phase vision on the grip configuration according to the movement duration and compared it between two different movement durations (approximately 500 and 1000 ms). We found that non-availability of early phase online vision affected the grip configuration (i.e., inducing a larger peak grip aperture) even in the shorter movement duration. The influential period for online vision for grasping control shifts to an earlier time when movement time is shorter (i.e., from approximately 214 to 106 ms after movement onset), indicating a flexible mechanism for grip configuration according to the movement duration and the available online vision.

Effect of visual and haptic feedback on grasping movements

Perceptual estimates of three-dimensional (3D) properties, such as the distance and depth of an object, are often inaccurate. Given the accuracy and ease with which we pick up objects, it may be expected that perceptual distortions do not affect how the brain processes 3D information for reach-to-grasp movements. Nonetheless, empirical results show that grasping accuracy is reduced when visual feedback of the hand is removed. Here we studied whether specific types of training could correct grasping behavior to perform adequately even when any form of feedback is absent. Using a block design paradigm, we recorded the movement kinematics of subjects grasping virtual objects located at different distances in the absence of visual feedback of the hand and haptic feedback of the object, before and after different training blocks with different feedback combinations (vision of the thumb and vision of thumb and index finger, with and without tactile feedback of the object). In the Pretraining block, we found systematic biases of the terminal hand position, the final grip aperture, and the maximum grip aperture like those reported in perceptual tasks. Importantly, the distance at which the object was presented modulated all these biases. In the Posttraining blocks only the hand position was partially adjusted, but final and maximum grip apertures remained unchanged. These findings show that when visual and haptic feedback are absent systematic distortions of 3D estimates affect reach-to-grasp movements in the same way as they affect perceptual estimates. Most importantly, accuracy cannot be learned, even after extensive training with feedback.