How removing visual information affects grasping movements

Grasping tiny objects

In grasping studies, maximum grip aperture (MGA) is commonly used as an indicator of the object size representation within the visuomotor system. However, a number of additional factors, such as movement safety, comfort, and efficiency, might affect the scaling of MGA with object size and potentially mask perceptual effects on actions. While unimanual grasping has been investigated for a wide range of object sizes, so far very small objects (<5 mm) have not been included. Investigating grasping of these tiny objects is particularly interesting because it allows us to evaluate the three most prominent explanatory accounts of grasping (the perception-action model, the digits-in-space hypothesis, and the biomechanical account) by comparing the predictions that they make for these small objects. In the first experiment, participants ( N = 26 ) grasped and manually estimated the height of square cuboids with heights from 0.5 to 5 mm. In the second experiment, a different sample of participants ( N = 24 ) performed the same tasks with square cuboids with heights from 5 to 20 mm. We determined MGAs, manual estimation apertures (MEA), and the corresponding just-noticeable differences (JND). In both experiments, MEAs scaled with object height and adhered to Weber's law. MGAs for grasping scaled with object height in the second experiment but not consistently in the first experiment. JNDs for grasping never scaled with object height. We argue that the digits-in-space hypothesis provides the most plausible account of the data. Furthermore, the findings highlight that the reliability of MGA as an indicator of object size is strongly task-dependent.

How prism adaptation reveals the distinct use of size and positions in grasping

The size of an object equals the distance between the positions of its opposite edges. However, human sensory processing for perceiving positions differs from that for perceiving size. Which of these two information sources is used to control grip aperture? In this paper, we answer this question by prism adaptation of single-digit movements of the index finger and thumb. We previously showed that it is possible to adapt the index finger and thumb in opposite directions and that this adaptation induces an aftereffect in grip aperture in grasping. This finding suggests that grasping is based on the perceived positions of the contact points. However, it might be compatible with grasping being controlled based on size provided that the opposing prism adaptation leads to changes in visually perceived size or proprioception of hand opening. In that case, one would predict a similar aftereffect in manually indicating the perceived size. In contrast, if grasping is controlled based on information about the positions of the edges, the aftereffect in grasping is due to altered position information, so one would predict no aftereffect in manually indicating the perceived size. Our present experiment shows that there was no aftereffect in manually indicating perceived size. We conclude that grip aperture during grasping is based on perceived positions rather than on perceived size.

Interaction of hand orientations during familiarization of a goal-directed aiming task

The purpose of the present study was to examine errors for an isometric goal-directed aiming task during familiarization at different hand orientation. Interaction between neutral and pronated hand orientations with and without directional feedback would provide insights into short-term adaptations and the nature of control. In this study, 30 healthy right-handed adults (age, 22.7 ± 3.1 years; weight, 69.4 ± 16.6 kg; height, 166.7 ± 7.9 cm) were randomly assigned to neutral or pronated hand orientation conditions. To assess familiarization, participants performed ten sets (16 targets/set) of goal-directed aiming task with continuous visual feedback towards targets symmetrically distributed about the origin. Following familiarization, participants then completed eight sets; four sets with and four sets without directional feedback, in an alternated order. For both hand orientations, directional errors were reduced in the first two sets (p < 0.05), suggesting only three sets were required for familiarization. Additionally, the learning rate was also similar for both hand orientations. Following familiarization, aiming errors without feedback were significantly higher than with feedback while no change between sets was observed, regardless of hand orientation. Aiming errors were reduced in the early phase with and without visual feedback, however, in the late phase, errors were corrected when visual feedback was provided. It suggests that hand orientation does not affect familiarization, and mechanisms similar to rapid learning may be involved. It is probable that learning is consolidated during familiarization along with feedforward input to maintain performance. In addition, proprioceptive feedback plays a role in reducing errors early, while the online visual feedback plays a role in reducing errors later, independent of hand orientation.

Stereopsis contributes to the predictive control of grip forces during prehension

Binocular viewing is associated with a superior prehensile performance, which is particularly evident in the latter part of the reach as the hand approaches and makes contact with the target object. However, the visuomotor mechanisms through which binocular vision serves prehension are not fully understood. This study assessed the role of stereopsis in the predictive control of grasping by measuring grip force. Twenty participants performed a precision reach-to-grasp task in four viewing conditions: binocular, monocular, and with reduced stereoacuity (200 arc sec, > 400 arc sec). Monocular, compared to binocular viewing, was associated with a fourfold increase in grasp errors, a 56% increase in grasp duration, 22% decrease in grip force at 50 ms following grasp initiation, and the time of peak force occurred 40% later after grasp initiation (all p < 0.05). Grasp performance was also disrupted when viewing with reduced stereoacuity. Notably, grip force at the time of object lift-off was comparable between all viewing conditions. These results demonstrate that binocular stereopsis contributes to the efficient programming of grip forces. Specifically, stereopsis may provide important sensory information that enables the central nervous system to engage in predictive control of grasping.

Looking away from a moving target does not disrupt the way in which the movement toward the target is guided

People usually follow a moving object with their gaze if they intend to interact with it. What would happen if they did not? We recorded eye and finger movements while participants moved a cursor toward a moving target. An unpredictable delay in updating the position of the cursor on the basis of that of the invisible finger made it essential to use visual information to guide the finger's ongoing movement. Decreasing the contrast between the cursor and the background from trial to trial made it difficult to see the cursor without looking at it. In separate experiments, either participants were free to hit the target anywhere along its trajectory or they had to move along a specified path. In the two experiments, participants tracked the cursor rather than the target with their gaze on 13% and 32% of the trials, respectively. They hit fewer targets when the contrast was low or a path was imposed. Not looking at the target did not disrupt the visual guidance that was required to deal with the delays that we imposed. Our results suggest that peripheral vision can be used to guide one item to another, irrespective of which item one is looking at.

A review of the neurobiomechanical processes underlying secure gripping in object manipulation

O'SHEA, H. and S. J. Redmond. A review of the neurobiomechanical processes underlying secure gripping in object manipulation. NEUROSCI BIOBEHAV REV 286-300, 2021. Humans display skilful control over the objects they manipulate, so much so that biomimetic systems have yet to emulate this remarkable behaviour. Two key control processes are assumed to facilitate such dexterity: predictive cognitive-motor processes that guide manipulation procedures by anticipating action outcomes; and reactive sensorimotor processes that provide important error-based information for movement adaptation. Notwithstanding increased interdisciplinary research interest in object manipulation behaviour, the complexity of the perceptual-sensorimotor-cognitive processes involved and the theoretical divide regarding the fundamentality of control mean that the essential mechanisms underlying manipulative action remain undetermined. In this paper, following a detailed discussion of the theoretical and empirical bases for understanding human dexterous movement, we emphasise the role of tactile-related sensory events in secure object handling, and consider the contribution of certain biophysical and biomechanical phenomena. We aim to provide an integrated account of the current state-of-art in skilled human-object interaction that bridges the literature in neuroscience, cognitive psychology, and biophysics. We also propose novel directions for future research exploration in this area.

The Role of Haptic Expectations in Reaching to Grasp: From Pantomime to Natural Grasps and Back Again

When we reach to pick up an object, our actions are effortlessly informed by the object's spatial information, the position of our limbs, stored knowledge of the object's material properties, and what we want to do with the object. A substantial body of evidence suggests that grasps are under the control of "automatic, unconscious" sensorimotor modules housed in the "dorsal stream" of the posterior parietal cortex. Visual online feedback has a strong effect on the hand's in-flight grasp aperture. Previous work of ours exploited this effect to show that grasps are refractory to cued expectations for visual feedback. Nonetheless, when we reach out to pretend to grasp an object (pantomime grasp), our actions are performed with greater cognitive effort and they engage structures outside of the dorsal stream, including the ventral stream. Here we ask whether our previous finding would extend to cued expectations for haptic feedback. Our method involved a mirror apparatus that allowed participants to see a "virtual" target cylinder as a reflection in the mirror at the start of all trials. On "haptic feedback" trials, participants reached behind the mirror to grasp a size-matched cylinder, spatially coincident with the virtual one. On "no-haptic feedback" trials, participants reached behind the mirror and grasped into "thin air" because no cylinder was present. To manipulate haptic expectation, we organized the haptic conditions into blocked, alternating, and randomized schedules with and without verbal cues about the availability of haptic feedback. Replicating earlier work, we found the strongest haptic effects with the blocked schedules and the weakest effects in the randomized uncued schedule. Crucially, the haptic effects in the cued randomized schedule was intermediate. An analysis of the influence of the upcoming and immediately preceding haptic feedback condition in the cued and uncued random schedules showed that cuing the upcoming haptic condition shifted the haptic influence on grip aperture from the immediately preceding trial to the upcoming trial. These findings indicate that, unlike cues to the availability of visual feedback, participants take advantage of cues to the availability of haptic feedback, flexibly engaging pantomime, and natural modes of grasping to optimize the movement.

Integration of haptics and vision in human multisensory grasping

Grasping actions are directed not only toward objects we see but also toward objects we both see and touch (multisensory grasping). In this latter case, the integration of visual and haptic inputs improves movement performance compared to each sense alone. This performance advantage could be due to the integration of all the redundant positional and size cues or to the integration of only a subset of these cues. Here we selectively provided specific cues to tease apart how these different sensory sources contribute to visuo-haptic multisensory grasping. We demonstrate that the availability of the haptic positional cue together with the visual cues is sufficient to achieve the same grasping performance as when all cues are available. These findings provide strong evidence that the human sensorimotor system relies on non-visual sensory inputs and open new perspectives on their role in supporting vision during both development and adulthood.

Predicting precision grip grasp locations on three-dimensional objects

We rarely experience difficulty picking up objects, yet of all potential contact points on the surface, only a small proportion yield effective grasps. Here, we present extensive behavioral data alongside a normative model that correctly predicts human precision grasping of unfamiliar 3D objects. We tracked participants' forefinger and thumb as they picked up objects of 10 wood and brass cubes configured to tease apart effects of shape, weight, orientation, and mass distribution. Grasps were highly systematic and consistent across repetitions and participants. We employed these data to construct a model which combines five cost functions related to force closure, torque, natural grasp axis, grasp aperture, and visibility. Even without free parameters, the model predicts individual grasps almost as well as different individuals predict one another's, but fitting weights reveals the relative importance of the different constraints. The model also accurately predicts human grasps on novel 3D-printed objects with more naturalistic geometries and is robust to perturbations in its key parameters. Together, the findings provide a unified account of how we successfully grasp objects of different 3D shape, orientation, mass, and mass distribution.

Why some size illusions affect grip aperture

There is extensive literature debating whether perceived size is used to guide grasping. A possible reason for not using judged size is that using judged positions might lead to more precise movements. As this argument does not hold for small objects and all studies showing an effect of the Ebbinghaus illusion on grasping used small objects, we hypothesized that size information is used for small objects but not for large ones. Using a modified diagonal illusion, we obtained an effect of about 10% on perceptual judgements, without an effect on grasping, irrespective of object size. We therefore reject our precision hypothesis. We discuss the results in the framework of grasping as moving digits to positions on an object. We conclude that the reported disagreement on the effect of illusions is because the Ebbinghaus illusion not only affects size, but—unlike most size illusions—also affects perceived positions.

A review of grasping as the movements of digits in space

It is tempting to describe human reach-to-grasp movements in terms of two, more or less independent visuomotor channels, one relating hand transport to the object’s location and the other relating grip aperture to the object’s size. Our review of experimental work questions this framework for reasons that go beyond noting the dependence between the two channels. Both the lack of effect of size illusions on grip aperture and the finding that the variability in grip aperture does not depend on the object’s size indicate that size information is not used to control grip aperture. An alternative is to describe grip formation as emerging from controlling the movements of the digits in space. Each digit’s trajectory when grasping an object is remarkably similar to its trajectory when moving to tap the same position on its own. The similarity is also evident in the fast responses when the object is displaced. This review develops a new description of the speed-accuracy trade-off for multiple effectors that is applied to grasping. The most direct support for the digit-in-space framework is that prism-induced adaptation of each digit’s tapping movements transfers to that digit’s movements when grasping, leading to changes in grip aperture for adaptation in opposite directions for the two digits. We conclude that although grip aperture and hand transport are convenient variables to describe grasping, treating grasping as movements of the digits in space is a more suitable basis for understanding the neural control of grasping.

Object Visibility, Not Energy Expenditure, Accounts For Spatial Biases in Human Grasp Selection

Humans exhibit spatial biases when grasping objects. These biases may be due to actors attempting to shorten their reaching movements and therefore minimize energy expenditures. An alternative explanation could be that they arise from actors attempting to minimize the portion of a grasped object occluded from view by the hand. We reanalyze data from a recent study, in which a key condition decouples these two competing hypotheses. The analysis reveals that object visibility, not energy expenditure, most likely accounts for spatial biases observed in human grasping.

Vision facilitates tactile perception when grasping an object

Tactile sensitivity measured on the hand is significantly decreased for a moving (MH), as opposed to a resting hand (RH). This process (i.e., tactile suppression) is affected by the availability of visual information during goal-directed action. However, the timing of the contribution of visual information is currently unclear for reach-to-grasp movements, especially in the period before the digits land on the object to grasp it. Here participants reached for, grasped, and lifted an object placed in front of them in conditions of full/limited vision. Tactile perception was assessed by measures of signal detection theory (d' & c'). Electro-cutaneous stimulation could be delivered/not at the MH/RH, either during movement preparation, execution, before grasping, or while lifting the object. Results confirm tactile gating at the MH. This result is accompanied by a significant conservative criterion shift at the MH for the latter movement stages. Importantly, visual information enhances MH sensitivity just before grasping the object, but also improves RH sensitivity, during object lift. These findings reveal that tactile suppression is shaped by visual inputs at critical action stages. Further, they indicate that such a time-dependent modulation from vision to touch extends beyond the MH, suggesting a dynamic monitoring of the grasp space.

Multiple distance cues do not prevent systematic biases in reach to grasp movements

The perceived distance of objects is biased depending on the distance from the observer at which objects are presented, such that the egocentric distance tends to be overestimated for closer objects, but underestimated for objects further away. This leads to the perceived depth of an object (i.e., the perceived distance from the front to the back of the object) also being biased, decreasing with object distance. Several studies have found the same pattern of biases in grasping tasks. However, in most of those studies, object distance and depth were solely specified by ocular vergence and binocular disparities. Here we asked whether grasping objects viewed from above would eliminate distance-dependent depth biases, since this vantage point introduces additional information about the object's distance, given by the vertical gaze angle, and its depth, given by contour information. Participants grasped objects presented at different distances (1) at eye-height and (2) 130 mm below eye-height, along their depth axes. In both cases, grip aperture was systematically biased by the object distance along most of the trajectory. The same bias was found whether the objects were seen in isolation or above a ground plane to provide additional depth cues. In two additional experiments, we verified that a consistent bias occurs in a perceptual task. These findings suggest that grasping actions are not immune to biases typically found in perceptual tasks, even when additional cues are available. However, online visual control can counteract these biases when direct vision of both digits and final contact points is available.