Center or side: biases in selecting grasp points on small bars

Human manipulation strategy when changing object deformability and task properties

Robotic literature widely addresses deformable object manipulation, but few studies analyzed human manipulation accounting for different levels of deformability and task properties. We asked participants to grasp and insert rigid and deformable objects into holes with varying tolerances and depths, and we analyzed the grasping behavior, the reaching velocity profile, and completion times. Results indicated that the more deformable the object is, the nearer the grasping point is to the extremity to be inserted. For insertions in the long hole, the selection of the grasping point is a trade-off between task accuracy and the number of re-grasps required to complete the insertion. The compliance of the deformable object facilitates the alignment between the object and the hole. The reaching velocity profile when increasing deformability recalls the one observed when task accuracy and precision decrease. Identifying human strategy allows the implementation of human-inspired high-level reasoning algorithms for robotic manipulation.

Healthy adults favor stable left/right hand choices over performance at an unconstrained reach-to-grasp task

Reach-to-grasp actions are fundamental to the daily activities of human life, but few methods exist to assess individuals' reaching and grasping actions in unconstrained environments. The Block Building Task (BBT) provides an opportunity to directly observe and quantify these actions, including left/right hand choices. Here we sought to investigate the motor and non-motor causes of left/right hand choices, and optimize the design of the BBT, by manipulating motor and non-motor difficulty in the BBT's unconstrained reach-to-grasp task. We hypothesized that greater motor and non-motor (e.g. cognitive/perceptual) difficulty would drive increased usage of the dominant hand. To test this hypothesis, we modulated block size (large vs. small) to influence motor difficulty, and model complexity (10 vs. 5 blocks per model) to influence non-motor difficulty, in healthy adults (n = 57). Our data revealed that increased motor and non-motor difficulty led to lower task performance (slower task speed), but participants only increased use of their dominant hand only under the most difficult combination of conditions: in other words, participants allowed their performance to degrade before changing hand choices, even though participants were instructed only to optimize performance. These results demonstrate that hand choices during reach-to grasp actions are more stable than motor performance in healthy right-handed adults, but tasks with multifaceted difficulties can drive individuals to rely more on their dominant hand.

Eye-hand coordination: memory-guided grasping during obstacle avoidance

When reaching to grasp previously seen, now out-of-view objects, we rely on stored perceptual representations to guide our actions, likely encoded by the ventral visual stream. So-called memory-guided actions are numerous in daily life, for instance, as we reach to grasp a coffee cup hidden behind our morning newspaper. Little research has examined obstacle avoidance during memory-guided grasping, though it is possible obstacles with increased perceptual salience will provoke exacerbated avoidance maneuvers, like exaggerated deviations in eye and hand position away from obtrusive obstacles. We examined the obstacle avoidance strategies adopted as subjects reached to grasp a 3D target object under visually-guided (closed loop or open loop with full vision prior to movement onset) and memory-guided (short- or long-delay) conditions. On any given trial, subjects reached between a pair of flanker obstacles to grasp a target object. The positions and widths of the obstacles were manipulated, though their inner edges remained a constant distance apart. While reach and grasp behavior was consistent with the obstacle avoidance literature, in that reach, grasp, and gaze positions were biased away from obstacles most obtrusive to the reaching hand, our results reveal distinctive avoidance approaches undertaken depend on the availability of visual feedback. Contrary to expectation, we found subjects reaching to grasp after a long delay in the absence of visual feedback failed to modify their final fixation and grasp positions to accommodate the different positions of obstacles, demonstrating a more moderate, rather than exaggerative, obstacle avoidance strategy.

Manipulation of physical 3-D and virtual 2-D stimuli: comparing digit placement and fixation position

The visuomotor processes involved in grasping a 2-D target are known to be fundamentally different than those involved in grasping a 3-D object, and this has led to concerns regarding the generalizability of 2-D grasping research. This study directly compared participants' fixation positions and digit placement during interaction with either physical square objects or 2-D virtual versions of these objects. Participants were instructed to either simply grasp the stimulus or grasp and slide it to another location. Participants' digit placement and fixation positions did not significantly differ as a function of stimulus type when grasping in the center of the display. However, gaze and grasp positions shifted toward the near side of non-central virtual stimuli, while consistently remaining close to the horizontal midline of the physical stimulus. Participants placed their digits at less stable locations when grasping the virtual stimulus in comparison to the physical stimulus on the right side of the display, but this difference disappeared when grasping in the center and on the left. Similar outward shifts in digit placement and lowered fixations were observed when sliding both stimulus types, suggesting participants incorporated similar adjustments in grasp selection in anticipation of manipulation in both Physical and Virtual stimulus conditions. These results suggest that while fixation position and grasp point selection differed between stimulus type as a function of stimulus position, certain eye-hand coordinated behaviours were maintained when grasping both physical and virtual stimuli.

Humans Can Visually Judge Grasp Quality and Refine Their Judgments Through Visual and Haptic Feedback

How humans visually select where to grasp objects is determined by the physical object properties (e.g., size, shape, weight), the degrees of freedom of the arm and hand, as well as the task to be performed. We recently demonstrated that human grasps are near-optimal with respect to a weighted combination of different cost functions that make grasps uncomfortable, unstable, or impossible, e.g., due to unnatural grasp apertures or large torques. Here, we ask whether humans can consciously access these rules. We test if humans can explicitly judge grasp quality derived from rules regarding grasp size, orientation, torque, and visibility. More specifically, we test if grasp quality can be inferred (i) by using visual cues and motor imagery alone, (ii) from watching grasps executed by others, and (iii) through performing grasps, i.e., receiving visual, proprioceptive and haptic feedback. Stimuli were novel objects made of 10 cubes of brass and wood (side length 2.5 cm) in various configurations. On each object, one near-optimal and one sub-optimal grasp were selected based on one cost function (e.g., torque), while the other constraints (grasp size, orientation, and visibility) were kept approximately constant or counterbalanced. Participants were visually cued to the location of the selected grasps on each object and verbally reported which of the two grasps was best. Across three experiments, participants were required to either (i) passively view the static objects and imagine executing the two competing grasps, (ii) passively view videos of other participants grasping the objects, or (iii) actively grasp the objects themselves. Our results show that, for a majority of tested objects, participants could already judge grasp optimality from simply viewing the objects and imagining to grasp them, but were significantly better in the video and grasping session. These findings suggest that humans can determine grasp quality even without performing the grasp-perhaps through motor imagery-and can further refine their understanding of how to correctly grasp an object through sensorimotor feedback but also by passively viewing others grasp objects.

Predicting precision grip grasp locations on three-dimensional objects

We rarely experience difficulty picking up objects, yet of all potential contact points on the surface, only a small proportion yield effective grasps. Here, we present extensive behavioral data alongside a normative model that correctly predicts human precision grasping of unfamiliar 3D objects. We tracked participants' forefinger and thumb as they picked up objects of 10 wood and brass cubes configured to tease apart effects of shape, weight, orientation, and mass distribution. Grasps were highly systematic and consistent across repetitions and participants. We employed these data to construct a model which combines five cost functions related to force closure, torque, natural grasp axis, grasp aperture, and visibility. Even without free parameters, the model predicts individual grasps almost as well as different individuals predict one another's, but fitting weights reveals the relative importance of the different constraints. The model also accurately predicts human grasps on novel 3D-printed objects with more naturalistic geometries and is robust to perturbations in its key parameters. Together, the findings provide a unified account of how we successfully grasp objects of different 3D shape, orientation, mass, and mass distribution.

Grasping a 2D virtual target: The influence of target position and movement on gaze and digit placement

While much has been learned about the visual pursuit and motor strategies used to intercept a moving object, less research has focused on the coordination of gaze and digit placement when grasping moving stimuli. Participants grasped 2D computer generated square targets that either encouraged placement of the index finger and thumb along the horizontal midline (Control targets) or had narrow "notches" in the top and bottom surfaces of the target, intended to discourage digit placement near the midline (Experimental targets). In Experiment 1, targets remained stationary at the left, middle, or right side of the screen. Gaze and digit placement were biased toward the closest side of non-central targets, and toward the midline of center targets. These locations were shifted rightward when grasping Experimental targets, suggesting participants prioritized visibility of the target. In Experiment 2, participants grasped horizontally translating targets at early, middle, or late stages of travel. Average gaze and digit placement were consistently positioned behind the moving target's horizontal midline when grasping. Gaze was directed farther behind the midline of Experimental targets, suggesting the absence of a flat central grasp location pulled participants' gaze toward the trailing edge. Participants placed their digits at positions closer to the horizontal midline of leftward moving targets, suggesting participants were compensating for the added mechanical constraints associated with grasping targets moving in a direction contralateral to the grasping hand. These results suggest participants minimize the effort associated with reaching to non-central targets by grasping the nearest side when the target is stationary, but grasp the trailing side of moving targets, even if this means placing the digits at locations on the far side of the target, potentially limiting visibility of the target.

The Visual Perception of Material Properties Affects Motor Planning in Prehension: An Analysis of Temporal and Spatial Components of Lifting Cups

The current study examined the role of visually perceived material properties in motor planning, where we analyzed the temporal and spatial components of motor movements during a seated reaching task. We recorded hand movements of 14 participants in three dimensions while they lifted and transported paper cups that differed in weight and glossiness. Kinematic- and spatial analysis revealed speed-accuracy trade-offs to depend on visual material properties of the objects, in which participants reached slower and grabbed closer to the center of mass for stimuli that required to be handled with greater precision. We found grasp-preparation during the first encounters with the cups was not only governed by the anticipated weight of the cups, but also by their visual material properties, namely glossiness. After a series of object lifting, the execution of reaching, the grip position, and the transportation of the cups from one location to another were preeminently guided by the object weight. We also found the planning phase in reaching to be guided by the expectation of hardness and surface gloss. The findings promote the role of general knowledge of material properties in reach-to-grasp movements, in which visual material properties are incorporated in the spatio-temporal components.

A review of grasping as the movements of digits in space

It is tempting to describe human reach-to-grasp movements in terms of two, more or less independent visuomotor channels, one relating hand transport to the object’s location and the other relating grip aperture to the object’s size. Our review of experimental work questions this framework for reasons that go beyond noting the dependence between the two channels. Both the lack of effect of size illusions on grip aperture and the finding that the variability in grip aperture does not depend on the object’s size indicate that size information is not used to control grip aperture. An alternative is to describe grip formation as emerging from controlling the movements of the digits in space. Each digit’s trajectory when grasping an object is remarkably similar to its trajectory when moving to tap the same position on its own. The similarity is also evident in the fast responses when the object is displaced. This review develops a new description of the speed-accuracy trade-off for multiple effectors that is applied to grasping. The most direct support for the digit-in-space framework is that prism-induced adaptation of each digit’s tapping movements transfers to that digit’s movements when grasping, leading to changes in grip aperture for adaptation in opposite directions for the two digits. We conclude that although grip aperture and hand transport are convenient variables to describe grasping, treating grasping as movements of the digits in space is a more suitable basis for understanding the neural control of grasping.

Object Visibility, Not Energy Expenditure, Accounts For Spatial Biases in Human Grasp Selection

Humans exhibit spatial biases when grasping objects. These biases may be due to actors attempting to shorten their reaching movements and therefore minimize energy expenditures. An alternative explanation could be that they arise from actors attempting to minimize the portion of a grasped object occluded from view by the hand. We reanalyze data from a recent study, in which a key condition decouples these two competing hypotheses. The analysis reveals that object visibility, not energy expenditure, most likely accounts for spatial biases observed in human grasping.

The target as an obstacle: Grasping an object at different heights

Humans use a stereotypical movement pattern to grasp a target object. What is the cause of this stereotypical pattern? One of the possible factors is that the target object is considered an obstacle at positions other than the envisioned goal positions for the digits: while each digit aims for a goal position on the target object, they avoid other positions on the target object even if these positions do not obstruct the movement. According to this hypothesis, the maximum grip aperture will be higher if the risk of colliding with the target object is larger. Based on this hypothesis, we made a set of two unique predictions for grasping a vertically oriented cuboid at its sides at different heights. For cuboids of the same height, the maximum grip aperture will be smaller when grasped higher. For cuboids whose height varies with grip height, the maximum grip aperture will be larger when grasped higher. Both predicted relations were experimentally confirmed. This result supports the idea that considering the target object as an obstacle at positions other than the envisioned goal positions for the digits is underlying the stereotypical movement patterns in grasping. The goal positions of the digits thus influence the maximum grip aperture even if the distance between the goal positions on the target object does not change.

Effects of material properties and object orientation on precision grip kinematics

Successfully picking up and handling objects requires taking into account their physical properties (e.g., material) and position relative to the body. Such features are often inferred by sight, but it remains unclear to what extent observers vary their actions depending on the perceived properties. To investigate this, we asked participants to grasp, lift and carry cylinders to a goal location with a precision grip. The cylinders were made of four different materials (Styrofoam, wood, brass and an additional brass cylinder covered with Vaseline) and were presented at six different orientations with respect to the participant (0°, 30°, 60°, 90°, 120°, 150°). Analysis of their grasping kinematics revealed differences in timing and spatial modulation at all stages of the movement that depended on both material and orientation. Object orientation affected the spatial configuration of index finger and thumb during the grasp, but also the timing of handling and transport duration. Material affected the choice of local grasp points and the duration of the movement from the first visual input until release of the object. We find that conditions that make grasping more difficult (orientation with the base pointing toward the participant, high weight and low surface friction) lead to longer durations of individual movement segments and a more careful placement of the fingers on the object.

The influence of object shape and center of mass on grasp and gaze

Recent experiments examining where participants look when grasping an object found that fixations favor the eventual index finger landing position on the object. Even though the act of picking up an object must involve complex high-level computations such as the visual analysis of object contours, surface properties, knowledge of an object's function and center of mass (COM) location, these investigations have generally used simple symmetrical objects - where COM and horizontal midline overlap. Less research has been aimed at looking at how variations in object properties, such as differences in curvature and changes in COM location, affect visual and motor control. The purpose of this study was to examine grasp and fixation locations when grasping objects whose COM was positioned to the left or right of the objects horizontal midline (Experiment 1) and objects whose COM was moved progressively further from the midline of the objects based on the alteration of the object's shape (Experiment 2). Results from Experiment 1 showed that object COM position influenced fixation locations and grasp locations differently, with fixations not as tightly linked to index finger grasp locations as was previously reported with symmetrical objects. Fixation positions were also found to be more central on the non-symmetrical objects. This difference in gaze position may provide a more holistic view, which would allow both index finger and thumb positions to be monitored while grasping. Finally, manipulations of COM distance (Experiment 2) exerted marked effects on the visual analysis of the objects when compared to its influence on grasp locations, with fixation locations more sensitive to these manipulations. Together, these findings demonstrate how object features differentially influence gaze vs. grasp positions during object interaction.

The hand grasps the center, while the eyes saccade to the top of novel objects

In the present study, we investigated whether indenting the sides of novel objects (e.g., product packaging) would influence where people grasp, and hence focus their gaze, under the assumption that gaze precedes grasping. In Experiment 1, the participants grasped a selection of custom-made objects designed to resemble typical packaging forms with an indentation in the upper, middle, or lower part. In Experiment 2, eye movements were recorded while the participants viewed differently-sized (small, medium, and large) objects with the same three indentation positions tested in Experiment 1, together with a control object lacking any indentation. The results revealed that irrespective of the location of the indentation, the participants tended to grasp the mid-region of the object, with their index finger always positioned slightly above its midpoint. Importantly, the first visual fixation tended to fall in the cap region of the novel object. The participants also fixated for longer in this region. Furthermore, participants saccaded more often, as well saccading more rapidly when directing their gaze to the upper region of the objects that they were required to inspect visually. Taken together, these results therefore suggest that different spatial locations on target objects are of interest to our eyes and hands.