Does visuomotor adaptation contribute to illusion-resistant grasping?

Revisiting the effect of visual illusions on grasping in left and right handers

Visual illusions have provided compelling evidence for a dissociation between perception and action. For example, when two different-sized objects are placed on opposite ends of the Ponzo illusion, people erroneously perceive the physically smaller object to be bigger than the physically larger one, but when they pick up the objects, their grip aperture reflects the real difference in size between the objects. This and similar findings have been demonstrated almost entirely for the right hand in right handers. The scarce research that has examined right and left-handed subjects in this context, has typically used only small samples. Here, we extended this research with a larger sample size (more than 50 in each group) in a version of the Ponzo illusion that allowed us to disentangle the effects of real and illusory size on action and perception in much more powerful way. We also collected a wide range of kinematic measures to assess possible differences in visuomotor control in left and right handers. The results showed that the dissociation between perception and action persisted for both hands in right handers, but only for the right hand in left handers. The left hand of left handers was sensitive to the illusion. Left handers also showed more variable and slower movements, as well as larger safety margins in both hands. These findings suggest that grasping in left handers may require more cognitive supervision, which could lead to greater sensitivity to visual context , particularly with their dominant left hand.

Coming to grips with reality: Real grasps, but not pantomimed grasps, resist a simultaneous tilt illusion

Investigations of grasping real, 3D objects subjected to illusory effects from a pictorial background often choose in-flight grasp aperture as the primary variable to test the hypothesis that the visuomotor system resists the illusion. Here we test an equally important feature of grasps that has received less attention: in-flight grasp orientation. The current study tested a variant of the simultaneous tilt illusion using a mirror-apparatus to manipulate the availability of haptic feedback. Participants performed grasps with haptic feedback (real grasps) and without it (pantomime grasps), reaching for the reflection of a real, 3D bar atop a background grating that induced a 1.1° bias in the perceived orientation of the bar in a separate sample of participants. Analysis of the hand's in-flight grasp orientation at early, late, and end stages of the reach showed that at no point were the real grasps biased by the illusion. In contrast, pantomimed grasps were affected by the illusion at the late and end stages of the reach. At each stage, the effect on the real grasps was significantly weaker than the effect of the illusion as measured by the mean point of subjective equality (PSE) in a two-alternative forced-choice task. In contrast, the effect on the pantomime grasps was statistically indistinguishable from the mean PSE at all three stages of the reach. These findings reinforce the idea that in-flight grasp orientation, like grasp aperture to pictorial illusions of target size, is refractory to pictorial backgrounds that bias perceived orientation.

Not only perception but also grasping actions can obey Weber's law

Weber's law, the principle that the uncertainty of perceptual estimates increases proportionally with object size, is regularly violated when considering the uncertainty of the grip aperture during grasping movements. The origins of this perception-action dissociation are debated and are attributed to various reasons, including different coding of visual size information for perception and action, biomechanical factors, the use of positional information to guide grasping, or, sensorimotor calibration. Here, we contrasted these accounts and compared perceptual and grasping uncertainties by asking people to indicate the visually perceived center of differently sized objects (Perception condition) or to grasp and lift the same objects with the requirement to achieve a balanced lift (Action condition). We found that the variability (uncertainty) of contact positions increased as a function of object size in both perception and action. The adherence of the Action condition to Weber's law and the consequent absence of a perception-action dissociation contradict the predictions based on different coding of visual size information and sensorimotor calibration. These findings provide clear evidence that human perceptual and visuomotor systems rely on the same visual information and suggest that the previously reported violations of Weber's law in grasping movements should be attributed to other factors.

Looking at the Ebbinghaus illusion: differences in neurocomputational requirements, not gaze-mediated attention, explain a classic perception-action dissociation

Perceiving and grasping an object present an animal with different sets of computational problems. The solution in primates entails the specialization of separate neural networks for visual processing with different object representations. This explains why the Ebbinghaus illusion minimally affects the grasping hand's in-flight aperture, which normally scales with target size, even though the size of the target disc remains misperceived. An attractive alternative account, however, posits that grasps are refractory to the illusion because participants fixate on the target and fail to attend to the surrounding context. To test this account, we tracked both limb and gaze while participants made forced-choice judgments of relative disc size in the Ebbinghaus illusion or did so in combination with grasping or manually estimating the size of one of the discs. We replicated the classic dissociation: grasp aperture was refractory to the measured illusory effect on perceived size, while judgments and manual estimates of disc size were not. Importantly, the number of display-wide saccades per second and the percentage of total fixation time or fixations directed at the selected disc failed to explain the dissociation. Our findings support the contention that object perception and goal-directed action rely on distinct visual representations. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Relative, not absolute, stimulus size is responsible for a correspondence effect between physical stimulus size and left/right responses

Recent studies have demonstrated a novel compatibility (or correspondence) effect between physical stimulus size and horizontally aligned responses: Left-hand responses are shorter and more accurate to a small stimulus, compared to a large stimulus, whereas the opposite is true for right-hand responses. The present study investigated whether relative or absolute size is responsible for the effect. If relative size was important, a particular stimulus would elicit faster left-hand responses if the other stimuli in the set were larger, but the same stimulus would elicit a faster right-hand response if the other stimuli in the set were smaller. In terms of two-visual-systems theory, our study explores whether “vision for perception” (i.e., the ventral system) or “vision for action” (i.e., the dorsal system) dominates the processing of stimulus size in our task. In two experiments, participants performed a discrimination task in which they responded to stimulus color (Experiment 1) or to stimulus shape (Experiment 2) with their left/right hand. Stimulus size varied as an irrelevant stimulus feature, thus leading to corresponding (small-left; large-right) and non-corresponding (small-right; large-left) conditions. Moreover, a set of smaller stimuli and a set of larger stimuli, with both sets sharing an intermediately sized stimulus, were used in different conditions. The consistently significant two-way interaction between stimulus size and response location demonstrated the presence of the correspondence effect. The three-way interaction between stimulus size, response location, and stimulus set, however, was never significant. The results suggest that participants are inadvertently classifying stimuli according to relative size in a context-specific manner.

Explicit and implicit depth-cue integration: Evidence of systematic biases with real objects

In previous studies using VR, we found evidence that 3D shape estimation agrees to a superadditivity rule of depth-cue combination, by which adding depth cues leads to greater perceived depth and, in principle, to depth overestimation. Superadditivity can be quantitatively accounted for by a normative theory of cue integration, via adapting a model termed Intrinsic Constraint (IC). As for its qualitative nature, it remains unclear whether superadditivity represents the genuine readout of depth-cue integration, as predicted by IC, or alternatively a byproduct of artificial virtual displays, because they carry flatness cues that can bias depth estimates in a Bayesian fashion, or even just a way for observers to express that a scene "looks deeper" with more depth cues by explicitly inflating their depth judgments. In the present study, we addressed this question by testing whether the IC model's prediction of superadditivity generalizes to real world settings. We asked participants to judge the perceived 3D shape of cardboard prisms through a matching task. To control for the potential interference of explicit reasoning, we also asked participants to reach-to-grasp the same objects and we analyzed the in-flight grip size throughout the reaching. We designed a novel technique to carefully control binocular and monocular 3D cues independently, allowing to add or remove depth information seamlessly. Even with real objects, participants exhibited a clear superadditivity effect in both tasks. Furthermore, the magnitude of this effect was accurately predicted by the IC model. These results confirm that superadditivity is an inherent feature of depth estimation.

Priming of the Sander Parallelogram illusion separates perception from action

The two-visual stream hypothesis posits that the dorsal stream is less susceptible than the ventral stream to the effects of illusions and visual priming. While previous studies have separately examined these perceptual manipulations, the present study combined the effects of a visual illusion and priming to examine the possibility of dorsally guided actions being susceptible to the perceptual stimuli due to interactions between the two streams. Thirty-four participants were primed with a 'long' or 'short' version of the Sander Parallelogram illusion and were asked to either reach out and grasp or manually estimate the length of a rod placed on a version of the illusion that was on some trials the same as the prime (congruent) and on other trials was the inverse (incongruent). Due to the context-focused nature of ventral processing, we predicted that estimations would be more susceptible to the effects of the illusion and priming than grasps. Results showed that while participants' manual estimations were susceptible to both priming and the illusion, the grasps were only affected by the illusion, not by priming. The influence of the illusion on grip aperture was greater during manual estimations than it was during grasping. These findings support the notion that the functionally distinct dorsal and ventral streams interact under the current experimental paradigm. Outcomes of the study help better understand the nature of stimuli that promote interactions between the dorsal and ventral streams.

A double dissociation between action and perception in bimanual grasping: evidence from the Ponzo and the Wundt-Jastrow illusions

Research on visuomotor control suggests that visually guided actions toward objects rely on functionally distinct computations with respect to perception. For example, a double dissociation between grasping and between perceptual estimates was reported in previous experiments that pit real against illusory object size differences in the context of the Ponzo illusion. While most previous research on the relation between action and perception focused on one-handed grasping, everyday visuomotor interactions also entail the simultaneous use of both hands to grasp objects that are larger in size. Here, we examined whether this double dissociation extends to bimanual movement control. In Experiment 1, participants were presented with different-sized objects embedded in the Ponzo Illusion. In Experiment 2, we tested whether the dissociation between perception and action extends to a different illusion, the Wundt–Jastrow illusion, which has not been previously used in grasping experiments. In both experiments, bimanual grasping trajectories reflected the differences in physical size between the objects; At the same time, perceptual estimates reflected the differences in illusory size between the objects. These results suggest that the double dissociation between action and perception generalizes to bimanual movement control. Unlike conscious perception, bimanual grasping movements are tuned to real-world metrics, and can potentially resist irrelevant information on relative size and depth.

Spatial updating of allocentric landmark information in real-time and memory-guided reaching

The 2-streams model of vision suggests that egocentric and allocentric reference frames are utilized by the dorsal and the ventral stream for real-time and memory-guided movements, respectively. Recent studies argue against such a strict functional distinction and suggest that real-time and memory-guided movements recruit the same spatial maps. In this study we focus on allocentric spatial coding and updating of targets by using landmark information in real-time and memory-guided reaching. We presented participants with a naturalistic scene which consisted of six objects on a table that served as potential reach targets. Participants were informed about the target object after scene encoding, and were prompted by a go cue to reach to its position. After target identification a brief air-puff was applied to the participant's right eye inducing an eye blink. During the blink the target object disappeared from the scene, and in half of the trials the remaining objects, that functioned as landmarks, were shifted horizontally in the same direction. We found that landmark shifts systematically influenced participants' reaching endpoints irrespective of whether the movements were controlled online based on available target information (real-time movement) or memory-guided based on remembered target information (memory-guided movement). Overall, the effect of landmark shift was stronger for memory-guided than real-time reaching. Our findings suggest that humans can encode and update reach targets in an allocentric reference frame for both real-time and memory-guided movements and show stronger allocentric coding when the movement is based on memory.

A review of grasping as the movements of digits in space

It is tempting to describe human reach-to-grasp movements in terms of two, more or less independent visuomotor channels, one relating hand transport to the object’s location and the other relating grip aperture to the object’s size. Our review of experimental work questions this framework for reasons that go beyond noting the dependence between the two channels. Both the lack of effect of size illusions on grip aperture and the finding that the variability in grip aperture does not depend on the object’s size indicate that size information is not used to control grip aperture. An alternative is to describe grip formation as emerging from controlling the movements of the digits in space. Each digit’s trajectory when grasping an object is remarkably similar to its trajectory when moving to tap the same position on its own. The similarity is also evident in the fast responses when the object is displaced. This review develops a new description of the speed-accuracy trade-off for multiple effectors that is applied to grasping. The most direct support for the digit-in-space framework is that prism-induced adaptation of each digit’s tapping movements transfers to that digit’s movements when grasping, leading to changes in grip aperture for adaptation in opposite directions for the two digits. We conclude that although grip aperture and hand transport are convenient variables to describe grasping, treating grasping as movements of the digits in space is a more suitable basis for understanding the neural control of grasping.

The endless visuomotor calibration of reach-to-grasp actions

It is reasonable to assume that when we grasp an object we carry out the movement based only on the currently available sensory information. Unfortunately, our senses are often prone to err. Here, we show that the visuomotor system exploits the mismatch between the predicted and sensory outcomes of the immediately preceding action (sensory prediction error) to attain a degree of robustness against the fallibility of our perceptual processes. Participants performed reach-to-grasp movements toward objects presented at eye level at various distances. Grip aperture was affected by the object distance, even though both visual feedback of the hand and haptic feedback were provided. Crucially, grip aperture as well as the trajectory of the hand were systematically influenced also by the immediately preceding action. These results are well predicted by a model that modifies an internal state of the visuomotor system by adjusting the visuomotor mapping based on the sensory prediction errors. In sum, the visuomotor system appears to be in a constant fine-tuning process which makes the generation and control of grasping movements more resistant to interferences caused by our perceptual errors.