The "What" and "How" of Pantomime Actions

Pantomimes are human actions that simulate ideas, objects, and events, commonly used in conversation, performance art, and gesture-based interfaces for computing and controlling robots. Yet, their underlying neurocognitive mechanisms are not well understood. In this review, we examine pantomimes through two parallel lines of research: (1) the two visual systems (TVS) framework for visually guided action, and (2) the neuropsychological literature on limb apraxia. Historically, the TVS framework has considered pantomime actions as expressions of conscious perceptual processing in the ventral stream, but an emerging view is that they are jointly influenced by ventral and dorsal stream processing. Within the apraxia literature, pantomimes were historically viewed as learned motor schemas, but there is growing recognition that they include creative and improvised actions. Both literatures now recognize that pantomimes are often created spontaneously, sometimes drawing on memory and always requiring online cognitive control. By highlighting this convergence of ideas, we aim to encourage greater collaboration across these two research areas, in an effort to better understand these uniquely human behaviors.

Grasping Weber's Law in a Virtual Environment: The Effect of Haptic Feedback

Recent findings suggest that the functional separation between vision-for-action and vision-for-perception does not generalize to situations in which virtual objects are used as targets. For instance, unlike actions toward real objects that violate Weber's law, a basic law of visual perception, actions toward virtual objects presented on flat-screens, or in remote virtual environments, obey to Weber's law. These results suggest that actions in virtual environments are performed in an inefficient manner and are subjected to perceptual effects. It is unclear, however, whether this inefficiency reflects extensive variation in the way in which visual information is processed in virtual environments or more local aspects related to the settings of the virtual environment. In the current study, we focused on grasping performance in a state-of-the-art virtual reality system that provides an accurate representation of the 3D space. Within this environment, we tested the effect of haptic feedback on grasping trajectories. Participants were asked to perform bimanual grasping movements toward the edges of virtual targets. In the haptic feedback condition, physical stimuli of matching dimensions were embedded in the virtual environment. Haptic feedback was not provided in the no-feedback condition. The results showed that grasping trajectories in the feedback, but not in the no-feedback condition, could be performed more efficiently, and evade the influence of Weber's law. These findings are discussed in relevance to previous literature on 2D and 3D grasping.

When perception intrudes on 2D grasping: evidence from Garner interference

When participants reach out to pick up a real 3-D object, their grip aperture reflects the size of the object well before contact is made. At the same time, the classical psychophysical laws and principles of relative size and shape that govern visual perception do not appear to intrude into the control of such movements, which are instead tuned only to the relevant dimension for grasping. In contrast, accumulating evidence suggests that grasps directed at flat 2D objects are not immune to perceptual effects. Thus, in 2D but not 3D grasping, the aperture of the fingers has been shown to be affected by relative and contextual information about the size and shape of the target object. A notable example of this dissociation comes from studies of Garner interference, which signals holistic processing of shape. Previous research has shown that 3D grasping shows no evidence for Garner interference but 2D grasping does (Freud & Ganel, 2015). In a recent study published in this journal (Löhr-Limpens et al., 2019), participants were presented with 2D objects in a Garner paradigm. The pattern of results closely replicated the previously published results with 2D grasping. Unfortunately, the authors, who appear to be unaware the potential differences between 2D and 3D grasping, used their findings to draw an overgeneralized and unwarranted conclusion about the relation between 3D grasping and perception. In this short methodological commentary, we discuss current literature on aperture shaping during 2D grasping and suggest that researchers should play close attention to the nature of the target stimuli they use before drawing conclusions about visual processing for perception and action.

Active visuomotor interactions with virtual objects on touchscreens adhere to Weber's law

Recent findings suggest that the functional separation between vision-for-action and vision-for-perception does not generalize to situations in which two-dimensional (2D), virtual objects, are used as targets. For example, unlike grasping movements directed at real, three-dimensional (3D) objects, the trajectories of grasping movements directed at 2D objects adhere to the psychophysical principle of Weber's law, indicating relative and less efficient processing of their size. Such inefficiency could be attributed to the fact that everyday interactions with touchscreens do not usually entail grasping movements. It is possible, therefore, that more typical interactions with virtual objects, which involve active manipulation of their size or location on a touchscreen, could be performed efficiently and in an absolute manner, and would violate Weber's law. We examined this hypothesis in three experiments in which participants performed active interactions with virtual objects. In Experiment 1, participants made swiping gestures to move virtual objects across the touchscreen. In Experiment 2, participants touched the edges of virtual objects to enlarge their size. In Experiment 3, participants freely enlarged the size of virtual objects, without being required to touch their edges upon contact. In all experiments, the resolution of grip aperture decreased with the size of the target object, adhering to Weber's law. These results suggest that active interactions with 2D objects on touchscreens are not performed in a natural, absolute manner which characterize visuomotor control of real objects.

Obeying the law: speed-precision tradeoffs and the adherence to Weber's law in 2D grasping

Visually guided actions toward two-dimensional (2D) and three-dimensional (3D) objects show different patterns of adherence to Weber's law. In 3D grasping, Just Noticeable Differences (JNDs) do not scale with object size, violating Weber's law. Conversely, JNDs in 2D grasping increase with size, showing a pattern of scaler variability between aperture and JND, as predicted by Weber's law. In the current study, we tested whether such scaler variability in 2D grasping reflects genuine adherence to Weber's law. Alternatively, it could be potentially accounted for by a speed-precision tradeoff effect due to an increase in aperture velocity with size. In two experiments, we modified the relation between aperture velocity and size in 2D grasping and tested whether movement trajectories still adhere to Weber's law. In Experiment 1, we aimed to equate aperture velocities between different-sized objects by pre-adjusting the initial finger aperture to match the target's size. In Experiment 2, we reversed the relation between size and velocity by asking participants to hold their fingers wide open prior to grasp, resulting in faster velocities for smaller rather than for larger objects. The results of the two experiments showed that although aperture velocities did not increase with size, adherence to Weber's law was still maintained. These results indicate that the adherence to Weber's law during 2D grasping cannot be accounted for by a speed-precision tradeoff effect, but rather represents genuine reliance on relative, perceptually based computations in visuomotor interactions with 2D objects.

Force feedback delay affects perception of stiffness but not action, and the effect depends on the hand used but not on the handedness

Interaction with an object often requires the estimation of its mechanical properties. We examined whether the hand that is used to interact with the object and their handedness affected people's estimation of these properties using stiffness estimation as a test case. We recorded participants' responses on a stiffness discrimination of a virtual elastic force field and the grip force applied on the robotic device during the interaction. In half of the trials, the robotic device delayed the participants' force feedback. Consistent with previous studies, delayed force feedback biased the perceived stiffness of the force field. Interestingly, in both left-handed and right-handed participants, for the delayed force field, there was even less perceived stiffness when participants used their left hand than their right hand. This result supports the idea that haptic processing is affected by laterality in the brain, not by handedness. Consistent with previous studies, participants adjusted their applied grip force according to the correct size and timing of the load force regardless of the hand that was used, the handedness, or the delay. This suggests that in all of these conditions, participants were able to form an accurate internal representation of the anticipated trajectory of the load force (size and timing) and that this representation was used for accurate control of grip force independently of the perceptual bias. Thus these results provide additional evidence for the dissociation between action and perception in the processing of delayed information. NEW & NOTEWORTHY Introducing delay to force feedback during interaction with an elastic force field biases the perceived stiffness of the force field. We show that this bias depends on the hand that was used for probing but not on handedness. At the same time, both left-handed and right-handed participants adjusted their applied grip force while using either their left or right hands in anticipation of the correct magnitude and timing despite the delay in load force.