A touchy subject: advancing the modulated visual pathways account of altered vision near the hand

Opposing influences of global and local stimulus-hand proximity on crosstalk interference in dual tasks

In contrast to traditional dualistic views of cognition, visual stimulus processing appears not independent of bodily factors such as hand positioning. For example, reduced crosstalk between two temporally overlapping tasks has been observed when the hands are moved into the attentional window alongside their respective stimuli (i.e., establishing global stimulus-hand proximity). This result indicates that hand-specific attentional processing enhancements support a more serial rather than parallel processing of the two tasks. To further elucidate the nature of these processing modulations and their effect on multitasking performance, the present study consisted of three interrelated crosstalk experiments. Experiment 1 manipulated global stimulus-hand proximity and stimulus-effect proximity orthogonally, with results demonstrating that hand proximity rather than effect proximity drives the crosstalk reduction. Experiment 2 manipulated the physical distance between both hands (i.e., varying local stimulus-hand proximity), with results showing weak evidence of increased crosstalk when both hands are close to each other. Experiment 3 tested opposing influences of global and local stimulus-hand proximity as observed in Experiment 1 and 2 rigorously within one experiment, by employing an orthogonal manipulation of these two proximity measures. Again, we observed slightly increased crosstalk for hands close to each other (replicating Experiment 2); however, in contrast to Experiment 1, the effect of global stimulus-hand proximity on the observed crosstalk was not significant this time. Taken together, the experiments support the notion of hand-specific modulations of perception-action coupling, which can either lead to more or less interference in multitasking, depending on the exact arrangement of hands and stimuli.

Does hand proximity enhance letter identification?

Adam et al. (2012) found that letters were identified more accurately when presented near, compared to away from, the hands. Participants performed the task in two conditions: with their hands held stationary and with their hands moving towards and away from the target letters. The near-hands effect included the contribution of both static and dynamic trials. Further studies showed that accuracy in letter discrimination was higher when hands were away from a target (a far-hands effect) and moving toward it, suggesting an interaction between hand position and movement direction. The present study aimed to test whether hand proximity affects letter identification when the hands are stationary, as it remains unclear if this effect can be reliably observed. Participants viewed strings of three consonants, briefly presented and masked, and had to verbally report their identity. Stimuli were presented under two different hand conditions: proximal and distal. The predicted effects of letter position and stimulus duration were all statistically significant and robust; however, we did not observe a hand proximity effect.

Visual attention around a hand location localized by proprioceptive information

Facilitation of visual processing has been reported in the space near the hand. To understand the underlying mechanism of hand proximity attention, we conducted experiments that isolated hand-related effects from top–down attention, proprioceptive information from visual information, the position effect from the influence of action, and the distance effect from the peripersonal effect. The flash-lag effect was used as an index of attentional modulation. Because the results showed that the flash-lag effect was smaller at locations near the hand, we concluded that there was a facilitation effect of the visual stimuli around the hand location identified through proprioceptive information. This was confirmed by conventional reaction time measures. We also measured steady-state visual evoked potential (SSVEP) in order to investigate the spatial properties of hand proximity attention and top–down attention. The results showed that SSVEP reflects the effect of top–down attention but not that of hand proximity attention. This suggests that the site of hand proximity attention is at a later stage of visual processing, assuming that SSVEP responds to neural activities at the early stages. The results of left-handers differed from those of right-handers, and this is discussed in relation to handedness variation.

Design choices: Empirical recommendations for designing two-dimensional finger-tracking experiments

The continuous tracking of mouse or finger movements has become an increasingly popular research method for investigating cognitive and motivational processes such as decision-making, action-planning, and executive functions. In the present paper, we evaluate and discuss how apparently trivial design choices of researchers may impact participants' behavior and, consequently, a study's results. We first provide a thorough comparison of mouse- and finger-tracking setups on the basis of a Simon task. We then vary a comprehensive set of design factors, including spatial layout, movement extent, time of stimulus onset, size of the target areas, and hit detection in a finger-tracking variant of this task. We explore the impact of these variations on a broad spectrum of movement parameters that are typically used to describe movement trajectories. Based on our findings, we suggest several recommendations for best practice that avoid some of the pitfalls of the methodology. Keeping these recommendations in mind will allow for informed decisions when planning and conducting future tracking experiments.

Altered EEG Signal Complexity Induced by Hand Proximity: A Multiscale Entropy Approach

Visual short-term memory (VSTM) is an important cognitive function that acts as a temporary storage for visual information. Previous studies have shown that VSTM capacity can be modulated by the location of one’s hands, where hand proximity enhances neural processing and memory of nearby visual stimuli. The present study used traditional event-related potentials (ERP) along with multiscale entropy (MSE) analysis to shed light on the neural mechanism(s) behind such near-hand effect. Participants’ electroencephalogram (EEG) data were recorded as they performed a VSTM task with their hands either proximal or distal to the display. ERP analysis showed altered memory processing in the 400–700 ms time window during memory retrieval period. Importantly, MSE analysis also showed significant EEG difference between hand proximal and distal conditions between scales 10 to 20, and such difference is clustered around the right parietal cortex – a region that is involved in VSTM processing and bimodal hand-eye integration. The implications of higher MSE time scale in the parietal cortex are discussed in the context of signal complexity and its possible relation to cognitive processing. To our knowledge, this study provides the first investigation using MSE to characterize the temporal characteristics and signal complexity behind the effect of hand proximity.

Change detection for real-world objects in perihand space

Recent evidence has demonstrated that observers experience visual-processing biases in perihand space that may be tied to the hands' relevance for grasping actions. Our previous work suggested that when the hands are positioned to afford a power-grasp action, observers show increased temporal sensitivity that could aid with fast and forceful action, whereas when the hands are instead at the ready to perform a precision-grasp action, observers show enhanced spatial sensitivity that benefits delicate and detail-oriented actions. In the present investigation we seek to extend these previous findings by examining how object affordances may interact with hand positioning to shape visual biases in perihand space. Across three experiments, we examined how long participants took to perform a change detection task on photos of real objects, while we manipulated hand position (near/far from display), grasp posture (power/precision), and change type (orientation/identity). Participants viewed objects that afforded either a power grasp or a precision grasp, or were ungraspable. Although we were unable to uncover evidence of altered vision in perihand space in our first experiment, mirroring previous findings, in Experiments 2 and 3 our participants showed grasp-dependent biases near the hands when detecting changes to target objects that afforded a power grasp. Interestingly, ungraspable target objects were not subject to the same perihand space biases. Taken together, our results suggest that the influence of hand position on change detection performance is mediated not only by the hands' grasp posture, but also by a target object's affordances for grasping.

Embodied cognition in multitasking: increased hand-specific task shielding when stimuli are presented near the hand

The proximity of hand position alters the processing of visual stimuli. Stimuli presented close (proximal) to hands receive an enhanced allocation of visual attention compared to stimuli presented far (distal) from hands. In the present dual-task study we studied the consequences of this preferential processing when the stimulus for Task 1 (S1) and the stimulus for Task 2 (S2) were presented together and were assigned to specific response hands (R1 and R2) located proximal (at the monitor) versus distal (in the lap) to the stimuli. In two experiments, we tested whether stimulus-hand proximity affected the extent of between-task interference (i.e., influence of additional Task 2 processing on primary Task 1 processing). In Experiment 1, results showed that stimulus-hand proximity reduced the amount of between-task interference compared to the distal stimulus-hand condition. Extending these findings, in Experiment 2 a further reduction of between-task interference was obtained when a single hand was located at the monitor instead of two hands. These results are inconsistent with the assumption of a generally increased attentional processing benefit for multiple stimuli within hand space. Instead, these findings speak for a hand-specific processing benefit that supports more separate processing of two tasks. Together these findings demonstrate that stimulus-hand proximity affects the quality of multiple task performance, which is discussed in the context of both, basic and applied cognitive research.

Touch the table before the target: contact with an underlying surface may assist the development of precise visually controlled reach and grasp movements in human infants

Multiple motor channel theory posits that skilled hand movements arise from the coordinated activation of separable neural circuits in parietofrontal cortex, each of which produces a distinct movement and responds to different sensory inputs. Prehension, the act of reaching to grasp an object, consists of at least two movements: a reach movement that transports the hand to a target location and a grasp movement that shapes and closes the hand for target acquisition. During early development, discrete pre-reach and pre-grasp movements are refined based on proprioceptive and tactile feedback, but are gradually coordinated together into a singular hand preshaping movement under feedforward visual control. The neural and behavioural factors that enable this transition are currently unknown. In an attempt to identify such factors, the present descriptive study used frame-by-frame video analysis to examine 9-, 12-, and 15-month-old infants, along with sighted and unsighted adults, as they reached to grasp small ring-shaped pieces of cereal (Cheerios) resting on a table. Compared to sighted adults, infants and unsighted adults were more likely to make initial contact with the underlying table before they contacted the target. The way in which they did so was also similar in that they generally contacted the table with the tip of the thumb and/or pinky finger, a relatively open hand, and poor reach accuracy. Despite this, infants were similar to sighted adults in that they tended to use a pincer digit, defined as the tip of the thumb or index finger, to subsequently contact the target. Only in infants was this ability related to their having made prior contact with the underlying table. The results are discussed in relation to the idea that initial contact with an underlying table or surface may assist infants in learning to use feedforward visual control to direct their digits towards a precise visual target.

Slower attentional disengagement but faster perceptual processing near the hand

Many recent studies have reported altered visual processing near the hands. However, there is no definitive agreement about the mechanisms responsible for this effect. One viewpoint is that the effect is predominantly attentional while others argue for the role of pre-attentive perceptual differences in the manifestation of the hand-proximity effect. However, in most of the studies pre-attentional and attentional effects have been conflated. We argue that it is important to dissociate the effect of hand proximity on perception and attention to better theorize and understand how visual processing is altered near the hands. We report two experiments using a visual search task where participants completed a visual search task with their hands either on the monitor or on their lap. When on the monitor, the target could appear near the hand or farther away. In experiment 1, a letter search task showed steeper search slope near the hand suggesting slower attentional disengagement. However, the intercept was smaller in the near hand condition suggesting faster perceptual processing. These results were also replicated in experiment 2 with a conjunction search task with target present and absent conditions and 4 set sizes. The results suggest that there are dissociable effects of hand proximity on perception and attention. Importantly, the pre-attentive advantage of hand proximity does not translate to attentional benefit, but a processing cost. The results of experiment 2 additionally indicate that the steeper slope does not arise from any spatial biases in how search proceeds, but an indicator of slower attentional processing near the hands. The results also suggest that the effect of hand proximity on attention is not spatially graded whereas its effect on perceptuo-motor processes seems to be.

Effects of Hand Proximity and Movement Direction in Spatial and Temporal Gap Discrimination

Previous research on the interplay between static manual postures and visual attention revealed enhanced visual selection near the hands (near-hand effect). During active movements there is also superior visual performance when moving toward compared to away from the stimulus (direction effect). The “modulated visual pathways” hypothesis argues that differential involvement of magno- and parvocellular visual processing streams causes the near-hand effect. The key finding supporting this hypothesis is an increase in temporal and a reduction in spatial processing in near-hand space (Gozli et al., 2012). Since this hypothesis has, so far, only been tested with static hand postures, we provide a conceptual replication of Gozli et al.’s (2012) result with moving hands, thus also probing the generality of the direction effect. Participants performed temporal or spatial gap discriminations while their right hand was moving below the display. In contrast to Gozli et al. (2012), temporal gap discrimination was superior at intermediate and not near hand proximity. In spatial gap discrimination, a direction effect without hand proximity effect suggests that pragmatic attentional maps overshadowed temporal/spatial processing biases for far/near-hand space.

Action Experience Drives Visual-Processing Biases Near the Hands

Observers experience affordance-specific biases in visual processing for objects within the hands' grasping space, but the mechanism that tunes visual cognition to facilitate action remains unknown. I investigated the hypothesis that altered vision near the hands is a result of experience-driven plasticity. Participants performed motion-detection and form-perception tasks-while their hands were either near the display, in atypical grasping postures, or positioned in their laps-both before and after learning novel grasp affordances. Participants showed enhanced temporal sensitivity for stimuli viewed near the backs of the hands after training to execute a power grasp using the backs of their hands (Experiment 1), but showed enhanced spatial sensitivity for stimuli viewed near the tips of their little fingers after training to use their little fingers to execute a precision grasp (Experiment 2). These results show that visual biases near the hands are plastic, facilitating processing of information relevant to learned grasp affordances.

Phantoms on the hands: Influence of the body on brief synchiric visual percepts

Recent studies have found preferential responses for brief, transient visual stimuli near the hands, suggesting a link between magnocellular visual processing and peripersonal representations. We report an individual with a right hemisphere lesion whose illusory phantom percepts may be attributable to an impairment in the peripersonal system specific to transient visual stimuli. When presented with a single, brief (250 ms) visual stimulus to her ipsilesional side, she reported visual percepts on both sides - synchiria. These contralesional phantoms were significantly more frequent when visual stimuli were presented on the hands versus off the hands. We next manipulated stimulus duration to examine the relationship between these phantom percepts and transient visual processing. We found a significant position by duration interaction, with substantially more phantom synchiric percepts on the hands for brief compared to sustained stimuli. This deficit provides novel evidence both for preferential processing of transient visual stimuli near the hands, and for mechanisms that, when damaged, result in phantom percepts.

Intentional control of visual processing benefits from referential objects

Arguments have been made that enhanced visual processing occurs in the area of the palms of the hands due to greater density of bimodal neurons. An alternative is that the hands serve as reference objects relative to which attentional resources are allocated. Two experiments were conducted to determine whether the palms are unique in speeding responses in an Eriksen flanker-type task compared with other parts of the hands and objects used as barriers. In Experiment 1, the hands were crossed and positioned so that the palms faced outward toward letters located in the outer positions. Trial blocks differed in whether the centrally located letter or outer letters were designated as the target for responding. Results yielded reductions in flanker interference much as obtained when the palms face inward. This reduction occurred regardless of whether the center or outer positions of the letters were designated as the target. Experiment 2 replicated these results using as reference objects wooden blocks that mimicked the hands' physical contours, positioned with a curve-edge facing outwards. The results lend support to the referential coding account of the reduction of flanker interference.

Hand position influences perceptual grouping

Over the past decade, evidence has accumulated that performance in attention, perception, and memory-related tasks are influenced by the distance between the hands and the stimuli (i.e., placing the observer's hands near or far from the stimuli). To account for existing findings, it has recently been proposed that processing of stimuli near the hands is dominated by the magnocellular visual pathway. The present study tests an implication of this hypothesis, whether perceptual grouping is reduced in hands-proximal space. Consistent with previous work on the object-based capture of attention, a benefit for the visual object in the hands-distal condition was observed in the present study. Interestingly, the object-based benefit did not emerge in the hands-proximal condition, suggesting perceptual grouping is impaired near the hands. This change in perceptual grouping processes provides further support for the hypothesis that visual processing near the hands is subject to increased magnocellular processing.

Joint attention for stimuli on the hands: ownership matters

The visual system treats the space near the hands with unique, action-related priorities. For example, attention orients slowly to stimuli on the hands (Taylor and Witt, 2014). In this article, we asked whether jointly attended hands are attended in the same way. Specifically, we examined whether ownership over the hand mattered: do we attend to our hands and the hands of others in the same way? Pairs of participants performed a spatial cueing task with stimuli that could be projected onto one partner’s hands or on a control surface. Results show delayed orienting of attention to targets appearing on the hands, but only for the owner of the hands. For an observer, others’ hands are like any other surface. This result emphasizes the importance of ownership for hand-based effects on vision, and in doing so, is inconsistent with some expectations of the joint action literature.