Trade-offs in visual processing for stimuli near the hands

Opposing influences of global and local stimulus-hand proximity on crosstalk interference in dual tasks

In contrast to traditional dualistic views of cognition, visual stimulus processing appears not independent of bodily factors such as hand positioning. For example, reduced crosstalk between two temporally overlapping tasks has been observed when the hands are moved into the attentional window alongside their respective stimuli (i.e., establishing global stimulus-hand proximity). This result indicates that hand-specific attentional processing enhancements support a more serial rather than parallel processing of the two tasks. To further elucidate the nature of these processing modulations and their effect on multitasking performance, the present study consisted of three interrelated crosstalk experiments. Experiment 1 manipulated global stimulus-hand proximity and stimulus-effect proximity orthogonally, with results demonstrating that hand proximity rather than effect proximity drives the crosstalk reduction. Experiment 2 manipulated the physical distance between both hands (i.e., varying local stimulus-hand proximity), with results showing weak evidence of increased crosstalk when both hands are close to each other. Experiment 3 tested opposing influences of global and local stimulus-hand proximity as observed in Experiment 1 and 2 rigorously within one experiment, by employing an orthogonal manipulation of these two proximity measures. Again, we observed slightly increased crosstalk for hands close to each other (replicating Experiment 2); however, in contrast to Experiment 1, the effect of global stimulus-hand proximity on the observed crosstalk was not significant this time. Taken together, the experiments support the notion of hand-specific modulations of perception-action coupling, which can either lead to more or less interference in multitasking, depending on the exact arrangement of hands and stimuli.

Virtual occlusion effects on the perception of self-initiated visual stimuli

Virtual reality (VR) has established itself as a useful tool in the study of human perception in the laboratory. A recent study introduced a new approach to examine visual sensory attenuation (SA) effects in VR. Hand movements triggered the appearance of Gabor stimuli, which were either presented behind the participant's hand - not rendered in VR ("virtual occlusion") - or elsewhere on the display. Virtual occlusion led to a rightward shift of the psychometric curve, suggesting that self-generated hand movements reduced the perceived contrast of the stimulus. Since such attenuation effects might provide a window into the predictive processing of the sensory and cognitive apparatus, we sought to better understand the nature of the virtual occlusion effects. In our study, the presentation of test stimuli was either self-initiated, self-initiated with a variable delay, or triggered externally; the test stimuli were occluded or not. In conflict with our hypothesis, we found moderate to strong evidence for an absence of any horizontal shifts between the psychometric curves. However, virtual occlusion was associated with a decrease in the slope of the psychometric function. Our results suggest that virtual occlusion attenuated the relative perceptual sensitivity, so that participants had more difficulty discriminating contrast differences when the test stimulus was presented behind the hand. We tentatively conclude that, in the visual domain, the discriminability of stimulus intensity is modified by internal predictive cues (i.e., proprioception), possibly linked to shifts in covert spatial attention.

Mugs and Plants: Object Semantic Knowledge Alters Perceptual Processing With Behavioral Ramifications

Neural processing of objects with action associations recruits dorsal visual regions more than the neural processing of objects without such associations. We hypothesized that because the dorsal and ventral visual pathways have differing proportions of magno- and parvocellular input, there should be behavioral differences in perceptual tasks between manipulable and nonmanipulable objects. This hypothesis was tested in college-age adults across five experiments (Ns = 26, 26, 30, 25, and 25) using a gap-detection task, suited to the spatial resolution of parvocellular processing, and an object-flicker-discrimination task, suited to the temporal resolution of magnocellular processing. Directly predicted from the cellular composition of each pathway, a strong nonmanipulable-object advantage was observed in gap detection, and a small manipulable-object advantage was observed in flicker discrimination. Additionally, these effects were modulated by reducing object recognition through inversion and by suppressing magnocellular processing using red light. These results establish perceptual differences between objects dependent on semantic knowledge.

Is semantic processing impaired near the hands?

In a sentence decision task, Davoli et al. (2010) found that the semantic analysis of sentences differed depending on whether the participants' hands were close to or far from the computer screen. According to the authors, the findings reflected an impoverishment of semantic processing near the hands. In the current study, we examine this explanation by asking whether hand position affects 1) other aspects of sentence processing, such as syntactic analysis, 2) semantic processing at the individual word level, and 3) performance in a picture naming task that requires access to meaning. In Experiment 1, participants judged the acceptability of sentences, half of which included semantic or syntactic violations. In Experiment 2, only semantically acceptable or nonacceptable sentences were presented. In Experiment 3, participants performed a go/nogo semantic categorization task on individual words. In Experiment 4, participants performed a picture naming task. Participants performed these tasks both with their hands near to and far from the computer screen. Regardless of the task, we found no evidence of impoverished semantic processing near the hands.

Using perceptual tasks to selectively measure magnocellular and parvocellular performance: Rationale and a user's guide

The visual system uses parallel pathways to process information. However, an ongoing debate centers on the extent to which the pathways from the retina, via the Lateral Geniculate nucleus to the visual cortex, process distinct aspects of the visual scene and, if they do, can stimuli in the laboratory be used to selectively drive them. These questions are important for a number of reasons, including that some pathologies are thought to be associated with impaired functioning of one of these pathways and certain cognitive functions have been preferentially linked to specific pathways. Here we examine the two main pathways that have been the focus of this debate: the magnocellular and parvocellular pathways. Specifically, we review the results of electrophysiological and lesion studies that have investigated their properties and conclude that while there is substantial overlap in the type of information that they process, it is possible to identify aspects of visual information that are predominantly processed by either the magnocellular or parvocellular pathway. We then discuss the types of visual stimuli that can be used to preferentially drive these pathways.

Simple action alters attention towards visual features

Recent studies have revealed an action effect, in which a simple action towards a prime stimulus biases attention in a subsequent visual search in favor of objects that match the prime. However, to date the majority of research on the phenomenon has studied search elements that are exact matches to the prime, and that vary only on the dimension of color, making it unclear how general the phenomenon is. Here, across a series of experiments, we show that action can also prioritize objects that match the shape of the prime. Additionally, action can prioritize attention to objects that match only one of either the color or the shape of the prime, suggesting that action enhances individual visual features present in the acted-on objects. The pattern of results suggests that the effect may be stronger for color matches – prioritization for shape only occurred when attention was not drawn to the color of the prime, whereas prioritization for color occurred regardless. Taken together, the results reveal that a prior action can exert a strong influence on subsequent attention towards features of the acted-on object.

An auditory hand-proximity effect: The auditory Simon effect is enhanced near the hands

Visual processing near the hands is altered compared with stimuli far from the hands. Here, we aimed to test whether this alteration can be found in auditory processing. Participants were required to perform an auditory Simon task either with their hands close to the loudspeakers or far from the loudspeakers. Two experiments consistently showed that the auditory Simon effect was enhanced when the hands were close to the speakers compared with far from the speakers. This is consistent with previous findings of an enhanced visual Simon effect near the hands. Furthermore, the hand-proximity effects in auditory and visual Simon tasks (an enhanced Simon effect near hands compared with far from hands) were comparable, indicating hand-proximity effect is reliable across visual and auditory modalities. Thus, the present study extended the hand-proximity effect from vision to audition by showing that the auditory Simon effect was enhanced near the hands compared with far from the hands.

A critical review of the cognitive and perceptual factors influencing attentional scaling and visual processing

An important mechanism used to selectively process relevant information in the environment is spatial attention. One fundamental way in which spatial attention is deployed is attentional scaling - the process of focusing attentional resources either narrowly or broadly across the visual field. Although early empirical work suggested that narrowing attention improves all aspects of visual processing, recent studies have demonstrated that narrowing attention can also have no effect or even a detrimental impact when it comes to vision that is thought to be mediated via the magnocellular pathway of the visual system. Here, for the first time, we synthesize empirical evidence measuring the behavioral effects of attentional scaling on tasks gauging the contribution of the major neural pathways of the visual system, with the purpose of determining the potential factors driving these contradictory empirical findings. This analysis revealed that attentional scaling could be best understood by considering the unique methodologies used in the research literature to date. The implications of this analysis for theoretical frameworks of attentional scaling are discussed, and methodological improvements for future research are proposed.

Embodied cognition in multitasking: increased hand-specific task shielding when stimuli are presented near the hand

The proximity of hand position alters the processing of visual stimuli. Stimuli presented close (proximal) to hands receive an enhanced allocation of visual attention compared to stimuli presented far (distal) from hands. In the present dual-task study we studied the consequences of this preferential processing when the stimulus for Task 1 (S1) and the stimulus for Task 2 (S2) were presented together and were assigned to specific response hands (R1 and R2) located proximal (at the monitor) versus distal (in the lap) to the stimuli. In two experiments, we tested whether stimulus-hand proximity affected the extent of between-task interference (i.e., influence of additional Task 2 processing on primary Task 1 processing). In Experiment 1, results showed that stimulus-hand proximity reduced the amount of between-task interference compared to the distal stimulus-hand condition. Extending these findings, in Experiment 2 a further reduction of between-task interference was obtained when a single hand was located at the monitor instead of two hands. These results are inconsistent with the assumption of a generally increased attentional processing benefit for multiple stimuli within hand space. Instead, these findings speak for a hand-specific processing benefit that supports more separate processing of two tasks. Together these findings demonstrate that stimulus-hand proximity affects the quality of multiple task performance, which is discussed in the context of both, basic and applied cognitive research.

Building Blocks of Psychology: on Remaking the Unkept Promises of Early Schools

The appeal and popularity of "building blocks", i.e., simple and dissociable elements of behavior and experience, persists in psychological research. We begin our assessment of this research strategy with an historical review of structuralism (as espoused by E. B. Titchener) and behaviorism (espoused by J. B. Watson and B. F. Skinner), two movements that held the assumption in their attempts to provide a systematic and unified discipline. We point out the ways in which the elementism of the two schools selected, framed, and excluded topics of study. After the historical review, we turn to contemporary literature and highlight the persistence of research into building blocks and the associated framing and exclusions in psychological research. The assumption that complex categories of human psychology can be understood in terms of their elementary components and simplest forms seems indefensible. In specific cases, therefore, reliance on the assumption requires justification. Finally, we review alternative strategies that bypass the commitment to building blocks.

The Fragility of the Near-Hand Effect

Recent literature has demonstrated that hand position can affect visual processing, a set of phenomena termed Near Hand Effects (NHEs). Across four studies we looked for single-hand NHEs on a large screen when participants were asked to discriminate stimuli based on size, colour, and orientation (Study 1), to detect stimuli after a manipulation of hand shaping (Study 2), to detect stimuli after the introduction of a peripheral cue (Study 3), and finally to detect stimuli after a manipulation of screen orientation (Study 4). Each study failed to find a NHE. Further examination of the pooled data using a Bayesian analysis also failed to reveal positive evidence for faster responses or larger cueing effects near a hand. These findings suggest that at least some NHEs may be surprisingly fragile, which dovetails with the recent proposition that NHEs may not form a unitary set of phenomena (Gozli & Deng, 2018). The implication is that visual processing may be less sensitive to hand position across measurement techniques than previously thought, and points to a need for well-powered, methodologically rigorous studies on this topic in the future.

Effector-based attention systems

Visual processing is known to be enhanced at the end point of eye movements. Feedback within the oculomotor system has been shown to drive these alterations in visual processing. However, we do not simply view the world; we also reach out and interact using our hands. Consequently, it is not surprising that visual processing has also been shown to be altered in near-hand space. A growing body of work documents a myriad of alterations in near-hand visual processing, with little consensus on the neural underpinnings of the effect of the hand. Since movement of the eyes and hands is governed by parallel frontoparietal networks and since within the oculomotor system feedback from these motor control regions has been shown to drive enhanced visual processing at saccade end points, it is plausible that a similar feedback mechanism is at play in near-hand improvements in visual processing. Here, we compare and contrast oculomotor-driven and hand-driven changes in visual processing and provide support for the hypothesis that feedback within the reaching and grasping systems enhances visual processing near the hand in a novel way.

Slower attentional disengagement but faster perceptual processing near the hand

Many recent studies have reported altered visual processing near the hands. However, there is no definitive agreement about the mechanisms responsible for this effect. One viewpoint is that the effect is predominantly attentional while others argue for the role of pre-attentive perceptual differences in the manifestation of the hand-proximity effect. However, in most of the studies pre-attentional and attentional effects have been conflated. We argue that it is important to dissociate the effect of hand proximity on perception and attention to better theorize and understand how visual processing is altered near the hands. We report two experiments using a visual search task where participants completed a visual search task with their hands either on the monitor or on their lap. When on the monitor, the target could appear near the hand or farther away. In experiment 1, a letter search task showed steeper search slope near the hand suggesting slower attentional disengagement. However, the intercept was smaller in the near hand condition suggesting faster perceptual processing. These results were also replicated in experiment 2 with a conjunction search task with target present and absent conditions and 4 set sizes. The results suggest that there are dissociable effects of hand proximity on perception and attention. Importantly, the pre-attentive advantage of hand proximity does not translate to attentional benefit, but a processing cost. The results of experiment 2 additionally indicate that the steeper slope does not arise from any spatial biases in how search proceeds, but an indicator of slower attentional processing near the hands. The results also suggest that the effect of hand proximity on attention is not spatially graded whereas its effect on perceptuo-motor processes seems to be.

Handedness and Graspability Modify Shifts of Visuospatial Attention to Near-Hand Objects

We examined how factors related to the internal representation of the hands (handedness and grasping affordances) influence the distribution of visuospatial attention near the body. Left and right handed participants completed a covert visual cueing task, discriminating between two target shapes. In Experiment 1, participants responded with either their dominant or non-dominant hand. In Experiment 2, the non-responding hand was positioned below one of two target placeholders, aligned with the shoulder. In Experiment 3 the near-monitor hand was positioned under the placeholder in the opposite region of hemispace, crossed over the body midline. For Experiments 2 & 3, in blocked trials the palmar and back-of hand surfaces were directed towards the target placeholder such that targets appeared towards either the graspable or non-graspable space of the hand respectively. In Experiment 2, both left and right handers displayed larger accuracy cueing effects for targets near versus distant from the graspable space of the right hand. Right handers also displayed larger response time cueing effects for objects near the graspable versus non-graspable region of their dominant hand but not for their non-dominant hands. These effects were not evident for left-handers. In Experiment 3, for right handers, accuracy biases for near hand targets were still evident when the hand was crossed over the body midline, and reflected hand proximity but not functional orientation biases. These findings suggest that biased visuospatial attention enhances object identity discrimination near hands and that these effects are particularly enhanced for right-handers.

Action Experience Drives Visual-Processing Biases Near the Hands

Observers experience affordance-specific biases in visual processing for objects within the hands' grasping space, but the mechanism that tunes visual cognition to facilitate action remains unknown. I investigated the hypothesis that altered vision near the hands is a result of experience-driven plasticity. Participants performed motion-detection and form-perception tasks-while their hands were either near the display, in atypical grasping postures, or positioned in their laps-both before and after learning novel grasp affordances. Participants showed enhanced temporal sensitivity for stimuli viewed near the backs of the hands after training to execute a power grasp using the backs of their hands (Experiment 1), but showed enhanced spatial sensitivity for stimuli viewed near the tips of their little fingers after training to use their little fingers to execute a precision grasp (Experiment 2). These results show that visual biases near the hands are plastic, facilitating processing of information relevant to learned grasp affordances.

The nature of altered vision near the hands: evidence for the magnocellular enhancement account from object correspondence through occlusion

A growing body of evidence indicates that the perception of visual stimuli is altered when they occur near the observer's hands, relative to other locations in space (see Brockmole, Davoli, Abrams, & Witt, 2013, for a review). Several accounts have been offered to explain the pattern of performance across different tasks. These have typically focused on attentional explanations (attentional prioritization and detailed attentional evaluation of stimuli in near-hand space), but more recently, it has been suggested that near-hand space enjoys enhanced magnocellular (M) input. Here we differentiate between the attentional and M-cell accounts, via a task that probes the roles of position consistency and color consistency in determining dynamic object correspondence through occlusion. We found that placing the hands near the visual display made observers use only position consistency, and not color, in determining object correspondence through occlusion, which is consistent with the fact that M cells are relatively insensitive to color. In contrast, placing observers' hands far from the stimuli allowed both color and position contribute. This provides evidence in favor of the M-cell enhancement account of altered vision near the hands.

Phantoms on the hands: Influence of the body on brief synchiric visual percepts

Recent studies have found preferential responses for brief, transient visual stimuli near the hands, suggesting a link between magnocellular visual processing and peripersonal representations. We report an individual with a right hemisphere lesion whose illusory phantom percepts may be attributable to an impairment in the peripersonal system specific to transient visual stimuli. When presented with a single, brief (250 ms) visual stimulus to her ipsilesional side, she reported visual percepts on both sides - synchiria. These contralesional phantoms were significantly more frequent when visual stimuli were presented on the hands versus off the hands. We next manipulated stimulus duration to examine the relationship between these phantom percepts and transient visual processing. We found a significant position by duration interaction, with substantially more phantom synchiric percepts on the hands for brief compared to sustained stimuli. This deficit provides novel evidence both for preferential processing of transient visual stimuli near the hands, and for mechanisms that, when damaged, result in phantom percepts.

Reduced object-based perception in the near-hand space

Previous studies have shown that hand proximity changes visual perception (Abrams et al. in Cognition 107(3):1035-1047, 2008). The present study examined the effects of hand proximity on object-based perception. In three experiments, participants viewed stimuli that were either near to or far from their hands. The target stimulus appeared, after a cue, in one of two rectangular objects: either at the location that had been previously cued, at the uncued end of the cued object, or in the uncued object. We found a significantly reduced same-object benefit in reaction time for stimuli near the hands in one experiment. Interestingly, we observed a same-object cost in sensitivity for stimuli near the hands in another experiment. The results reveal that object-based perception is disrupted in the near-hand space. This is consistent with previous findings revealing altered visual processing near the hands.

On the use of spatial frequency to isolate contributions from the magnocellular and parvocellular systems and the dorsal and ventral cortical streams

Many authors have claimed that suprathreshold achromatic stimuli of low and high spatial frequency can be used to separate responses from different entities in the visual system. Most prominently, it has been proposed that such stimuli can differentiate responses from the magnocellular and parvocellular systems. As is reviewed here, investigators who have examined stimulus specificity of neurons in these systems have found little difference between magno- and parvocellular cells. It has also been proposed that spatial frequency can be used to selectively activate the "magnocellular-dorsal stream". The present review indicates that cells in Area MT of the dorsal stream do prefer very low spatial frequencies. However, the review also shows that cells in Area V4 of the ventral stream respond, not only to relatively high spatial frequencies, but also to low frequency stimuli. Thus, low spatial frequencies cannot be relied upon to selectively activate the dorsal stream.

Hand position influences perceptual grouping

Over the past decade, evidence has accumulated that performance in attention, perception, and memory-related tasks are influenced by the distance between the hands and the stimuli (i.e., placing the observer's hands near or far from the stimuli). To account for existing findings, it has recently been proposed that processing of stimuli near the hands is dominated by the magnocellular visual pathway. The present study tests an implication of this hypothesis, whether perceptual grouping is reduced in hands-proximal space. Consistent with previous work on the object-based capture of attention, a benefit for the visual object in the hands-distal condition was observed in the present study. Interestingly, the object-based benefit did not emerge in the hands-proximal condition, suggesting perceptual grouping is impaired near the hands. This change in perceptual grouping processes provides further support for the hypothesis that visual processing near the hands is subject to increased magnocellular processing.

Joint attention for stimuli on the hands: ownership matters

The visual system treats the space near the hands with unique, action-related priorities. For example, attention orients slowly to stimuli on the hands (Taylor and Witt, 2014). In this article, we asked whether jointly attended hands are attended in the same way. Specifically, we examined whether ownership over the hand mattered: do we attend to our hands and the hands of others in the same way? Pairs of participants performed a spatial cueing task with stimuli that could be projected onto one partner’s hands or on a control surface. Results show delayed orienting of attention to targets appearing on the hands, but only for the owner of the hands. For an observer, others’ hands are like any other surface. This result emphasizes the importance of ownership for hand-based effects on vision, and in doing so, is inconsistent with some expectations of the joint action literature.

Grasp posture alters visual processing biases near the hands

Observers experience biases in visual processing for objects within easy reach of their hands; these biases may assist them in evaluating items that are candidates for action. I investigated the hypothesis that hand postures that afford different types of actions differentially bias vision. Across three experiments, participants performed global-motion-detection and global-form-perception tasks while their hands were positioned (a) near the display in a posture affording a power grasp, (b) near the display in a posture affording a precision grasp, or (c) in their laps. Although the power-grasp posture facilitated performance on the motion-detection task, the precision-grasp posture instead facilitated performance on the form-perception task. These results suggest that the visual system weights processing on the basis of an observer's current affordances for specific actions: Fast and forceful power grasps enhance temporal sensitivity, whereas detail-oriented precision grasps enhance spatial sensitivity.

Interacting hands: the role of attention for the joint Simon effect

Recent research in monkeys and humans has shown that the presence of the hands near an object enhances spatial processing for objects presented near the hand. This study aimed to test the effect of hand position on the joint Simon effect. In Experiment 1, two human co-actors shared a Simon task while placing their response hands either near the objects appearing on the monitor or away from the monitor. Experiment 2 varied each co-actor’s hand position independently. Experiment 3 tested whether enhanced spatial processing for objects presented near the hand is obtained when replacing one of the two co-actors by a non-human event-producing rubber hand. Experiment 1 provided evidence for a Simon effect. Hand position significantly modulated the size of the Simon effect in the joint Simon task showing an increased Simon effect when the hands of both actors were located near the objects on the monitor, than when they were located away from the monitor. Experiment 2 replicated this finding showing an increased Simon effect when the actor’s hand was located near the objects on the monitor, but only when the co-actor also produced action events in spatial reference. A similar hand position effect was observed in Experiment 3 when a non-human rubber hand replaced the human co-actor. These findings suggest that external action events that are produced in spatial reference bias the distribution of attention to the area near the hand. This strengthens the weight of the spatial response codes (referential coding) and hence increases the joint Simon effect.

A touchy subject: advancing the modulated visual pathways account of altered vision near the hand

A growing body of evidence demonstrates that human vision operates differently in the space near and on the hands; for example, early findings in this literature reported that rapid onsets are detected faster near the hands, and that objects are searched more thoroughly. These and many other effects were attributed to enhanced attention via the recruitment of bimodal visual-tactile neurons representing the hand and near-hand space. However, recent research supports an alternative account: stimuli near the hands are preferentially processed by the action-oriented magnocellular visual pathway at the expense of processing in the parvocellular pathway. This Modulated Visual Pathways (MVP) account of altered vision near the hands describes a hand position-dependent trade-off between the two main retinal-cortical visual pathways between the eye and brain. The MVP account explains past findings and makes new predictions regarding near-hand vision supported by new research.

Hand proximity facilitates spatial discrimination of auditory tones

The effect of hand proximity on vision and visual attention has been well documented. In this study we tested whether such effect(s) would also be present in the auditory modality. With hands placed either near or away from the audio sources, participants performed an auditory-spatial discrimination (Experiment 1: left or right side), pitch discrimination (Experiment 2: high, med, or low tone), and spatial-plus-pitch (Experiment 3: left or right; high, med, or low) discrimination task. In Experiment 1, when hands were away from the audio source, participants consistently responded faster with their right hand regardless of stimulus location. This right hand advantage, however, disappeared in the hands-near condition because of a significant improvement in left hand's reaction time (RT). No effect of hand proximity was found in Experiments 2 or 3, where a choice RT task requiring pitch discrimination was used. Together, these results that the perceptual and attentional effect of hand proximity is not limited to one specific modality, but applicable to the entire “space” near the hands, including stimuli of different modality (at least visual and auditory) within that space. While these findings provide evidence from auditory attention that supports the multimodal account originally raised by Reed et al. (2006), we also discuss the possibility of a dual mechanism hypothesis to reconcile findings from the multimodal and magno/parvocellular account.