A bias to detail: how hand position modulates visual learning and visual memory

Contextual cueing during lethal force training: How target design and repetition can alter threat assessments

Lethal force training requires individuals to make threat assessments, which involves holistic scenario processing to identify potential threats. Photorealistic targets can make threat/non-threat judgments substantially more genuine and challenging compared to simple cardboard or silhouette targets. Unfortunately, repeated target use also brings unintended consequences that could invalidate threat assessment processes conducted during training. Contextually rich or unique targets could be implicitly memorable in a way that allows observers to recall weapon locations rather than forcing observers to conduct a naturalistic assessment. Experiment 1 demonstrated robust contextual cueing effects in a well-established shoot/don't-shoot stimulus set, and Experiment 2 extended this finding from complex scene stimuli to simple actor-only stimuli. Experiment 3 demonstrated that these effects also occurred among trained professionals using rifles rather than computer-based tasks. Taken together, these findings demonstrate the potential for uncontrolled target repetition to alter the fundamental processes of threat assessment during lethal force training.

Altered EEG Signal Complexity Induced by Hand Proximity: A Multiscale Entropy Approach

Visual short-term memory (VSTM) is an important cognitive function that acts as a temporary storage for visual information. Previous studies have shown that VSTM capacity can be modulated by the location of one’s hands, where hand proximity enhances neural processing and memory of nearby visual stimuli. The present study used traditional event-related potentials (ERP) along with multiscale entropy (MSE) analysis to shed light on the neural mechanism(s) behind such near-hand effect. Participants’ electroencephalogram (EEG) data were recorded as they performed a VSTM task with their hands either proximal or distal to the display. ERP analysis showed altered memory processing in the 400–700 ms time window during memory retrieval period. Importantly, MSE analysis also showed significant EEG difference between hand proximal and distal conditions between scales 10 to 20, and such difference is clustered around the right parietal cortex – a region that is involved in VSTM processing and bimodal hand-eye integration. The implications of higher MSE time scale in the parietal cortex are discussed in the context of signal complexity and its possible relation to cognitive processing. To our knowledge, this study provides the first investigation using MSE to characterize the temporal characteristics and signal complexity behind the effect of hand proximity.

The Impact of a Self-Avatar, Hand Collocation, and Hand Proximity on Embodiment and Stroop Interference

Understanding the effects of hand proximity to objects and tasks is critical for hand-held and near-hand objects. Even though self-avatars have been shown to be beneficial for various tasks in virtual environments, little research has investigated the effect of avatar hand proximity on working memory. This paper presents a between-participants user study investigating the effects of self-avatars and physical hand proximity on a common working memory task, the Stroop interference task. Results show that participants felt embodied when a self-avatar was in the scene, and that the subjective level of embodiment decreased when a participant's hands were not collocated with the avatar's hands. Furthermore, a participant's physical hand placement was significantly related to Stroop interference: proximal hands produced a significant increase in accuracy compared to non-proximal hands. Surprisingly, Stroop interference was not mediated by the existence of a self-avatar or level of embodiment.

Hand function, not proximity, biases visuotactile integration later in object processing: An ERP study

Behavioral studies document a functional hand proximity effect: objects near the palm, but not the back of the hand, affect visual processing. Although visuotactile bimodal neurons integrate visual and haptic inputs, their receptive fields in monkey cortex encompass the whole hand, not just the palm. Using ERPs, we investigated whether hand function influenced the topology of integrated space around the hand. In a visual detection paradigm, target and non-target stimuli appeared equidistantly in front or in back of the hand. Equivalent N1 amplitudes were found for both conditions. P3 target versus non-target amplitude differences were greater for palm conditions. Hand proximity biased processing of visual targets equidistant from the hand early in processing. However, hand function biases emerged later when targets were selected for potential action. Thus, early hand proximity effects on object processing depend on sensory-reliant neural responses, whereas later multisensory integration depend more on the hand's functional expertise.

Impaired conflict monitoring near the hands: Neurophysiological evidence

Recent evidence suggests that hand-stimulus proximity enhances the visuo-motor Simon effect. The present study used event-related potentials (ERP) to examine the timing at which hand-stimulus proximity modulates the Simon effect. The results show that the P1 and N1 components were not modulated by hand-stimulus proximity, suggesting that early sensory processing is not altered by hand-stimulus proximity. However, the interference effect (the difference between incompatible versus compatible trials) on the N2 component was significantly attenuated near the hands compared to far from the hands, indicating that hand-stimulus proximity impairs conflict monitoring. We also found significant effects on a later component, as the P3 was reduced and had a shorter latency for the hand-proximal condition relative to the hand-distal condition. These new findings suggest that the critical stage at which hand-stimulus proximity affects cognitive processing lies past the early perceptual processing, acting instead on later stages of processing related to executive functioning.

Grounding by Attention Simulation in Peripersonal Space: Pupils Dilate to Pinch Grip But Not Big Size Nominal Classifier

Grammatical categories represent implicit knowledge, and it is not known if such abstract linguistic knowledge can be continuously grounded in real-life experiences, nor is it known what types of mental states can be simulated. A former study showed that attention bias in peripersonal space (PPS) affects reaction times in grammatical congruency judgments of nominal classifiers, suggesting that simulated semantics may include reenactment of attention. In this study, we contrasted a Chinese nominal classifier used with nouns denoting pinch grip objects with a classifier for nouns with big object referents in a pupil dilation experiment. Twenty Chinese native speakers read grammatical and ungrammatical classifier-noun combinations and made grammaticality judgment while their pupillary responses were measured. It was found that their pupils dilated significantly more to the pinch grip classifier than to the big object classifier, indicating attention simulation in PPS. Pupil dilations were also significantly larger with congruent trials on the whole than in incongruent trials, but crucially, congruency and classifier semantics were independent of each other. No such effects were found in controls.

Are Older Adults Less Embodied? A Review of Age Effects through the Lens of Embodied Cognition

Embodied cognition is a theoretical framework which posits that cognitive function is intimately intertwined with the body and physical actions. Although the field of psychology is increasingly accepting embodied cognition as a viable theory, it has rarely been employed in the gerontological literature. However, embodied cognition would appear to have explanatory power for aging research given that older adults typically manifest concurrent physical and mental changes, and that research has indicated a correlative relationship between such changes. The current paper reviews age-related changes in sensory processing, mental representation, and the action-perception relationship, exploring how each can be understood through the lens of embodied cognition. Compared to younger adults, older adults exhibit across all three domains an increased tendency to favor visual processing over bodily factors, leading to the conclusion that older adults are less embodied than young adults. We explore the significance of this finding in light of existing theoretical models of aging and argue that embodied cognition can benefit gerontological research by identifying further factors that can explain the cause of age-related declines.

Walking through doorways causes forgetting: Event structure or updating disruption?

According to event cognition theory, people segment experience into separate event models. One consequence of this segmentation is that when people transport objects from one location to another, memory is worse than if people move across a large location. In two experiments participants navigated through a virtual environment, and recognition memory was tested in either the presence or the absence of a location shift for objects that were recently interacted with (i.e., just picked up or set down). Of particular concern here is whether this location updating effect is due to (a) differences in retention intervals as a result of the navigation process, (b) a temporary disruption in cognitive processing that may occur as a result of the updating processes, or (c) a need to manage multiple event models, as has been suggested in prior research. Experiment 1 explored whether retention interval is driving this effect by recording travel times from the acquisition of an object and the probe time. The results revealed that travel times were similar, thereby rejecting a retention interval explanation. Experiment 2 explored whether a temporary disruption in processing is producing the effect by introducing a 3-second delay prior to the presentation of a memory probe. The pattern of results was not affected by adding a delay, thereby rejecting a temporary disruption account. These results are interpreted in the context of the event horizon model, which suggests that when there are multiple event models that contain common elements there is interference at retrieval, which compromises performance.

Phantoms on the hands: Influence of the body on brief synchiric visual percepts

Recent studies have found preferential responses for brief, transient visual stimuli near the hands, suggesting a link between magnocellular visual processing and peripersonal representations. We report an individual with a right hemisphere lesion whose illusory phantom percepts may be attributable to an impairment in the peripersonal system specific to transient visual stimuli. When presented with a single, brief (250 ms) visual stimulus to her ipsilesional side, she reported visual percepts on both sides - synchiria. These contralesional phantoms were significantly more frequent when visual stimuli were presented on the hands versus off the hands. We next manipulated stimulus duration to examine the relationship between these phantom percepts and transient visual processing. We found a significant position by duration interaction, with substantially more phantom synchiric percepts on the hands for brief compared to sustained stimuli. This deficit provides novel evidence both for preferential processing of transient visual stimuli near the hands, and for mechanisms that, when damaged, result in phantom percepts.

Reduced object-based perception in the near-hand space

Previous studies have shown that hand proximity changes visual perception (Abrams et al. in Cognition 107(3):1035-1047, 2008). The present study examined the effects of hand proximity on object-based perception. In three experiments, participants viewed stimuli that were either near to or far from their hands. The target stimulus appeared, after a cue, in one of two rectangular objects: either at the location that had been previously cued, at the uncued end of the cued object, or in the uncued object. We found a significantly reduced same-object benefit in reaction time for stimuli near the hands in one experiment. Interestingly, we observed a same-object cost in sensitivity for stimuli near the hands in another experiment. The results reveal that object-based perception is disrupted in the near-hand space. This is consistent with previous findings revealing altered visual processing near the hands.

Investigating implicit statistical learning mechanisms through contextual cueing

Since its inception, the contextual cueing (CC) paradigm has generated considerable interest in various fields of cognitive sciences because it constitutes an elegant approach to understanding how statistical learning (SL) mechanisms can detect contextual regularities during a visual search. In this article we review and discuss five aspects of CC: (i) the implicit nature of learning, (ii) the mechanisms involved in CC, (iii) the mediating factors affecting CC, (iv) the generalization of CC phenomena, and (v) the dissociation between implicit and explicit CC phenomena. The findings suggest that implicit SL is an inherent component of ongoing processing which operates through clustering, associative, and reinforcement processes at various levels of sensory-motor processing, and might result from simple spike-timing-dependent plasticity.

Hand position influences perceptual grouping

Over the past decade, evidence has accumulated that performance in attention, perception, and memory-related tasks are influenced by the distance between the hands and the stimuli (i.e., placing the observer's hands near or far from the stimuli). To account for existing findings, it has recently been proposed that processing of stimuli near the hands is dominated by the magnocellular visual pathway. The present study tests an implication of this hypothesis, whether perceptual grouping is reduced in hands-proximal space. Consistent with previous work on the object-based capture of attention, a benefit for the visual object in the hands-distal condition was observed in the present study. Interestingly, the object-based benefit did not emerge in the hands-proximal condition, suggesting perceptual grouping is impaired near the hands. This change in perceptual grouping processes provides further support for the hypothesis that visual processing near the hands is subject to increased magnocellular processing.

Grasp posture alters visual processing biases near the hands

Observers experience biases in visual processing for objects within easy reach of their hands; these biases may assist them in evaluating items that are candidates for action. I investigated the hypothesis that hand postures that afford different types of actions differentially bias vision. Across three experiments, participants performed global-motion-detection and global-form-perception tasks while their hands were positioned (a) near the display in a posture affording a power grasp, (b) near the display in a posture affording a precision grasp, or (c) in their laps. Although the power-grasp posture facilitated performance on the motion-detection task, the precision-grasp posture instead facilitated performance on the form-perception task. These results suggest that the visual system weights processing on the basis of an observer's current affordances for specific actions: Fast and forceful power grasps enhance temporal sensitivity, whereas detail-oriented precision grasps enhance spatial sensitivity.

From body shadows to bodily attention: automatic orienting of tactile attention driven by cast shadows

Body shadows orient attention to the body-part casting the shadow. We have investigated the automaticity of this phenomenon, by addressing its time-course and its resistance to contextual manipulations. When targets were tactile stimuli at the hands (Exp.1) or visual stimuli near the body-shadow (Exp.2), cueing effects emerged regardless of the delay between shadow and target onset (100, 600, 1200, 2400ms). This suggests a fast and sustained attention orienting to body-shadows, that involves both the space occupied by shadows (extra-personal space) and the space the shadow refers to (own body). When target type became unpredictable (tactile or visual), shadow-cueing effects remained robust only for tactile targets, as visual stimuli showed no overall reliable effects, regardless of whether they occurred near the shadow (Exp.3) or near the body (Exp.4). We conclude that mandatory attention shifts triggered by body-shadows are limited to tactile targets and, instead, are less automatic for visual stimuli.

Trade-offs in visual processing for stimuli near the hands

It is known that stimuli near the hands receive preferential processing. In the present study, we explored changes in early vision near the hands. Participants were more sensitive to low-spatial-frequency information and less sensitive to high-spatial-frequency information for stimuli presented close to the hands. This pattern suggests enhanced processing in the magnocellular visual pathway for such stimuli, and impaired processing in the parvocellular pathway. Consistent with that possibility, we found that the effects of hand proximity in several tasks were eliminated by illumination with red diffuse light-a manipulation known to impair magnocellular processing. These results help clarify how the hands affect vision.

Hand proximity facilitates spatial discrimination of auditory tones

The effect of hand proximity on vision and visual attention has been well documented. In this study we tested whether such effect(s) would also be present in the auditory modality. With hands placed either near or away from the audio sources, participants performed an auditory-spatial discrimination (Experiment 1: left or right side), pitch discrimination (Experiment 2: high, med, or low tone), and spatial-plus-pitch (Experiment 3: left or right; high, med, or low) discrimination task. In Experiment 1, when hands were away from the audio source, participants consistently responded faster with their right hand regardless of stimulus location. This right hand advantage, however, disappeared in the hands-near condition because of a significant improvement in left hand's reaction time (RT). No effect of hand proximity was found in Experiments 2 or 3, where a choice RT task requiring pitch discrimination was used. Together, these results that the perceptual and attentional effect of hand proximity is not limited to one specific modality, but applicable to the entire “space” near the hands, including stimuli of different modality (at least visual and auditory) within that space. While these findings provide evidence from auditory attention that supports the multimodal account originally raised by Reed et al. (2006), we also discuss the possibility of a dual mechanism hypothesis to reconcile findings from the multimodal and magno/parvocellular account.

Hand proximity differentially affects visual working memory for color and orientation in a binding task

Observers determined whether two sequentially presented arrays of six lines were the same or different. Differences, when present, involved either a swap in the color of two lines or a swap in the orientation of two lines. Thus, accurate change detection required the binding of color and orientation information for each line within visual working memory. Holding viewing distance constant, the proximity of the arrays to the hands was manipulated. Placing the hands near the to-be-remembered array decreased participants’ ability to remember color information, but increased their ability to remember orientation information. This pair of results indicates that hand proximity differentially affects the processing of various types of visual information, a conclusion broadly consistent with functional and anatomical differences in the magnocellular and parvocellular pathways. It further indicates that hand proximity affects the likelihood that various object features will be encoded into integrated object files.

Shorter lines facilitate reading in those who struggle

People with dyslexia, who ordinarily struggle to read, sometimes remark that reading is easier when e-readers are used. Here, we used eye tracking to observe high school students with dyslexia as they read using these devices. Among the factors investigated, we found that reading using a small device resulted in substantial benefits, improving reading speeds by 27%, reducing the number of fixations by 11%, and importantly, reducing the number of regressive saccades by more than a factor of 2, with no cost to comprehension. Given that an expected trade-off between horizontal and vertical regression was not observed when line lengths were altered, we speculate that these effects occur because sluggish attention spreads perception to the left as the gaze shifts during reading. Short lines eliminate crowded text to the left, reducing regression. The effects of attention modulation by the hand, and of increased letter spacing to reduce crowding, were also found to modulate the oculomotor dynamics in reading, but whether these factors resulted in benefits or costs depended on characteristics, such as visual attention span, that varied within our sample.

Grasp posture modulates attentional prioritization of space near the hands

Changes in visual processing near the hands may assist observers in evaluating items that are candidates for actions. If altered vision near the hands reflects adaptations linked to effective action production, then positioning the hands for different types of actions could lead to different visual biases. I examined the influence of hand posture on attentional prioritization to test this hypothesis. Participants placed one of their hands on a visual display and detected targets appearing either near or far from the hand. Replicating previous findings, detection near the hand was facilitated when participants positioned their hand on the display in a standard open palm posture affording a power grasp (Experiments 1 and 3). However, when participants instead positioned their hand in a pincer grasp posture with the thumb and forefinger resting on the display, they were no faster to detect targets appearing near their hand than targets appearing away from their hand (Experiments 2 and 3). These results demonstrate that changes in visual processing near the hands rely on the hands' posture. Although hands positioned to afford power grasps facilitate rapid onset detection, a pincer grasp posture that affords more precise action does not.

The World Within Reach

Our mental processing of the visual world is not independent of our physical actions within it. Placing objects near one’s hands and interacting with objects using tools can enhance visual perception, bias and prolong the allocation of attention, and distort memory in systematic ways. This suggests that the world within our reach is cognitively different from the world beyond reach. In this review, we examine the evidence supporting this conclusion, focusing on the cognitive and neural mechanisms that underlie these effects, the parameters that may control their emergence, and their potential practical applications.

Take the matter into your own hands: a brief review of the effect of nearby-hands on visual processing

An exciting new line of research that investigates the impact of one's own hands on visual perception and attention has flourished in the past several years. Specifically, several studies have demonstrated that the nearness of one's hands can modulate visual perception, visual attention, and even visual memory. These studies together shed new light on how the brain prioritizes certain information to be processed first. This review first outlines the recent progress that has been made to uncover various characteristics of the nearby-hand effect, including how they may be transferred to a familiar tool. We then summarize the findings into four specific characteristics of the nearby-hand effect, and conclude with a possible neural mechanism that may account for all the findings.