Grasping preparation enhances orientation change detection

Increased visual alpha-band activity during self-paced finger tapping does not affect early visual stimulus processing

Alpha-band activity is thought to be involved in orchestrating neural processing within and across brain regions relevant to various functions such as perception, cognition, and motor activity. Across different studies, attenuated alpha-band activity has been linked to increased neural excitability. Yet, there have been conflicting results concerning the consequences of alpha-band modulations for early sensory processing. We here examined whether movement-related alterations in visual alpha-band activity affected the early sensory processing of visual stimuli. For this purpose, in an EEG experiment, participants were engaged in a voluntary finger-tapping task while passively viewing flickering dots. We found extensive and expected movement-related amplitude modulations of motor alpha- and beta-band activity with event-related-desynchronization (ERD) before and during, and event-related-synchronization (ERS) after single voluntary finger taps. Crucially, while a visual alpha-band ERS accompanied the motor alpha-ERD before and during each finger tap, flicker-evoked Steady-State-Visually-Evoked-Potentials (SSVEPs), as a marker of early visual sensory gain, were not modulated in amplitude. As early sensory stimulus processing was unaffected by amplitude-modulated visual alpha-band activity, this argues against the idea that alpha-band activity represents a mechanism by which early sensory gain modulation is implemented. The distinct neural dynamics of visual alpha-band activity and early sensory processing may point to distinct and multiplexed neural selection processes in visual processing.

Action-specific feature processing in the human cortex: An fMRI study

Sensorimotor integration involves feedforward and reentrant processing of sensory input. Grasp-related motor activity precedes and is thought to influence visual object processing. Yet, while the importance of reentrant feedback is well established in perception, the top-down modulations for action and the neural circuits involved in this process have received less attention. Do action-specific intentions influence the processing of visual information in the human cortex? Using a cue-separation fMRI paradigm, we found that action-specific instruction processing (manual alignment vs. grasp) became apparent only after the visual presentation of oriented stimuli, and occurred as early as in the primary visual cortex and extended to the dorsal visual stream, motor and premotor areas. Further, dorsal stream area aIPS, known to be involved in object manipulation, and the primary visual cortex showed task-related functional connectivity with frontal, parietal and temporal areas, consistent with the idea that reentrant feedback from dorsal and ventral visual stream areas modifies visual inputs to prepare for action. Importantly, both the task-dependent modulations and connections were linked specifically to the object presentation phase of the task, suggesting a role in processing the action goal. Our results show that intended manual actions have an early, pervasive, and differential influence on the cortical processing of vision.

Neural correlates of virtual reality-based attention training: An fMRI study

THEORETICAL BACKGROUND

Virtual Reality technology is increasingly used in attention rehabilitation for functional training purposes. However, the neural mechanisms by which Virtual Reality can affect attentional functioning are still unclear. The current study's objective is to examine the effects of stereoscopic vs. monoscopic presentation on neural processing during a visual attention task.

METHOD

Thirty-two healthy participants performed a visual attention task in an immersive virtual environment that was displayed via MR-compatible video goggles in an MRI scanner. The paradigm altered between trials that required active engagement with the task and mere observation trials. Furthermore, the form of binocular presentation switched between monoscopic and stereoscopic presentation.

RESULTS

Analyses yielded evidence for increased activation in stereoscopic compared to monoscopic trials in the tertiary visual cortex area V3A as well as elevated activation in the dorsal attention network when engaging in the attention task. An additional ROI analysis of area V3A revealed significantly lower attentional engagement costs in stereoscopic conditions.

DISCUSSION

Results support previous findings suggesting that V3A is involved in binocular depth perception. Furthermore, heightened activation in V3A following stereoscopic presentation seemed to facilitate attentional engagement with the task. Considering that V3A is the origin of the dorso-dorsal, ventro-dorsal, and ventral visual processing pathways, we regard it as a gating area that decides which kind of visual perception is processed.

Orientation representation in human visual cortices: contributions of non-visual information and action-related process

Orientation processing in the human brain plays a crucial role in guiding grasping actions toward an object. Remarkably, despite the absence of visual input, the human visual cortex can still process orientation information. Instead of visual input, non-visual information, including tactile and proprioceptive sensory input from the hand and arm, as well as feedback from action-related processes, may contribute to orientation processing. However, the precise mechanisms by which the visual cortices process orientation information in the context of non-visual sensory input and action-related processes remain to be elucidated. Thus, our study examined the orientation representation within the visual cortices by analyzing the blood-oxygenation-level-dependent (BOLD) signals under four action conditions: direct grasp (DG), air grasp (AG), non-grasp (NG), and uninformed grasp (UG). The images of the cylindrical object were shown at +45° or - 45° orientations, corresponding to those of the real object to be grasped with the whole-hand gesture. Participants judged their orientation under all conditions. Grasping was performed without online visual feedback of the hand and object. The purpose of this design was to investigate the visual areas under conditions involving tactile feedback, proprioception, and action-related processes. To address this, a multivariate pattern analysis was used to examine the differences among the cortical patterns of the four action conditions in orientation representation by classification. Overall, significant decoding accuracy over chance level was discovered for the DG; however, during AG, only the early visual areas showed significant accuracy, suggesting that the object's tactile feedback influences the orientation process in higher visual areas. The NG showed no statistical significance in any area, indicating that without the grasping action, visual input does not contribute to cortical pattern representation. Interestingly, only the dorsal and ventral divisions of the third visual area (V3d and V3v) showed significant decoding accuracy during the UG despite the absence of visual instructions, suggesting that the orientation representation was derived from action-related processes in V3d and visual recognition of object visualization in V3v. The processing of orientation information during non-visually guided grasping of objects relies on other non-visual sources and is specifically divided by the purpose of action or recognition.

The influence of action on perception spans different effectors

Perception and action are fundamental processes that characterize our life and our possibility to modify the world around us. Several pieces of evidence have shown an intimate and reciprocal interaction between perception and action, leading us to believe that these processes rely on a common set of representations. The present review focuses on one particular aspect of this interaction: the influence of action on perception from a motor effector perspective during two phases, action planning and the phase following execution of the action. The movements performed by eyes, hands, and legs have a different impact on object and space perception; studies that use different approaches and paradigms have formed an interesting general picture that demonstrates the existence of an action effect on perception, before as well as after its execution. Although the mechanisms of this effect are still being debated, different studies have demonstrated that most of the time this effect pragmatically shapes and primes perception of relevant features of the object or environment which calls for action; at other times it improves our perception through motor experience and learning. Finally, a future perspective is provided, in which we suggest that these mechanisms can be exploited to increase trust in artificial intelligence systems that are able to interact with humans.

Visual working memory representations bias attention more when they are the target of an action plan

Attention has frequently been regarded as an emergent property of linking sensory representations to action plans. It has recently been proposed that similar mechanisms may operate within visual working memory (VWM), such that linking an object in VWM to an action plan strengthens its sensory memory representation, which then expresses as an attentional bias. Here we directly tested this hypothesis by comparing attentional biases induced by VWM representations which were the target of a future action, to those induced by VWM representations that were equally task-relevant, but not the direct target of action. We predicted that the first condition would result in a more prioritized memory state and hence stronger attentional biases. Specifically, participants memorized a geometric shape for a subsequent memory test. At test, in case of a match, participants either had to perform a grip movement on the matching object (action condition), or perform the same movement, but on an unrelated object (control condition). To assess any attentional biases, during the delay period between memorandum and test, participants performed a visual selection task in which either the target was surrounded by the memorized shape (congruent trials) or a distractor (incongruent trials). Eye movements were measured as a proxy for attentional priority. We found a significant interaction for saccade latencies between action condition and shape congruency, reflecting more pronounced VWM-based attentional biases in the action condition. Our results are consistent with the idea that action plans prioritize sensory representations in VWM.

Action planning modulates the representation of object features in human fronto-parietal and occipital cortex

The visual cortex has been extensively studied to investigate its role in object recognition but to a lesser degree to determine how action planning influences the representation of objects' features. We used functional MRI and pattern classification methods to determine if during action planning, object features (orientation and location) could be decoded in an action‐dependent way. Sixteen human participants used their right dominant hand to perform movements (Align or Open reach) towards one of two 3D‐real oriented objects that were simultaneously presented and placed on either side of a fixation cross. While both movements required aiming towards target location, Align but not Open reach movements required participants to precisely adjust hand orientation. Therefore, we hypothesized that if the representation of object features is modulated by the upcoming action, pre‐movement activity pattern would allow more accurate dissociation between object features in Align than Open reach tasks. We found such dissociation in the anterior and posterior parietal cortex, as well as in the dorsal premotor cortex, suggesting that visuomotor processing is modulated by the upcoming task. The early visual cortex showed significant decoding accuracy for the dissociation between object features in the Align but not Open reach task. However, there was no significant difference between the decoding accuracy in the two tasks. These results demonstrate that movement‐specific preparatory signals modulate object representation in the frontal and parietal cortex, and to a lesser extent in the early visual cortex, likely through feedback functional connections.

Horizontal target size perturbations during grasping movements are described by subsequent size perception and saccade amplitude

Perception and action are essential in our day-to-day interactions with the environment. Despite the dual-stream theory of action and perception, it is now accepted that action and perception processes interact with each other. However, little is known about the impact of unpredicted changes of target size during grasping actions on perception. We assessed whether size perception and saccade amplitude were affected before and after grasping a target that changed its horizontal size during the action execution under the presence or absence of tactile feedback. We have tested twenty-one participants in 4 blocks of 30 trials. Blocks were divided into two experimental tactile feedback paradigms: tactile and non-tactile. Trials consisted of 3 sequential phases: pre-grasping size perception, grasping, and post-grasping size perception. During pre- and post-phases, participants executed a saccade towards a horizontal bar and performed a manual size estimation of the bar size. During grasping phase, participants were asked to execute a saccade towards the bar and to make a grasping action towards the screen. While grasping, 3 horizontal size perturbation conditions were applied: non-perturbation, shortening, and lengthening. 30% of the trials presented perturbation, meaning a symmetrically shortened or lengthened by 33% of the original size. Participants’ hand and eye positions were assessed by a motion capture system and a mobile eye-tracker, respectively. After grasping, in both tactile and non-tactile feedback paradigms, size estimation was significantly reduced in lengthening (p = 0.002) and non-perturbation (p<0.001), whereas shortening did not induce significant adjustments (p = 0.86). After grasping, saccade amplitude became significantly longer in shortening (p<0.001) and significantly shorter in lengthening (p<0.001). Non-perturbation condition did not display adjustments (p = 0.95). Tactile feedback did not generate changes in the collected perceptual responses, but horizontal size perturbations did so, suggesting that all relevant target information used in the movement can be extracted from the post-action target perception.

Visual attention around a hand location localized by proprioceptive information

Facilitation of visual processing has been reported in the space near the hand. To understand the underlying mechanism of hand proximity attention, we conducted experiments that isolated hand-related effects from top–down attention, proprioceptive information from visual information, the position effect from the influence of action, and the distance effect from the peripersonal effect. The flash-lag effect was used as an index of attentional modulation. Because the results showed that the flash-lag effect was smaller at locations near the hand, we concluded that there was a facilitation effect of the visual stimuli around the hand location identified through proprioceptive information. This was confirmed by conventional reaction time measures. We also measured steady-state visual evoked potential (SSVEP) in order to investigate the spatial properties of hand proximity attention and top–down attention. The results showed that SSVEP reflects the effect of top–down attention but not that of hand proximity attention. This suggests that the site of hand proximity attention is at a later stage of visual processing, assuming that SSVEP responds to neural activities at the early stages. The results of left-handers differed from those of right-handers, and this is discussed in relation to handedness variation.

Grasp aperture corrections in reach-to-grasp movements do not reliably alter size perception

When grasping an object, the opening between the fingertips (grip aperture) scales with the size of the object. If an object changes in size, the grip aperture has to be corrected. In this study, it was investigated whether such corrections would influence the perceived size of objects. The grasping plan was manipulated with a preview of the object, after which participants initiated their reaching movement without vision. In a minority of the grasps, the object changed in size after the preview and participants had to adjust their grasping movement. Visual feedback was manipulated in two experiments. In experiment 1, vision was restored during reach and both visual and haptic information was available to correct the grasp and lift the object. In experiment 2, no visual information was provided during the movement and grasps could only be corrected using haptic information. Participants made reach-to-grasp movements towards two objects and compared these in size. Results showed that participants adjusted their grasp to a change in object size from preview to grasped object in both experiments. However, a change in object size did not bias the perception of object size or alter discrimination performance. In experiment 2, a small perceptual bias was found when objects changed from large to small. However, this bias was much smaller than the difference that could be discriminated and could not be considered meaningful. Therefore, it can be concluded that the planning and execution of reach-to-grasp movements do not reliably affect the perception of object size.

Slowing the body slows down time perception

Interval timing is a fundamental component of action and is susceptible to motor-related temporal distortions. Previous studies have shown that concurrent movement biases temporal estimates, but have primarily considered self-modulated movement only. However, real-world encounters often include situations in which movement is restricted or perturbed by environmental factors. In the following experiments, we introduced viscous movement environments to externally modulate movement and investigated the resulting effects on temporal perception. In two separate tasks, participants timed auditory intervals while moving a robotic arm that randomly applied four levels of viscosity. Results demonstrated that higher viscosity led to shorter perceived durations. Using a drift-diffusion model and a Bayesian observer model, we confirmed these biasing effects arose from perceptual mechanisms, instead of biases in decision making. These findings suggest that environmental perturbations are an important factor in movement-related temporal distortions, and enhance the current understanding of the interactions of motor activity and cognitive processes.

The effect of motor engagement on memory: Testing a motor-induced encoding account

The motor system is traditionally thought to reflect the output of cognition. However, the inverse relationship of how the motor system impacts cognitive processes is less known. Work on this interaction has demonstrated that recognition memory for stimuli presented in combination with the inhibition of a prepared action is weaker compared to stimuli associated with the execution of an action (Chiu & Egner, Psychological Science, 26, 27-38, 2015a). This effect has been explained through competition for common neural resources: to the extent that response inhibition processes are recruited, fewer resources are available for memory encoding (Chiu & Egner, Journal of Neuroscience, 35, 11936-11945, 2015b). Alternatively, it has been proposed that action execution enhances memory encoding (Yebra et al., Nature Communications, 10(1), 1-12, 2019). In this report, we examined how recognition memory for stimuli paired with both the preparation and execution of a motor response compare to stimuli absent of any motor processes. We first replicated Chiu and Egner (2015a, 2015b). Next, we added a motor-neutral condition as a baseline comparison. Across three experiments, recognition memory for stimuli associated with action execution was superior to stimuli absent of motor demands. More importantly, we found that recognition memory for stimuli associated with motor preparation, but no subsequent execution, was also superior to stimuli that did not engage the motor system (Experiments 2a and 2b). These results support a motor-induced encoding effect, in which the degree of motor processing (both action preparation and action execution) enhanced memory encoding.

Action affects perception through modulation of attention

The present study explored the origin of perceptual changes repeatedly observed in the context of actions. In Experiment 1, participants tried to hit a circular target with a stylus movement under restricted feedback conditions. We measured the perception of target size during action planning and observed larger estimates for larger movement distances. In Experiment 2, we then tested the hypothesis that this action specific influence on perception is due to changes in the allocation of spatial attention. For this purpose, we replaced the hitting task by conditions of focused and distributed attention and measured the perception of the former target stimulus. The results revealed changes in the perceived stimulus size very similar to those observed in Experiment 1. These results indicate that action’s effects on perception root in changes of spatial attention.

Selective Modulation of Early Visual Cortical Activity by Movement Intention

The primate visual system contains myriad feedback projections from higher- to lower-order cortical areas, an architecture that has been implicated in the top-down modulation of early visual areas during working memory and attention. Here we tested the hypothesis that these feedback projections also modulate early visual cortical activity during the planning of visually guided actions. We show, across three separate human functional magnetic resonance imaging (fMRI) studies involving object-directed movements, that information related to the motor effector to be used (i.e., limb, eye) and action goal to be performed (i.e., grasp, reach) can be selectively decoded-prior to movement-from the retinotopic representation of the target object(s) in early visual cortex. We also find that during the planning of sequential actions involving objects in two different spatial locations, that motor-related information can be decoded from both locations in retinotopic cortex. Together, these findings indicate that movement planning selectively modulates early visual cortical activity patterns in an effector-specific, target-centric, and task-dependent manner. These findings offer a neural account of how motor-relevant target features are enhanced during action planning and suggest a possible role for early visual cortex in instituting a sensorimotor estimate of the visual consequences of movement.

Faster recognition of graspable targets defined by orientation in a visual search task

Peri-hand space is the area surrounding the hand. Objects within this space may be subject to increased visuospatial perception, increased attentional prioritization, and slower attentional disengagement compared to more distal objects. This may result from kinesthetic and visual feedback about the location of the hand that projects from the reach and grasp networks of the dorsal visual stream back to occipital visual areas, which in turn, refines cortical visual processing that can subsequently guide skilled motor actions. Thus, we hypothesized that visual stimuli that afford action, which are known to potentiate activity in the dorsal visual stream, would be associated with greater alterations in visual processing when presented near the hand. To test this, participants held their right hand near or far from a touchscreen that presented a visual array containing a single target object that differed from 11 distractor objects by orientation only. The target objects and their accompanying distractors either strongly afforded grasping or did not. Participants identified the target among the distractors by reaching out and touching it with their left index finger while eye-tracking was used to measure visual search times, target recognition times, and search accuracy. The results failed to support the theory of enhanced visual processing of graspable objects near the hand as participants were faster at recognizing graspable compared to non-graspable targets, regardless of the position of the right hand. The results are discussed in relation to the idea that, in addition to potentiating appropriate motor responses, object affordances may also potentiate early visual processes necessary for object recognition.

Temporal dynamics of a perceptual decision

Previous research suggests that cognitive factors acting in a top-down manner influence the perceptual interpretation of ambiguous stimuli. To examine the temporal unfolding of these influences as a perceptual decision evolves, we have implemented a modified version of the stream-bounce display. Our novel approach allows us to track responses to stream-bounce stimuli dynamically over the entire course of the motion sequence rather than collecting a subjective report after the fact. Using a trackpad, we had participants control a cursor to track a stream-bounce target actively from start to end and measured tracking speed throughout as the dependent variable. Our paradigm replicated the typical effect of visual-only displays being associated with a streaming bias and audiovisual displays with a bouncing bias. Our main finding is a significant behavioral change preceding a perceptual decision that then predicts that decision. Specifically, for trials in which the sound was presented, tracking speeds were significantly slower starting 500 ms before the point of coincidence and presentation of the sound for bounce compared to stream responses. We suggest that behavioral response may reflect a cognitive expectation of a perceptual outcome that then biases action and the interpretation of sensory input to favor that forthcoming percept in a manner consistent with both the predictive-coding and common-coding theoretical frameworks. Our approach provides a novel behavioral corroboration of recent imaging studies that are suggestive of early brain activity in perception and action.

Action Fluency Facilitates Perceptual Discrimination

Perception and action interact in nearly every moment of daily life. Previous studies have demonstrated not only that perceptual input shapes action but also that various factors associated with action-including individual abilities and biomechanical costs-influence perceptual decisions. However, it is unknown how action fluency affects the sensitivity of early-stage visual perception, such as orientation. To address this question, we used a dual-task paradigm: Participants prepared an action (e.g., grasping), while concurrently performing an orientation-change-detection task. We demonstrated that as actions became more fluent (e.g., as grasping errors decreased), perceptual-discrimination performance also improved. Importantly, we found that grasping training prior to discrimination enhanced subsequent perceptual sensitivity, supporting the notion of a reciprocal relation between perception and action.

The influence of motor preparation on the processing of action-relevant visual features

Action preparation can facilitate performance in tasks of visual perception, for instance by speeding up responses to action-relevant stimulus features. However, it is unknown whether this facilitation reflects an influence on early perceptual processing, or instead post-perceptual processes. In three experiments, a combination of psychophysics and electroencephalography was used to investigate whether visual features are influenced by action preparation at the perceptual level. Participants were cued to prepare oriented reach-to-grasp actions before discriminating target stimuli oriented in the same direction as the prepared grasping action (congruent) or not (incongruent). As expected, stimuli were discriminated faster if their orientation was congruent, compared to incongruent, with the prepared action. However, action-congruency had no influence on perceptual sensitivity, regardless of cue-target interval and discrimination difficulty. The reaction time effect was not accompanied by modulations of early visual-evoked potentials. Instead, beta-band (13-30 Hz) synchronization over sensorimotor brain regions was influenced by action preparation, indicative of improved response preparation. Together, the results suggest that action preparation may not modulate early visual processing of orientation, but likely influences higher order response or decision related processing. While early effects of action on spatial perception are well documented, separate mechanisms appear to govern non-spatial feature selection.

A Rhythmic Theory of Attention

Recent evidence has demonstrated that environmental sampling is a fundamentally rhythmic process. Both perceptual sensitivity during covert spatial attention and the probability of overt exploratory movements are tethered to theta-band activity (3-8Hz) in the attention network. The fronto-parietal part of this network is positioned at the nexus of sensory and motor functions, directing two tightly coupled processes related to environmental exploration: preferential routing of sensory input and saccadic eye movements. We propose that intrinsic theta rhythms temporally resolve potential functional conflicts by periodically reweighting functional connections between higher-order brain regions and either sensory or motor regions. This rhythmic reweighting alternately promotes either sampling at a behaviorally relevant location (i.e., sensory functions) or shifting to another location (i.e., motor functions).

Dynamic large-scale network synchronization from perception to action

Sensory-guided actions entail the processing of sensory information, generation of perceptual decisions, and the generation of appropriate actions. Neuronal activity underlying these processes is distributed into sensory, fronto-parietal, and motor brain areas, respectively. How the neuronal processing is coordinated across these brain areas to support functions from perception to action remains unknown. We investigated whether phase synchronization in large-scale networks coordinate these processes. We recorded human cortical activity with magnetoencephalography (MEG) during a task in which weak somatosensory stimuli remained unperceived or were perceived. We then assessed dynamic evolution of phase synchronization in large-scale networks from source-reconstructed MEG data by using advanced analysis approaches combined with graph theory. Here we show that perceiving and reporting of weak somatosensory stimuli is correlated with sustained strengthening of large-scale synchrony concurrently in delta/theta (3-7 Hz) and gamma (40-60 Hz) frequency bands. In a data-driven network localization, we found this synchronization to dynamically connect the task-relevant, that is, the fronto-parietal, sensory, and motor systems. The strength and temporal pattern of interareal synchronization were also correlated with the response times. These data thus show that key brain areas underlying perception, decision-making, and actions are transiently connected by large-scale dynamic phase synchronization in the delta/theta and gamma bands.

Passive sensorimotor stimulation triggers long lasting alpha-band fluctuations in visual perception

Movement planning and execution rely on the anticipation and online control of the incoming sensory input. Evidence suggests that sensorimotor processes may synchronize visual rhythmic activity in preparation of action performance. Indeed, we recently reported periodic fluctuations of visual contrast sensitivity that are time-locked to the onset of an intended movement of the arm. However, the origin of the observed visual modulations has so far remained unclear because of the endogenous (and thus temporally undetermined) activation of the sensorimotor system that is associated with voluntary movement initiation. In this study, we activated the sensorimotor circuitry involved in the hand control in an exogenous and controlled way by means of peripheral stimulation of the median nerve and characterized the spectrotemporal dynamics of the ensuing visual perception. The stimulation of the median nerve triggers robust and long-lasting (∼1 s) alpha-band oscillations in visual perception, whose strength is temporally modulated in a way that is consistent with the changes in alpha power described at the neurophysiological level after sensorimotor stimulation. These findings provide evidence in support of a causal role of the sensorimotor system in modulating oscillatory activity in visual areas with consequences for visual perception. NEW & NOTEWORTHY This study shows that the peripheral activation of the somatomotor hand system triggers long-lasting alpha periodicity in visual perception. This demonstrates that not only the endogenous sensorimotor processes involved in movement preparation but also the passive stimulation of the sensorimotor system can synchronize visual activity. The present work suggests that oscillation-based mechanisms may subserve core (task independent) sensorimotor integration functions.

Grasp preparation modulates early visual processing of size and detection of local/global stimulus features

Preparing to grasp objects facilitates visual processing of object location, orientation and size, compared to preparing actions such as pointing. This influence of action on perception reflects mechanisms of selection in visual perception tuned to current action goals, such that action relevant sensory information is prioritized relative to less relevant information. In three experiments, rather than varying movement type (grasp vs point), the magnitude of a prepared movement (power vs precision grasps) was manipulated while visual processing of object size, as well as local/global target detection was measured. Early event-related potentials (ERP) elicited by task-irrelevant visual probes were enhanced for larger probes during power grasp preparation and smaller probes during precision grasp preparation. Local targets were detected faster following precision, relative to power grasp cues. The results demonstrate a direct influence of grasp preparation on sensory processing of size and suggest that the hierarchical dimension of objects may be a relevant perceptual feature for grasp programming. To our knowledge, this is the first evidence that preparing different magnitudes of the same basic action has systematic effects on visual processing.

Effector-based attention systems

Visual processing is known to be enhanced at the end point of eye movements. Feedback within the oculomotor system has been shown to drive these alterations in visual processing. However, we do not simply view the world; we also reach out and interact using our hands. Consequently, it is not surprising that visual processing has also been shown to be altered in near-hand space. A growing body of work documents a myriad of alterations in near-hand visual processing, with little consensus on the neural underpinnings of the effect of the hand. Since movement of the eyes and hands is governed by parallel frontoparietal networks and since within the oculomotor system feedback from these motor control regions has been shown to drive enhanced visual processing at saccade end points, it is plausible that a similar feedback mechanism is at play in near-hand improvements in visual processing. Here, we compare and contrast oculomotor-driven and hand-driven changes in visual processing and provide support for the hypothesis that feedback within the reaching and grasping systems enhances visual processing near the hand in a novel way.

Rhythmic modulation of visual contrast discrimination triggered by action

Recent evidence suggests that ongoing brain oscillations may be instrumental in binding and integrating multisensory signals. In this experiment, we investigated the temporal dynamics of visual-motor integration processes. We show that action modulates sensitivity to visual contrast discrimination in a rhythmic fashion at frequencies of about 5 Hz (in the theta range), for up to 1 s after execution of action. To understand the origin of the oscillations, we measured oscillations in contrast sensitivity at different levels of luminance, which is known to affect the endogenous brain rhythms, boosting the power of alpha-frequencies. We found that the frequency of oscillation in sensitivity increased at low luminance, probably reflecting the shift in mean endogenous brain rhythm towards higher frequencies. Importantly, both at high and at low luminance, contrast discrimination showed a rhythmic motor-induced suppression effect, with the suppression occurring earlier at low luminance. We suggest that oscillations play a key role in sensory-motor integration, and that the motor-induced suppression may reflect the first manifestation of a rhythmic oscillation.

A different kind of weapon focus: simulated training with ballistic weapons reduces change blindness

Attentional allocation is flexibly altered by action-related priorities. Given that tools - and specifically weapons - can affect attentional allocation, we asked whether training with a weapon or holding a weapon during search would affect change detection. In three experiments, participants searched for changes to agents, shootable objects, or environments in the popular flicker paradigm. Participants trained with a simulated weapon or watched a video from the same training perspective and then searched for changes while holding a weapon or a control object. Results show an effect of training, highlighting the importance of sensorimotor experience for the action-relevant allocation of attention, and a possible interaction between training and the object held during search. Simulated training with ballistic weapons reduces change blindness. This result has implications for the interaction between tool use and attentional allocation.

Non-uniform transformation of subjective time during action preparation

Although many studies have reported a distortion of subjective (internal) time during preparation and execution of actions, it is highly controversial whether actions cause a dilation or compression of time. In the present study, we tested a hypothesis that the previous controversy (dilation vs. compression) partly resulted from a mixture of two types of sensory inputs on which a time length was estimated; some studies asked subjects to measure the time of presentation for a single continuous stimulus (stimulus period, e.g. the duration of a long-lasting visual stimulus on a monitor) while others required estimation of a period without continuous stimulations (no-stimulus period, e.g. an inter-stimulus interval between two flashes). Results of our five experiments supported this hypothesis, showing that action preparation induced a dilation of a stimulus period, whereas a no-stimulus period was not subject to this dilation and sometimes can be compressed by action preparation. Those results provided a new insight into a previous view assuming a uniform dilation or compression of subjective time by actions. Our findings about the distinction between stimulus and no-stimulus periods also might contribute to a resolution of mixed results (action-induced dilation vs. compression) in a previous literature.

Action potential influences spatial perception: Evidence for genuine top-down effects on perception

The action-specific account of spatial perception asserts that a perceiver's ability to perform an action, such as hitting a softball or walking up a hill, impacts the visual perception of the target object. Although much evidence is consistent with this claim, the evidence has been challenged as to whether perception is truly impacted, as opposed to the responses themselves. These challenges have recently been organized as six pitfalls that provide a framework with which to evaluate the empirical evidence. Four case studies of action-specific effects are offered as evidence that meets the framework's high bar, and thus that demonstrates genuine perceptual effects. That action influences spatial perception is evidence that perceptual and action-related processes are intricately and bidirectionally linked.

Functional coupling between frontoparietal and occipitotemporal pathways during action and perception

Several lines of evidence point to areas in the occipitotemporal pathway as being critical in the processes of visual perception and object recognition. Much less appreciated, however, is the role that this pathway plays in object-related processing for the purposes of visually guided action. Here, using functional MRI (fMRI) and functional connectivity (FC) measures, we examined interactions between areas in frontoparietal cortex (FPC) involved in grasping, reaching, eye movements, and tool use and areas in occipitotemporal cortex (OTC) involved in object-, face-, scene-, body-, tool-, and motion-related processing, both during the performance of sensorimotor and visual-perceptual tasks, as well as during passive fixation (resting-state). Cluster analysis of regional time course data identified correspondence in the patterns of FPC and OTC connectivity during the visual-perceptual tasks and rest that both tended to segregate regions along traditional dorsal/ventral pathway boundaries. During the sensorimotor tasks, however, we observed a notable separation in functional coupling between ventral-medial and ventral-lateral regions of OTC, with several of the latter areas often being clustered together with sensorimotor-defined areas in parietal cortex. These findings indicate that the functional coupling of ventral-lateral OTC areas to dorsal parietal and ventral-medial structures is flexible and task-dependent, and suggests that regions in lateral occipital cortex, in particular, may play an important role in mediating interactions between the dorsal and ventral pathways during tasks involving sensorimotor control.

An Eye in the Palm of Your Hand: Alterations in Visual Processing Near the Hand, a Mini-Review

Feedback within the oculomotor system improves visual processing at eye movement end points, also termed a visual grasp. We do not just view the world around us however, we also reach out and grab things with our hands. A growing body of literature suggests that visual processing in near-hand space is altered. The control systems for moving either the eyes or the hands rely on parallel networks of fronto-parietal regions, which have feedback connections to visual areas. Since the oculomotor system effects on visual processing occur through feedback, both through the motor plan and the motor efference copy, a parallel system where reaching and/or grasping motor-related activity also affects visual processing is likely. Areas in the posterior parietal cortex, for example, receive proprioceptive and visual information used to guide actions, as well as motor efference signals. This trio of information channels is all that would be necessary to produce spatial allocation of reach-related visual attention. We review evidence from behavioral and neurophysiological studies that support the hypothesis that feedback from the reaching and/or grasping motor control networks affects visual processing while noting ways in which it differs from that seen within the oculomotor system. We also suggest that object affordances may represent the neural mechanism through which certain object features are selected for preferential processing when stimuli are near the hand. Finally, we summarize the two effector-based feedback systems and discuss how having separate but parallel effector systems allows for efficient decoupling of eye and hand movements.

Rhythmic oscillations of visual contrast sensitivity synchronized with action

It is well known that the motor and the sensory systems structure sensory data collection and cooperate to achieve an efficient integration and exchange of information. Increasing evidence suggests that both motor and sensory functions are regulated by rhythmic processes reflecting alternating states of neuronal excitability, and these may be involved in mediating sensory-motor interactions. Here we show an oscillatory fluctuation in early visual processing time locked with the execution of voluntary action, and, crucially, even for visual stimuli irrelevant to the motor task. Human participants were asked to perform a reaching movement toward a display and judge the orientation of a Gabor patch, near contrast threshold, briefly presented at random times before and during the reaching movement. When the data are temporally aligned to the onset of movement, visual contrast sensitivity oscillates with periodicity within the theta band. Importantly, the oscillations emerge during the motor planning stage, ∼500 ms before movement onset. We suggest that brain oscillatory dynamics may mediate an automatic coupling between early motor planning and early visual processing, possibly instrumental in linking and closing up the visual-motor control loop.

Action preparation shapes processing in early visual cortex

Preparation for an action, such as grasping an object, is accompanied by an enhanced perception of the object's action-relevant features, such as orientation and size. Cortical feedback from motor planning areas to early visual areas may drive this enhanced perception. To examine whether action preparation modulates activity in early human visual cortex, subjects grasped or pointed to oriented objects while high-resolution fMRI data were acquired. Using multivoxel pattern analysis techniques, we could decode with >70% accuracy whether a grasping or pointing action was prepared from signals in visual cortex as early as V1. These signals in early visual cortex were observed even when actions were only prepared but not executed. Anterior parietal cortex, on the other hand, showed clearest modulation for actual movements. This demonstrates that preparation of actions, even without execution, modulates relevant neuronal populations in early visual areas.

Impact of action planning on spatial perception: attention matters

Previous research suggested that perception of spatial location is biased towards spatial goals of planned hand movements. In the present study I show that an analogous perceptual distortion can be observed if attention is paid to a spatial location in the absence of planning a hand movement. Participants judged the position of a target during preparation of a mouse movement, the end point of which could deviate from the target by a varying degree in Exp. 1. Judgments of target position were systematically affected by movement characteristics consistent with perceptual assimilation between the target and the planned movement goal. This effect was neither due to an impact of motor execution on judgments (Exp. 2) nor due to characteristics of the movement cues or of certain target positions (Exp. 3, Exp. 5A). When the task included deployment of attention to spatial positions (former movement goals) in preparation for a secondary perceptual task, an effect emerged that was comparable with the bias associated with movement planning (Exp. 4, Exp. 5B). These results indicate that visual distortions accompanying manipulations of variables related to action could be mediated by attentional mechanisms.

Hand placement near the visual stimulus improves orientation selectivity in V2 neurons

Often, the brain receives more sensory input than it can process simultaneously. Spatial attention helps overcome this limitation by preferentially processing input from a behaviorally-relevant location. Recent neuropsychological and psychophysical studies suggest that attention is deployed to near-hand space much like how the oculomotor system can deploy attention to an upcoming gaze position. Here we provide the first neuronal evidence that the presence of a nearby hand enhances orientation selectivity in early visual processing area V2. When the hand was placed outside the receptive field, responses to the preferred orientation were significantly enhanced without a corresponding significant increase at the orthogonal orientation. Consequently, there was also a significant sharpening of orientation tuning. In addition, the presence of the hand reduced neuronal response variability. These results indicate that attention is automatically deployed to the space around a hand, improving orientation selectivity. Importantly, this appears to be optimal for motor control of the hand, as opposed to oculomotor mechanisms which enhance responses without sharpening orientation selectivity. Effector-based mechanisms for visual enhancement thus support not only the spatiotemporal dissociation of gaze and reach, but also the optimization of vision for their separate requirements for guiding movements.

Weighing the evidence for a dorsal processing bias under continuous flash suppression

With the introduction of continuous flash suppression (CFS) as a method to render stimuli invisible and study unconscious visual processing, a novel hypothesis has gained popularity. It states that processes typically ascribed to the dorsal visual stream can escape CFS and remain functional, while ventral stream processes are suppressed when stimuli are invisible under CFS. This notion of a CFS-specific "dorsal processing bias" has been argued to be in line with core characteristics of the influential dual-stream hypothesis of visual processing which proposes a dissociation between dorsally mediated vision-for-action and ventrally mediated vision-for-perception. Here, we provide an overview of neuroimaging and behavioral studies that either examine this dorsal processing bias or base their conclusions on it. We show that both evidence for preserved ventral processing as well as lack of dorsal processing can be found in studies using CFS. To reconcile the diverging results, differences in the paradigms and their effects are worthy of future research. We conclude that given the current level of information a dorsal processing bias under CFS cannot be universally assumed.

Catching ease influences perceived speed: evidence for action-specific effects from action-based measures

According to the action-specific account of perception, people perceive the environment in terms of their ability to act. Here, we directly tested this claim by using an action-based measure of perceived speed: Participants attempted to catch a virtual fish by releasing a virtual net. The net varied in size, making the task easier or harder. We measured perceived speed by using explicit judgment-based measures and an action-based measure (time to release the net). Participants released the net later when playing with the big as compared with the small net, indicating that the fish looked to be moving more slowly when participants played with the big net. Explicit judgments of fish speed were similarly influenced by net size. These results provide converging evidence from both explicit and action-based measures that a perceiver's ability to act influences a common underlying process, most likely perceived speed, rather than postperceptual processes such as response formation.

Pointing treatments are task relevant: a visuomotor priming study

The present study focused on priming effects on pointing with everyday objects. In a set of four experiments, a visuomotor priming paradigm was used to investigate the nature of visuomotor processing (automatic versus task relevant). By manipulating congruency of orientation and location we found that location congruency facilitates the initiation time of pointing whereas orientation congruency does not. We provide evidence to show that motor planning is influenced by the goal of the action, and that how visual information is processed and held in memory depends on the task relevance. These data are consistent with the proposed interaction between visuomotor and higher processes during the planning and execution of actions such as pointing.

The Effect of Visual Degradation on Anticipatory and Compensatory Steering Control

It has long been held that steering a vehicle is subserved by two distinct visual processes, a compensatory one for maintaining lane position and an anticipatory one for previewing the curvature of the upcoming road. In this study, we investigated the robustness of these two steering control processes by systematically degrading their visual inputs. Performance was measured at the level of vehicle position and at the level of the actions on the steering wheel. The results show that the compensatory process is more robust to visual degradation than the anticipatory process. The results are also consistent with the idea that steering is under the supervision of a combination of compensatory and anticipatory mechanisms, although they suggest that the quality of the sensory information will determine how information is combined.

Two action systems in the human brain

The distinction between dorsal and ventral visual processing streams, first proposed by Ungerleider and Mishkin (1982) and later refined by Milner and Goodale (1995) has been elaborated substantially in recent years, spurred by two developments. The first was proposed in large part by Rizzolatti and Matelli (2003) and is a more detailed description of the multiple neural circuits connecting the frontal, temporal, and parietal cortices. Secondly, there are a number of behavioral observations that the classic "two visual systems" hypothesis is unable to accommodate without additional assumptions. The notion that the Dorsal stream is specialized for "where" or "how" actions and the Ventral stream for "What" knowledge cannot account for two prominent disorders of action, limb apraxia and optic ataxia, that represent a double dissociation in terms of the types of actions that are preserved and impaired. A growing body of evidence, instead, suggests that there are at least two distinct Dorsal routes in the human brain, referred to as the "Grasp" and "Use" systems. Both of these may be differentiated from the Ventral route in terms of neuroanatomic localization, representational specificity, and time course of information processing.

Impact of planned movement direction on judgments of visual locations

The present study examined if and how the direction of planned hand movements affects the perceived direction of visual stimuli. In three experiments participants prepared hand movements that deviated regarding direction ("Experiment 1" and "2") or distance relative to a visual target position ("Experiment 3"). Before actual execution of the movement, the direction of the visual stimulus had to be estimated by means of a method of adjustment. The perception of stimulus direction was biased away from planned movement direction, such that with leftward movements stimuli appeared somewhat more rightward than with rightward movements. Control conditions revealed that this effect was neither a mere response bias, nor a result of processing or memorizing movement cues. Also, shifting the focus of attention toward a cued location in space was not sufficient to induce the perceptual bias observed under conditions of movement preparation ("Experiment 4"). These results confirm that characteristics of planned actions bias visual perception, with the direction of bias (contrast or assimilation) possibly depending on the type of the representations (categorical or metric) involved.

TMS of the anterior intraparietal area selectively modulates orientation change detection during action preparation

Perception of relevant visual object features can be modulated by the preparation of an action toward it (“action-modulated perception”). For instance, the perception of the orientation of a book can be enhanced when preparing to grasp it (but not when pointing to it). However, the underlying neuronal mechanisms are poorly understood. We argue that brain areas controlling arm movements are involved in establishing this effect through top-down feedback to early visual areas, similar to the neuronal mechanisms linking visual attention and eye movements. To investigate this involvement, we applied transcranial magnetic stimulation to a grasping motor area, the left anterior intraparietal sulcus (aIPS), during grasping or pointing preparation. Concurrently, an orientation change detection task was performed. As a control area, the vertex was stimulated. We found that stimulation of aIPS selectively modulates orientation sensitivity during action preparation compared with control stimulation (vertex), negating the increased orientation sensitivity with grasping preparation over pointing preparation. We argue that aIPS is a critical part of the mechanism underlying perceptual modulations during action preparation. The present results and recent literature suggest that this action-modulated perception for hand movements is implemented through a cortical feedback connection between aIPS and early visual areas.

Response bias cannot explain action-specific effects: evidence from compliant and non-compliant participants

According to the action-specific account of perception, the perceiver's ability to act influences perception of the target. For example, targets that are easier to acquire are reported to look closer, bigger, and slower. However, an alternative explanation for these effects is that they are due to response bias, rather than to changes in perception. To test the role of response bias, we employed two separate manipulations. We manipulated people's abilities to block a ball and measured the corresponding effects on estimated ball speed. We also created an explicit task demand by giving participants instructions that emphasized responding either slow or fast. Participants were grouped, based on whether they were compliant or non-compliant with the instructions. Regardless of their compliance, we found an action-specific effect of blocking ability on estimated speed. Given that non-compliant participants still showed the effect, the results provide strong evidence against a response-bias explanation of this action-specific effect. Paired with earlier research, we conclude that blocking ability influences perceived speed. Perception expresses the relationship between the environment and the perceiver, and this view is consistent with emerging neural and behavioral evidence for an interconnected perceptual-motor system.

Moving further moves things further away in visual perception: position-based movement planning affects distance judgments

We examined how different characteristics of planned hand movements affect visual perception of distances in reachable space. Participants planned hand movements of certain amplitude. Before execution of the movement, certain visual distances had to be judged. Distances were judged as larger the larger the amplitude of the concurrently prepared hand movements was. On top of that, with constant movement amplitude, distances were judged as larger, the further away the start point of the planned movement was located from the body. These results indicate that distinct variables specified during motor planning, such as effector's final position, are linked to the visual perception of environmental characteristics.

Incidental and context-responsive activation of structure- and function-based action features during object identification

Previous studies suggest that action representations are activated during object processing, even when task-irrelevant. In addition, there is evidence that lexical-semantic context may affect such activation during object processing. Finally, prior work from our laboratory and others indicates that function-based ("use") and structure-based ("move") action subtypes may differ in their activation characteristics. Most studies assessing such effects, however, have required manual object-relevant motor responses, thereby plausibly influencing the activation of action representations. The present work uses eyetracking and a Visual World Paradigm task without object-relevant actions to assess the time course of activation of action representations, as well as their responsiveness to lexical-semantic context. In two experiments, participants heard a target word and selected its referent from an array of four objects. Gaze fixations on nontarget objects signal activation of features shared between targets and nontargets. The experiments assessed activation of structure-based (Experiment 1) or function-based (Experiment 2) distractors, using neutral sentences ("S/he saw the....") or sentences with a relevant action verb (Experiment 1: "S/he picked up the...."; Experiment 2: "S/he used the...."). We observed task-irrelevant activations of action information in both experiments. In neutral contexts, structure-based activation was relatively faster-rising but more transient than function-based activation. Additionally, action verb contexts reliably modified patterns of activation in both Experiments. These data provide fine-grained information about the dynamics of activation of function-based and structure-based actions in neutral and action-relevant contexts, in support of the "Two Action System" model of object and action processing (e.g., Buxbaum & Kalénine, 2010).

Action-Specific Effects Underwater

Action-specific effects on perception are apparent in terrestrial environments. For example, targets that require more effort to walk, jump, or throw to look farther away than when the targets require less effort. Here, we examined whether action-specific effects would generalize to an underwater environment. Instead, perception might be geometrically precise, rather than action-specific, in an environment that is novel from an evolutionary perspective. We manipulated ease to swim by giving participants swimming flippers or taking them away. Those who estimated distance while wearing the flippers judged underwater targets to be closer than did participants who had taken them off. In addition, participants with better swimming ability judged the targets to be closer than did those with worse swimming ability. These results suggest perceived distance underwater is a function of the perceiver's ability to swim to the targets.