Deployment of visual attention before sequences of goal-directed hand movements

Simple actions modulate context-dependent visual size perception at late processing stages

A simple button press towards a prime stimulus enhances subsequent visual search for objects that match the prime. The present study investigated whether this action effect is a general phenomenon across different task domains, and the underlying neural mechanisms. The action effect was measured in an unspeeded size-matching task, with the presentation of the central target and the surrounding inducers of the Ebbinghaus illusion together to one eye or separately to each eye, and when repetitive TMS was applied over right primary motor cortex (M1). The results showed that a prior key-press significantly reduced the illusion effect compared to passive viewing. Notably, the action effect persisted with dichoptic presentation of the Ebbinghaus configuration, but disappeared with the right M1 disruption. These results suggest that action guides visual perception to influence human behavior, which mainly affects the late visual processing stage and probably relies on feedback projections from the motor cortex.

Minimal impact of chronic proprioceptive loss on implicit sensorimotor adaptation and perceived movement outcome

Implicit sensorimotor adaptation keeps our movements well calibrated amid changes in the body and environment. We have recently postulated that implicit adaptation is driven by a perceptual error: the difference between the desired and perceived movement outcome. According to this perceptual realignment model, implicit adaptation ceases when the perceived movement outcome-a multimodal percept determined by a prior belief conveying the intended action, the motor command, and feedback from proprioception and vision-is aligned with the desired movement outcome. Here, we examined the role of proprioception in implicit motor adaptation and perceived movement outcome by examining individuals who experience deafferentation (i.e., individuals with impaired proprioception and touch). We used a modified visuomotor rotation task designed to isolate implicit adaptation and probe perceived movement outcomes throughout the experiment. Surprisingly, both implicit adaptation and perceived movement outcome were minimally impacted by chronic deafferentation, posing a challenge to the perceptual realignment model of implicit adaptation.NEW & NOTEWORTHY We tested six individuals with chronic somatosensory deafferentation on a novel task that isolates implicit sensorimotor adaptation and probes perceived movement outcome. Strikingly, both implicit motor adaptation and perceptual movement outcome were not significantly impacted by chronic deafferentation, posing a challenge for theoretical models of adaptation that involve proprioception.

Sensorimotor prediction is used to direct gaze toward task-relevant locations in a goal-directed throwing task

Previous research has shown that action effects of self-generated movements are internally predicted before outcome feedback becomes available. To test whether these sensorimotor predictions are used to facilitate visual information uptake for feedback processing, we measured eye movements during the execution of a goal-directed throwing task. Participants could fully observe the effects of their throwing actions (ball trajectory and either hitting or missing a target) in most of the trials. In a portion of the trials, the ball trajectory was not visible, and participants only received static information about the outcome. We observed a large proportion of predictive saccades, shifting gaze toward the goal region before the ball arrived and outcome feedback became available. Fixation locations after predictive saccades systematically covaried with future ball positions in trials with continuous ball flight information, but notably also in trials with static outcome feedback and only efferent and proprioceptive information about the movement that could be used for predictions. Fixation durations at the chosen positions after feedback onset were modulated by action outcome (longer durations for misses than for hits) and outcome uncertainty (longer durations for narrow vs. clear outcomes). Combining both effects, durations were longest for narrow errors and shortest for clear hits, indicating that the chosen locations offer informational value for feedback processing. Thus, humans are able to use sensorimotor predictions to direct their gaze toward task-relevant feedback locations. Outcome-dependent saccade latency differences (miss vs. hit) indicate that also predictive valuation processes are involved in planning predictive saccades.NEW & NOTEWORTHY We elucidate the potential benefits of sensorimotor predictions, focusing on how the system actually uses this information to optimize feedback processing in goal-directed actions. Sensorimotor information is used to predict spatial parameters of movement outcomes, guiding predictive saccades toward future action effects. Saccade latencies and fixation durations are modulated by outcome quality, indicating that predictive valuation processes are considered and that the locations chosen are of high informational value for feedback processing.

Computation on Demand: Action-Specific Representations of Visual Task Features Arise during Distinct Movement Phases

The intention to act influences the computations of various task-relevant features. However, little is known about the time course of these computations. Furthermore, it is commonly held that these computations are governed by conjunctive neural representations of the features. But, support for this view comes from paradigms arbitrarily combining task features and affordances, thus requiring representations in working memory. Therefore, the present study used electroencephalography and a well-rehearsed task with features that afford minimal working memory representations to investigate the temporal evolution of feature representations and their potential integration in the brain. Female and male human participants grasped objects or touched them with a knuckle. Objects had different shapes and were made of heavy or light materials with shape and weight being relevant for grasping, not for "knuckling." Using multivariate analysis showed that representations of object shape were similar for grasping and knuckling. However, only for grasping did early shape representations reactivate at later phases of grasp planning, suggesting that sensorimotor control signals feed back to the early visual cortex. Grasp-specific representations of material/weight only arose during grasp execution after object contact during the load phase. A trend for integrated representations of shape and material also became grasp-specific but only briefly during the movement onset. These results suggest that the brain generates action-specific representations of relevant features as required for the different subcomponents of its action computations. Our results argue against the view that goal-directed actions inevitably join all features of a task into a sustained and unified neural representation.

Influence of hand starting position on radial line bisection

When normal individuals are asked to localize and mark the midpoint of a radial line, they tend to bisect it farther than the true center. It has been suggested that radial misbisection depends on the presence of a visual attentional bias directed toward the far space. The aim of the present study was to investigate whether the localization of the center of radial lines was affected by the starting position of the hand. There were two starting positions: one between the body and the radial line ("near"), the other beyond the radial line ("far"). Thirty-four subjects participated in the experiment. The results showed that (i) participants bisected radial lines farther than the true center, measured with reference to their body, in both near and far condition, and (ii) bisection errors in the near condition were greater than those in the far condition. We suggest that hand starting position and direction of ongoing movement influenced radial line misbisection by modulating visual attentional bias directed to far space.

Visual working memory representations bias attention more when they are the target of an action plan

Attention has frequently been regarded as an emergent property of linking sensory representations to action plans. It has recently been proposed that similar mechanisms may operate within visual working memory (VWM), such that linking an object in VWM to an action plan strengthens its sensory memory representation, which then expresses as an attentional bias. Here we directly tested this hypothesis by comparing attentional biases induced by VWM representations which were the target of a future action, to those induced by VWM representations that were equally task-relevant, but not the direct target of action. We predicted that the first condition would result in a more prioritized memory state and hence stronger attentional biases. Specifically, participants memorized a geometric shape for a subsequent memory test. At test, in case of a match, participants either had to perform a grip movement on the matching object (action condition), or perform the same movement, but on an unrelated object (control condition). To assess any attentional biases, during the delay period between memorandum and test, participants performed a visual selection task in which either the target was surrounded by the memorized shape (congruent trials) or a distractor (incongruent trials). Eye movements were measured as a proxy for attentional priority. We found a significant interaction for saccade latencies between action condition and shape congruency, reflecting more pronounced VWM-based attentional biases in the action condition. Our results are consistent with the idea that action plans prioritize sensory representations in VWM.

Probing the deployment of peripheral visual attention during obstacle-crossing planning

Gaze is directed to one location at a time, making peripheral visual input important for planning how to negotiate different terrain during walking. Whether and how the brain attends to this input is unclear. We developed a novel paradigm to probe the deployment of sustained covert visual attention by testing orientation discrimination of a Gabor patch at stepping and non-stepping locations during obstacle-crossing planning. Compared to remaining stationary, obstacle-crossing planning decreased visual performance (percent correct) and sensitivity (d') at only the first of two stepping locations. Given the timing of the first and second steps before obstacle crossing relative to the Gabor patch presentation, the results suggest the brain uses peripheral vision to plan one step at a time during obstacle crossing, in contrast to how it uses central vision to plan two or more steps in advance. We propose that this protocol, along with multiple possible variations, presents a novel behavioral approach to identify the role of covert visual attention during obstacle-crossing planning and other goal-directed walking tasks.

Saccadic landing positions reveal that eye movements are affected by distractor-based retrieval

Binding theories assume that stimulus and response features are integrated into short-lasting episodes and that upon repetition of any feature the whole episode is retrieved, thereby affecting performance. Such binding theories are nowadays the standard explanation for a wide range of action control tasks and aim to explain all simple actions, without making assumptions of effector specificity. Yet, it is unclear if eye movements are affected by integration and retrieval in the same way as manual responses. We asked participants to discriminate letters framed by irrelevant shapes. In Experiment 1, participants gave their responses with eye movements. Saccade landing positions showed a spatial error pattern consistent with predictions of binding theories. Saccadic latencies were not affected. In Experiment 2 with an increased interval between prime and probe, the error pattern diminished, again congruent with predictions of binding theories presuming quickly decaying retrieval effects. Experiment 3 used the same task as in Experiment 1, but participants executed their responses with manual key presses; again, we found a binding pattern in response accuracy. We conclude that eye movements and manual responses are affected by the same integration and retrieval processes, supporting the tacit assumption of binding theories to apply to any effector.

Motor engagement enhances incidental memory for task-irrelevant items

Actions shape what we see and memorize. A previous study suggested the interaction between motor and memory systems by showing that memory encoding for task-irrelevant items was enhanced when presented with motor-response cues. However, in the studies on the attentional boost effect, it has been revealed that detection of the target stimulus can lead to memory enhancement without requiring overt action. Thus, the direct link between the action and memory remains unclear. To exclude the effect of the target detection process as a potential confounder, this study assessed the benefit of action for memory by separating items from the response cue in time. In our pre-registered online experiment (N = 142), participants responded to visual Go cues by pressing a key (i.e., motor task) or counting (i.e., motor-neutral cognitive task) while ignoring No-go cues. In each trial, two task-irrelevant images were sequentially presented after the cue disappearance. After encoding the Go/No-go tasks, participants performed a surprise recognition memory test for those images. Importantly, we quantified the impact of overt execution of the action by comparing memories with and without motor response and the impact of covert motor processes (e.g., preparation and planning of action) by comparing memory between the motor and cognitive tasks. The results showed no memory differences between Go and No-go trials in the motor task. This means that the execution itself was not critical for memory enhancement. However, the memory performance in the motor No-go trials was higher than that in the cognitive No-go trials, only for the items presented away from the cues in time. Therefore, engaging the motor task itself could increase incidental memory for the task-irrelevant items compared to a passive viewing situation. We added empirical evidence on the online interaction between action and memory encoding. These memory advantages could be especially brought in action preparation and planning. We believe this fact may expand our present understanding of everyday memory, such as active learning.

Tactile motor attention induces sensory attenuation for sounds

Sensory events appear reduced in intensity when we actively produce them. Here, we investigated sensory attenuation in a virtual reality setup that allowed us to manipulate the time of tactile feedback when pressing a virtual button. We asked whether tactile motor attention might shift to the tactile location that makes contact with the button. In experiment one, we found that a tactile impulse was perceived as more intense when button pressing. In a second experiment, participants pushed a button and estimated the intensity of sounds. We found sensory attenuation for sounds only when tactile feedback was provided at the time the movement goal was reached. These data indicate that attentional prioritization for the tactile modality during a goal-directed hand movement might lead to a transient reduction in sensitivity in other modalities, resulting in sensory attenuation for sounds.

Catch the star! Spatial information activates the manual motor system

Previous research demonstrated a close bidirectional relationship between spatial attention and the manual motor system. However, it is unclear whether an explicit hand movement is necessary for this relationship to appear. A novel method with high temporal resolution–bimanual grip force registration–sheds light on this issue. Participants held two grip force sensors while being presented with lateralized stimuli (exogenous attentional shifts, Experiment 1), left- or right-pointing central arrows (endogenous attentional shifts, Experiment 2), or the words "left" or "right" (endogenous attentional shifts, Experiment 3). There was an early interaction between the presentation side or arrow direction and grip force: lateralized objects and central arrows led to a larger increase of the ipsilateral force and a smaller increase of the contralateral force. Surprisingly, words led to the opposite pattern: larger force increase in the contralateral hand and smaller force increase in the ipsilateral hand. The effect was stronger and appeared earlier for lateralized objects (60 ms after stimulus presentation) than for arrows (100 ms) or words (250 ms). Thus, processing visuospatial information automatically activates the manual motor system, but the timing and direction of this effect vary depending on the type of stimulus.

Advances in the application of a computational Theory of Visual Attention (TVA): Moving towards more naturalistic stimuli and game-like tasks

The theory of visual attention, “TVA”, is an influential and formal theory of attentional selection. It is widely applied in clinical assessment of attention and fundamental attention research. However, most TVA-based research is based on accuracy data from letter report experiments performed in controlled laboratory environments. While such basic approaches to questions regarding attentional selection are undoubtedly useful, recent technological advances have enabled the use of increasingly sophisticated experimental paradigms involving more realistic scenarios. Notably, these studies have in many cases resulted in different estimates of capacity limits than those found in studies using traditional TVA-based assessment. Here we review recent developments in TVA-based assessment of attention that goes beyond the use of letter report experiments and experiments performed in controlled laboratory environments. We show that TVA can be used with other tasks and new stimuli, that TVA-based parameter estimation can be embedded into complex scenarios, such as games that can be used to investigate particular problems regarding visual attention, and how TVA-based simulations of “visual foraging” can elucidate attentional control in more naturalistic tasks. We also discuss how these developments may inform future advances of TVA.

Visual attention around a hand location localized by proprioceptive information

Facilitation of visual processing has been reported in the space near the hand. To understand the underlying mechanism of hand proximity attention, we conducted experiments that isolated hand-related effects from top–down attention, proprioceptive information from visual information, the position effect from the influence of action, and the distance effect from the peripersonal effect. The flash-lag effect was used as an index of attentional modulation. Because the results showed that the flash-lag effect was smaller at locations near the hand, we concluded that there was a facilitation effect of the visual stimuli around the hand location identified through proprioceptive information. This was confirmed by conventional reaction time measures. We also measured steady-state visual evoked potential (SSVEP) in order to investigate the spatial properties of hand proximity attention and top–down attention. The results showed that SSVEP reflects the effect of top–down attention but not that of hand proximity attention. This suggests that the site of hand proximity attention is at a later stage of visual processing, assuming that SSVEP responds to neural activities at the early stages. The results of left-handers differed from those of right-handers, and this is discussed in relation to handedness variation.

Eye and hand movements disrupt attentional control

Voluntary attentional control is the ability to selectively focus on a subset of visual information in the presence of other competing stimuli–a marker of cognitive control enabling flexible, goal-driven behavior. To test its robustness, we contrasted attentional control with the most common source of attentional orienting in daily life: attention shifts prior to goal-directed eye and hand movements. In a multi-tasking paradigm, human participants attended at a location while planning eye or hand movements elsewhere. Voluntary attentional control suffered with every simultaneous action plan, even under reduced task difficulty and memory load–factors known to interfere with attentional control. Furthermore, the performance cost was limited to voluntary attention: We observed simultaneous attention benefits at two movement targets without attentional competition between them. This demonstrates that the visual system allows for the concurrent representation of multiple attentional foci. Since attentional control is extremely fragile and dominated by premotor attention shifts, we propose that action-driven selection plays the superordinate role for visual selection.

Misdirected attentional focus in functional tremor

A characteristic and intriguing feature of functional neurological disorder is that symptoms typically manifest with attention and improve or disappear with distraction. Attentional phenomena are therefore likely to be important in functional neurological disorder, but exactly how this manifests is unknown. The aim of the study was to establish whether in functional tremor the attentional focus is misdirected, and if this misdirection is detrimental to the movement, or rather reflects a beneficial compensatory strategy. Patients with a functional action tremor, between the ages of 21-75, were compared to two age and gender matched control groups: healthy controls and patients with an organic action tremor. The groups included between 17 and 28 participants. First, we compared the natural attentional focus on different aspects of a reaching movement (target, ongoing visual feedback, proprioceptive-motor aspect). This revealed that the attentional focus in the functional tremor group, in contrast to both control groups, was directed to ongoing visual feedback from the movement. Next, we established that all groups were able to shift their attentional focus to different aspects of the reaching movement when instructed. Subsequently, the impact of attentional focus on the ongoing visual feedback on movement performance was evaluated under several conditions: the reaching movement was performed with direct, or indirect visual feedback, without any visual feedback, under three different instruction conditions (as accurately as possible/very slowly/very quickly), and finally as a preparatory movement that was supposedly of no importance. Low trajectory length and low movement duration were taken as measures of good motor performance. For all three groups, motor performance deteriorated with attention to indirect visual feedback, to accuracy, and when instructed to move slowly. It improved without visual feedback and when instructed to move fast. Motor performance improved, in participants with functional tremor only, when the movement was performed as a preparatory movement without any apparent importance. In addition to providing experimental evidence for improvement with distraction, we found that the normal allocation of attention during aimed movement is altered in functional tremor. Attention is disproportionately directed towards the ongoing visual feedback from the moving hand. This altered attentional focus may be partly responsible for the tremor, since it also worsens motor performance in healthy controls and patients with an organic action tremor. It may have its detrimental impact through interference with automatic movement processes, due to a maladaptive shift from lower- to higher-level motor control circuitry.

The effect of motor engagement on memory: Testing a motor-induced encoding account

The motor system is traditionally thought to reflect the output of cognition. However, the inverse relationship of how the motor system impacts cognitive processes is less known. Work on this interaction has demonstrated that recognition memory for stimuli presented in combination with the inhibition of a prepared action is weaker compared to stimuli associated with the execution of an action (Chiu & Egner, Psychological Science, 26, 27-38, 2015a). This effect has been explained through competition for common neural resources: to the extent that response inhibition processes are recruited, fewer resources are available for memory encoding (Chiu & Egner, Journal of Neuroscience, 35, 11936-11945, 2015b). Alternatively, it has been proposed that action execution enhances memory encoding (Yebra et al., Nature Communications, 10(1), 1-12, 2019). In this report, we examined how recognition memory for stimuli paired with both the preparation and execution of a motor response compare to stimuli absent of any motor processes. We first replicated Chiu and Egner (2015a, 2015b). Next, we added a motor-neutral condition as a baseline comparison. Across three experiments, recognition memory for stimuli associated with action execution was superior to stimuli absent of motor demands. More importantly, we found that recognition memory for stimuli associated with motor preparation, but no subsequent execution, was also superior to stimuli that did not engage the motor system (Experiments 2a and 2b). These results support a motor-induced encoding effect, in which the degree of motor processing (both action preparation and action execution) enhanced memory encoding.

Attention attracts action in healthy participants: An insight into optic ataxia?

Patients with optic ataxia following lesions to superior parts of the posterior parietal cortex make large errors when reaching to targets in the peripheral visual field. These errors are characterised by a contraction, or attraction, towards the point of fixation. These patients also have a reduced ability to allocate visual attention away from the point of fixation, but it is unclear whether the core symptom of misreaching is related to these attentional problems. In neurologically-intact adults, we tested the effect of an attention-demanding dual-task performed at fixation upon visually-guided reaching to peripheral targets. The dual task was associated with delayed movement initiation, and a shortened deceleration phase of movement suggesting a reduced ability to benefit from online control. It also induced a small but consistent shift of reaching endpoints towards the side of fixation. Our experimental restriction of visual attention thus impaired both the programming and control of reaching, and induced a spatial pattern of errors that was qualitatively reminiscent of optic ataxia, albeit much less severe. These findings are consistent with a close functional link between attention and action in the healthy brain, and suggest that attentional disturbances could be a core component of optic ataxia following parietal lesions.

Selective attention to real-world objects drives their emotional appraisal

Attentional manipulations have been shown to influence subsequent evaluations of objects and images. For example, images used as distractors in a visual search task are subsequently rated more negatively than are target images. One powerful manipulation of attention occurs when we plan and execute movements toward objects in our environment. Here, in two experiments, we show that selective attention to real-world objects subsequently improves emotional appraisal of those objects-an effect we term "target appreciation." Participants were presented with abstract images on three-dimensional objects, and were cued to either reach and grasp one of the two objects, or to respond to the cued object with a keyboard. Images presented on target objects were appraised more positively when compared with novel images. In contrast, images associated with obstacles or distractor objects were not appraised differently than novel images, despite the attentional suppression thought to be required to successfully avoid or ignore these objects. We speculate that this automatic appreciation of the objects of selective attention may be adaptive for organisms acting in complex environments.

Selective Modulation of Early Visual Cortical Activity by Movement Intention

The primate visual system contains myriad feedback projections from higher- to lower-order cortical areas, an architecture that has been implicated in the top-down modulation of early visual areas during working memory and attention. Here we tested the hypothesis that these feedback projections also modulate early visual cortical activity during the planning of visually guided actions. We show, across three separate human functional magnetic resonance imaging (fMRI) studies involving object-directed movements, that information related to the motor effector to be used (i.e., limb, eye) and action goal to be performed (i.e., grasp, reach) can be selectively decoded-prior to movement-from the retinotopic representation of the target object(s) in early visual cortex. We also find that during the planning of sequential actions involving objects in two different spatial locations, that motor-related information can be decoded from both locations in retinotopic cortex. Together, these findings indicate that movement planning selectively modulates early visual cortical activity patterns in an effector-specific, target-centric, and task-dependent manner. These findings offer a neural account of how motor-relevant target features are enhanced during action planning and suggest a possible role for early visual cortex in instituting a sensorimotor estimate of the visual consequences of movement.

Preparation of saccade sequences and eye programming affect endogenous covert attention

Endogenous attention can be allocated in parallel to at least two saccade target locations of a planned sequence, but attentional resources are larger in the location of the first than the second saccade. The meridian effect that is observed in endogenous attention can be explained by eye programming, but it is not known how eye-movement preparation and eye programming can together affect endogenous attention during sequential saccades. We used a double-task paradigm to investigate this issue. In two experiments, we confirmed the relation between the preparation of sequential saccades and attentional selection and also showed that eye programming could eliminate deterioration of attentional resources in the second saccade location. The finding of the meridian effect in both the saccadic task and the target discrimination task additionally indicates the important role of eye programming in endogenous attention. The results were discussed in terms of the Premotor Theory of Attention.

Shared attention for action selection and action monitoring in goal-directed reaching

Dual-task studies have shown higher sensitivity for stimuli presented at the targets of upcoming actions. We examined whether attention is directed to action targets for the purpose of action selection, or if attention is directed to these locations because they are expected to provide feedback about movement outcomes. In our experiment, endpoint accuracy feedback was spatially separated from the action targets to determine whether attention would be allocated to (a) the action targets, (b) the expected source of feedback, or (c) to both locations. Participants reached towards a location indicated by an arrow while identifying a discrimination target that could appear in any one of eight possible locations. Discrimination target accuracy was used as a measure of attention allocation. Participants were unable to see their hand during reaching and were provided with a small monetary reward for each accurate movement. Discrimination target accuracy was best at action targets but was also enhanced at the spatially separated feedback locations. Separating feedback from the reaching targets did not diminish discrimination accuracy at the movement targets but did result in delayed movement initiation and reduced reaching accuracy, relative to when feedback was presented at the reaching target. The results suggest attention is required for both action planning and monitoring movement outcomes. Dividing attention between these functions negatively impacts action performance.

The spatial and temporal properties of attentional selectivity for saccades and reaches

The preparation and execution of saccades and goal-directed movements elicits an accompanying shift in attention at the locus of the impending movement. However, some key aspects of the spatiotemporal profile of this attentional shift between eye and hand movements are not resolved. While there is evidence that attention is improved at the target location when making a reach, it is not clear how attention shifts over space and time around the movement target as a saccade and a reach are made to that target. Determining this spread of attention is an important aspect in understanding how attentional resources are used in relation to movement planning and guidance in real world tasks. We compared performance on a perceptual discrimination paradigm during a saccade-alone task, reach-alone task, and a saccade-plus-reach task to map the temporal profile of the premotor attentional shift at the goal of the movement and at three surrounding locations. We measured performance relative to a valid baseline level to determine whether motor planning induces additional attentional facilitation compared to mere covert attention. Sensitivity increased relative to movement onset at the target and at the surrounding locations, for both the saccade-alone and saccade-plus-reach conditions. The results suggest that the temporal profile of the attentional shift is similar for the two tasks involving saccades (saccade-alone and saccade-plus-reach tasks), but is very different when the influence of the saccade is removed. In this case, performance in the saccade-plus-reach task reflects the lower sensitivity observed when a reach-alone task is being conducted. In addition, the spatial profile of this spread of attention is not symmetrical around the target. This suggests that when a saccade and reach are being planned together, the saccade drives the attentional shift, and the reach-alone carries little attentional weight.

Spatial attention enhances cortical tracking of quasi-rhythmic visual stimuli

Successfully interpreting and navigating our natural visual environment requires us to track its dynamics constantly. Additionally, we focus our attention on behaviorally relevant stimuli to enhance their neural processing. Little is known, however, about how sustained attention affects the ongoing tracking of stimuli with rich natural temporal dynamics. Here, we used MRI-informed source reconstructions of magnetoencephalography (MEG) data to map to what extent various cortical areas track concurrent continuous quasi-rhythmic visual stimulation. Further, we tested how top-down visuo-spatial attention influences this tracking process. Our bilaterally presented quasi-rhythmic stimuli covered a dynamic range of 4-20 ​Hz, subdivided into three distinct bands. As an experimental control, we also included strictly rhythmic stimulation (10 vs 12 ​Hz). Using a spectral measure of brain-stimulus coupling, we were able to track the neural processing of left vs. right stimuli independently, even while fluctuating within the same frequency range. The fidelity of neural tracking depended on the stimulation frequencies, decreasing for higher frequency bands. Both attended and non-attended stimuli were tracked beyond early visual cortices, in ventral and dorsal streams depending on the stimulus frequency. In general, tracking improved with the deployment of visuo-spatial attention to the stimulus location. Our results provide new insights into how human visual cortices process concurrent dynamic stimuli and provide a potential mechanism - namely increasing the temporal precision of tracking - for boosting the neural representation of attended input.

No one knows what attention is

In this article, we challenge the usefulness of "attention" as a unitary construct and/or neural system. We point out that the concept has too many meanings to justify a single term, and that "attention" is used to refer to both the explanandum (the set of phenomena in need of explanation) and the explanans (the set of processes doing the explaining). To illustrate these points, we focus our discussion on visual selective attention. It is argued that selectivity in processing has emerged through evolution as a design feature of a complex multi-channel sensorimotor system, which generates selective phenomena of "attention" as one of many by-products. Instead of the traditional analytic approach to attention, we suggest a synthetic approach that starts with well-understood mechanisms that do not need to be dedicated to attention, and yet account for the selectivity phenomena under investigation. We conclude that what would serve scientific progress best would be to drop the term "attention" as a label for a specific functional or neural system and instead focus on behaviorally relevant selection processes and the many systems that implement them.

Beyond one's body parts: Remote object movement with sense of agency involuntarily biases spatial attention

Humans' own body postures and/or actions bias visual attention. This study examined whether an object controlled remotely with a sense of agency (SoA) can bias attentional allocation, even if the mappings of body movement and object movement do not correspond (i.e., with a less extended body representation). In the experiments, participants shifted a circle toward an instructed direction via keypress. On some trials, the task changed to identification of visual characters instead of the circle shift. Strength of SoA was manipulated based on the probability that the circle actually moved to the participant's intended location, with high probability producing a strong SoA. Results showed that visual identification performance became higher for appearance of the target at the intended location than at the unintended location, but only when observers felt a strong SoA. This suggests that the object's movement attracts attention when the observer feels a strong SoA over its control.

Is the premotor theory of attention essentially about pre-reflective intentionality?

The Premotor Theory of Attention (PToA) is a prominent, albeit controversial, modern experimental account of attentional processes. According to the PToA, motor preparation is both necessary and sufficient for spatial attention. Explaining the cognitive process of attention in terms of sensori-motor machinery can be considered as embedded in the idea of embodied cognition. The vocabulary adopted by the PToA seems to bear a particular resemblance to Merleau-Ponty’s phenomenological notion of pre-reflective intentionality. He articulates it by means of directness towards the lived world, which is constituted in the spatial motility of the body-subject. In this epistemological state of affairs, we come up with two leading questions: (a) can the main tenets of PToA be essentially reconstructed in terms of the notion of pre-reflective intentionality and since the bodily motility is meant by the French phenomenologist to be at the root of all forms of intentionality, (b) can the PToA be expanded to account for all kinds of attention? In conclusion, we advocate a positive answer for the former question and point to serious doubts as to why it can rather not be retained regarding the latter.

Visual attention and action: How cueing, direct mapping, and social interactions drive orienting

Despite considerable interest in both action perception and social attention over the last 2 decades, there has been surprisingly little investigation concerning how the manual actions of other humans orient visual attention. The present review draws together studies that have measured the orienting of attention, following observation of another's goal-directed action. Our review proposes that, in line with the literature on eye gaze, action is a particularly strong orienting cue for the visual system. However, we additionally suggest that action may orient visual attention using mechanisms, which gaze direction does not (i.e., neural direct mapping and corepresentation). Finally, we review the implications of these gaze-independent mechanisms for the study of attention to action. We suggest that our understanding of attention to action may benefit from being studied in the context of joint action paradigms, where the role of higher level action goals and social factors can be investigated.

Visual Selection of the Future Reach Path in Obstacle Avoidance

In two EEG experiments, we studied the role of visual attention during the preparation of manual movements around an obstacle. Participants performed rapid hand movements to a goal position avoiding a central obstacle either on the left or right side, depending on the pitch of the acoustical go signal. We used a dot probe paradigm to analyze the deployment of spatial attention in the visual field during the motor preparation. Briefly after the go signal but still before the hand movement actually started, a visual transient was flashed either on the planned pathway of the hand (congruent trials) or on the opposite, movement-irrelevant side (incongruent trials). The P1/N1 components that were evoked by the onset of the dot probe were enhanced in congruent trials where the visual transient was presented on the planned path of the hand. The results indicate that, during movement preparation, attention is allocated selectively to the planned trajectory the hand is going to take around the obstacle.

The profile of attention differs between locations orthogonal to and in line with reach direction

People make movements in a variety of directions when interacting with the world around them. It has been well documented that attention shifts to the goal of an upcoming movement, whether the movement is a saccade or a reach. However, recent evidence suggests that the direction of a movement may influence the spatial spread of attention (Stewart & Ma-Wyatt, 2015, Journal of Vision, 15(5), 10). We investigated whether the spatiotemporal profile of attention differs depending on where that location is situated relative to the direction of movement, and if this pattern is consistent across different movement effectors. We compared attentional facilitation at locations in line with or orthogonal to the movement, for reach-only, reach-plus-saccade, and saccade-only conditions. Results show that the spatiotemporal profile of attention differs across different movement combinations, and is also different at target locations orthogonal to and in line with the movement direction. Specifically, when a reach alone was made, there was a general decrease in attention at all locations during the movement and a general increase in attention at all locations with a saccade only. However, the concurrent reach and saccade condition showed a premovement attentional facilitation at locations orthogonal to movement direction, but not those in line with the movement direction. These results suggest attentional guidance may be more important at differing time points, depending on the type of movement.

Effector-based attention systems

Visual processing is known to be enhanced at the end point of eye movements. Feedback within the oculomotor system has been shown to drive these alterations in visual processing. However, we do not simply view the world; we also reach out and interact using our hands. Consequently, it is not surprising that visual processing has also been shown to be altered in near-hand space. A growing body of work documents a myriad of alterations in near-hand visual processing, with little consensus on the neural underpinnings of the effect of the hand. Since movement of the eyes and hands is governed by parallel frontoparietal networks and since within the oculomotor system feedback from these motor control regions has been shown to drive enhanced visual processing at saccade end points, it is plausible that a similar feedback mechanism is at play in near-hand improvements in visual processing. Here, we compare and contrast oculomotor-driven and hand-driven changes in visual processing and provide support for the hypothesis that feedback within the reaching and grasping systems enhances visual processing near the hand in a novel way.

Grasping remaps the distribution of visuospatial attention and enhances competing action activation

We examined how action goals influence the distribution of visuospatial attention near the body (Experiment 1) and how the temporal relationship between distractors and targets modifies shifts in visuospatial attention (Experiment 2). Targets were light emitting diodes (LEDs) in the left and right hemispace of a visual display. Following left or right target illumination, participants reached to point-to or grasp target object in blocked trials. Coincident with target onset, a distractor LED illuminated in the same or opposite hemispace between the initiation point and target, or no distractor appeared. In Experiment 1, during grasping there was a larger temporal interference effect (slower reach initiation) than with pointing. When grasping versus pointing, participants deviated more towards same-side distractors and away from opposite-side distractors. In Experiment 2, distractors onset 200ms prior to (-200-ms), coincident with (0 ms), or 200ms following (+200 ms) the target. For both reach types, -200-ms distractors had greater onset temporal interference than 0 ms and +200-ms distractors. For grasping, +200 ms distractors had larger temporal interference than 0 ms distractors. For -200-ms, reach trajectories deviated more towards opposite-side distractors and away from same-side distractors, the reverse of the pattern for 0 ms and +200-ms distractors.

Perceived visual time depends on motor preparation and direction of hand movements

Perceived time undergoes distortions when we prepare and perform movements, showing compression and/or expansion for visual, tactile and auditory stimuli. However, the actual motor system contribution to these time distortions is far from clear. In this study we investigated visual time perception during preparation of isometric contractions and real movements of the hand in two different directions (right/left). Comparable modulations of visual event-timing are found in the isometric and in the movement condition, excluding explanations based on movement-induced sensory masking or attenuation. Most importantly, and surprisingly, visual time depends on the movement direction, being expanded for hand movements pointing away from the body and compressed in the other direction. Furthermore, the effect of movement direction is not constant, but rather undergoes non-monotonic modulations in the brief moments preceding movement initiation. Our findings indicate that time distortions are strongly linked to the motor system, and they may be unavoidable consequences of the mechanisms subserving sensory-motor integration.

An Eye in the Palm of Your Hand: Alterations in Visual Processing Near the Hand, a Mini-Review

Feedback within the oculomotor system improves visual processing at eye movement end points, also termed a visual grasp. We do not just view the world around us however, we also reach out and grab things with our hands. A growing body of literature suggests that visual processing in near-hand space is altered. The control systems for moving either the eyes or the hands rely on parallel networks of fronto-parietal regions, which have feedback connections to visual areas. Since the oculomotor system effects on visual processing occur through feedback, both through the motor plan and the motor efference copy, a parallel system where reaching and/or grasping motor-related activity also affects visual processing is likely. Areas in the posterior parietal cortex, for example, receive proprioceptive and visual information used to guide actions, as well as motor efference signals. This trio of information channels is all that would be necessary to produce spatial allocation of reach-related visual attention. We review evidence from behavioral and neurophysiological studies that support the hypothesis that feedback from the reaching and/or grasping motor control networks affects visual processing while noting ways in which it differs from that seen within the oculomotor system. We also suggest that object affordances may represent the neural mechanism through which certain object features are selected for preferential processing when stimuli are near the hand. Finally, we summarize the two effector-based feedback systems and discuss how having separate but parallel effector systems allows for efficient decoupling of eye and hand movements.

Rhythmic oscillations of visual contrast sensitivity synchronized with action

It is well known that the motor and the sensory systems structure sensory data collection and cooperate to achieve an efficient integration and exchange of information. Increasing evidence suggests that both motor and sensory functions are regulated by rhythmic processes reflecting alternating states of neuronal excitability, and these may be involved in mediating sensory-motor interactions. Here we show an oscillatory fluctuation in early visual processing time locked with the execution of voluntary action, and, crucially, even for visual stimuli irrelevant to the motor task. Human participants were asked to perform a reaching movement toward a display and judge the orientation of a Gabor patch, near contrast threshold, briefly presented at random times before and during the reaching movement. When the data are temporally aligned to the onset of movement, visual contrast sensitivity oscillates with periodicity within the theta band. Importantly, the oscillations emerge during the motor planning stage, ∼500 ms before movement onset. We suggest that brain oscillatory dynamics may mediate an automatic coupling between early motor planning and early visual processing, possibly instrumental in linking and closing up the visual-motor control loop.

The role of audience participation and task relevance on change detection during a card trick

Magicians utilize many techniques for misdirecting audience attention away from the secret sleight of a trick. One technique is to ask an audience member to participate in a trick either physically by asking them to choose a card or cognitively by having them keep track of a card. While such audience participation is an established part of most magic the cognitive mechanisms by which it operates are unknown. Failure to detect changes to objects while passively viewing magic tricks has been shown to be conditional on the changing feature being irrelevant to the current task. How change blindness operates during interactive tasks is unclear but preliminary evidence suggests that relevance of the changing feature may also play a role (Triesch et al., 2003). The present study created a simple on-line card trick inspired by Triesch et al.’s (2003) that allowed playing cards to be instantaneously replaced without distraction or occlusion as participants were either actively sorting the cards (Doing condition) or watching another person perform the task (Watching conditions). Participants were given one of three sets of instructions. The relevance of the card color to the task increased across the three instructions. During half of the trials a card changed color (but retained its number) as it was moving to the stack. Participants were instructed to immediately report such changes. Analysis of the probability of reporting a change revealed that actively performing the sorting task led to more missed changes than passively watching the same task but only when the changing feature was irrelevant to the sorting task. If the feature was relevant during either the pick-up or put-down action change detection was as good as during the watching block. These results confirm the ability of audience participation to create subtle dynamics of attention and perception during a magic trick and hide otherwise striking changes at the center of attention.

Planning Ahead: Object-Directed Sequential Actions Decoded from Human Frontoparietal and Occipitotemporal Networks

Object-manipulation tasks (e.g., drinking from a cup) typically involve sequencing together a series of distinct motor acts (e.g., reaching toward, grasping, lifting, and transporting the cup) in order to accomplish some overarching goal (e.g., quenching thirst). Although several studies in humans have investigated the neural mechanisms supporting the planning of visually guided movements directed toward objects (such as reaching or pointing), only a handful have examined how manipulatory sequences of actions-those that occur after an object has been grasped-are planned and represented in the brain. Here, using event-related functional MRI and pattern decoding methods, we investigated the neural basis of real-object manipulation using a delayed-movement task in which participants first prepared and then executed different object-directed action sequences that varied either in their complexity or final spatial goals. Consistent with previous reports of preparatory brain activity in non-human primates, we found that activity patterns in several frontoparietal areas reliably predicted entire action sequences in advance of movement. Notably, we found that similar sequence-related information could also be decoded from pre-movement signals in object- and body-selective occipitotemporal cortex (OTC). These findings suggest that both frontoparietal and occipitotemporal circuits are engaged in transforming object-related information into complex, goal-directed movements.

The Influences of Goal and Laterality of a Prepared Hand Response on a Co-Occurring Visual Search

Computer use generally requires manual interaction with human-computer interfaces. In this experiment, we studied the influence of manual response preparation on co-occurring shifts of attention to information on a computer screen. The participants were to carry out a visual search task on a computer screen while simultaneously preparing to reach for either a proximal or distal switch on a horizontal device, with either their right or left hand. The response properties were not predictive of the target’s spatial position. The results mainly showed that the preparation of a manual response influenced visual search: (1) The visual target whose location was congruent with the goal of the prepared response was found faster; (2) the visual target whose location was congruent with the laterality of the response hand was found faster; (3) these effects have a cumulative influence on visual search performance; (4) the magnitude of the influence of the response goal on visual search is marginally negatively correlated with the rapidity of response execution. These results are discussed in the general framework of structural coupling between perception and motor planning.

Translating working memory into action: behavioral and neural evidence for using motor representations in encoding visuo-spatial sequences

The neurobiological organization of action-oriented working memory is not well understood. To elucidate the neural correlates of translating visuo-spatial stimulus sequences into delayed (memory-guided) sequential actions, we measured brain activity using functional magnetic resonance imaging while participants encoded sequences of four to seven dots appearing on fingers of a left or right schematic hand. After variable delays, sequences were to be reproduced with the corresponding fingers. Recall became less accurate with longer sequences and was initiated faster after long delays. Across both hands, encoding and recall activated bilateral prefrontal, premotor, superior and inferior parietal regions as well as the basal ganglia, whereas hand-specific activity was found (albeit to a lesser degree during encoding) in contralateral premotor, sensorimotor, and superior parietal cortex. Activation differences after long versus short delays were restricted to motor-related regions, indicating that rehearsal during long delays might have facilitated the conversion of the memorandum into concrete motor programs at recall. Furthermore, basal ganglia activity during encoding selectively predicted correct recall. Taken together, the results suggest that to-be-reproduced visuo-spatial sequences are encoded as prospective action representations (motor intentions), possibly in addition to retrospective sensory codes. Overall, our study supports and extends multi-component models of working memory, highlighting the notion that sensory input can be coded in multiple ways depending on what the memorandum is to be used for.

Look where you're going! Perceptual attention constrains the online guidance of action

Action guidance, like perceptual discrimination, requires selective attention. Perception is enhanced at the target of a reaching movement, but it is not known whether selecting an object for perception reciprocally prioritises it for action. Two theoretical frameworks, the premotor theory and the Visual Attention Model, predict that this reciprocal relation should hold. We tested the influence of perceptual attention on the online control of reaching. In Experiment 1, participants attended covertly to a flanker on one or other side of a fixated target, prior to reaching for that target, which occasionally jumped, after reach onset, to the attended or non-attended side. Participants corrected their reaches for almost all target jumps. In Experiment 2, we required covert monitoring of the flanker during reaching. This concurrent perceptual task globally reduced correction behaviour, indicating that perception and action share a common attentional resource. Corrections were especially unlikely toward the attended side. This is explained by assuming that perceptual attention primed an action toward the attended location and that the participant inhibited this primed action. The data thus imply that perceptual selection constrains online action guidance, as predicted by the premotor theory and the VAM. We further argue that the fact that participants can inhibit a location within the action system but simultaneously maintain its prioritisation for perceptual monitoring, is easier to reconcile with the VAM than with the premotor theory.