Intentional control of attention: action planning primes action-related stimulus dimensions

Computation on Demand: Action-Specific Representations of Visual Task Features Arise during Distinct Movement Phases

The intention to act influences the computations of various task-relevant features. However, little is known about the time course of these computations. Furthermore, it is commonly held that these computations are governed by conjunctive neural representations of the features. But, support for this view comes from paradigms arbitrarily combining task features and affordances, thus requiring representations in working memory. Therefore, the present study used electroencephalography and a well-rehearsed task with features that afford minimal working memory representations to investigate the temporal evolution of feature representations and their potential integration in the brain. Female and male human participants grasped objects or touched them with a knuckle. Objects had different shapes and were made of heavy or light materials with shape and weight being relevant for grasping, not for "knuckling." Using multivariate analysis showed that representations of object shape were similar for grasping and knuckling. However, only for grasping did early shape representations reactivate at later phases of grasp planning, suggesting that sensorimotor control signals feed back to the early visual cortex. Grasp-specific representations of material/weight only arose during grasp execution after object contact during the load phase. A trend for integrated representations of shape and material also became grasp-specific but only briefly during the movement onset. These results suggest that the brain generates action-specific representations of relevant features as required for the different subcomponents of its action computations. Our results argue against the view that goal-directed actions inevitably join all features of a task into a sustained and unified neural representation.

TAT-HUM: Trajectory analysis toolkit for human movements in Python

Human movement trajectories can reveal useful insights regarding the underlying mechanisms of human behaviors. Extracting information from movement trajectories, however, can be challenging because of their complex and dynamic nature. The current paper presents a Python toolkit developed to help users analyze and extract meaningful information from the trajectories of discrete rapid aiming movements executed by humans. This toolkit uses various open-source Python libraries, such as NumPy and SciPy, and offers a collection of common functionalities to analyze movement trajectory data. To ensure flexibility and ease of use, the toolkit offers two approaches: an automated approach that processes raw data and generates relevant measures automatically, and a manual approach that allows users to selectively use different functions based on their specific needs. A behavioral experiment based on the spatial cueing paradigm was conducted to illustrate how one can use this toolkit in practice. Readers are encouraged to access the publicly available data and relevant analysis scripts as an opportunity to learn about kinematic analysis for human movements.

Exploring the internal forward model: action-effect prediction and attention in sensorimotor processing

Action-effect predictions are believed to facilitate movement based on its association with sensory objectives and suppress the neurophysiological response to self- versus externally generated stimuli (i.e. sensory attenuation). However, research is needed to explore theorized differences in the use of action-effect prediction based on whether movement is uncued (i.e. volitional) or in response to external cues (i.e. stimulus-driven). While much of the sensory attenuation literature has examined effects involving the auditory N1, evidence is also conflicted regarding this component's sensitivity to action-effect prediction. In this study (n = 64), we explored the influence of action-effect contingency on event-related potentials associated with visually cued and uncued movement, as well as resultant stimuli. Our findings replicate recent evidence demonstrating reduced N1 amplitude for tones produced by stimulus-driven movement. Despite influencing motor preparation, action-effect contingency was not found to affect N1 amplitudes. Instead, we explore electrophysiological markers suggesting that attentional mechanisms may suppress the neurophysiological response to sound produced by stimulus-driven movement. Our findings demonstrate lateralized parieto-occipital activity that coincides with the auditory N1, corresponds to a reduction in its amplitude, and is topographically consistent with documented effects of attentional suppression. These results provide new insights into sensorimotor coordination and potential mechanisms underlying sensory attenuation.

Perception and action as viewed from the Theory of Event Coding: a multi-lab replication and effect size estimation of common experimental designs

The Theory of Event Coding (TEC) has influenced research on action and perception across the past two decades. It integrates several seminal empirical phenomena and it has continued to stimulate novel experimental approaches on the representational foundations of action control and perceptual experience. Yet, many of the most notable results surrounding TEC originate from an era of psychological research that relied on rather small sample sizes as judged by today's standards. This state hampers future research aiming to build on previous phenomena. We, therefore, provide a multi-lab re-assessment of the following six classical observations: response-effect compatibility, action-induced blindness, response-effect learning, stimulus-response binding, code occupation, and short-term response-effect binding. Our major goal is to provide precise estimates of corresponding effect sizes to facilitate future scientific endeavors. These effect sizes turned out to be considerably smaller than in the original reports, thus allowing for informed decisions on how to address each phenomenon in future work. Of note, the most relevant results of the original observations were consistently obtained in the present experiments as well.

The influence of action on perception spans different effectors

Perception and action are fundamental processes that characterize our life and our possibility to modify the world around us. Several pieces of evidence have shown an intimate and reciprocal interaction between perception and action, leading us to believe that these processes rely on a common set of representations. The present review focuses on one particular aspect of this interaction: the influence of action on perception from a motor effector perspective during two phases, action planning and the phase following execution of the action. The movements performed by eyes, hands, and legs have a different impact on object and space perception; studies that use different approaches and paradigms have formed an interesting general picture that demonstrates the existence of an action effect on perception, before as well as after its execution. Although the mechanisms of this effect are still being debated, different studies have demonstrated that most of the time this effect pragmatically shapes and primes perception of relevant features of the object or environment which calls for action; at other times it improves our perception through motor experience and learning. Finally, a future perspective is provided, in which we suggest that these mechanisms can be exploited to increase trust in artificial intelligence systems that are able to interact with humans.

Visual working memory representations bias attention more when they are the target of an action plan

Attention has frequently been regarded as an emergent property of linking sensory representations to action plans. It has recently been proposed that similar mechanisms may operate within visual working memory (VWM), such that linking an object in VWM to an action plan strengthens its sensory memory representation, which then expresses as an attentional bias. Here we directly tested this hypothesis by comparing attentional biases induced by VWM representations which were the target of a future action, to those induced by VWM representations that were equally task-relevant, but not the direct target of action. We predicted that the first condition would result in a more prioritized memory state and hence stronger attentional biases. Specifically, participants memorized a geometric shape for a subsequent memory test. At test, in case of a match, participants either had to perform a grip movement on the matching object (action condition), or perform the same movement, but on an unrelated object (control condition). To assess any attentional biases, during the delay period between memorandum and test, participants performed a visual selection task in which either the target was surrounded by the memorized shape (congruent trials) or a distractor (incongruent trials). Eye movements were measured as a proxy for attentional priority. We found a significant interaction for saccade latencies between action condition and shape congruency, reflecting more pronounced VWM-based attentional biases in the action condition. Our results are consistent with the idea that action plans prioritize sensory representations in VWM.

Advances in the application of a computational Theory of Visual Attention (TVA): Moving towards more naturalistic stimuli and game-like tasks

The theory of visual attention, “TVA”, is an influential and formal theory of attentional selection. It is widely applied in clinical assessment of attention and fundamental attention research. However, most TVA-based research is based on accuracy data from letter report experiments performed in controlled laboratory environments. While such basic approaches to questions regarding attentional selection are undoubtedly useful, recent technological advances have enabled the use of increasingly sophisticated experimental paradigms involving more realistic scenarios. Notably, these studies have in many cases resulted in different estimates of capacity limits than those found in studies using traditional TVA-based assessment. Here we review recent developments in TVA-based assessment of attention that goes beyond the use of letter report experiments and experiments performed in controlled laboratory environments. We show that TVA can be used with other tasks and new stimuli, that TVA-based parameter estimation can be embedded into complex scenarios, such as games that can be used to investigate particular problems regarding visual attention, and how TVA-based simulations of “visual foraging” can elucidate attentional control in more naturalistic tasks. We also discuss how these developments may inform future advances of TVA.

Horizontal target size perturbations during grasping movements are described by subsequent size perception and saccade amplitude

Perception and action are essential in our day-to-day interactions with the environment. Despite the dual-stream theory of action and perception, it is now accepted that action and perception processes interact with each other. However, little is known about the impact of unpredicted changes of target size during grasping actions on perception. We assessed whether size perception and saccade amplitude were affected before and after grasping a target that changed its horizontal size during the action execution under the presence or absence of tactile feedback. We have tested twenty-one participants in 4 blocks of 30 trials. Blocks were divided into two experimental tactile feedback paradigms: tactile and non-tactile. Trials consisted of 3 sequential phases: pre-grasping size perception, grasping, and post-grasping size perception. During pre- and post-phases, participants executed a saccade towards a horizontal bar and performed a manual size estimation of the bar size. During grasping phase, participants were asked to execute a saccade towards the bar and to make a grasping action towards the screen. While grasping, 3 horizontal size perturbation conditions were applied: non-perturbation, shortening, and lengthening. 30% of the trials presented perturbation, meaning a symmetrically shortened or lengthened by 33% of the original size. Participants’ hand and eye positions were assessed by a motion capture system and a mobile eye-tracker, respectively. After grasping, in both tactile and non-tactile feedback paradigms, size estimation was significantly reduced in lengthening (p = 0.002) and non-perturbation (p<0.001), whereas shortening did not induce significant adjustments (p = 0.86). After grasping, saccade amplitude became significantly longer in shortening (p<0.001) and significantly shorter in lengthening (p<0.001). Non-perturbation condition did not display adjustments (p = 0.95). Tactile feedback did not generate changes in the collected perceptual responses, but horizontal size perturbations did so, suggesting that all relevant target information used in the movement can be extracted from the post-action target perception.

Flexible use of post-saccadic visual feedback in oculomotor learning

Saccadic eye movements bring objects of interest onto our fovea. These gaze shifts are essential for visual perception of our environment and the interaction with the objects within it. They precede our actions and are thus modulated by current goals. It is assumed that saccadic adaptation, a recalibration process that restores saccade accuracy in case of error, is mainly based on an implicit comparison of expected and actual post-saccadic position of the target on the retina. However, there is increasing evidence that task demands modulate saccade adaptation and that errors in task performance may be sufficient to induce changes to saccade amplitude. We investigated if human participants are able to flexibly use different information sources within the post-saccadic visual feedback in task-dependent fashion. Using intra-saccadic manipulation of the visual input, participants were either presented with congruent post-saccadic information, indicating the saccade target unambiguously, or incongruent post-saccadic information, creating conflict between two possible target objects. Using different task instructions, we found that participants were able to modify their saccade behavior such that they achieved the goal of the task. They succeeded in decreasing saccade gain or maintaining it, depending on what was necessary for the task, irrespective of whether the post-saccadic feedback was congruent or incongruent. It appears that action intentions prime task-relevant feature dimensions and thereby facilitated the selection of the relevant information within the post-saccadic image. Thus, participants use post-saccadic feedback flexibly, depending on their intentions and pending actions.

The Me-File: An Event-Coding Approach to Self-Representation

Numerous authors have taken it for granted that people represent themselves or even have something like “a self”, but the underlying mechanisms remain a mystery. How do people represent themselves? Here I propose that they do so not any differently from how they represent other individuals, events, and objects: by binding codes representing the sensory consequences of being oneself into a Me-File, that is, into an event file integrating all the codes resulting from the behaving me. This amounts to a Humean bundle-self theory of selfhood, and I will explain how recent extensions of the Theory of Event Coding, a general theory of human perception and action control, provide all the necessary ingredients for specifying the mechanisms underlying such a theory. The Me-File concept is likely to provide a useful mechanistic basis for more specific and more theoretically productive experimentation, as well as for the construction of artificial agents with human-like selves.

Simple action planning can affect attentional allocation in subsequent visual search

Previous research has shown that mental representations of actions can influence performance on relevant tasks or dimensions even when there is no overt execution of the action. In this study, we examined whether cognitive processes prior to the physical execution of an action can elicit attentional bias towards irrelevant tasks or dimensions of that action. Participants performed two independent tasks-an action task and a search task-where they were instructed to plan an action and execute the action following the visual search task. We found that the same features of the object were prioritized in the subsequent search task when participants had planned an action response on the object in comparison to when they had not. This effect occurred even when the feature was irrelevant to the tasks or required action. Furthermore, the effect of action planning without physical response was found to be comparable to the effect of physical response. These results suggest that planning of a simple action can induce attentional bias to irrelevant features of objects even without physical action.

Tool-number interaction during a prospective memory task

Theoretical views suggest that tool use and numerical magnitude processing can interact during prospective actions. For example, if a person intends to make a meal for several persons the next week, she/he will have to keep in mind during the homework-week large dish and large food portions for this event. Here, the magnitude 'large' can influence the future choice for large dishes and other related large tools. This study presents the first empirical evidence supporting this hypothesis. During a prospective memory task that implied to keep in mind a future action, participants were required to use a tool after processing Arabic numbers. Small (less than 5) and large (more than 5) magnitudes were employed as cues for the initiation of the tool-use task, which required participants to use inverse pliers with a small or a large object, but only for some prospective trials. The inverse pliers were used to dissociate the hand action from the tool action with the object (for example, opening the hand produced the closing action of the tool). The results revealed that during prospective trials, number processing interacted only with the tool action toward the object and not with the hand action. Specifically, after the processing of large magnitudes, the initiation of the closing action of the tool (i.e. the opening action of the hand) was reduced. This finding is discussed in the light of theories on the emergence of semantics through tool actions.

Choices in a key press decision-making task are more optimal after gaining both aiming and reward experience

When presented with two different target-penalty configurations of similar maximum expected gain (MEG), participants prefer aiming to configurations with more advantageous spatial, rather than more advantageous gain parameters-perhaps due to the motor system's inherent prioritisation of spatial information during movements with high accuracy demands such as aiming. To test this hypothesis, participants in the present studies chose between target-penalty configurations via key presses to reduce the importance of spatial parameters of the response and performance-related feedback. Configurations varied in spatial (target-penalty region overlap) and gain parameters (negative penalty values) and could have similar or different MEG. Choices were made without prior aiming experience (Experiment 1), after aiming experience provided information of movement variability (Experiment 2), or after aiming experience provided information of movement variability and outcome feedback (Experiment 3). Overall, configurations with advantageous spatial or gain parameters were chosen equally (Both-Similar condition) in all experiments. However, average behaviour at the group level was not reflective of the behaviour of most individual participants with three subgroups emerging: those with a value preference, distance preference, or no preference. In Experiments 1 and 2, these individual differences cannot be explained by MEG differences between configurations or participants' movement variability, but these variables predicted choice behaviour in Experiment 3. Further in the Both-Different condition, participants only selected the larger MEG configuration at a level above chance when both variability and outcome information were given prior to the key press task (Experiment 3). In sum, the data indicate that prioritisation of spatial information did not emerge at the group level when performing key presses and more optimal behaviour emerged when information regarding movement variability and outcome feedback were given.

Rediscovering Richard Held: Activity and Passivity in Perceptual Learning

Understanding the role of self-generated movements in perceptual learning is central to action-based theories of perception. Pioneering work on sensory adaptation by Richard M. Held during the 1950s and 1960s can still shed light on this question. In a variety of rich experiments Held and his team demonstrated the need for self-generated movements in sensory adaptation and perceptual learning. This body of work received different critical interpretations, was then forgotten for some time, and saw a surge of revived interest within embodied cognitive science. Through a brief review of Held’s work and reactions to it, we seek to contribute to discussions on the role of activity and passivity in perceptual learning. We classify different positions according to whether this role is considered to be contextual (facilitatory, but not necessary), enabling (causally necessary), or constitutive (an inextricable part of the learning process itself). We also offer a critique of the notions of activity and passivity and how they are operationalized in experimental studies. The active-passive distinction is not a binary but involves a series of dimensions and relative degrees that can make it difficult to interpret and replicate experimental results. We introduce three of these dimensions drawing on work on the sense of agency: action initiation, control, and monitoring. These refinements in terms of causal relations and dimensions of activity-passivity should help illuminate open questions concerning the role of activity in perception and perceptual learning and clarify the convergences and differences between enaction and ecological psychology.

Theory of Event Coding (TEC) V2.0: Representing and controlling perception and action

This article provides an update of the Theory of Event Coding (TEC), which claims that perception and action are identical processes operating on the same codes - event files consisting of integrated networks of sensorimotor feature codes. The original version of the theory emphasized its representational underpinnings, but recent theoretical developments provide the basis for a more integrated view consisting of both the codes that are shared between perception and action in the control processes operating on these codes. Four developments are discussed in more detail: The degree to which the integration and retrieval of event files depends on current goals, how metacontrol states impact the handling of event files, how feature binding relates to event learning, and how the integration of non-social events relates to the integration of social events. Case examples using various versions of the Simon task are used to explain how the new version of TEC explains interactions between perception and action in non-social and social situations.

No one knows what attention is

In this article, we challenge the usefulness of "attention" as a unitary construct and/or neural system. We point out that the concept has too many meanings to justify a single term, and that "attention" is used to refer to both the explanandum (the set of phenomena in need of explanation) and the explanans (the set of processes doing the explaining). To illustrate these points, we focus our discussion on visual selective attention. It is argued that selectivity in processing has emerged through evolution as a design feature of a complex multi-channel sensorimotor system, which generates selective phenomena of "attention" as one of many by-products. Instead of the traditional analytic approach to attention, we suggest a synthetic approach that starts with well-understood mechanisms that do not need to be dedicated to attention, and yet account for the selectivity phenomena under investigation. We conclude that what would serve scientific progress best would be to drop the term "attention" as a label for a specific functional or neural system and instead focus on behaviorally relevant selection processes and the many systems that implement them.

Change detection for real-world objects in perihand space

Recent evidence has demonstrated that observers experience visual-processing biases in perihand space that may be tied to the hands' relevance for grasping actions. Our previous work suggested that when the hands are positioned to afford a power-grasp action, observers show increased temporal sensitivity that could aid with fast and forceful action, whereas when the hands are instead at the ready to perform a precision-grasp action, observers show enhanced spatial sensitivity that benefits delicate and detail-oriented actions. In the present investigation we seek to extend these previous findings by examining how object affordances may interact with hand positioning to shape visual biases in perihand space. Across three experiments, we examined how long participants took to perform a change detection task on photos of real objects, while we manipulated hand position (near/far from display), grasp posture (power/precision), and change type (orientation/identity). Participants viewed objects that afforded either a power grasp or a precision grasp, or were ungraspable. Although we were unable to uncover evidence of altered vision in perihand space in our first experiment, mirroring previous findings, in Experiments 2 and 3 our participants showed grasp-dependent biases near the hands when detecting changes to target objects that afforded a power grasp. Interestingly, ungraspable target objects were not subject to the same perihand space biases. Taken together, our results suggest that the influence of hand position on change detection performance is mediated not only by the hands' grasp posture, but also by a target object's affordances for grasping.

The influence of motor preparation on the processing of action-relevant visual features

Action preparation can facilitate performance in tasks of visual perception, for instance by speeding up responses to action-relevant stimulus features. However, it is unknown whether this facilitation reflects an influence on early perceptual processing, or instead post-perceptual processes. In three experiments, a combination of psychophysics and electroencephalography was used to investigate whether visual features are influenced by action preparation at the perceptual level. Participants were cued to prepare oriented reach-to-grasp actions before discriminating target stimuli oriented in the same direction as the prepared grasping action (congruent) or not (incongruent). As expected, stimuli were discriminated faster if their orientation was congruent, compared to incongruent, with the prepared action. However, action-congruency had no influence on perceptual sensitivity, regardless of cue-target interval and discrimination difficulty. The reaction time effect was not accompanied by modulations of early visual-evoked potentials. Instead, beta-band (13-30 Hz) synchronization over sensorimotor brain regions was influenced by action preparation, indicative of improved response preparation. Together, the results suggest that action preparation may not modulate early visual processing of orientation, but likely influences higher order response or decision related processing. While early effects of action on spatial perception are well documented, separate mechanisms appear to govern non-spatial feature selection.

Does the motor system contribute to the perception of changes in objects visual attributes? The neural dynamics of sensory binding by action

The contribution of the motor system to perception has been highlighted in research investigating the effect of performing an action on the conscious processing of information received from the sensory systems. For example, the perceptual temporal asynchrony observed when passively reporting changes in visual object attributes (e.g., colour and position) was found to disappear virtually when the changes resulted from a voluntary motor action. Although the spatio-temporal constraints of sensory binding by action have been broadly investigated, the underlying neural correlates are still largely unknown. In the present study, participants performed temporal order judgments of changes in the colour and position of a visual stimulus, while adapting to a 750 ms delay between a sound (perceptual condition) or the end of a manual reaching action (motor condition), and the visual changes. Behavioural observations indicated that temporal asynchrony (-30.2 ms) decreased in the motor condition (2.7 ms), as a result of sensorimotor adaptation, but not in the perceptual condition (-29.6 ms). EEG-evoked potentials on posterior visual regions showed that early components were altered by sensorimotor adaptation, with in particular a broad reduction in the amplitude of the early P1 component. Furthermore, time-frequency analysis of EEG signals during the 350 ms period preceding the visual changes revealed an increase of the 15-25 Hz frequency band amplitude in the central region and a decrease of the 8-12 Hz frequency band amplitude in the posterior region. Overall the results suggest that sensory binding by action depends on an early top-down modulation of the visual regions by the motor system - in agreement with the pre-activation theory of action-perception coupling - associated with an increase of attentional resources.

Sensorimotor contingency modulates breakthrough of virtual 3D objects during a breaking continuous flash suppression paradigm

To investigate how embodied sensorimotor interactions shape subjective visual experience, we developed a novel combination of Virtual Reality (VR) and Augmented Reality (AR) within an adapted breaking continuous flash suppression (bCFS) paradigm. In a first experiment, participants manipulated novel virtual 3D objects, viewed through a head-mounted display, using three interlocking cogs. This setup allowed us to manipulate the sensorimotor contingencies governing interactions with virtual objects, while characterising the effects on subjective visual experience by measuring breakthrough times from bCFS. We contrasted the effects of the congruency (veridical versus reversed sensorimotor coupling) and contingency (live versus replayed interactions) using a motion discrimination task. The results showed that the contingency but not congruency of sensorimotor coupling affected breakthrough times, with live interactions displaying faster breakthrough times. In a second experiment, we investigated how the contingency of sensorimotor interactions affected object category discrimination within a more naturalistic setting, using a motion tracker that allowed object interactions with increased degrees of freedom. We again found that breakthrough times were faster for live compared to replayed interactions (contingency effect). Together, these data demonstrate that bCFS breakthrough times for unfamiliar 3D virtual objects are modulated by the contingency of the dynamic causal coupling between actions and their visual consequences, in line with theories of perception that emphasise the influence of sensorimotor contingencies on visual experience. The combination of VR/AR and motion tracking technologies with bCFS provides a novel methodology extending the use of binocular suppression paradigms into more dynamic and realistic sensorimotor environments.

Visual attention and action: How cueing, direct mapping, and social interactions drive orienting

Despite considerable interest in both action perception and social attention over the last 2 decades, there has been surprisingly little investigation concerning how the manual actions of other humans orient visual attention. The present review draws together studies that have measured the orienting of attention, following observation of another's goal-directed action. Our review proposes that, in line with the literature on eye gaze, action is a particularly strong orienting cue for the visual system. However, we additionally suggest that action may orient visual attention using mechanisms, which gaze direction does not (i.e., neural direct mapping and corepresentation). Finally, we review the implications of these gaze-independent mechanisms for the study of attention to action. We suggest that our understanding of attention to action may benefit from being studied in the context of joint action paradigms, where the role of higher level action goals and social factors can be investigated.

Visuomotor and motorvisual priming with different types of set-level congruency: evidence in support of ideomotor theory, and the planning and control model (PCM)

Perception can prime action (visuomotor priming), and action can prime perception (motorvisual priming). According to ideomotor theory both effects rely on the overlap of mental representations between perception and action. This implies that both effects get more pronounced the more features they share. We tested this hypothesis by employing in a motorvisual (Exp. 1) and in a visuomotor (Exp. 2) setting, three different pairs of left/right target stimuli (hand pictures, arrows, and words) varying in how strongly they overlap with the pair of left/right responses. For two stimulus pairs (hands and words) the hypothesis was confirmed: hand pictures share more features with the responses than words, consequently hand pictures produced a stronger visuomotor and a stronger motorvisual priming effect than words. However, arrow stimuli showed a different pattern: the temporal dynamics of both priming effects, as well as the direction of the effect seen in motorvisual priming, were significant but opposite to that of the hand and word stimuli. This suggests that the arrows' representations were not involved in ideomotor processes, and we propose instead that they were represented in a spatial or scalar fashion, outside the representations assumed in ideomotor theory. The results are discussed in the context of ideomotor theory, and the planning and control model of motorvisual priming.

How movement direction shapes the spatial representation of its effects: About the consequence of the ideomotor bidirectional association

Ideomotor theories assume that action and perception share a common representational system in which a movement and its effect are equally represented and integrated by a bidirectional association. However, there is no mention of how this association leads to influence the representational content of each part. In this article, we investigated the influence of movement properties on the spatial representation of auditory effects. In line with the Action Constrains Theory of space perception, we suggest that changes in the movement direction leads to correlative changes in the spatial representation of the effect. In a pre-experiment, we replicated traditional ideomotor results with a response-effect (R-E) compatibility procedure. In two experiments, we used one condition of this procedure (i.e., the corresponding R-E mapping) to manipulate the movement properties associated to a non-spatialised effect. In the first experiment, the effect was associated with horizontal outward movements or with forward-backward movements. In the second experiment, we tested some alternative explanations for the results obtained in the first experiment. Globally, we showed that rightward movements led to localised auditory effect more on the right space than leftward movements and that backward movements led to localisation of the effect closer from the subjects than forward movements. In accordance with the Action Constrains Theory of space perception, these data suggest that movement shapes the spatial organisation of the effect representation.

Selective weighting of action-related feature dimensions in visual working memory

Planning an action primes feature dimensions that are relevant for that particular action, increasing the impact of these dimensions on perceptual processing. Here, we investigated whether action planning also affects the short-term maintenance of visual information. In a combined memory and movement task, participants were to memorize items defined by size or color while preparing either a grasping or a pointing movement. Whereas size is a relevant feature dimension for grasping, color can be used to localize the goal object and guide a pointing movement. The results showed that memory for items defined by size was better during the preparation of a grasping movement than during the preparation of a pointing movement. Conversely, memory for color tended to be better when a pointing movement rather than a grasping movement was being planned. This pattern was not only observed when the memory task was embedded within the preparation period of the movement, but also when the movement to be performed was only indicated during the retention interval of the memory task. These findings reveal that a weighting of information in visual working memory according to action relevance can even be implemented at the representational level during maintenance, demonstrating that our actions continue to influence visual processing beyond the perceptual stage.

Grasp preparation modulates early visual processing of size and detection of local/global stimulus features

Preparing to grasp objects facilitates visual processing of object location, orientation and size, compared to preparing actions such as pointing. This influence of action on perception reflects mechanisms of selection in visual perception tuned to current action goals, such that action relevant sensory information is prioritized relative to less relevant information. In three experiments, rather than varying movement type (grasp vs point), the magnitude of a prepared movement (power vs precision grasps) was manipulated while visual processing of object size, as well as local/global target detection was measured. Early event-related potentials (ERP) elicited by task-irrelevant visual probes were enhanced for larger probes during power grasp preparation and smaller probes during precision grasp preparation. Local targets were detected faster following precision, relative to power grasp cues. The results demonstrate a direct influence of grasp preparation on sensory processing of size and suggest that the hierarchical dimension of objects may be a relevant perceptual feature for grasp programming. To our knowledge, this is the first evidence that preparing different magnitudes of the same basic action has systematic effects on visual processing.

Integration of Action and Size Perception Through Practice

Size perception is known to influence our usual interactions with environment. Numerous studies highlighted that during the visual presentation of an object, the properties of manual actions vary as a function of this object's size. In order to better understand the dynamic variations of relationships between size perception and action, we used an experimental paradigm consisting in two phases. During a previous implicit learning phase, a manual response (right or left) was specifically associated with the appearance of a large or small stimulus. During further test phase, participants were required to prepare a response while discriminating the color of a stimulus (GO/No GO task). We observed that the response execution was faster when the size of the stimulus was congruent with the size that had been associated to this response (during implicit learning phase). These results suggest that when a response usually co-occurs with visual stimuli characterized by a specific size pattern, the response and the size pattern become integrated. Any subsequent preparation and execution of this action are therefore influenced by the reactivation of this visual pattern. This result brings out new insights on how sensorimotor interactions may modulate the ability to anticipate perceptive size variations in the environment.

The effect of task difficulty on motor performance and frontal-striatal connectivity in cocaine users

BACKGROUND

There is growing recognition that chronic cocaine users have alterations in sensorimotor control that are positively related to low frontal-striatal connectivity within the motor system. These frontal-striatal motor circuits however, are modulated by circuits governing attention, which are also disrupted in cocaine users. This study's aim was to determine if sensorimotor control deficits are positively related to the difficulty of a motor task or exist independent of the increasing cognitive demand.

METHODS

Functional MRI data was collected from 40 individuals (20 non-treatment seeking chronic cocaine users, 20 age and gender matched non-drug using controls) as they mimicked an unpredictable finger-tapping sequence at various speeds. Dependent measures included task accuracy, percent BOLD signal change in sensorimotor regions of interest (ROIs), and functional connectivity (temporal correlations) between ROIs.

RESULTS

In both groups, as speed increased, the BOLD signal change increased in the primary motor cortex, supplementary motor area (SMA), cerebellum, and anterior cingulate cortex. Compared to controls, cocaine user SMA-Caudate and ACC-Putamen connectivity was lower at all speeds in the contralateral hemisphere. Furthermore, as speed increased there was a decrease in connectivity between additional ROI pairs among users.

CONCLUSIONS

These data support previous observations of sensorimotor performance deficits and dorsal frontal-striatal connectivity impairments among cocaine users. While previous studies demonstrate these deficits when performing a finger-tapping task at a single speed, we show that these same impairments exist at multiple levels of task difficulty. These data suggest that previously observed frontal-striatal connectivity in cocaine users during sensorimotor task performance are stable and not directly related to cognitive demands of the task.

Situational and person-related factors influencing momentary and retrospective soundscape evaluations in day-to-day life

Soundscape research draws on both experiments conducted in laboratory settings and studies in the field to explore peoples' perception and understanding of their acoustic environments. One opportunity to combine the strength of both approaches is the so-called Experience Sampling Method (ESM). This method was used to investigate the influence of situational and person-related variables on soundscape evaluations. Further, the relationship between momentary and retrospective soundscape judgments was explored. In the course of the 7-day ESM study, 32 participants were prompted ten times per day by a smartphone application to evaluate their soundscape and report on situational factors. Additionally, they performed summary retrospective judgments evaluating the whole of each day and their whole week. Upon completion, an exit interview probed personality traits (e.g., Big Five, information processing styles). Results revealed that both situational and person-related factors significantly contributed to the judgments of three soundscape dimensions (pleasantness, eventfulness, familiarity). Retrospective judgments of soundscape pleasantness were not only the average of the momentary judgments, but were also affected by the peak moment, the linear trend of the experience, and a person's mood while performing the judgment. Hence, the study provides valuable insights into the complex structure of momentary and retrospective soundscape evaluations.

Visuomotor integration of relevant and irrelevant angry and fearful facial expressions

Our brain codes the features of perceptual events in a distributed fashion, raising the question of how information belonging to one event is processed without any interference of features from other events. Hommel (1998) suggested the "event file" concept to elucidate these mechanisms: an episodic memory trace "binding" together perceptual features and actions related to an object. Using a similar paradigm, we designed a pilot experiment and four additional experiments to investigate whether emotion, similarly than perceptual features, could bind with a motor response when the emotion was relevant and irrelevant for the task. Few studies have revealed this to be the case. We investigated how angry and fearful faces expressed by avatars and humans might be subject to a binding phenomenon. Our results show that at least three degrees of visuomotor binding seem to coexist: one implicating the relevant feature of the task with a strong effect on behavior, another implicating the location with a smaller behavioral effect (even if not task related), and a third implicating non-task-related features with behavioral effects only under specific conditions in which emotion could play a role. Our adaptation of Hommel's paradigm showed that emotional percepts can be subject to visuomotor binding effects even if the emotion is not task related confirming the important role of emotional information for the central nervous system. These findings offer new perspective in the investigation of the emotion-action binding at the neuronal level.

HiTEC: a connectionist model of the interaction between perception and action planning

Increasing evidence suggests that perception and action planning do not represent separable stages of a unidirectional processing sequence, but rather emerging properties of highly interactive processes. To capture these characteristics of the human cognitive system, we have developed a connectionist model of the interaction between perception and action planning: HiTEC, based on the Theory of Event Coding (Hommel et al. in Behav Brain Sci 24:849–937, 2001). The model is characterized by representations at multiple levels and by shared representations and processes. It complements available models of stimulus–response translation by providing a rationale for (1) how situation-specific meanings of motor actions emerge, (2) how and why some aspects of stimulus–response translation occur automatically and (3) how task demands modulate sensorimotor processing. The model is demonstrated to provide a unitary account and simulation of a number of key findings with multiple experimental paradigms on the interaction between perception and action such as the Simon effect, its inversion (Hommel in Psychol Res 55:270–279, 1993), and action–effect learning.

Grasping remaps the distribution of visuospatial attention and enhances competing action activation

We examined how action goals influence the distribution of visuospatial attention near the body (Experiment 1) and how the temporal relationship between distractors and targets modifies shifts in visuospatial attention (Experiment 2). Targets were light emitting diodes (LEDs) in the left and right hemispace of a visual display. Following left or right target illumination, participants reached to point-to or grasp target object in blocked trials. Coincident with target onset, a distractor LED illuminated in the same or opposite hemispace between the initiation point and target, or no distractor appeared. In Experiment 1, during grasping there was a larger temporal interference effect (slower reach initiation) than with pointing. When grasping versus pointing, participants deviated more towards same-side distractors and away from opposite-side distractors. In Experiment 2, distractors onset 200ms prior to (-200-ms), coincident with (0 ms), or 200ms following (+200 ms) the target. For both reach types, -200-ms distractors had greater onset temporal interference than 0 ms and +200-ms distractors. For grasping, +200 ms distractors had larger temporal interference than 0 ms distractors. For -200-ms, reach trajectories deviated more towards opposite-side distractors and away from same-side distractors, the reverse of the pattern for 0 ms and +200-ms distractors.

Action Effects and Task Knowledge: The Influence of Anticipatory Priming on the Identification of Task-Related Stimuli in Experts

The purpose of the present study was to examine the extent to which anticipation of an action’s perceptual effect primes identification of task-related stimuli. Specifically, skilled (n = 16) and novice (n = 24) tennis players performed a choice-reaction time (CRT) test in which they identified whether the presented stimulus was a picture of a baseball bat or tennis racket. Following their response, auditory feedback associated with either baseball or tennis was presented. The CRT test was performed in blocks in which participants predictably received the baseball sound or tennis sound irrespective of which stimulus picture was displayed. Results indicated that skilled tennis players responded quicker to tennis stimuli when the response was predictably followed by the tennis auditory effect compared to the baseball auditory effect. These findings imply that, within an individual’s area of expertise, domain-relevant knowledge is primed by anticipation of an action’s perceptual effect, thus allowing the cognitive system to more quickly identify environmental information. This finding provides a more complete picture of the influence that anticipation can have on the cognitive-motor system. No differences existed for novices.

Embodied artificial agents for understanding human social cognition

In this paper, we propose that experimental protocols involving artificial agents, in particular the embodied humanoid robots, provide insightful information regarding social cognitive mechanisms in the human brain. Using artificial agents allows for manipulation and control of various parameters of behaviour, appearance and expressiveness in one of the interaction partners (the artificial agent), and for examining effect of these parameters on the other interaction partner (the human). At the same time, using artificial agents means introducing the presence of artificial, yet human-like, systems into the human social sphere. This allows for testing in a controlled, but ecologically valid, manner human fundamental mechanisms of social cognition both at the behavioural and at the neural level. This paper will review existing literature that reports studies in which artificial embodied agents have been used to study social cognition and will address the question of whether various mechanisms of social cognition (ranging from lower- to higher-order cognitive processes) are evoked by artificial agents to the same extent as by natural agents, humans in particular. Increasing the understanding of how behavioural and neural mechanisms of social cognition respond to artificial anthropomorphic agents provides empirical answers to the conundrum 'What is a social agent?'

An object-identity probability cueing paradigm during grasping observation: the facilitating effect is present only when the observed kinematics is suitable for the cued object

Electrophysiological and psychophysical data indicate that grasping observation automatically orients attention toward the incoming interactions between the actor’s hand and the object. The aim of the present study was to clarify if this effect facilitates the detection of a graspable object with the observed action as compared to an ungraspable one. We submitted participants to an object-identity probability cueing experiment in which the two possible targets were of the same dimensions but one of them presented sharp tips at one extreme while the other presented flat faces. At the beginning of each trial the most probable target was briefly shown. After a variable interval, at the same position, the same (75%) or a different target (25%) was presented. Participants had to press a key in response to target appearance. Superimposed to the video showing cue and target, an agent performing the reaching and grasping of the target was presented. The kinematics of the action was or was not suitable for grasping the cued target, according to the absence or presence of the sharp tips. Results showed that response was modulated by the probability of target identity but only when the observed kinematics was suitable to grasp the attended target. A further experiment clarified that response modulation was never present when the superimposed video always showed the agent at a rest position. These findings are discussed at the light of neurophysiological and psychophysical literature, considering the relationship between the motor system and the perception of objects and of others’ actions. We conclude that the prediction of the mechanical events that arise from the interactions between the hand and the attended object is at the basis of the capability to select a graspable object in space.

Interocular suppression prevents interference in a flanker task

Executive control of attention refers to processes that detect and resolve conflict among competing thoughts and actions. Despite the high-level nature of this faculty, the role of awareness in executive control of attention is not well understood. In this study, we used interocular suppression to mask the flankers in an arrow flanker task, in which the flankers and the target arrow were presented simultaneously in order to elicit executive control of attention. Participants were unable to detect the flanker arrows or to reliably identify their direction when masked. There was a typical conflict effect (prolonged reaction time and increased error rate under flanker-target incongruent condition compared to congruent condition) when the flanker arrows were unmasked, while the conflict effect was absent when the flanker arrows were masked with interocular suppression. These results suggest that blocking awareness of competing stimuli with interocular suppression prevents the involvement of executive control of attention.

Action preparation shapes processing in early visual cortex

Preparation for an action, such as grasping an object, is accompanied by an enhanced perception of the object's action-relevant features, such as orientation and size. Cortical feedback from motor planning areas to early visual areas may drive this enhanced perception. To examine whether action preparation modulates activity in early human visual cortex, subjects grasped or pointed to oriented objects while high-resolution fMRI data were acquired. Using multivoxel pattern analysis techniques, we could decode with >70% accuracy whether a grasping or pointing action was prepared from signals in visual cortex as early as V1. These signals in early visual cortex were observed even when actions were only prepared but not executed. Anterior parietal cortex, on the other hand, showed clearest modulation for actual movements. This demonstrates that preparation of actions, even without execution, modulates relevant neuronal populations in early visual areas.