Using goal- and grip-related information for understanding the correctness of other's actions: an ERP study

The pain hidden in your hands: Facial expression of pain reduces the influence of goal-related information in action recognition

The involvement of the sensorimotor system in the perception of painful actions has been repeatedly demonstrated. Yet the cognitive processes corresponding to sensorimotor activations have not been identified. In particular, the respective role of higher-level and lower-level action representations such as goals and grips in the recognition of painful actions is not clear. Previous research has shown that in a neutral context, higher-level action representations (goals) are prioritized over lower-level action representations (grips) and guide action recognition. The present study evaluates to what extent the general priority given to goal-related information in the processing of visual actions can be modulated by a context of pain. We used the action violation paradigm developed by van Elk et al. (2008). In the present action tasks, participants had to judge whether the grip or the goal of object-directed actions displayed in photographs was correct or not. The actress in the photograph could show either a neutral facial expression or a facial expression of pain. In the control task, they had to judge whether the actress expressed pain. In the action tasks, goals influenced grip judgements more than grips influenced goal judgements overall, corroborating the priority given to goal-related information previously reported. Critically, the impact of irrelevant goal-related information on the identification of incorrect grips disappeared in the pain context. Moreover, judgements in the control task were similarly influenced by grip and goal-related information. Results suggest that a context of pain reduces the reliance on higher-level action for action judgments. Findings provide novel directions regarding the cognitive and brain mechanisms involved in action processing in painful situations and support pluralist views of action understanding.

Error observation as a window on performance monitoring in social contexts? A systematic review

Living in a social world requires social monitoring, i.e., the ability to keep track of others' actions and mistakes. Here, we demonstrate the good reliability of the behavioral and neurophysiological indexes ascribed to social monitoring. We also show that no consensus exists on the cognitive bases of this phenomenology and discuss three alternative hypotheses: (i) the direct matching hypothesis, postulating that observed errors are processed through automatic simulation; (ii) the attentional hypothesis, considering errors as unexpected events that take resources away from task processing; and (iii) the goal representation hypothesis, which weighs social error monitoring depending on how relevant the other's task is to the observer's goals. To date, evidence on the role played by factors that could help to disentangle these hypotheses (e.g., the human vs. non-human nature of the actor, the error rate, and the reward context) is insufficient, although the goal representation hypothesis seems to receive more support. Theory-driven experimental designs are needed to enlighten this debate and clarify the role of error monitoring during interactive exchanges.

Effects of Cognitive Reappraisal on Subjective and Neural Reactivity to Angry Faces in Children with Social Anxiety Disorder, Clinical Controls with Mixed Anxiety Disorders and Healthy Children

Cognitive models of social anxiety suggest that social anxiety disorder (SAD) is characterized by both enhanced emotional reactivity and deficits in emotion regulation. Emotional reactivity to socially threatening children's faces and their modulation through reappraisal were measured via subjective ratings and electrocortical responses in children (age 10-13) with SAD (n = 28), clinical controls with mixed anxiety disorders (n = 28), and healthy controls (n = 29). Children with SAD showed higher subjective reactivity to the images of angry children's faces while all children reported reduced reactivity in their subjective ratings following reappraisal. Reduced electrocortical reactivity after reappraisal was only evident in older children and boys and was unrelated to anxiety. The present study indicates that cognitive reappraisal may be beneficial in reducing subjective reactivity in children with anxiety disorders, while neural effects of reappraisal may emerge at older ages.

A predictive processing framework of tool use

In this paper I introduce the theory of predictive processing as a unifying conceptual framework to account for the human ability to use and innovate tools. I explain the basic concepts of predictive processing and illustrate how this framework accounts for the development of tool use in young infants and for findings in the neuropsychological and neuroscientific literature. Then, I argue that the predictive processing model needs to be complemented with a functional-evolutionary perspective, according to which the developmental and neurocognitive mechanisms should be understood in relation to the adaptive function that tools subserve. I discuss cross-cultural and comparative studies on tool use to illustrate how tools could facilitate a process of cumulative cultural and technological evolution. Furthermore, I illustrate how central premises of the predictive processing framework, such as the notion of Bayesian inference as a general principle and the role of prediction-error-updating, speak to central debates in evolutionary psychology, such as the massive modularity hypothesis and the trade-off between exploitation and innovation. Throughout the paper I make several concrete suggestions for future studies that could be used to put the predictive processing model of tool use to the test.

Neural dynamics of grip and goal integration during the processing of others' actions with objects: An ERP study

Recent behavioural evidence suggests that when processing others’ actions, motor acts and goal-related information both contribute to action recognition. Yet the neuronal mechanisms underlying the dynamic integration of the two action dimensions remain unclear. This study aims to elucidate the ERP components underlying the processing and integration of grip and goal-related information. The electrophysiological activity of 28 adults was recorded during the processing of object-directed action photographs (e.g., writing with pencil) containing either grip violations (e.g. upright pencil grasped with atypical-grip), goal violations (e.g., upside-down pencil grasped with typical-grip), both grip and goal violations (e.g., upside-down pencil grasped with atypical-grip), or no violations. Participants judged whether actions were overall typical or not according to object typical use. Brain activity was sensitive to the congruency between grip and goal information on the N400, reflecting the semantic integration between the two dimensions. On earlier components, brain activity was affected by grip and goal typicality independently. Critically, goal typicality but not grip typicality affected brain activity on the N300, supporting an earlier role of goal-related representations in action recognition. Findings provide new insights on the neural temporal dynamics of the integration of motor acts and goal-related information during the processing of others’ actions.

Role of sensorimotor areas in early detection of motor errors: An EEG and TMS study

Action execution is prone to errors and, while engaged in interaction, our brain is tuned to detect deviations from what one expects from other's action. Prior research has shown that Event-Related-Potentials (ERPs) are specifically modulated by the observation of action mistakes interfering with goal achievement. However, in complex and modular actions, embedded motor errors do not necessarily produce an immediate effect on the global goal. Here we dissociate embedded motor goals from global action goals by asking subjects to observe familiar but untrained knotting actions. During knotting an embedded motor error (i.e. the rope is inserted top-down instead of bottom-up during the formation of a loop) while not producing any immediate mistake, may strongly affect the final result. We found that embedded errors elicit in the observer specific early fronto-central negativity (120-180 ms). In a second experiment, we online administered exicitatory transcranial magnetic stimulation (TMS) over central (C3) or occipital (Oz) scalp locations, at the timing of the ERP components observed in the first experiment. C3 stimulation produced a significant improvement in embedded error discrimination performance. These results show that sensorimotor areas are instrumental in the early detection of embedded motor errors. We conclude that others' embedded errors provide fundamental cues which, inserted within a complex hierarchical action plan, might be used by the observer to anticipate whether an action will eventually fail.

Theta synchronization over occipito-temporal cortices during visual perception of body parts

Categorical clustering in the visual system is thought to have evolved as a function of intrinsic (intra-areal) and extrinsic (interareal) connectivity and experience. In the visual system, the extrastriate body area (EBA), an occipito-temporal region, responds to full body and body part images under the organizational principle of their functional/semantic meaning. Although frequency-specific modulations of neural activity associated with perceptive and cognitive functions are increasingly attracting the interest of neurophysiologists and cognitive neuroscientists, perceiving single body parts with different functional meaning and full body images induces time-frequency modulations over occipito-temporal electrodes are yet to be described. Here, we studied this issue by measuring EEG in participants who passively observed fingers, hands, arms and faceless full body images with four control plant stimuli, each bearing hierarchical analogy with the body stimuli. We confirmed that occipito-temporal electrodes (compatible with the location of EBA) show a larger event-related potential (ERP, N190) for body-related images. Furthermore, we identified a body part-specific (i.e. selective for hands and arms) theta event-related synchronization increase under the same electrodes. This frequency modulation associated with the perception of body effectors over occipito-temporal cortices is in line with recent findings of categorical organization of neural responses to human effectors in the visual system.

Not so secret agents: Event-related potentials to semantic roles in visual event comprehension

Research across domains has suggested that agents, the doers of actions, have a processing advantage over patients, the receivers of actions. We hypothesized that agents as "event builders" for discrete actions (e.g., throwing a ball, punching) build on cues embedded in their preparatory postures (e.g., reaching back an arm to throw or punch) that lead to (predictable) culminating actions, and that these cues afford frontloading of event structure processing. To test this hypothesis, we compared event-related brain potentials (ERPs) to averbal comic panels depicting preparatory agents (ex. reaching back an arm to punch) that cued specific actions with those to non-preparatory agents (ex. arm to the side) and patients that did not cue any specific actions. We also compared subsequent completed action panels (ex. agent punching patient) across conditions, where we expected an inverse pattern of ERPs indexing the differential costs of processing completed actions asa function of preparatory cues. Preparatory agents evoked a greater frontal positivity (600-900ms) relative to non-preparatory agents and patients, while subsequent completed actions panels following non-preparatory agents elicited a smaller frontal positivity (600-900ms). These results suggest that preparatory (vs. non-) postures may differentially impact the processing of agents and subsequent actions in real time.

Embodying Others in Immersive Virtual Reality: Electro-Cortical Signatures of Monitoring the Errors in the Actions of an Avatar Seen from a First-Person Perspective

Brain monitoring of errors in one's own and other's actions is crucial for a variety of processes, ranging from the fine-tuning of motor skill learning to important social functions, such as reading out and anticipating the intentions of others. Here, we combined immersive virtual reality and EEG recording to explore whether embodying the errors of an avatar by seeing it from a first-person perspective may activate the error monitoring system in the brain of an onlooker. We asked healthy participants to observe, from a first- or third-person perspective, an avatar performing a correct or an incorrect reach-to-grasp movement toward one of two virtual mugs placed on a table. At the end of each trial, participants reported verbally how much they embodied the avatar's arm. Ratings were maximal in first-person perspective, indicating that immersive virtual reality can be a powerful tool to induce embodiment of an artificial agent, even through mere visual perception and in the absence of any cross-modal boosting. Observation of erroneous grasping from a first-person perspective enhanced error-related negativity and medial-frontal theta power in the trials where human onlookers embodied the virtual character, hinting at the tight link between early, automatic coding of error detection and sense of embodiment. Error positivity was similar in 1PP and 3PP, suggesting that conscious coding of errors is similar for self and other. Thus, embodiment plays an important role in activating specific components of the action monitoring system when others' errors are coded as if they are one's own errors.

SIGNIFICANCE STATEMENT

Detecting errors in other's actions is crucial for social functions, such as reading out and anticipating the intentions of others. Using immersive virtual reality and EEG recording, we explored how the brain of an onlooker reacted to the errors of an avatar seen from a first-person perspective. We found that mere observation of erroneous actions enhances electrocortical markers of error detection in the trials where human onlookers embodied the virtual character. Thus, the cerebral system for action monitoring is maximally activated when others' errors are coded as if they are one's own errors. The results have important implications for understanding how the brain can control the external world and thus creating new brain-computer interfaces.

The notion of the motion: the neurocognition of motion lines in visual narratives

Motion lines appear ubiquitously in graphic representation to depict the path of a moving object, most popularly in comics. Some researchers have argued that these graphic signs directly tie to the "streaks" appearing in the visual system when a viewer tracks an object (Burr, 2000), despite the fact that previous studies have been limited to offline measurements. Here, we directly examine the cognition of motion lines by comparing images in comic strips that depicted normal motion lines with those that either had no lines or anomalous, reversed lines. In Experiment 1, shorter viewing times appeared to images with normal lines than those with no lines, which were shorter than those with anomalous lines. In Experiment 2, measurements of event-related potentials (ERPs) showed that, compared to normal lines, panels with no lines elicited a posterior positivity that was distinct from the frontal positivity evoked by anomalous lines. These results suggested that motion lines aid in the comprehension of depicted events. LORETA source localization implicated greater activation of visual and language areas when understanding was made more difficult by anomalous lines. Furthermore, in both experiments, participants' experience reading comics modulated these effects, suggesting motion lines are not tied to aspects of the visual system, but rather are conventionalized parts of the "vocabulary" of the visual language of comics.

A saw is first identified as an object used on wood: ERP evidence for temporal differences between Thematic and Functional similarity relations

The role of functional and motor information in manipulable artifact object semantic organization is still poorly understood. In particular, several types of semantic relations involving object functional knowledge may be distinguished. Functional similarity relations group objects with similar functions at relatively specific (e.g. saw-axe, both used to cut wood) or general (saw-knife, both used to cut) levels. Thematic relations group objects based on their complementarity in events (saw used upon/with wood). Recent eye-tracking data showed distinct temporal time courses for the different semantic relations, with fastest processing of thematic relations and slowest processing of general function similarity relations. Behavioral data suggest the involvement of distinct cognitive mechanisms in manipulable artifact object semantic processing. The aim of the present study was to assess the neural correlates of thematic, and specific and general function similarity relation processing. Specifically, we investigated whether time course differences between semantic relations could be highlighted at the neurophysiological level. We used a protocol combining semantic priming with electroencephalography, and manipulated the type of semantic relation and the duration of the interval between prime and target objects. Two consistent and complementary results were shown. On N1 and P3 components, semantic priming was observed for thematic relations only. On N400 component, the type of semantic relation interacted with interval duration, and semantic priming was visible for all 3 relations after the longest interval only. Results revealed graded processing time courses for thematic, specific function similarity, and general function similarity relations at the neural level, and further indicate that thematic relations impact object processing during the early stages of object recognition. Findings suggest a hierarchical organization of three types of semantic relations based on functional knowledge. The parallel between semantic relations involving manipulable artifact objects and levels of action representations is discussed.

Action representations activated by task-irrelevant information: is it really irrelevant?

Accessing action knowledge is believed to rely on the activation of action representations through the retrieval of functional, manipulative, and spatial information associated with objects. However, it remains unclear whether action representations can be activated in this way when the object information is irrelevant to the current judgment. The present study investigated this question by independently manipulating the correctness of three types of action-related information: the functional relation between the two objects, the grip applied to the objects, and the orientation of the objects. In each of three tasks in Experiment 1, participants evaluated the correctness of only one of the three information types (function, grip or orientation). Similar results were achieved with all three tasks: "correct" judgments were facilitated when the other dimensions were correct; however, "incorrect" judgments were facilitated when the other two dimensions were both correct and also when they were both incorrect. In Experiment 2, when participants attended to an action-irrelevant feature (object color), there was no interaction between function, grip, and orientation. These results clearly indicate that action representations can be activated by retrieval of functional, manipulative, and spatial knowledge about objects, even though this is task-irrelevant information.

Watching novice action degrades expert motor performance: causation between action production and outcome prediction of observed actions by humans

Our social skills are critically determined by our ability to understand and appropriately respond to actions performed by others. However despite its obvious importance, the mechanisms enabling action understanding in humans have remained largely unclear. A popular but controversial belief is that parts of the motor system contribute to our ability to understand observed actions. Here, using a novel behavioral paradigm, we investigated this belief by examining a causal relation between action production, and a component of action understanding - outcome prediction, the ability of a person to predict the outcome of observed actions. We asked dart experts to watch novice dart throwers and predict the outcome of their throws. We modulated the feedbacks provided to them, caused a specific improvement in the expert's ability to predict watched actions while controlling the other experimental factors, and exhibited that a change (improvement) in their outcome prediction ability results in a progressive and proportional deterioration in the expert's own darts performance. This causal relationship supports involvement of the motor system in outcome prediction by humans of actions observed in others.

Infants' object location and identity processing in spatial scenes: an ERP study

BACKGROUND

Fast detection and identification of objects in an environment is important for using objects as landmarks during navigation. While adults rapidly process objects within an environment and use landmarks during navigation, infants do not routinely use distal landmarks below the age of 18 months. In the current event-related potential (ERP) study we adopted an oddball paradigm to examine whether infants are capable of processing objects in environments, which is a prerequisite for using objects as landmarks.

METHODS

We measured the electrophysiological correlates and time courses related to the processing of changes in object location, object identity, and a switch of two objects.

RESULTS

Twelve-month-old infants showed an Nc (negative central) effect reflecting increased attention likely caused by initial change detection within 300 msec for all three manipulations. In addition, they showed conscious processing of an object change and a location change as evidenced by a positive slow wave (PSW).

CONCLUSION

This study is the first to show that infants are capable of rapidly detecting changes in single objects when these are presented in an environment, but lack conscious detection of a switch. These results indicate that 12-month-old infants as yet lack the ability to rapidly bind the identity and location of multiple objects within an environment.

Event-related brain potentials for goal-related power grips

Recent research has shown that neurophysiological activation during action planning depends on the orientation to initial or final action goals for precision grips. However, the neural signature for a distinct class of grasping, power grips, is still unknown. The aim of the present study was to differentiate between cerebral activity, by means of event-related potentials (ERPs), and its temporal organization during power grips executed with an emphasis on either the initial or final parts of movement sequences. In a grasp and transportation task, visual cues emphasized either the grip (the immediate goal) or the target location (the final goal). ERPs differed between immediate and final goal-cued conditions, suggesting different means of operation dependent on goal-relatedness. Differences in mean amplitude occurred earlier for power grips than for recently reported precision grips time-locked to grasping over parieto-occipital areas. Time-locked to final object placement, differences occurred within a similar time window for power and precision grips over frontal areas. These results suggest that a parieto-frontal network of activation is of crucial importance for grasp planning and execution. Our results indicate that power grip preparation and execution for goal-related actions are controlled by similar neural mechanisms as have been observed during precision grips, but with a distinct temporal pattern.