My partner is also on my mind: social context modulates the N1 response

Impact of hierarchical processing levels on division of labor: neural electrophysiological evidence from a joint rule shifting

OBJECTIVES

Although previous studies have revealed the behavioral and neural mechanisms underlying cognitive control, the neural electrophysiological process in joint task switching has rarely been discussed. Moreover, the hierarchy process remains unclear. Therefore, the present study aimed to explore the neural mechanism of a target when two actors use division of labor between action selections in a joint hierarchical rule-shifting situation.

METHODS

The present study employed a joint hierarchical rule shifting paradigm, and each participant was responsible for the selection of one type of action (left key and right key). The cue was the letter 'R' with three features, which indicated the hierarchical rule of the current trial. The target was an Arabic numeral (1-9, except 5). Participants made a magnitude judgement or a parity judgment based on the high and low hierarchical features of cues.

RESULTS

The P2 component was smaller in the high-shift condition than in the repeat condition; however, the high-shift condition elicited a larger N2 amplitude than the low-shift and repeat conditions. We believe that the current P2 is related to selective attention to the hierarchical features of rules, and N2 is related to conflict control. In addition, the high-shift condition had the largest P3 amplitude, which indicates that this condition has the largest extent of update and leads to the most unstable association between rule and stimuli.

CONCLUSIONS

The study demonstrated that participants had dynamic brain responses to target stimuli in the joint rule-shifting situation. In other words, participants first confirmed the actor of the current target and then processed the target stimuli themselves.

Color me impressed: A partner's target feature captures visual attention

Attending to a target is more difficult in the presence of a salient distractor. The present study investigated whether social value can modulate the extent to which distractors capture attention. Two participants sitting side-by-side performed a visual search task in cooperative and competitive conditions. Search displays contained either both targets, one target and a neutral stimulus or an ambiguous and a neutral stimulus. Results showed that agents took longer to respond to targets presented together with the partner's target compared to a neutral stimulus of equal salience. Agents also produced more false alarms in response to stimuli whose color lay between their own and the partner's target color compared to stimuli lying between the colors of their target and a neutral stimulus. These results suggest that stimuli with features relevant to a partner can capture attention more than neutral but equally salient stimuli, indicating that social value affects selective attention in a similar way as task goals and selection history.

Your action does matter to me: Examining the role of the co-actor's action-effects in resolving the self-other discrimination problem

Sharing a task with another person can introduce the need to discriminate representations that refer to our own action from that of the other person's. The current understanding is that information about the stimulus event drives the self-other discrimination process, as it promotes (via the reactivation of feature codes) the representation that encodes the corresponding action. However, this mechanistic explanation relies on experimental situations in which stimulus event information (e.g., spatial location) is always and directly available. Thus, it remains unclear whether and how we could successfully discriminate between self- and other related action representations in the absence of such information. The present study addressed this unanswered question using a novel joint Simon task-based paradigm. We report the results of three experiments in which we manipulated the availability of stimulus event information into the contralateral space. Our findings demonstrate that participants are able to compensate for the absence of stimulus event information by relying on temporal features of their co-actor's action-effects (Experiment 1). Even more surprising was that participants continued to monitor the temporal features of their co-actor's actions even when given a verbal signal by their co-actor (Experiments 2a), or full access to the common workspace (Experiment 2b). Our results are strong evidence that the representation of actions is not purely stimulus driven. They suggest that the temporal dimension of the other person's actions is able to drive the self-other discrimination process, in the same way as other perceptual dimensions and feature codes that are shared with the stimulus event.

The hierarchical sensitivity to social misalignment during decision-making under uncertainty

Social misalignment occurs when a person’s attitudes and opinions deviate from those of others. We investigated how individuals react to social misalignment in risky (outcome probabilities are known) or ambiguous (outcome probabilities are unknown) decision contexts. During each trial, participants played a forced-choice gamble, and they observed the decisions of four other players after they made a tentative decision, followed by an opportunity to keep or change their initial decision. Behavioral and event-related potential data were collected. Behaviorally, the stronger the participants’ initial preference, the less likely they were to switch their decisions, whereas the more their decisions were misaligned with the majority, the more likely they were to switch. Electrophysiological results showed a hierarchical processing pattern of social misalignment. Misalignment was first detected binarily (i.e. match/mismatch) at an early stage, as indexed by the N1 component. During the second stage, participants became sensitive to low levels of misalignment, which were indexed by the feedback-related negativity. The degree of social misalignment was processed in greater detail, as indexed by the P3 component. Moreover, such hierarchical neural sensitivity is generalizable across different decision contexts (i.e. risky and ambiguous). These findings demonstrate a fine-grained neural sensitivity to social misalignment during decision-making under uncertainty.

Let's Talk About Each Other: Neural Responses to Dissenting Personality Evaluations Based on Real Dyadic Interactions

Dyadic interactions are associated with the exchange of personality-related messages, which can be congruent or incongruent with one’s self-view. In the current preregistered study (N = 52), we investigated event-related potentials (ERPs) toward real social evaluations in order to uncover the neural mechanisms underlying the processing of congruent and incongruent evaluative feedback. Participants interacted first, and then during an electroencephalogram (EEG) session, they received evaluations from their interaction partner that were either congruent or incongruent with their own ratings. Findings show potentiated processing of self-related incongruent negative evaluations at early time points (N1) followed by increased processing of both incongruent negative and positive evaluations at midlatency time windows (early posterior negativity) and a prioritized processing of self-related incongruent positive evaluations at late time points (feedback-related P3, late positive potential). These findings reveal that, after real social interactions, evaluative feedback about oneself that violates one’s self-view modulates all processing stages with an early negativity and a late positivity bias.

Processing and utilization of auditory action effects in individual and social tasks

The influence of action-effect integration on motor control and sensory processing is often investigated in arrangements featuring human-machine interactions. Such experiments focus on predictable sensory events produced through participants' interactions with simple response devices. Action-effect integration may, however, also occur when we interact with human partners. The current study examined the similarities and differences in perceptual and motor control processes related to generating sounds with or without the involvement of a human partner. We manipulated the complexity of the causal chain of events between the initial motor and the final sensory event. In the self-induced condition participants generated sounds directly by pressing a button, while in the interactive condition sounds resulted from a paired reaction-time task, that is, the final sound was generated indirectly, by relying on the contribution of the partner. Auditory event-related potentials (ERPs) and force application patterns were similar in the two conditions, suggesting that social action effects produced with the involvement of a second human agent in the causal sequence are processed, and utilized as action feedback in the same way as direct consequences of one's actions. The only reflection of a processing difference between the two conditions was a slow, posterior ERP waveform that started before the presentation of the auditory stimulus, which may reflect differences in stimulus expectancy or task difficulty.

ERP markers of action planning and outcome monitoring in human - robot interaction

The present study aimed to examine event-related potentials (ERPs) of action planning and outcome monitoring in human-robot interaction. To this end, participants were instructed to perform costly actions (i.e. losing points) to stop a balloon from inflating and to prevent its explosion. They performed the task alone (individual condition) or with a robot (joint condition). Similar to findings from human-human interactions, results showed that action planning was affected by the presence of another agent, robot in this case. Specifically, the early readiness potential (eRP) amplitude was larger in the joint, than in the individual, condition. The presence of the robot affected also outcome perception and monitoring. Our results showed that the P1/N1 complex was suppressed in the joint, compared to the individual condition when the worst outcome was expected, suggesting that the presence of the robot affects attention allocation to negative outcomes of one's own actions. Similarly, results also showed that larger losses elicited smaller feedback-related negativity (FRN) in the joint than in the individual condition. Taken together, our results indicate that the social presence of a robot may influence the way we plan our actions and also the way we monitor their consequences. Implications of the study for the human-robot interaction field are discussed.

Anticipation in sociomotor actions: Similar effects for in- and outgroup interactions

In social interactions, own actions often trigger a particular response from another person. The sociomotor framework proposes that this consistent behavior of others can become incorporated into own action control. In line with this idea, recent studies have shown that own motor actions are facilitated if they are predictably being imitated rather than counterimitated by a social interaction partner. In the present study, we investigated whether this finding is influenced by the relationship between the interacting persons. To that end, we manipulated whether a participant was being imitated and counterimitated by an ingroup or by an outgroup member. In two experiments, we found a beneficial influence of being imitated irrespective of group membership. The results suggest that, while people incorporated their partner's behavior into own action control, this was not further qualified by group membership as a higher-order social variable. This finding points to a universal account of action control for actions with social action effects and actions with inanimate action effects alike.

The Multimodal Go-Nogo Simon Effect: Signifying the Relevance of Stimulus Features in the Go-Nogo Simon Paradigm Impacts Event Representations and Task Performance

Numerous studies have shown that stimulus-response-compatibility (SRC) effects in the go-nogo version of the Simon task can be elicited as a result of performing the task together with another human or non-human agent (e.g., a Japanese-waving-cat, a working-clock, or a ticking-metronome). A parsimonious explanation for both social and non-social SRC effects is that highlighting the spatial significance of alternative (non-/social) action events makes action selection more difficult. This holds even when action events are task-irrelevant. Recent findings, however, suggest that this explanation holds only for cases of a modality correspondence between the Simon task as such (i.e., auditory or visual) and the alternative (non-/social) action event that needs to be discriminated. However, based on the fact that perception and action are represented by the same kind of codes, an event that makes the go-nogo decision more challenging should impact go-nogo Simon task performance. To tackle this issue, the present study tested if alternative stimulus events that come from a different sensory modality do impact SRC effects in the go-nogo version of the Simon task. This was tested in the presence and absence of alternative action events of a human co-actor. In a multimodal (auditory–visual) go-nogo Simon paradigm, participants responded to their assigned stimulus – e.g., a single auditory stimulus while ignoring the alternative visual stimulus or vice versa – in the presence or absence of a human co-actor (i.e., joint and single go-nogo condition). Results showed reliable SRCs in both, single and joint go-nogo Simon task conditions independent of the modality participants had to respond to. Although a correspondence between stimulus material and attention-grabbing event might be an efficient condition for SRCs to emerge, the driving force underlying the emergence of SRCs rather appears to be whether the attentional focus prevents or facilitates alternative events to be integrated. Thus, under task conditions in which the attentional focus is sufficiently broad to enable the integration and thus cognitive representation of alternative events, go-nogo decisions become more difficult, resulting in reliable SRCs in single and joint go-nogo Simon tasks.

Are You Keeping an Eye on Me? The Influence of Competition and Cooperation on Joint Simon Task Performance

Social interaction plays an important role in human life. While there are instances that require cooperation, there are others that force people to compete rather than to cooperate, in order to achieve certain goals. A key question is how the deployment of attention differs between cooperative and competitive situation; however, empirical investigations have yielded inconsistent results. By manipulating the (in-)dependence of individuals via performance-contingent incentives, in a visual go–nogo Simon task the current study aimed at improving our understanding of complementary task performance in a joint action context. In the independent condition each participant received what s/he achieves; in the cooperative condition each participant received the half of what both achieved, and in the competitive condition participants were instructed that the winner takes it all. Extending previous findings, we found sequential processing adjustments of the Simon effect as a function of the interdependency (i.e., competition, cooperation) and transition between (i.e., go–nogo requirements) interacting individuals. While sequential processing adjustments of the Simon effect in both the competition and cooperation condition were unaffected when alternating between responsible actors (i.e., nogo–go transition), sequential processing adjustments were enlarged under competition for repeating responsibilities of one and the same actor (i.e., go–go transitions). In other words, the prospect of performance-contingent reward in a competitive context exclusively impacts flexible behavioral adjustments of one’s own actions. Rather than fostering the consideration and differentiation of the other actor, pushing one’s own performance to the limit appears to be the suitable strategy in competitive instances of complementary tasks. Therefore, people keep their eyes on themselves when aiming at beating a co-actor and emerging as the winner.

Idiosyncratic representation of peripersonal space depends on the success of one's own motor actions, but also the successful actions of others!

Peripersonal space is a multisensory representation of the environment around the body in relation to the motor system, underlying the interactions with the physical and social world. Although changing body properties and social context have been shown to alter the functional processing of space, little is known about how changing the value of objects influences the representation of peripersonal space. In two experiments, we tested the effect of modifying the spatial distribution of reward-yielding targets on manual reaching actions and peripersonal space representation. Before and after performing a target-selection task consisting of manually selecting a set of targets on a touch-screen table, participants performed a two-alternative forced-choice reachability-judgment task. In the target-selection task, half of the targets were associated with a reward (change of colour from grey to green, providing 1 point), the other half being associated with no reward (change of colour from grey to red, providing no point). In Experiment 1, the target-selection task was performed individually with the aim of maximizing the point count, and the distribution of the reward-yielding targets was either 50%, 25% or 75% in the proximal and distal spaces. In Experiment 2, the target-selection task was performed in a social context involving cooperation between two participants to maximize the point count, and the distribution of the reward-yielding targets was 50% in the proximal and distal spaces. Results showed that changing the distribution of the reward-yielding targets or introducing the social context modified concurrently the amplitude of self-generated manual reaching actions and the representation of peripersonal space. Moreover, a decrease of the amplitude of manual reaching actions caused a reduction of peripersonal space when resulting from the distribution of reward-yielding targets, while this effect was not observed in a social interaction context. In that case, the decreased amplitude of manual reaching actions was accompanied by an increase of peripersonal space representation, which was not due to the mere presence of a confederate (control experiment). We conclude that reward-dependent modulation of objects values in the environment modifies the representation of peripersonal space, when resulting from either self-generated motor actions or observation of motor actions performed by a confederate.

It's not me, it's you - Differential neural processing of social and non-social nogo cues in joint action

This study used a joint flanker task to investigate differences in processing of social and non-social nogo cues, i.e., between cues indicating that a co-actor should respond and cues signaling that neither actor nor co-actor should respond, using event-related potentials (ERPs) and trial-to-trial response times (RTs). It was hypothesized that a social co-actor's response should be reflected in stronger modulation (slower RTs on subsequent trials; augmented neural responses) for social compared to non-social nogo. RTs and ERPs replicated flanker compatibility effects, with faster responses and increased P3a on compatible trials. In line with the hypotheses, ERPs revealed distinct coding of social and non-social nogo in the conflict-sensitive N2 which showed a compatibility effect only for social nogo, and in the attention/memory-related P3b which was larger for social relative to non-social nogo. The P3a did not distinguish between social and non-social nogo, but was larger for compatible and smaller for go trials. Contrary to our hypotheses, RTs were faster after social relative to non-social nogo. Hence, the representation of the co-actor's response in joint action modulates conflict processing reflected in the N2 and response discrimination and evaluation reflected in the P3b and may facilitate subsequent responses in the context of social versus non-social nogo.

Whose turn is it anyway? The moderating role of response-execution certainty on the joint Simon effect

When a two-choice "Simon task" is distributed between two people, performance in the shared go/no-go task resembles performance in the whole task alone. This finding has been described as the joint Simon effect (JSE). Unlike the individual go/no-go task, not only is the typical joint Simon task shared with another person, but also the imperative stimuli dictate whose turn it is to respond. Therefore, in the current study, we asked whether removing the agent discrimination component of the joint Simon task influences co-representation. Participants performed the typical joint Simon task, which was compared to two turn-taking versions of the task. For these turn-taking tasks, pairs predictably alternated turns on consecutive trials, with their respective imperative stimulus presented either on 100% of their turns (fully predictable group) or on 83% of their turns (response-uncertainty group, 17% no-go catch trials). The JSE was absent in the fully predictable, turn-taking task, but emerged similarly under the response-uncertainty condition and the typical joint Simon task condition where there is both turn and response-execution-related uncertainty. These results demonstrate that conflict related to agent discrimination is likely not a critical factor driving the JSE, whereas conflict surrounding the need to execute a response (and hence the degree of preparation) appears fundamental to co-representation.

How You Move Is What I See: Planning an Action Biases a Partner's Visual Search

Activating action representations can modulate perceptual processing of action-relevant dimensions, indicative of a common-coding of perception and action. When two or more agents work together in joint action, individual agents often need to consider not only their own actions and their effects on the world, but also predict the actions of a co-acting partner. If in these situations the action of a partner is represented in a functionally equivalent way to the agent's own actions, one may also expect interaction effects between action and perception across jointly acting individuals. The present study investigated whether the action of a co-acting partner may modulate an agent's perception. The "performer" prepared a grasping or pointing movement toward a physical target while the "searcher" performed a visual search task. The performer's planned action impaired the searcher's perceptual performance when the search target dimension was relevant to the performer's movement execution. These results demonstrate an action-induced modulation of perceptual processes across participants and indicate that agents represent their partner's action by employing the same perceptual system they use to represent an own action. We suggest that task representations in joint action operate along multiple levels of a cross-brain predictive coding system, which provides agents with information about a partner's actions when they coordinate to reach a common goal.

Cultural Differences in Perceiving Sounds Generated by Others: Self Matters

Sensory consequences resulting from own movements receive different neural processing compared to externally generated sensory consequences (e.g., by a computer), leading to sensory attenuation, i.e., a reduction in perceived intensity or brain evoked responses. However, discrepant findings exist from different cultural regions about whether sensory attenuation is also present for sensory consequences generated by others. In this study, we performed a cross culture (between Chinese and British) comparison on the processing of sensory consequences (perceived loudness) from self and others compared to an external source in the auditory domain. We found a cultural difference in processing sensory consequences generated by others, with only Chinese and not British showing the sensory attenuation effect. Sensory attenuation in this case was correlated with independent self-construal scores. The sensory attenuation effect for self-generated sensory consequences was not replicated. However, a correlation with delusional ideation was observed for British. These findings are discussed with respects to mechanisms of sensory attenuation.

Motor interference in interactive contexts

Action observation and execution share overlapping neural substrates, so that simultaneous activation by observation and execution modulates motor performance. Previous literature on simple prehension tasks has revealed that motor influence can be two-sided: facilitation for observed and performed congruent actions and interference for incongruent actions. But little is known of the specific modulations of motor performance in complex forms of interaction. Is it possible that the very same observed movement can lead either to interference or facilitation effects on a temporally overlapping congruent executed action, depending on the context? To answer this question participants were asked to perform a reach-to-grasp movement adopting a precision grip (PG) while: (i) observing a fixation cross, (ii) observing an actor performing a PG with interactive purposes, (iii) observing an actor performing a PG without interactive purposes. In particular, in the interactive condition the actor was shown trying to pour some sugar on a large cup located out of her reach but close to the participant watching the video, thus eliciting in reaction a complementary whole-hand grasp. Notably, fine-grained kinematic analysis for this condition revealed a specific delay in the grasping and reaching components and an increased trajectory deviation despite the observed and executed movement’s congruency. Moreover, early peaks of trajectory deviation seem to indicate that socially relevant stimuli are acknowledged by the motor system very early. These data suggest that interactive contexts can determine a prompt modulation of stimulus–response compatibility effects.

It's all in your head - how anticipating evaluation affects the processing of emotional trait adjectives

Language has an intrinsically evaluative and communicative function. Words can serve to describe emotional traits and states in others and communicate evaluations. Using electroencephalography (EEG), we investigate how the cerebral processing of emotional trait adjectives is modulated by their perceived communicative sender in anticipation of an evaluation. 16 students were videotaped while they described themselves. They were told that a stranger would evaluate their personality based on this recording by endorsing trait adjectives. In a control condition a computer program supposedly randomly selected the adjectives. Actually, both conditions were random. A larger parietal N1 was found for adjectives in the supposedly human-generated condition. This indicates that more visual attention is allocated to the presented adjectives when putatively interacting with a human. Between 400 and 700 ms a fronto-central main effect of emotion was found. Positive, and in tendency also negative adjectives, led to a larger late positive potential (LPP) compared to neutral adjectives. A centro-parietal interaction in the LPP-window was due to larger LPP amplitudes for negative compared to neutral adjectives within the ‘human sender’ condition. Larger LPP amplitudes are related to stimulus elaboration and memory consolidation. Participants responded more to emotional content particularly when presented in a meaningful ‘human’ context. This was first observed in the early posterior negativity window (210–260 ms). But the significant interaction between sender and emotion reached only trend-level on post hoc tests. Our results specify differential effects of even implied communicative partners on emotional language processing. They show that anticipating evaluation by a communicative partner alone is sufficient to increase the relevance of particularly emotional adjectives, given a seemingly realistic interactive setting.