Engagement of amygdala in third-person view of face-to-face interaction

Neural Representations of Observed Interpersonal Synchrony/Asynchrony in the Social Perception Network

The visual perception of individuals is thought to be mediated by a network of regions in the occipitotemporal cortex that supports specialized processing of faces, bodies, and actions. In comparison, we know relatively little about the neural mechanisms that support the perception of multiple individuals and the interactions between them. The present study sought to elucidate the visual processing of social interactions by identifying which regions of the social perception network represent interpersonal synchrony. In an fMRI study with 32 human participants (26 female, 6 male), we used multivoxel pattern analysis to investigate whether activity in face-selective, body-selective, and interaction-sensitive regions across the social perception network supports the decoding of synchronous versus asynchronous head-nodding and head-shaking. Several regions were found to support significant decoding of synchrony/asynchrony, including extrastriate body area (EBA), face-selective and interaction-sensitive mid/posterior right superior temporal sulcus, and occipital face area. We also saw robust cross-classification across actions in the EBA, suggestive of movement-invariant representations of synchrony/asynchrony. Exploratory whole-brain analyses also identified a region of the right fusiform cortex that responded more strongly to synchronous than to asynchronous motion. Critically, perceiving interpersonal synchrony/asynchrony requires the simultaneous extraction and integration of dynamic information from more than one person. Hence, the representation of synchrony/asynchrony cannot be attributed to augmented or additive processing of individual actors. Our findings therefore provide important new evidence that social interactions recruit dedicated visual processing within the social perception network that extends beyond that engaged by the faces and bodies of the constituent individuals.

The role of motion in the neural representation of social interactions in the posterior temporal cortex

Humans are an inherently social species, with multiple focal brain regions sensitive to various visual social cues such as faces, bodies, and biological motion. More recently, research has begun to investigate how the brain responds to more complex, naturalistic social scenes, identifying a region in the posterior superior temporal sulcus (SI-pSTS; i.e., social interaction pSTS), amongst others, as an important region for processing social interaction. This research, however, has presented images or videos, and thus the contribution of motion to social interaction perception in these brain regions is not yet understood. In the current study, 22 participants viewed videos, image sequences, scrambled image sequences and static images of either social interactions or non-social independent actions. Combining univariate and multivariate analyses, we confirm that bilateral SI-pSTS plays a central role in dynamic social interaction perception but is much less involved when 'interactiveness' is conveyed solely with static cues. Regions in the social brain, including SI-pSTS and extrastriate body area (EBA), showed sensitivity to both motion and interactive content. While SI-pSTS is somewhat more tuned to video interactions than is EBA, both bilateral SI-pSTS and EBA showed a greater response to social interactions compared to non-interactions and both regions responded more strongly to videos than static images. Indeed, both regions showed higher responses to interactions than independent actions in videos and intact sequences, but not in other conditions. Exploratory multivariate regression analyses suggest that selectivity for simple visual motion does not in itself drive interactive sensitivity in either SI-pSTS or EBA. Rather, selectivity for interactions expressed in point-light animations, and selectivity for static images of bodies, make positive and independent contributions to this effect across the LOTC region. Our results strongly suggest that EBA and SI-pSTS work together during dynamic interaction perception, at least when interactive information is conveyed primarily via body information. As such, our results are also in line with proposals of a third visual stream supporting dynamic social scene perception.

Seeing things differently: Gaze shapes neural signal during mentalizing according to emotional awareness

Studies on social cognition often use complex visual stimuli to asses neural processes attributed to abilities like "mentalizing" or "Theory of Mind" (ToM). During the processing of these stimuli, eye gaze, however, shapes neural signal patterns. Individual differences in neural operations on social cognition may therefore be obscured if individuals' gaze behavior differs systematically. These obstacles can be overcome by the combined analysis of neural signal and natural viewing behavior. Here, we combined functional magnetic resonance imaging (fMRI) with eye-tracking to examine effects of unconstrained gaze on neural ToM processes in healthy individuals with differing levels of emotional awareness, i.e. alexithymia. First, as previously described for emotional tasks, people with higher alexithymia levels look less at eyes in both ToM and task-free viewing contexts. Further, we find that neural ToM processes are not affected by individual differences in alexithymia per se. Instead, depending on alexithymia levels, gaze on critical stimulus aspects reversely shapes the signal in medial prefrontal cortex (MPFC) and anterior temporoparietal junction (TPJ) as distinct nodes of the ToM system. These results emphasize that natural selective attention affects fMRI patterns well beyond the visual system. Our study implies that, whenever using a task with multiple degrees of freedom in scan paths, ignoring the latter might obscure important conclusions.

Fronto-temporal brain activity and connectivity track implicit attention to positive and negative social words in a novel socio-emotional Stroop task

Previous inconsistencies on the effects of implicitly processing positively - vs. negatively - connotated emotional words might reflect the influence of uncontrolled psycholinguistic dimensions, and/or social facets inherent in putative "emotional" stimuli. Based on the relevance of social features in semantic cognition, we developed a socio-emotional Stroop task to assess the influence of social vs. individual (non-social) emotional content, besides negative vs. positive valence, on implicit word processing. The effect of these variables was evaluated in terms of performance and RTs, alongside associated brain activity/connectivity. We matched conditions for several psycholinguistic variables, and assessed a modulation of brain activity/connectivity by trial-wise RT, to characterize the maximum of condition- and subject-specific variability. RTs were tracked by insular and anterior cingulate activations likely reflecting implicit attention to stimuli, interfering with task-performance based on condition-specific processing of their subjective salience. Slower performance for negative than neutral/positive words was tracked by left-hemispheric structures processing negative stimuli and emotions, such as fronto-insular cortex, while the lack of specific activations for positively-connotated words supported their marginal facilitatory effect. The speeding/slowing effects of processing positive/negative individual emotional stimuli were enhanced by social words, reflecting in specific activations of the right anterior temporal and orbitofrontal cortex, respectively. RTs to social positive and negative words modulated connectivity from these regions to fronto-striatal and sensorimotor structures, respectively, likely promoting approach vs. avoidance dispositions shaping their facilitatory vs. inhibitory effect. These results might help assessing the neural correlates of impaired social cognition and emotional regulation, and the effects of rehabilitative interventions.

Sex Differences in Neural Responses to the Perception of Social Interactions

Social interaction is critical to emotional well-being. Previous studies have suggested sex differences in the perception of social interaction. However, the findings depend on the nature of interactions and whether it involves facial emotions. Here, we explored sex differences in neural responses to the perception of social interaction using the Human Connectome Project data. Participants (n = 969, 505 women) were engaged in a social cognition task with geometric objects moving and colliding to simulate social interaction. Behaviorally, men relative to women demonstrated higher accuracy in perceiving social vs. random interactions. Men vs. women showed higher activation in the right superior temporal gyrus, bilateral occipital and posterior cingulate cortex and precuneus, and women vs. men showed higher activation in the right inferior frontal cortex, during exposure to social vs. random interactions. In whole-brain regressions, the differences in accuracy rate in identifying social vs. random interactions (AR SOC - AR RAN ) were associated with higher activation in the paracentral lobule (PCL) and lower activation in bilateral anterior insula (AI), pre-supplementary motor area (preSMA), and left middle frontal gyrus (MFG) in men and women combined, lower activation in bilateral AI, preSMA and left MFG in men alone, and higher activation in the PCL and the medial orbitofrontal cortex in women alone. The latter sex differences were confirmed by slope tests. Further, the PCL activity mediated the correlation between an internalizing syndromal score, as assessed by the Achenbach Self-Report, and (AR SOC - AR RAN ) across all subjects. These findings highlighted sex differences in the behavioral and neural processes underlying the perception of social interaction, as well as the influence of internalizing traits on these processes.

Developmental changes in visual responses to social interactions

•

Children show less interaction selectivity in the pSTS than adults.

•

Adults show bilateral pSTS selectivity, while children are more right-lateralized.

•

Exploratory findings suggest interaction selectivity in pSTS is more focally tuned in adults.

Recent evidence demonstrates that a region of the posterior superior temporal sulcus (pSTS) is selective to visually observed social interactions in adults. In contrast, little is known about neural responses to social interactions in children. Here, we used fMRI to ask whether the pSTS is ‘tuned’ to social interactions in children at all, and if so, how selectivity might differ from adults. This was investigated in the pSTS, along with several other socially-tuned regions in neighbouring temporal cortex: extrastriate body area, face selective STS, fusiform face area, and mentalizing selective temporo-parietal junction.

Both children and adults showed selectivity to social interaction within right pSTS, while only adults showed selectivity on the left. Adults also showed both more focal and greater selectivity than children (6–12 years) bilaterally. Exploratory sub-group analyses showed that younger children (6–8), but not older children (9–12), are less selective than adults on the right, while there was a continuous developmental trend (adults > older > younger) in left pSTS. These results suggest that, over development, the neural response to social interactions is characterized by increasingly more selective, focal, and bilateral pSTS responses, a process that likely continues into adolescence.

Electrophysiological Correlates of Racial In-group Bias in Observing Nonverbal Social Encounters

Despite evidence identifying the role of group membership in social cognition, the neural mechanisms associated with the perception and evaluation of nonverbal behaviors displayed by in-group versus out-group members remain unclear. Here, 42 white participants underwent electroencephalographic recording while observing social encounters involving dynamic displays of nonverbal behaviors by racial in-group and out-group avatar characters. Dynamic behaviors included approach and avoidance poses and expressions, followed by the participants' ratings of the avatars displaying them. Behaviorally, participants showed longer RTs when evaluating in-group approach behavior compared with other behaviors, possibly suggesting increased interest and attention devoted to processing positive social encounters with their in-group members. Analyses of ERPs revealed differential sensitivity of the N450 and late positivity components to social cues, with the former showing initial sensitivity to the presence of a humanoid avatar character at the beginning of social encounters and the latter showing sensitivity to dynamic nonverbal behaviors displayed by the avatars. Moreover, time-frequency analysis of electroencephalography data also identified suppression of beta-range power linked to the observation of dynamic nonverbal behaviors. Notably, the magnitude of these responses was modulated by the degree of behavioral racial in-group bias. This suggests that differential neural sensitivity to nonverbal cues while observing social encounters is associated with subsequent in-group bias manifested in the evaluation of such encounters. Collectively, these findings shed light on the mechanisms of racial in-group bias in social cognition and have implications for understanding factors related to successful interactions with individuals from diverse racial backgrounds.

Epilepsy, language, and social skills

Language and social skills are essential for intrapersonal and interpersonal functioning and quality of life. Since epilepsy impacts these important domains of individuals' functioning, understanding the psychosocial and biological factors involved in the relationship among epilepsy, language, and social skills has important theoretical and clinical implications. This review first describes the psychosocial and biological factors involved in the association between language and social behavior in children and in adults and their relevance for epilepsy. It reviews the findings of studies of social skills and the few studies conducted on the inter-relationship of language and social skills in pediatric and adult epilepsy. The paper concludes with suggested future research and clinical directions that will enhance early identification and treatment of epilepsy patients at risk for impaired language and social skills.

Neural processing of social interaction: Coordinate-based meta-analytic evidence from human neuroimaging studies

While the action observation and mentalizing networks are considered to play complementary roles in understanding others' goals and intentions, they might be concurrently engaged when processing social interactions. We assessed this hypothesis via three activation-likelihood-estimation meta-analyses of neuroimaging studies on the neural processing of: (a) social interactions, (b) individual actions by the action observation network, and (c) mental states by the mentalizing network. Conjunction analyses and direct comparisons unveiled overlapping and specific regions among the resulting maps. We report quantitative meta-analytic evidence for a "social interaction network" including key nodes of the action observation and mentalizing networks. An action-social interaction-mentalizing gradient of activity along the posterior temporal cortex highlighted a hierarchical processing of interactions, from visuomotor analyses decoding individual and shared intentions to in-depth inferences on actors' intentional states. The medial prefrontal cortex, possibly in conjunction with the amygdala, might provide additional information concerning the affective valence of the interaction. This evidence suggests that the functional architecture underlying the neural processing of interactions involves the joint involvement of the action observation and mentalizing networks. These data might inform the design of rehabilitative treatments for social cognition disorders in pathological conditions, and the assessment of their outcome in randomized controlled trials.

Neural Correlates of Racial Ingroup Bias in Observing Computer-Animated Social Encounters

Despite evidence for the role of group membership in the neural correlates of social cognition, the mechanisms associated with processing non-verbal behaviors displayed by racially ingroup vs. outgroup members remain unclear. Here, 20 Caucasian participants underwent fMRI recording while observing social encounters with ingroup and outgroup characters displaying dynamic and static non-verbal behaviors. Dynamic behaviors included approach and avoidance behaviors, preceded or not by a handshake; both dynamic and static behaviors were followed by participants’ ratings. Behaviorally, participants showed bias toward their ingroup members, demonstrated by faster/slower reaction times for evaluating ingroup static/approach behaviors, respectively. At the neural level, despite overall similar responses in the action observation network to ingroup and outgroup encounters, the medial prefrontal cortex showed dissociable activation, possibly reflecting spontaneous processing of ingroup static behaviors and positive evaluations of ingroup approach behaviors. The anterior cingulate and superior frontal cortices also showed sensitivity to race, reflected in coordinated and reduced activation for observing ingroup static behaviors. Finally, the posterior superior temporal sulcus showed uniquely increased activity to observing ingroup handshakes. These findings shed light on the mechanisms of racial ingroup bias in observing social encounters, and have implications for understanding factors related to successful interactions with individuals from diverse backgrounds.

Affective and cooperative social interactions modulate effective connectivity within and between the mirror and mentalizing systems

Decoding the meaning of others' actions, a crucial step for social cognition, involves different neural mechanisms. While the "mirror" and "mentalizing" systems have been associated with, respectively, the processing of biological actions versus more abstract information, their respective contribution to intention understanding is debated. Processing social interactions seems to recruit both neural systems, with a different weight depending on cues emphasizing either shared action goals or shared mental states. We have previously shown that observing cooperative and affective social interactions elicits stronger activity in key nodes of, respectively, the mirror (left posterior superior temporal sulcus (pSTS), superior parietal cortex (SPL), and ventral/dorsal premotor cortex (vPMC/dPMC)) and mentalizing (ventromedial prefrontal cortex (vmPFC)) systems. To unveil their causal organization, we investigated the effective connectivity underlying the observation of human social interactions expressing increasing cooperativity (involving left pSTS, SPL, and vPMC) versus affectivity (vmPFC) via dynamic causal modeling in 36 healthy human subjects. We found strong evidence for a model including the pSTS and vPMC as input nodes for the observed interactions. The extrinsic connectivity of this model undergoes oppositely valenced modulations, with cooperativity promoting positive modulations of connectivity between pSTS and both SPL (forward) and vPMC (mainly backward), and affectivity promoting reciprocal positive modulations of connectivity between pSTS and vmPFC (mainly backward). Alongside fMRI data, such divergent effective connectivity suggests that different dimensions underlying the processing of social interactions recruit distinct, although strongly interconnected, neural pathways associated with, respectively, the bottom-up visuomotor processing of motor intentions, and the top-down attribution of affective/mental states.

The Emerging Science of People-Watching: Forming Impressions From Third-Party Encounters

Traditional impression formation studies have focused almost exclusively on the perception and evaluation of isolated individuals. In recent years, however, portrayals of third-party encounters between two (or more) people have been used increasingly often to probe impressions about the interactions and relations between individuals. This tacit paradigm change has revealed an intriguing scope of judgments that concern how and why people relate to one another. Though these judgments recruit well-known neural networks of impression formation, their underlying cognitive operations and functional significance remain largely speculative. By providing an overview of recent theoretical and empirical approaches on encounter-based impressions, this article highlights their prevalent role in human social cognition.

Systematic Comparison of Brain Imaging Meta-Analyses of ToM with vPT

In visual perspective taking (vPT) one has to concern oneself with what other people see and how they see it. Since seeing is a mental state, developmental studies have discussed vPT within the domain of “theory of mind (ToM)” but imaging studies have not treated it as such. Based on earlier results from several meta-analyses, we tested for the overlap of visual perspective taking studies with 6 different kinds of ToM studies: false belief, trait judgments, strategic games, social animations, mind in the eyes, and rational actions. Joint activation was observed between the vPT task and some kinds of ToM tasks in regions involving the left temporoparietal junction (TPJ), anterior precuneus, left middle occipital gyrus/extrastriate body area (EBA), and the left inferior frontal and precentral gyrus. Importantly, no overlap activation was found for the vPT tasks with the joint core of all six kinds of ToM tasks. This raises the important question of what the common denominator of all tasks that fall under the label of “theory of mind” is supposed to be if visual perspective taking is not one of them.

Personality traits predict brain activation and connectivity when witnessing a violent conflict

As observers we excel in decoding the emotional signals telling us that a social interaction is turning violent. The neural substrate and its modulation by personality traits remain ill understood. We performed an fMRI experiment in which participants watched videos displaying a violent conflict between two people. Observers’ attention was directed to either the aggressor or the victim. Focusing on the aggressor (vs. focusing on the victim) activated the superior temporal sulcus (STS), extra-striate body area (EBA), occipital poles and centro-medial amygdala (CMA). Stronger instantaneous connectivity occurred between these and the EBA, insula, and the red nucleus. When focusing on the victim, basolateral amygdala (BLA) activation was related to trait empathy and showed increased connectivity with the insula and red nucleus. STS activation was associated with trait aggression and increased connectivity with the hypothalamus. The findings reveal that focusing on the aggressor of a violent conflict triggers more activation in categorical (EBA) and emotion (CMA, STS) areas. This is associated with increased instantaneous connectivity among emotion areas (CMA-insula) and between categorical and emotion (EBA-STS) areas. When the focus is on the victim, personality traits (aggression/empathy) modulate activity in emotion areas (respectively STS and postcentral gyrus/ BLA), along with connectivity in the emotional diencephalon (hypothalamus) and early visual areas (occipital pole).

In our own image? Emotional and neural processing differences when observing human-human vs human-robot interactions

Notwithstanding the significant role that human-robot interactions (HRI) will play in the near future, limited research has explored the neural correlates of feeling eerie in response to social robots. To address this empirical lacuna, the current investigation examined brain activity using functional magnetic resonance imaging while a group of participants (n = 26) viewed a series of human-human interactions (HHI) and HRI. Although brain sites constituting the mentalizing network were found to respond to both types of interactions, systematic neural variation across sites signaled diverging social-cognitive strategies during HHI and HRI processing. Specifically, HHI elicited increased activity in the left temporal-parietal junction indicative of situation-specific mental state attributions, whereas HRI recruited the precuneus and the ventromedial prefrontal cortex (VMPFC) suggestive of script-based social reasoning. Activity in the VMPFC also tracked feelings of eeriness towards HRI in a parametric manner, revealing a potential neural correlate for a phenomenon known as the uncanny valley. By demonstrating how understanding social interactions depends on the kind of agents involved, this study highlights pivotal sub-routes of impression formation and identifies prominent challenges in the use of humanoid robots.

From personal fear to mass panic: The neurological basis of crowd perception

Recent studies have investigated the neural correlates of how we perceive emotions of individuals or a group of individuals using images of individual bodily expressions. However, it is still largely unknown how we perceive the emotion of a dynamic crowd. This fMRI study used realistic videos of a large group of people expressing fearful, happy or neutral emotions. Furthermore, the emotions were expressed by either unrelated individuals in the group or by an interacting group. It was hypothesized that the dynamics between the people in a crowd is a more salient signal than merely the emotion of the crowd. Second, it was expected that the group interaction is of special importance in a fearful or "panic" situation, as opposed to a happy or neutral situation. Using a fast-event related design, it was revealed that observing interactive individuals, more so than independently expressive individuals, activated networks related to the perception, execution and integration of action and emotion. Most importantly, the interactive or panicked crowds, as opposed to the individually fearful crowds, triggered more anticipatory and action preparation activity, whereas the brain was less sensitive to the dynamics of individuals in a happy or neutral crowd. This is the first study to assess the effect of the dynamics between people and the collectively displayed emotion as an important aspect of emotional crowd perception.

Look at those two!: The precuneus role in unattended third-person perspective of social interactions

Human beings often observe other people's social interactions without being a part of them. Whereas the implications of some brain regions (e.g. amygdala) have been extensively examined, the implication of the precuneus remains yet to be determined. Here we examined the implication of the precuneus in third-person perspective of social interaction using functional magnetic resonance imaging (fMRI). Participants performed a socially irrelevant task while watching the biological motion of two agents acting in either typical (congruent to social conventions) or atypical (incongruent to social conventions) ways. When compared to typical displays, the atypical displays elicited greater activation in the central and posterior bilateral precuneus, and in frontoparietal and occipital regions. Whereas the right precuneus responded with greater activation also to upside down than upright displays, the left precuneus did not. Correlations and effective connectivity analysis added consistent evidence of an interhemispheric asymmetry between the right and left precuneus. These findings suggest that the precuneus reacts to violations of social expectations, and plays a crucial role in third-person perspective of others' interaction even when the social context is unattended.

Perceiving nonverbal behavior: neural correlates of processing movement fluency and contingency in dyadic interactions

Despite the fact that nonverbal dyadic social interactions are abundant in the environment, the neural mechanisms underlying their processing are not yet fully understood. Research in the field of social neuroscience has suggested that two neural networks appear to be involved in social understanding: (1) the action observation network (AON) and (2) the social neural network (SNN). The aim of this study was to determine the differential contributions of the AON and the SNN to the processing of nonverbal behavior as observed in dyadic social interactions. To this end, we used short computer animation sequences displaying dyadic social interactions between two virtual characters and systematically manipulated two key features of movement activity, which are known to influence the perception of meaning in nonverbal stimuli: (1) movement fluency and (2) contingency of movement patterns. A group of 21 male participants rated the "naturalness" of the observed scenes on a four-point scale while undergoing fMRI. Behavioral results showed that both fluency and contingency significantly influenced the "naturalness" experience of the presented animations. Neurally, the AON was preferentially engaged when processing contingent movement patterns, but did not discriminate between different degrees of movement fluency. In contrast, regions of the SNN were engaged more strongly when observing dyads with disturbed movement fluency. In conclusion, while the AON is involved in the general processing of contingent social actions, irrespective of their kinematic properties, the SNN is preferentially recruited when atypical kinematic properties prompt inferences about the agents' intentions.

The Power of a Handshake: Neural Correlates of Evaluative Judgments in Observed Social Interactions

Effective social interactions require the ability to evaluate other people's actions and intentions, sometimes only on the basis of such subtle factors as body language, and these evaluative judgments may lead to powerful impressions. However, little is known about the impact of affective body language on evaluative responses in social settings and the associated neural correlates. This study investigated the neural correlates of observing social interactions in a business setting, in which whole-body dynamic stimuli displayed approach and avoidance behaviors that were preceded or not by a handshake and were followed by participants' ratings of these behaviors. First, approach was associated with more positive evaluations than avoidance behaviors, and a handshake preceding social interaction enhanced the positive impact of approach and diminished the negative impact of avoidance behavior on the evaluation of social interaction. Second, increased sensitivity to approach than to avoidance behavior in the amygdala and STS was linked to a positive evaluation of approach behavior and a positive impact of handshake. Third, linked to the positive effect of handshake on social evaluation, nucleus accumbens showed greater activity for Handshake than for No-handshake conditions. These findings shed light on the neural correlates of observing and evaluating nonverbal social interactions and on the role of handshake as a way of formal greeting.

Dog experts' brains distinguish socially relevant body postures similarly in dogs and humans

We read conspecifics' social cues effortlessly, but little is known about our abilities to understand social gestures of other species. To investigate the neural underpinnings of such skills, we used functional magnetic resonance imaging to study the brain activity of experts and non-experts of dog behavior while they observed humans or dogs either interacting with, or facing away from a conspecific. The posterior superior temporal sulcus (pSTS) of both subject groups dissociated humans facing toward each other from humans facing away, and in dog experts, a distinction also occurred for dogs facing toward vs. away in a bilateral area extending from the pSTS to the inferior temporo-occipital cortex: the dissociation of dog behavior was significantly stronger in expert than control group. Furthermore, the control group had stronger pSTS responses to humans than dogs facing toward a conspecific, whereas in dog experts, the responses were of similar magnitude. These findings suggest that dog experts' brains distinguish socially relevant body postures similarly in dogs and humans.