Emotional expressions evoke a differential response in the fusiform face area

Phenomena of hypo- and hyperconnectivity in basal ganglia-thalamo-cortical circuits linked to major depression: a 7T fMRI study

Major depressive disorder (MDD) typically manifests itself in depressed affect, anhedonia, low energy, and additional symptoms. Despite its high global prevalence, its pathophysiology still gives rise to questions. Current research places alterations in functional connectivity among MDD's most promising biomarkers. However, given the heterogeneity of previous findings, the use of higher-resolution imaging techniques, like ultra-high field (UHF) fMRI (≥7 Tesla, 7T), may offer greater specificity in delineating fundamental impairments. In this study, 7T UHF fMRI scans were conducted on 31 MDD patients and 27 age-gender matched healthy controls to exploratorily contrast cerebral resting-state functional connectivity patterns between both groups. The CONN toolbox was used to generate functional network connectivity (FNC) analysis based on the region of interest (ROI)-to-ROI correlations in order to enable the identification of clusters of significantly different connections. Correction for multiple comparisons was implemented at the cluster level using a false discovery rate (FDR). The analysis revealed three significant clusters differentiating MDD patients and healthy controls. In Clusters 1 and 2, MDD patients exhibited between-network hypoconnectivity in basal ganglia-cortical pathways as well as hyperconnectivity in thalamo-cortical pathways, including several individual ROI-to-ROI connections. In Cluster 3, they showed increased occipital interhemispheric within-network connectivity. These findings suggest that alterations in basal ganglia-thalamo-cortical circuits play a substantial role in the pathophysiology of MDD. Furthermore, they indicate potential MDD-related deficits relating to a combination of perception (vision, audition, and somatosensation) as well as more complex functions, especially social-emotional processing, modulation, and regulation. It is anticipated that these findings might further inform more accurate clinical procedures for addressing MDD.

Early life adversity is associated with greater similarity in neural representations of ambiguous and threatening stimuli

Exposure to early life adversity (ELA) is hypothesized to sensitize threat-responsive neural circuitry. This may lead individuals to overestimate threat in the face of ambiguity, a cognitive-behavioral phenotype linked to poor mental health. The tendency to process ambiguity as threatening may stem from difficulty distinguishing between ambiguous and threatening stimuli. However, it is unknown how exposure to ELA relates to neural representations of ambiguous and threatening stimuli, or how processing of ambiguity following ELA relates to psychosocial functioning. The current fMRI study examined multivariate representations of threatening and ambiguous social cues in 41 emerging adults (aged 18 to 19 years). Using representational similarity analysis, we assessed neural representations of ambiguous and threatening images within affective neural circuitry and tested whether similarity in these representations varied by ELA exposure. Greater exposure to ELA was associated with greater similarity in neural representations of ambiguous and threatening images. Moreover, individual differences in processing ambiguity related to global functioning, an association that varied as a function of ELA. By evidencing reduced neural differentiation between ambiguous and threatening cues in ELA-exposed emerging adults and linking behavioral responses to ambiguity to psychosocial wellbeing, these findings have important implications for future intervention work in at-risk, ELA-exposed populations.

No influence of emotional expression on size underestimation of upright faces

Faces are a primary means of conveying social information between humans. One important factor modulating the perception of human faces is emotional expression. Face inversion also affects perception, including judgments of emotional expression, possibly through the disruption of configural processing. One intriguing inversion effect is an illusion whereby faces appear to be physically smaller when upright than when inverted. This illusion appears to be highly selective for faces. In this study, we investigated whether the emotional expression of a face (neutral, happy, afraid, and angry) modulates the magnitude of this size illusion. Results showed that for all four expressions, there was a clear bias for inverted stimuli to be judged as larger than upright ones. This demonstrates that there is no influence of emotional expression on the size underestimation of upright faces, a surprising result given that recognition of different emotional expressions is known to be affected unevenly by inversion. Results are discussed considering recent neuroimaging research which used population receptive field (pRF) mapping to investigate the neural mechanisms underlying face perception features and which may provide an explanation for how an upright face appears smaller than an inverted one. Elucidation of this effect would lead to a greater understanding of how humans communicate.

TMS disruption of the lateral prefrontal cortex increases neural activity in the default mode network when naming facial expressions

Recognizing facial expressions is dependent on multiple brain networks specialized for different cognitive functions. In the current study, participants (N = 20) were scanned using functional magnetic resonance imaging (fMRI), while they performed a covert facial expression naming task. Immediately prior to scanning thetaburst transcranial magnetic stimulation (TMS) was delivered over the right lateral prefrontal cortex (PFC), or the vertex control site. A group whole-brain analysis revealed that TMS induced opposite effects in the neural responses across different brain networks. Stimulation of the right PFC (compared to stimulation of the vertex) decreased neural activity in the left lateral PFC but increased neural activity in three nodes of the default mode network (DMN): the right superior frontal gyrus, right angular gyrus and the bilateral middle cingulate gyrus. A region of interest analysis showed that TMS delivered over the right PFC reduced neural activity across all functionally localised face areas (including in the PFC) compared to TMS delivered over the vertex. These results suggest that visually recognizing facial expressions is dependent on the dynamic interaction of the face-processing network and the DMN. Our study also demonstrates the utility of combined TMS/fMRI studies for revealing the dynamic interactions between different functional brain networks.

Dimensional emotions are represented by distinct topographical brain networks

The ability to recognize others' facial emotions has become increasingly important after the COVID-19 pandemic, which causes stressful situations in emotion regulation. Considering the importance of emotion in maintaining a social life, emotion knowledge to perceive and label emotions of oneself and others requires an understanding of affective dimensions, such as emotional valence and emotional arousal. However, limited information is available about whether the behavioral representation of affective dimensions is similar to their neural representation. To explore the relationship between the brain and behavior in the representational geometries of affective dimensions, we constructed a behavioral paradigm in which emotional faces were categorized into geometric spaces along the valence, arousal, and valence and arousal dimensions. Moreover, we compared such representations to neural representations of the faces acquired by functional magnetic resonance imaging. We found that affective dimensions were similarly represented in the behavior and brain. Specifically, behavioral and neural representations of valence were less similar to those of arousal. We also found that valence was represented in the dorsolateral prefrontal cortex, frontal eye fields, precuneus, and early visual cortex, whereas arousal was represented in the cingulate gyrus, middle frontal gyrus, orbitofrontal cortex, fusiform gyrus, and early visual cortex. In conclusion, the current study suggests that dimensional emotions are similarly represented in the behavior and brain and are presented with differential topographical organizations in the brain.

Three weeks of exercise therapy altered brain functional connectivity in fibromyalgia inpatients

Background

Fibromyalgia (FM) is a chronic pain syndrome characterized by widespread pain, tenderness, and fatigue. Patients with FM have no effective medication so far, and their activity of daily living and quality of life are remarkably impaired. Therefore, new therapeutic approaches are awaited. Recently, exercise therapy has been gathering much attention as a promising treatment for FM. However, the underlying mechanisms are not fully understood, particularly, in the central nervous system, including the brain. Therefore, we investigated functional connectivity changes and their relationship with clinical improvement in patients with FM after exercise therapy to investigate the underlying mechanisms in the brain using resting-state fMRI (rs-fMRI) and functional connectivity (FC) analysis.

Methods

Seventeen patients with FM participated in this study. They underwent a 3-week exercise therapy on in-patient basis and a 5-min rs-fMRI scan before and after the exercise therapy. We compared the FC strength of sensorimotor regions and the mesocortico-limbic system between two scans. We also performed a multiple regression analysis to examine the relationship between pre-post differences in FC strength and improvement of patients' clinical symptoms or motor abilities.

Results

Patients with FM showed significant improvement in clinical symptoms and motor abilities. They also showed a significant pre-post difference in FC of the anterior cingulate cortex and a significant correlation between pre-post FC changes and improvement of clinical symptoms and motor abilities. Although sensorimotor regions tended to be related to the improvement of general disease severity and depression, brain regions belonging to the mesocortico-limbic system tended to be related to the improvement of motor abilities.

Conclusion

Our 3-week exercise therapy could ameliorate clinical symptoms and motor abilities of patients with FM, and lead to FC changes in sensorimotor regions and brain regions belonging to the mesocortico-limbic system. Furthermore, these changes were related to improvement of clinical symptoms and motor abilities. Our findings suggest that, as predicted by previous animal studies, spontaneous brain activities modified by exercise therapy, including the mesocortico-limbic system, improve clinical symptoms in patients with FM.

Functional organization of social perception in the human brain

Humans rapidly extract diverse and complex information from ongoing social interactions, but the perceptual and neural organization of the different aspects of social perception remains unresolved. We showed short movie clips with rich social content to 97 healthy participants while their haemodynamic brain activity was measured with fMRI. The clips were annotated moment-to-moment for a large set of social features and 45 of the features were evaluated reliably between annotators. Cluster analysis of the social features revealed that 13 dimensions were sufficient for describing the social perceptual space. Three different analysis methods were used to map the social perceptual processes in the human brain. Regression analysis mapped regional neural response profiles for different social dimensions. Multivariate pattern analysis then established the spatial specificity of the responses and intersubject correlation analysis connected social perceptual processing with neural synchronization. The results revealed a gradient in the processing of social information in the brain. Posterior temporal and occipital regions were broadly tuned to most social dimensions and the classifier revealed that these responses showed spatial specificity for social dimensions; in contrast Heschl gyri and parietal areas were also broadly associated with different social signals, yet the spatial patterns of responses did not differentiate social dimensions. Frontal and subcortical regions responded only to a limited number of social dimensions and the spatial response patterns did not differentiate social dimension. Altogether these results highlight the distributed nature of social processing in the brain.

Dissociation and hierarchy of human visual pathways for simultaneously coding facial identity and expression

Humans have an extraordinary ability to recognize facial expression and identity from a single face simultaneously and effortlessly, however, the underlying neural computation is not well understood. Here, we optimized a multi-task deep neural network to classify facial expression and identity simultaneously. Under various optimization training strategies, the best-performing model consistently showed 'share-separate' organization. The two separate branches of the best-performing model also exhibited distinct abilities to categorize facial expression and identity, and these abilities increased along the facial expression or identity branches toward high layers. By comparing the representational similarities between the best-performing model and functional magnetic resonance imaging (fMRI) responses in the human visual cortex to the same face stimuli, the face-selective posterior superior temporal sulcus (pSTS) in the dorsal visual cortex was significantly correlated with layers in the expression branch of the model, and the anterior inferotemporal cortex (aIT) and anterior fusiform face area (aFFA) in the ventral visual cortex were significantly correlated with layers in the identity branch of the model. Besides, the aFFA and aIT better matched the high layers of the model, while the posterior FFA (pFFA) and occipital facial area (OFA) better matched the middle and early layers of the model, respectively. Overall, our study provides a task-optimization computational model to better understand the neural mechanism underlying face recognition, which suggest that similar to the best-performing model, the human visual system exhibits both dissociated and hierarchical neuroanatomical organization when simultaneously coding facial identity and expression.

Association between fractional amplitude of low-frequency fluctuation (fALFF) and facial emotion recognition ability in first-episode schizophrenia patients: a fMRI study

It was still unclear that the correlation between the resting-state intrinsic activity in brain regions and facial emotion recognition (FER) ability in patients with first-episode schizophrenia (FSZ). Our aim was to analyse the correlation between the fractional amplitude of low-frequency fluctuation (fALFF) and FER ability in FSZ patients. A total of 28 patients with FSZ and 33 healthy controls (HCs) completed visual search tasks for FER ability. Regions of interest (ROIs) related to facial emotion were obtained from a previous meta-analysis. Pearson correlation analysis was performed to understand the correlation between fALFF and FER ability. Our results indicated that the patients performed worse than the HCs in the accuracy performances of happy FER and fearful FER. The previous meta-analysis results showed that the brain regions related to FER included the bilateral amygdala (AMY)/hippocampus (HIP), right fusiform gyrus (FFG), and right supplementary motor area (SMA). Partial correlation analysis showed that the fALFF of the right FFG was associated with high-load fearful FER accuracy (r = - 0.60, p = 0.004). Our study indicated that FER ability is correlated with resting-state intrinsic activity in brain regions related to facial emotion, which may provide a reference for the study of FER deficiency in schizophrenia.

Dynamic influences on the neural encoding of social valence

Social signals can serve as potent emotional triggers with powerful impacts on processes from cognition to valence processing. How are social signals dynamically and flexibly associated with positive or negative valence? How do our past social experiences and present social standing shape our motivation to seek or avoid social contact? We discuss a model in which social attributes, social history, social memory, social rank and social isolation can flexibly influence valence assignment to social stimuli, termed here as 'social valence'. We emphasize how the brain encodes each of these four factors and highlight the neural circuits and mechanisms that play a part in the perception of social attributes, social memory and social rank, as well as how these factors affect valence systems associated with social stimuli. We highlight the impact of social isolation, dissecting the neural and behavioural mechanisms that mediate the effects of acute versus prolonged periods of social isolation. Importantly, we discuss conceptual models that may account for the potential shift in valence of social stimuli from positive to negative as the period of isolation extends in time. Collectively, this Review identifies factors that control the formation and attribution of social valence - integrating diverse areas of research and emphasizing their unique contributions to the categorization of social stimuli as positive or negative.

Neural Mechanisms of Perceiving and Subsequently Recollecting Emotional Facial Expressions in Young and Older Adults

It is known that emotional facial expressions modulate the perception and subsequent recollection of faces and that aging alters these modulatory effects. Yet, the underlying neural mechanisms are not well understood, and they were the focus of the current fMRI study. We scanned healthy young and older adults while perceiving happy, neutral, or angry faces paired with names. Participants were then provided with the names of the faces and asked to recall the facial expression of each face. fMRI analyses focused on the fusiform face area (FFA), the posterior superior temporal sulcus (pSTS), the OFC, the amygdala, and the hippocampus (HC). Univariate activity, multivariate pattern (MVPA), and functional connectivity analyses were performed. The study yielded two main sets of findings. First, in pSTS and the amygdala, univariate activity and MVPA discrimination during the processing of facial expressions were similar in young and older adults, whereas in FFA and OFC, MVPA discriminated facial expressions less accurately in older than young adults. These findings suggest that facial expression representations in FFA and OFC reflect age-related dedifferentiation and positivity effect. Second, HC-OFC connectivity showed subsequent memory effects (SMEs) for happy expressions in both age groups, HC-FFA connectivity exhibited SMEs for happy and neutral expressions in young adults, and HC-pSTS interactions displayed SMEs for happy expressions in older adults. These results could be related to compensatory mechanisms and positivity effects in older adults. Taken together, the results clarify the effects of aging on the neural mechanisms in perceiving and encoding facial expressions.

Medial prefrontal and occipito-temporal activity at encoding determines enhanced recognition of threatening faces after 1.5 years

Studies demonstrated that faces with threatening emotional expressions are better remembered than non-threatening faces. However, whether this memory advantage persists over years and which neural systems underlie such an effect remains unknown. Here, we employed an individual difference approach to examine whether the neural activity during incidental encoding was associated with differential recognition of faces with emotional expressions (angry, fearful, happy, sad and neutral) after a retention interval of > 1.5 years (N = 89). Behaviorally, we found a better recognition for threatening (angry, fearful) versus non-threatening (happy and neutral) faces after a delay of > 1.5 years, which was driven by forgetting of non-threatening faces compared with immediate recognition after encoding. Multivariate principal component analysis (PCA) on the behavioral responses further confirmed the discriminative recognition performance between threatening and non-threatening faces. A voxel-wise whole-brain analysis on the concomitantly acquired functional magnetic resonance imaging (fMRI) data during incidental encoding revealed that neural activity in bilateral inferior occipital gyrus (IOG) and ventromedial prefrontal/orbitofrontal cortex (vmPFC/OFC) was associated with the individual differences in the discriminative emotional face recognition performance measured by an innovative behavioral pattern similarity analysis (BPSA). The left fusiform face area (FFA) was additionally determined using a regionally focused analysis. Overall, the present study provides evidence that threatening facial expressions lead to persistent face recognition over periods of > 1.5 years, and that differential encoding-related activity in the medial prefrontal cortex and occipito-temporal cortex may underlie this effect.

Acquired Prosopagnosia with Structurally Intact and Functional Fusiform Face Area and with Face Identity-Specific Configuration Processing Deficits

Prosopagnosia or loss of face perception and recognition is still poorly understood and rare single cases of acquired prosopagnosia can provide a unique window on the behavioural and brain basis of normal face perception. The present study of a new case of acquired prosopagnosia with bilateral occipito-temporal lesions but a structurally intact FFA and OFA investigated whether the lesion overlapped with the face network and whether the structurally intact FFA showed a face selective response. We also investigated the behavioral correlates of the neural findings and assessed configural processing in the context of facial and non-facial identity recognition, expression recognition and memory, also focusing on the face-selectivity of each specific deficit. The findings reveal a face-selective response in the FFA, despite lesions in the face perception network. At the behavioural level, the results showed impaired configural processing for facial identity, but not for other stimulus categories and not for facial expression recognition. These findings challenge a critical role of the FFA for face identity processing and support a domain-specific account of configural processing.

Resting state functional brain networks associated with emotion processing in frontotemporal lobar degeneration

This study investigated the relationship between emotion processing and resting-state functional connectivity (rs-FC) of the brain networks in frontotemporal lobar degeneration (FTLD). Eighty FTLD patients (including cases with behavioral variant of frontotemporal dementia, primary progressive aphasia, progressive supranuclear palsy syndrome, motor neuron disease) and 65 healthy controls underwent rs-functional MRI. Emotion processing was tested using the Comprehensive Affect Testing System (CATS). In patients and controls, correlations were investigated between each emotion construct and rs-FC changes within critical networks. Mean rs-FC of the clusters significantly associated with CATS scoring were compared among FTLD groups. FTLD patients had pathological CATS scores compared with controls. In controls, increased rs-FC of the cerebellar and visuo-associative networks correlated with better scores in emotion-matching and discrimination tasks, respectively; while decreased rs-FC of the visuo-spatial network was related with better performance in the affect-matching and naming. In FTLD, the associations between rs-FC and CATS scores involved more brain regions, such as orbitofrontal and middle frontal gyri within anterior networks (i.e., salience and default-mode), parietal and somatosensory regions within visuo-spatial and sensorimotor networks, caudate and thalamus within basal-ganglia network. Rs-FC changes associated with CATS were similar among all FTLD groups. In FTLD compared to controls, the pattern of rs-FC associated with emotional processing involves a larger number of brain regions, likely due to functional specificity loss and compensatory attempts. These associations were similar across all FTLD groups, suggesting a common physiopathological mechanism of emotion processing breakdown, regardless the clinical presentation and pattern of atrophy.

The role of stimulus-based cues and conceptual information in processing facial expressions of emotion

Face shape and surface textures are two important cues that aid in the perception of facial expressions of emotion. Additionally, this perception is also influenced by high-level emotion concepts. Across two studies, we use representational similarity analysis to investigate the relative roles of shape, surface, and conceptual information in the perception, categorisation, and neural representation of facial expressions. In Study 1, 50 participants completed a perceptual task designed to measure the perceptual similarity of expression pairs, and a categorical task designed to measure the confusability between expression pairs when assigning emotion labels to a face. We used representational similarity analysis and constructed three models of the similarities between emotions using distinct information. Two models were based on stimulus-based cues (face shapes and surface textures) and one model was based on emotion concepts. Using multiple linear regression, we found that behaviour during both tasks was related with the similarity of emotion concepts. The model based on face shapes was more related with behaviour in the perceptual task than in the categorical, and the model based on surface textures was more related with behaviour in the categorical than the perceptual task. In Study 2, 30 participants viewed facial expressions while undergoing fMRI, allowing for the measurement of brain representational geometries of facial expressions of emotion in three core face-responsive regions (the Fusiform Face Area, Occipital Face Area, and Superior Temporal Sulcus), and a region involved in theory of mind (Medial Prefrontal Cortex). Across all four regions, the representational distances between facial expression pairs were related to the similarities of emotion concepts, but not to either of the stimulus-based cues. Together, these results highlight the important top-down influence of high-level emotion concepts both in behavioural tasks and in the neural representation of facial expressions.

The rapid and automatic categorization of facial expression changes in highly variable natural images

Emotional expressions are quickly and automatically read from human faces under natural viewing conditions. Yet, categorization of facial expressions is typically measured in experimental contexts with homogenous sets of face stimuli. Here we evaluated how the 6 basic facial emotions (Fear, Disgust, Happiness, Anger, Surprise or Sadness) can be rapidly and automatically categorized with faces varying in head orientation, lighting condition, identity, gender, age, ethnic origin and background context. High-density electroencephalography was recorded in 17 participants viewing 50 s sequences with natural variable images of neutral-expression faces alternating at a 6 Hz rate. Every five stimuli (1.2 Hz), variable natural images of one of the six basic expressions were presented. Despite the wide physical variability across images, a significant F/5 = 1.2 Hz response and its harmonics (e.g., 2F/5 = 2.4 Hz, etc.) was observed for all expression changes at the group-level and in every individual participant. Facial categorization responses were found mainly over occipito-temporal sites, with distinct hemispheric lateralization and cortical topographies according to the different expressions. Specifically, a stronger response was found to Sadness categorization, especially over the left hemisphere, as compared to Fear and Happiness, together with a right hemispheric dominance for categorization of Fearful faces. Importantly, these differences were specific to upright faces, ruling out the contribution of low-level visual cues. Overall, these observations point to robust rapid and automatic facial expression categorization processes in the human brain.

Mask exposure during COVID-19 changes emotional face processing

Faces are one of the key ways that we obtain social information about others. They allow people to identify individuals, understand conversational cues, and make judgements about others' mental states. When the COVID-19 pandemic hit the United States, widespread mask-wearing practices were implemented, causing a shift in the way Americans typically interact. This introduction of masks into social exchanges posed a potential challenge-how would people make these important inferences about others when a large source of information was no longer available? We conducted two studies that investigated the impact of mask exposure on emotion perception. In particular, we measured how participants used facial landmarks (visual cues) and the expressed valence and arousal (affective cues), to make similarity judgements about pairs of emotion faces. Study 1 found that in August 2020, participants with higher levels of mask exposure used cues from the eyes to a greater extent when judging emotion similarity than participants with less mask exposure. Study 2 measured participants' emotion perception in both April and September 2020 -before and after widespread mask adoption-in the same group of participants to examine changes in the use of facial cues over time. Results revealed an overall increase in the use of visual cues from April to September. Further, as mask exposure increased, people with the most social interaction showed the largest increase in the use of visual facial cues. These results provide evidence that a shift has occurred in how people process faces such that the more people are interacting with others that are wearing masks, the more they have learned to focus on visual cues from the eye area of the face.

Reexamining the neural network involved in perception of facial expression: A meta-analysis

Perception of facial expression is essential for social interactions. Although a few competing models have enjoyed some success to map brain regions, they are also facing difficult challenges. The current study used an updated activation likelihood estimation (ALE) method of meta-analysis to explore the involvement of brain regions in facial expression processing. The sample contained 96 functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) studies of healthy adults with the results of whole-brain analyses. The key findings revealed that the ventral pathway, especially the left fusiform face area (FFA) region, was more responsive to facial expression. The left posterior FFA showed strong involvement when participants passively viewing emotional faces without being asked to judge the type of expression or other attributes of the stimuli. Through meta-analytic connectivity modeling (MACM) of the main brain regions in the ventral pathway, we constructed a co-activating neural network as a revised model of facial expression processing that assigns prominent roles to the amygdala, FFA, the occipital gyrus, and the inferior frontal gyrus.

Computational approaches to the neuroscience of social perception

Across multiple domains of social perception-including social categorization, emotion perception, impression formation and mentalizing-multivariate pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data has permitted a more detailed understanding of how social information is processed and represented in the brain. As in other neuroimaging fields, the neuroscientific study of social perception initially relied on broad structure-function associations derived from univariate fMRI analysis to map neural regions involved in these processes. In this review, we trace the ways that social neuroscience studies using MVPA have built on these neuroanatomical associations to better characterize the computational relevance of different brain regions, and discuss how MVPA allows explicit tests of the correspondence between psychological models and the neural representation of social information. We also describe current and future advances in methodological approaches to multivariate fMRI data and their theoretical value for the neuroscience of social perception.

Statistical pattern recognition reveals shared neural signatures for displaying and recognizing specific facial expressions

Human neuroimaging and behavioural studies suggest that somatomotor ‘mirroring’ of seen facial expressions may support their recognition. Here we show that viewing specific facial expressions triggers the representation corresponding to that expression in the observer’s brain. Twelve healthy female volunteers underwent two separate fMRI sessions: one where they observed and another where they displayed three types of facial expressions (joy, anger and disgust). Pattern classifier based on Bayesian logistic regression was trained to classify facial expressions (i) within modality (trained and tested with data recorded while observing or displaying expressions) and (ii) between modalities (trained with data recorded while displaying expressions and tested with data recorded while observing the expressions). Cross-modal classification was performed in two ways: with and without functional realignment of the data across observing/displaying conditions. All expressions could be accurately classified within and also across modalities. Brain regions contributing most to cross-modal classification accuracy included primary motor and somatosensory cortices. Functional realignment led to only minor increases in cross-modal classification accuracy for most of the examined ROIs. Substantial improvement was observed in the occipito-ventral components of the core system for facial expression recognition. Altogether these results support the embodied emotion recognition model and show that expression-specific somatomotor neural signatures could support facial expression recognition.

Modality-general and modality-specific audiovisual valence processing

A fundamental question in affective neuroscience is whether there is a common hedonic system for valence processing independent of modality, or there are distinct neural systems for different modalities. To address this question, we used both region of interest and whole-brain representational similarity analyses on functional magnetic resonance imaging data to identify modality-general and modality-specific brain areas involved in valence processing across visual and auditory modalities. First, region of interest analyses showed that the superior temporal cortex was associated with both modality-general and auditory-specific models, while the primary visual cortex was associated with the visual-specific model. Second, the whole-brain searchlight analyses also identified both modality-general and modality-specific representations. The modality-general regions included the superior temporal, medial superior frontal, inferior frontal, precuneus, precentral, postcentral, supramarginal, paracentral lobule and middle cingulate cortices. The modality-specific regions included both perceptual cortices and higher-order brain areas. The valence representations derived from individualized behavioral valence ratings were consistent with these results. Together, these findings suggest both modality-general and modality-specific representations of valence.

Neural processing of facial identity and expression in adults with and without autism: A multi-method approach

The ability to recognize faces and facial expressions is a common human talent. It has, however, been suggested to be impaired in individuals with autism spectrum disorder (ASD). The goal of this study was to compare the processing of facial identity and emotion between individuals with ASD and neurotypicals (NTs). Behavioural and functional magnetic resonance imaging (fMRI) data from 46 young adults (aged 17-23 years, N_ASD = 22, N_NT = 24) was analysed. During fMRI data acquisition, participants discriminated between short clips of a face transitioning from a neutral to an emotional expression. Stimuli included four identities and six emotions. We performed behavioural, univariate, multi-voxel, adaptation and functional connectivity analyses to investigate potential group differences. The ASD-group did not differ from the NT-group on behavioural identity and expression processing tasks. At the neural level, we found no differences in average neural activation, neural activation patterns and neural adaptation to faces in face-related brain regions. In terms of functional connectivity, we found that amygdala seems to be more strongly connected to inferior occipital cortex and V1 in individuals with ASD. Overall, the findings indicate that neural representations of facial identity and expression have a similar quality in individuals with and without ASD, but some regions containing these representations are connected differently in the extended face processing network.

Cross-Subject Commonality of Emotion Representations in Dorsal Motion-Sensitive Areas

Emotion perception is a crucial question in cognitive neuroscience and the underlying neural substrates have been the subject of intense study. One of our previous studies demonstrated that motion-sensitive areas are involved in the perception of facial expressions. However, it remains unclear whether emotions perceived from whole-person stimuli can be decoded from the motion-sensitive areas. In addition, if emotions are represented in the motion-sensitive areas, we may further ask whether the representations of emotions in the motion-sensitive areas can be shared across individual subjects. To address these questions, this study collected neural images while participants viewed emotions (joy, anger, and fear) from videos of whole-person expressions (contained both face and body parts) in a block-design functional magnetic resonance imaging (fMRI) experiment. Multivariate pattern analysis (MVPA) was conducted to explore the emotion decoding performance in individual-defined dorsal motion-sensitive regions of interest (ROIs). Results revealed that emotions could be successfully decoded from motion-sensitive ROIs with statistically significant classification accuracies for three emotions as well as positive versus negative emotions. Moreover, results from the cross-subject classification analysis showed that a person’s emotion representation could be robustly predicted by others’ emotion representations in motion-sensitive areas. Together, these results reveal that emotions are represented in dorsal motion-sensitive areas and that the representation of emotions is consistent across subjects. Our findings provide new evidence of the involvement of motion-sensitive areas in the emotion decoding, and further suggest that there exists a common emotion code in the motion-sensitive areas across individual subjects.

Mapping neural activity patterns to contextualized fearful facial expressions onto callous-unemotional (CU) traits: intersubject representational similarity analysis reveals less variation among high-CU adolescents

Callous-unemotional (CU) traits are early-emerging personality features characterized by deficits in empathy, concern for others, and remorse following social transgressions. One of the interpersonal deficits most consistently associated with CU traits is impaired behavioral and neurophysiological responsiveness to fearful facial expressions. However, the facial expression paradigms traditionally employed in neuroimaging are often ambiguous with respect to the nature of threat (i.e., is the perceiver the threat, or is something else in the environment?). In the present study, 30 adolescents with varying CU traits viewed fearful facial expressions cued to three different contexts ("afraid for you," "afraid of you," "afraid for self") while undergoing functional magnetic resonance imaging (fMRI). Univariate analyses found that mean right amygdala activity during the "afraid for self" context was negatively associated with CU traits. With the goal of disentangling idiosyncratic stimulus-driven neural responses, we employed intersubject representational similarity analysis to link intersubject similarities in multivoxel neural response patterns to contextualized fearful expressions with differential intersubject models of CU traits. Among low-CU adolescents, neural response patterns while viewing fearful faces were most consistently similar early in the visual processing stream and among regions implicated in affective responding, but were more idiosyncratic as emotional face information moved up the cortical processing hierarchy. By contrast, high-CU adolescents' neural response patterns consistently aligned along the entire cortical hierarchy (but diverged among low-CU youths). Observed patterns varied across contexts, suggesting that interpretations of fearful expressions depend to an extent on neural response patterns and are further shaped by levels of CU traits.

Supramodal neural networks support top-down processing of social signals

The perception of facial and vocal stimuli is driven by sensory input and cognitive top‐down influences. Important top‐down influences are attentional focus and supramodal social memory representations. The present study investigated the neural networks underlying these top‐down processes and their role in social stimulus classification. In a neuroimaging study with 45 healthy participants, we employed a social adaptation of the Implicit Association Test. Attentional focus was modified via the classification task, which compared two domains of social perception (emotion and gender), using the exactly same stimulus set. Supramodal memory representations were addressed via congruency of the target categories for the classification of auditory and visual social stimuli (voices and faces). Functional magnetic resonance imaging identified attention‐specific and supramodal networks. Emotion classification networks included bilateral anterior insula, pre‐supplementary motor area, and right inferior frontal gyrus. They were pure attention‐driven and independent from stimulus modality or congruency of the target concepts. No neural contribution of supramodal memory representations could be revealed for emotion classification. In contrast, gender classification relied on supramodal memory representations in rostral anterior cingulate and ventromedial prefrontal cortices. In summary, different domains of social perception involve different top‐down processes which take place in clearly distinguishable neural networks.

Posterior Fusiform and Midfusiform Contribute to Distinct Stages of Facial Expression Processing

Though the fusiform is well-established as a key node in the face perception network, its role in facial expression processing remains unclear, due to competing models and discrepant findings. To help resolve this debate, we recorded from 17 subjects with intracranial electrodes implanted in face sensitive patches of the fusiform. Multivariate classification analysis showed that facial expression information is represented in fusiform activity and in the same regions that represent identity, though with a smaller effect size. Examination of the spatiotemporal dynamics revealed a functional distinction between posterior fusiform and midfusiform expression coding, with posterior fusiform showing an early peak of facial expression sensitivity at around 180 ms after subjects viewed a face and midfusiform showing a later and extended peak between 230 and 460 ms. These results support the hypothesis that the fusiform plays a role in facial expression perception and highlight a qualitative functional distinction between processing in posterior fusiform and midfusiform, with each contributing to temporally segregated stages of expression perception.

Brain Responses to Emotional Infant Faces in New Mothers and Nulliparous Women

The experience of motherhood is one of the most salient events in a woman’s life. Motherhood is associated with a series of neurophysiological, psychological, and behavioral changes that allow women to better adapt to their new role as mothers. Infants communicate their needs and physiological states mainly through salient emotional expressions, and maternal responses to infant signals are critical for infant survival and development. In this study, we investigated the whole brain functional response to emotional infant faces in 20 new mothers and 22 nulliparous women during functional magnetic resonance imaging scans. New mothers showed higher brain activation in regions involved in infant facial expression processing and empathic and mentalizing networks than nulliparous women. Furthermore, magnitudes of the activation of the left parahippocampal gyrus and the left fusiform gyrus, recruited during facial expression processing, were positively correlated with empathic concern (EC) scores in new mothers when viewing emotional (happy-sad) faces contrasted to neutral faces. Taken together, these results indicate that the experience of being a mother affects human brain responses in visual and social cognitive brain areas and in brain areas associated with theory-of-mind related and empathic processing.

Neural Correlates of Evidence and Urgency During Human Perceptual Decision-Making in Dynamically Changing Conditions

Current models of decision-making assume that the brain gradually accumulates evidence and drifts toward a threshold that, once crossed, results in a choice selection. These models have been especially successful in primate research; however, transposing them to human fMRI paradigms has proved it to be challenging. Here, we exploit the face-selective visual system and test whether decoded emotional facial features from multivariate fMRI signals during a dynamic perceptual decision-making task are related to the parameters of computational models of decision-making. We show that trial-by-trial variations in the pattern of neural activity in the fusiform gyrus reflect facial emotional information and modulate drift rates during deliberation. We also observed an inverse-urgency signal based in the caudate nucleus that was independent of sensory information but appeared to slow decisions, particularly when information in the task was ambiguous. Taken together, our results characterize how decision parameters from a computational model (i.e., drift rate and urgency signal) are involved in perceptual decision-making and reflected in the activity of the human brain.

The Preponderant Role of Fusiform Face Area for the Facial Expression Confusion Effect: An MEG Study

Although the recognition of facial expressions seems automatic and effortless, discrimination of expressions can still be error prone. Common errors are often due to visual similarities between some expressions (e.g., fear and surprise). However, little is known about the neural mechanisms underlying such a confusion effect. To address this question, we recorded the magnetoencephalography (MEG) while participants judged facial expressions that were either easily confused with or easily distinguished from other expressions. The results showed that the fusiform face area (FFA), rather than the posterior superior temporal sulcus (pSTS), played a preponderant role in discriminating confusable facial expressions. No difference between high confusion and low confusion conditions was observed on the M170 component in either the FFA or the pSTS, whilst a difference between two conditions started to emerge in the late positive potential (LPP), with the low confusion condition eliciting a larger LPP amplitude in the FFA. In addition, the power of delta was stronger in the time window of LPP component. This confusion effect was reflected in the FFA, which might be associated with the perceptual-to-conceptual shift.

The pontine-driven somatic gaze tract contributes to affective processing in humans

The relevance of subcortical structures for affective processing is not fully understood. Inspired by the gerbil retino-raphe pathway that has been shown to regulate affective behavior and previous human work showing that the pontine region is important for processing emotion, we asked whether well-established tracts in humans traveling between the eye and the brain stem contribute to functions beyond their conventionally understood roles. Here we report neuroimaging findings showing that optic chiasm-brain stem diffusivity predict responses reflecting perceived arousal and valence. Analyses of subsequent task-evoked connectivity further revealed that visual affective processing implicates the brain stem, particularly the pontine region at an early stage of the cascade, projecting to cortico-limbic regions in a feedforward manner. The optimal model implies that all intrinsic connections between the regions of interest are unidirectional and outwards from the pontine region. These findings suggest that affective processing implicates regions outside the cortico-limbic network. The involvement of a phylogenetically older locus in the pons that has consequences in oculomotor control may imply adaptive consequences of affect detection.

Gender-Related Brain Activations during an Emotional Task: An fMRI Study

BACKGROUND

Women have twice the rate of depression and anxiety disorders as men. Some studies suggest that this could be caused by women's greater sensitivity to negative emotions. Few brain imaging studies have compared the brain activity of men to women during a presentation of emotional stimuli. Our objective was to investigate brain activations in men and women during an emotional task. We hypothesized that the pattern of brain activations would differ by gender and valence of the stimuli.

METHODS

We conducted a functional magnetic resonance imaging study in 30 healthy participants (15 men and 15 women). Positive, negative and neutral photographs were presented to the subjects. Participants subjectively rated the valence and intensity of the stimuli.

RESULTS

No significant gender-by-category interaction effect was observed for the intensity or valence of the stimuli. We found that, during the presentation of negative photographs, there was a higher activity in women's right fusiform gyrus compared to men's.

CONCLUSION

Given the involvement of the fusiform gyrus in anxiety disorders, this study yields promising findings in order to better understand women's vulnerability to anxiety disorders.

Network Representations of Facial and Bodily Expressions: Evidence From Multivariate Connectivity Pattern Classification

Emotions can be perceived from both facial and bodily expressions. Our previous study has found the successful decoding of facial expressions based on the functional connectivity (FC) patterns. However, the role of the FC patterns in the recognition of bodily expressions remained unclear, and no neuroimaging studies have adequately addressed the question of whether emotions perceiving from facial and bodily expressions are processed rely upon common or different neural networks. To address this, the present study collected functional magnetic resonance imaging (fMRI) data from a block design experiment with facial and bodily expression videos as stimuli (three emotions: anger, fear, and joy), and conducted multivariate pattern classification analysis based on the estimated FC patterns. We found that in addition to the facial expressions, bodily expressions could also be successfully decoded based on the large-scale FC patterns. The emotion classification accuracies for the facial expressions were higher than that for the bodily expressions. Further contributive FC analysis showed that emotion-discriminative networks were widely distributed in both hemispheres, containing regions that ranged from primary visual areas to higher-level cognitive areas. Moreover, for a particular emotion, discriminative FCs for facial and bodily expressions were distinct. Together, our findings highlight the key role of the FC patterns in the emotion processing, indicating how large-scale FC patterns reconfigure in processing of facial and bodily expressions, and suggest the distributed neural representation for the emotion recognition. Furthermore, our results also suggest that the human brain employs separate network representations for facial and bodily expressions of the same emotions. This study provides new evidence for the network representations for emotion perception and may further our understanding of the potential mechanisms underlying body language emotion recognition.

The Cortical Network of Emotion Regulation: Insights From Advanced EEG-fMRI Integration Analysis

The ability to perceive and regulate emotion is a key component of cognition that is often disrupted by disease. Current neuroimaging studies regarding emotion regulation have implicated a number of cortical regions and identified several EEG features of interest, including the late positive potential and frontal asymmetry. Unfortunately, currently applied methods generally lack in the resolution necessary to capture focal cortical activity and explore the causal interactions between brain regions. In this paper, electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data were simultaneously recorded from 20 subjects undergoing emotion processing and regulation tasks. Cortical activity with high-spatiotemporal resolution and accuracy was reconstructed using a novel multimodal EEG/fMRI integration method. A detailed causal brain network associated with emotion processing and regulation was then identified, and the network changes that facilitate different emotion conditions were investigated. The cortical activity of the ventrolateral prefrontal (VLPFC) and posterior parietal cortices depicted conditionally-sensitive spike and wave patterns evidenced in inter-regional communication. The VLPFC was found to behave as a main network source, with conditionally-specific interactions supporting emotional shifts. The results provide unique insight into the cortical activity that supports emotional perception and regulation, the origins of known EEG phenomena, and the manner in which brain regions coordinate to affect behavior.

How do you perceive threat? It's all in your pattern of brain activity

Whether subtle differences in the emotional context during threat perception can be detected by multi-voxel pattern analysis (MVPA) remains a topic of debate. To investigate this question, we compared the ability of pattern recognition analysis to discriminate between patterns of brain activity to a threatening versus a physically paired neutral stimulus in two different emotional contexts (the stimulus being directed towards or away from the viewer). The directionality of the stimuli is known to be an important factor in activating different defensive responses. Using multiple kernel learning (MKL) classification models, we accurately discriminated patterns of brain activation to threat versus neutral stimuli in the directed towards context but not during the directed away context. Furthermore, we investigated whether it was possible to decode an individual’s subjective threat perception from patterns of whole-brain activity to threatening stimuli in the different emotional contexts using MKL regression models. Interestingly, we were able to accurately predict the subjective threat perception index from the pattern of brain activation to threat only during the directed away context. These results show that subtle differences in the emotional context during threat perception can be detected by MVPA. In the directed towards context, the threat perception was more intense, potentially producing more homogeneous patterns of brain activation across individuals. In the directed away context, the threat perception was relatively less intense and more variable across individuals, enabling the regression model to successfully capture the individual differences and predict the subjective threat perception.

The neural representation of facial-emotion categories reflects conceptual structure

Humans reliably categorize configurations of facial actions into specific emotion categories, leading some to argue that this process is invariant between individuals and cultures. However, growing behavioral evidence suggests that factors such as emotion-concept knowledge may shape the way emotions are visually perceived, leading to variability-rather than universality-in facial-emotion perception. Understanding variability in emotion perception is only emerging, and the neural basis of any impact from the structure of emotion-concept knowledge remains unknown. In a neuroimaging study, we used a representational similarity analysis (RSA) approach to measure the correspondence between the conceptual, perceptual, and neural representational structures of the six emotion categories Anger, Disgust, Fear, Happiness, Sadness, and Surprise. We found that subjects exhibited individual differences in their conceptual structure of emotions, which predicted their own unique perceptual structure. When viewing faces, the representational structure of multivoxel patterns in the right fusiform gyrus was significantly predicted by a subject's unique conceptual structure, even when controlling for potential physical similarity in the faces themselves. Finally, cross-cultural differences in emotion perception were also observed, which could be explained by individual differences in conceptual structure. Our results suggest that the representational structure of emotion expressions in visual face-processing regions may be shaped by idiosyncratic conceptual understanding of emotion categories.

Enhanced Early Visual Responses During Implicit Emotional Faces Processing in Autism Spectrum Disorder

Research on Autism Spectrum Disorder (ASD) has focused on processing of socially-relevant stimuli, such as faces. Nonetheless, before being 'social', faces are visual stimuli. The present magnetoencephalography study investigated the time course of brain activity during an implicit emotional task in visual emotion-related regions in 19 adults with ASD (mean age 26.3 ± 4.4) and 19 typically developed controls (26.4 ± 4). The results confirmed previously-reported differences between groups in brain responses to emotion and a hypo-activation in the ASD group in the right fusiform gyrus around 150 ms. However, the ASD group also presented early enhanced activity in the occipital region. These results support that impaired face processing in ASD might be sustained by atypical responses in primary visual areas.

Behavioral and neural indices of affective coloring for neutral social stimuli

Emotional processing often continues beyond the presentation of emotionally evocative stimuli, which can result in affective biasing or coloring of subsequently encountered events. Here, we describe neural correlates of affective coloring and examine how individual differences in affective style impact the magnitude of affective coloring. We conducted functional magnetic resonance imaging in 117 adults who passively viewed negative, neutral and positive pictures presented 2 s prior to neutral faces. Brain responses to neutral faces were modulated by the valence of preceding pictures, with greater activation for faces following negative (vs positive) pictures in the amygdala, dorsomedial and lateral prefrontal cortex, ventral visual cortices, posterior superior temporal sulcus, and angular gyrus. Three days after the magnetic resonance imaging scan, participants rated their memory and liking of previously encountered neutral faces. Individuals higher in trait positive affect and emotional reappraisal rated faces as more likable when preceded by emotionally arousing (negative or positive) pictures. In addition, greater amygdala responses to neutral faces preceded by positively valenced pictures were associated with greater memory for these faces 3 days later. Collectively, these results reveal individual differences in how emotions spill over onto the processing of unrelated social stimuli, resulting in persistent and affectively biased evaluations of such stimuli.

Cortical responses to dynamic emotional facial expressions generalize across stimuli, and are sensitive to task-relevance, in adults with and without Autism

Individuals with Autism Spectrum Disorders (ASD) report difficulties extracting meaningful information from dynamic and complex social cues, like facial expressions. The nature and mechanisms of these difficulties remain unclear. Here we tested whether that difficulty can be traced to the pattern of activity in "social brain" regions, when viewing dynamic facial expressions. In two studies, adult participants (male and female) watched brief videos of a range of positive and negative facial expressions, while undergoing functional magnetic resonance imaging (Study 1: ASD n = 16, control n = 21; Study 2: ASD n = 22, control n = 30). Patterns of hemodynamic activity differentiated among facial emotional expressions in left and right superior temporal sulcus, fusiform gyrus, and parts of medial prefrontal cortex. In both control participants and high-functioning individuals with ASD, we observed (i) similar responses to emotional valence that generalized across facial expressions and animated social events; (ii) similar flexibility of responses to emotional valence, when manipulating the task-relevance of perceived emotions; and (iii) similar responses to a range of emotions within valence. Altogether, the data indicate that there was little or no group difference in cortical responses to isolated dynamic emotional facial expressions, as measured with fMRI. Difficulties with real-world social communication and social interaction in ASD may instead reflect differences in initiating and maintaining contingent interactions, or in integrating social information over time or context.

Influence of task instructions and stimuli on the neural network of face processing: An ALE meta-analysis

Many neuroimaging studies have investigated the neural correlates of face processing. However, the location of face-preferential regions differs considerably between studies, possibly due to the use of different stimuli or tasks. By using Activation likelihood estimation meta-analyses, we aimed to a) delineate regions consistently involved in face processing and b) to assess the influence of stimuli and task on convergence of activation patterns. In total, we included 77 neuroimaging experiments in healthy subjects comparing face processing to a control condition. Results revealed a core face-processing network encompassing bilateral fusiform gyrus (FFG), inferior occipital (IOG) gyrus, superior temporal sulcus/middle temporal gyrus (STS/MTG), amygdala, inferior frontal junction (IFJ) and gyrus (IFG), left anterior insula as well as pre-supplementary motor area (pre-SMA). Furthermore, separate meta-analyses showed, that while significant convergence across all task and stimuli conditions was found in bilateral amygdala, right IOG, right mid-FFG, and right IFG, convergence in IFJ, STS/MTG, right posterior FFG, left FFG and pre-SMA differed between conditions. Thus, our results point to an occipito-frontal-amygdalae system that is involved regardless of stimulus and attention, whereas the remaining regions of the face-processing network are influenced by the task-dependent focus on specific facial characteristics as well as the type of stimuli processed.

Multivariate Pattern Classification of Facial Expressions Based on Large-Scale Functional Connectivity

It is an important question how human beings achieve efficient recognition of others’ facial expressions in cognitive neuroscience, and it has been identified that specific cortical regions show preferential activation to facial expressions in previous studies. However, the potential contributions of the connectivity patterns in the processing of facial expressions remained unclear. The present functional magnetic resonance imaging (fMRI) study explored whether facial expressions could be decoded from the functional connectivity (FC) patterns using multivariate pattern analysis combined with machine learning algorithms (fcMVPA). We employed a block design experiment and collected neural activities while participants viewed facial expressions of six basic emotions (anger, disgust, fear, joy, sadness, and surprise). Both static and dynamic expression stimuli were included in our study. A behavioral experiment after scanning confirmed the validity of the facial stimuli presented during the fMRI experiment with classification accuracies and emotional intensities. We obtained whole-brain FC patterns for each facial expression and found that both static and dynamic facial expressions could be successfully decoded from the FC patterns. Moreover, we identified the expression-discriminative networks for the static and dynamic facial expressions, which span beyond the conventional face-selective areas. Overall, these results reveal that large-scale FC patterns may also contain rich expression information to accurately decode facial expressions, suggesting a novel mechanism, which includes general interactions between distributed brain regions, and that contributes to the human facial expression recognition.

Task-dependent enhancement of facial expression and identity representations in human cortex

What cortical mechanisms allow humans to easily discern the expression or identity of a face? Subjects detected changes in expression or identity of a stream of dynamic faces while we measured BOLD responses from topographically and functionally defined areas throughout the visual hierarchy. Responses in dorsal areas increased during the expression task, whereas responses in ventral areas increased during the identity task, consistent with previous studies. Similar to ventral areas, early visual areas showed increased activity during the identity task. If visual responses are weighted by perceptual mechanisms according to their magnitude, these increased responses would lead to improved attentional selection of the task-appropriate facial aspect. Alternatively, increased responses could be a signature of a sensitivity enhancement mechanism that improves representations of the attended facial aspect. Consistent with the latter sensitivity enhancement mechanism, attending to expression led to enhanced decoding of exemplars of expression both in early visual and dorsal areas relative to attending identity. Similarly, decoding identity exemplars when attending to identity was improved in dorsal and ventral areas. We conclude that attending to expression or identity of dynamic faces is associated with increased selectivity in representations consistent with sensitivity enhancement.

Linear Representation of Emotions in Whole Persons by Combining Facial and Bodily Expressions in the Extrastriate Body Area

Our human brain can rapidly and effortlessly perceive a person’s emotional state by integrating the isolated emotional faces and bodies into a whole. Behavioral studies have suggested that the human brain encodes whole persons in a holistic rather than part-based manner. Neuroimaging studies have also shown that body-selective areas prefer whole persons to the sum of their parts. The body-selective areas played a crucial role in representing the relationships between emotions expressed by different parts. However, it remains unclear in which regions the perception of whole persons is represented by a combination of faces and bodies, and to what extent the combination can be influenced by the whole person’s emotions. In the present study, functional magnetic resonance imaging data were collected when participants performed an emotion distinction task. Multi-voxel pattern analysis was conducted to examine how the whole person-evoked responses were associated with the face- and body-evoked responses in several specific brain areas. We found that in the extrastriate body area (EBA), the whole person patterns were most closely correlated with weighted sums of face and body patterns, using different weights for happy expressions but equal weights for angry and fearful ones. These results were unique for the EBA. Our findings tentatively support the idea that the whole person patterns are represented in a part-based manner in the EBA, and modulated by emotions. These data will further our understanding of the neural mechanism underlying perceiving emotional persons.