Sixteen facial expressions occur in similar contexts worldwide

Observer-generated maps of diagnostic facial features enable categorization and prediction of emotion expressions

According to one prominent model, facial expressions of emotion can be categorized into depicting happiness, disgust, anger, sadness, fear and surprise. One open question is which facial features observers use to recognize the different expressions and whether the features indicated by observers can be used to predict which expression they saw. We created fine-grained maps of diagnostic facial features by asking participants to use mouse clicks to highlight those parts of a face that they deem useful for recognizing its expression. We tested how well the resulting maps align with models of emotion expressions (based on Action Units) and how the maps relate to the accuracy with which observers recognize full or partly masked faces. As expected, observers focused on the eyes and mouth regions in all faces. However, each expression deviated from this global pattern in a unique way, allowing to create maps of diagnostic face regions. Action Units considered most important for expressing an emotion were highlighted most often, indicating their psychological validity. The maps of facial features also allowed to correctly predict which expression a participant had seen, with above-chance accuracies for all expressions. For happiness, fear and anger, the face half which was highlighted the most was also the half whose visibility led to higher recognition accuracies. The results suggest that diagnostic facial features are distributed in unique patterns for each expression, which observers seem to intuitively extract and use when categorizing facial displays of emotion.

Structural neuroanatomy of human facial behaviors

The human face plays a central role in emotions and social communication. The emotional and somatic motor networks generate facial behaviors, but whether facial behaviors have representations in the structural anatomy of the human brain is unknown. We coded 16 facial behaviors in 55 healthy older adults who viewed five videos that elicited emotions and examined whether individual differences in facial behavior were related to regional variation in gray matter volume. Voxel-based morphometry analyses revealed that greater emotional facial behavior during the disgust trial (i.e. greater brow furrowing and eye tightening as well as nose wrinkling and upper lip raising) and the amusement trial (i.e. greater smiling and eye tightening) was associated with larger gray matter volume in midcingulate cortex, supplementary motor area, and precentral gyrus, areas spanning both the emotional and somatic motor networks. When measured across trials, however, these facial behaviors (and others) only related to gray matter volume in the precentral gyrus, a somatic motor network hub. These findings suggest that the emotional and somatic motor networks store structural representations of facial behavior and that the midcingulate cortex is critical for generating the predictable movements in the face that arise during emotions.

Digital assessment of nonverbal behaviors forecasts first onset of depression

BACKGROUND

Adolescence is marked by a sharp increase in the incidence of depression, especially in females. Identification of risk for depressive disorders (DD) in this key developmental stage can help prevention efforts, mitigating the clinical and public burden of DD. While frequently used in diagnosis, nonverbal behaviors are relatively understudied as risk markers for DD. Digital technology, such as facial recognition, may provide objective, fast, efficient, and cost-effective means of measuring nonverbal behavior.

METHOD

Here, we analyzed video-recorded clinical interviews of 359 never-depressed adolescents females via commercially available facial emotion recognition software.

RESULTS

We found that average head and facial movements forecast future first onset of depression (AUC = 0.70) beyond the effects of other established self-report and physiological markers of DD risk.

CONCLUSIONS

Overall, these findings suggest that digital assessment of nonverbal behaviors may provide a promising risk marker for DD, which could aid in early identification and intervention efforts.

Commonalities and variations in emotion representation across modalities and brain regions

Humans express emotions through various modalities such as facial expressions and natural language. However, the relationships between emotions expressed through different modalities and their correlations with neural activities remain uncertain. Here, we aimed to unveil some of these uncertainties by investigating the similarity of emotion representations across modalities and brain regions. First, we represented various emotion categories as multi-dimensional vectors derived from visual (face), linguistic, and visio-linguistic data, and used representational similarity analysis to compare these modalities. Second, we examined the linear transferability of emotion representation from other modalities to the visual modality. Third, we compared the representational structure derived in the first step with those from brain activities across 360 regions. Our findings revealed that emotion representations share commonalities across modalities with modality-type dependent variations, and they can be linearly mapped from other modalities to the visual modality. Additionally, emotion representations in uni-modalities showed relatively higher similarity with specific brain regions, while multi-modal emotion representation was most similar to representations across the entire brain region. These findings suggest that emotional experiences are represented differently across various brain regions with varying degrees of similarity to different modality types, and that they may be multi-modally conveyable in visual and linguistic domains.

New method for analysing spatial relationships of facial muscles on MRI: a pilot study

Dysfunction of the facial musculature can have significant physical, social, and psychological consequences. In surgeries such as cleft surgery or craniofacial bimaxillary osteotomies, the perioral facial muscles may be detached or severed, potentially altering their functional vectors and mimicry capabilities. Ensuring correct reconstruction and maintenance of anatomical sites and muscle vectors is crucial in these procedures. However, a standardized method for perioperative assessment of the facial musculature and function is currently lacking. The aim of this study was to develop a workflow to analyse the three-dimensional vectors of the facial musculature using magnetic resonance imaging (MRI) scans. A protocol for localizing the origins and insertions of these muscles was established. The protocol was implemented using the 3DMedX computer program and tested on 7 Tesla MRI scans obtained from 10 healthy volunteers. Inter- and intra-observer variability were assessed to validate the protocol. The absolute intra-observer variability was 2.6 mm (standard deviation 2.0 mm), and absolute inter-observer variability was 2.6 mm (standard deviation 1.5 mm). This study presents a reliable and reproducible method for analysing the spatial relationships and functional significance of the facial muscles. The workflow developed facilitates perioperative assessment of the facial musculature, potentially aiding clinicians in surgical planning and potentially enhancing the outcomes of midface surgery.

Emotional palette: a computational mapping of aesthetic experiences evoked by visual art

Despite the evolutionary history and cultural significance of visual art, the structure of aesthetic experiences it evokes has only attracted recent scientific attention. What kinds of experience does visual art evoke? Guided by Semantic Space Theory, we identify the concepts that most precisely describe people's aesthetic experiences using new computational techniques. Participants viewed 1457 artworks sampled from diverse cultural and historical traditions and reported on the emotions they felt and their perceived artwork qualities. Results show that aesthetic experiences are high-dimensional, comprising 25 categories of feeling states. Extending well beyond hedonism and broad evaluative judgments (e.g., pleasant/unpleasant), aesthetic experiences involve emotions of daily social living (e.g., "sad", "joy"), the imagination (e.g., "psychedelic", "mysterious"), profundity (e.g., "disgust", "awe"), and perceptual qualities attributed to the artwork (e.g., "whimsical", "disorienting"). Aesthetic emotions and perceptual qualities jointly predict viewers' liking of the artworks, indicating that we conceptualize aesthetic experiences in terms of the emotions we feel but also the qualities we perceive in the artwork. Aesthetic experiences are often mixed and lie along continuous gradients between categories rather than within discrete clusters. Our collection of artworks is visualized within an interactive map ( https://barradeau.com/2021/emotions-map/ ), revealing the high-dimensional space of aesthetic experiences associated with visual art.

Behavioral meaures of psychotic disorders: Using automatic facial coding to detect nonverbal expressions in video

Emotional deficits in psychosis are prevalent and difficult to treat. In particular, much remains unknown about facial expression abnormalities, and a key reason is that expressions are very labor-intensive to code. Automatic facial coding (AFC) can remove this barrier. The current study sought to both provide evidence for the utility of AFC in psychosis for research purposes and to provide evidence that AFC are valid measures of clinical constructs. Changes of facial expressions and head position of participants-39 with schizophrenia/schizoaffective disorder (SZ), 46 with other psychotic disorders (OP), and 108 never psychotic individuals (NP)-were assessed via FaceReader, a commercially available automated facial expression analysis software, using video recorded during a clinical interview. We first examined the behavioral measures of the psychotic disorder groups and tested if they can discriminate between the groups. Next, we evaluated links of behavioral measures with clinical symptoms, controlling for group membership. We found the SZ group was characterized by significantly less variation in neutral expressions, happy expressions, arousal, and head movements compared to NP. These measures discriminated SZ from NP well (AUC = 0.79, sensitivity = 0.79, specificity = 0.67) but discriminated SZ from OP less well (AUC = 0.66, sensitivity = 0.77, specificity = 0.46). We also found significant correlations between clinician-rated symptoms and most behavioral measures (particularly happy expressions, arousal, and head movements). Taken together, these results suggest that AFC can provide useful behavioral measures of psychosis, which could improve research on non-verbal expressions in psychosis and, ultimately, enhance treatment.

FaceMotionPreserve: a generative approach for facial de-identification and medical information preservation

Telemedicine and video-based diagnosis have raised significant concerns regarding the protection of facial privacy. Effective de-identification methods require the preservation of diagnostic information related to normal and pathological facial movements, which play a crucial role in the diagnosis of various movement, neurological, and psychiatric disorders. In this work, we have developed FaceMotionPreserve , a deep generative model-based approach that transforms patients' facial identities while preserving facial dynamics with a novel face dynamic similarity module to enhance facial landmark consistency. We collected test videos from patients with Parkinson's disease recruited via telemedicine for evaluation of model performance and clinical applicability. The performance of FaceMotionPreserve was quantitatively evaluated based on neurologist diagnostic consistency, critical facial behavior fidelity, and correlation of general facial dynamics. In addition, we further validated the robustness and advancements of our model in preserving medical information with clinical examination videos from a different cohort of patients. FaceMotionPreserve is applicable to real-time integration, safeguarding facial privacy while retaining crucial medical information associated with facial movements to address concerns in telemedicine, and facilitating safer and more collaborative medical data sharing.

How emotion is experienced and expressed in multiple cultures: a large-scale experiment across North America, Europe, and Japan

Core to understanding emotion are subjective experiences and their expression in facial behavior. Past studies have largely focused on six emotions and prototypical facial poses, reflecting limitations in scale and narrow assumptions about the variety of emotions and their patterns of expression. We examine 45,231 facial reactions to 2,185 evocative videos, largely in North America, Europe, and Japan, collecting participants' self-reported experiences in English or Japanese and manual and automated annotations of facial movement. Guided by Semantic Space Theory, we uncover 21 dimensions of emotion in the self-reported experiences of participants in Japan, the United States, and Western Europe, and considerable cross-cultural similarities in experience. Facial expressions predict at least 12 dimensions of experience, despite massive individual differences in experience. We find considerable cross-cultural convergence in the facial actions involved in the expression of emotion, and culture-specific display tendencies-many facial movements differ in intensity in Japan compared to the U.S./Canada and Europe but represent similar experiences. These results quantitatively detail that people in dramatically different cultures experience and express emotion in a high-dimensional, categorical, and similar but complex fashion.

Being facially expressive is socially advantageous

Individuals vary in how they move their faces in everyday social interactions. In a first large-scale study, we measured variation in dynamic facial behaviour during social interaction and examined dyadic outcomes and impression formation. In Study 1, we recorded semi-structured video calls with 52 participants interacting with a confederate across various everyday contexts. Video clips were rated by 176 independent participants. In Study 2, we examined video calls of 1315 participants engaging in unstructured video-call interactions. Facial expressivity indices were extracted using automated Facial Action Coding Scheme analysis and measures of personality and partner impressions were obtained by self-report. Facial expressivity varied considerably across participants, but little across contexts, social partners or time. In Study 1, more facially expressive participants were more well-liked, agreeable, and successful at negotiating (if also more agreeable). Participants who were more facially competent, readable, and perceived as readable were also more well-liked. In Study 2, we replicated the findings that facial expressivity was associated with agreeableness and liking by their social partner, and additionally found it to be associated with extraversion and neuroticism. Findings suggest that facial behaviour is a stable individual difference that proffers social advantages, pointing towards an affiliative, adaptive function.

Unveiling the underlying structure of awe in virtual reality and in autobiographical recall: an exploratory study

Over the last two decades, awe has attracted the attention of an increasing number of researchers. The use of virtual reality has been identified as one of the most effective techniques for eliciting awe, in addition to more personalized methods for inducing emotion, such as autobiographical recall. However, previous measures of awe were unable to uncover the hidden structure of this experience. Awe experience scale (AWE-S) has been validated as a comprehensive measure of contingent awe in English, providing new opportunities for analysis. In this two-phases study, we investigated whether the latent structure of the experience of awe evoked by the autobiographical recall technique (Study 1) overlapped with that induced by exposing participants to a validated virtual reality awe-eliciting training (Study 2). The original English AWE-S structure held both in autobiographical recall induction and virtual reality-based elicitation. Despite evidence of overlap between English and Italian structures, low correlations were found between Italian trait measures used to test the concurrent validity of the AWE-S in the Italian sample and AWE-S state dimensions. This study highlights cultural differences in awe experience, trait, and state variations, and provides new insights into the standardized induction of this emotion through simulated environments.

Automating the analysis of facial emotion expression dynamics: A computational framework and application in psychotic disorders

Facial emotion expressions play a central role in interpersonal interactions; these displays are used to predict and influence the behavior of others. Despite their importance, quantifying and analyzing the dynamics of brief facial emotion expressions remains an understudied methodological challenge. Here, we present a method that leverages machine learning and network modeling to assess the dynamics of facial expressions. Using video recordings of clinical interviews, we demonstrate the utility of this approach in a sample of 96 people diagnosed with psychotic disorders and 116 never-psychotic adults. Participants diagnosed with schizophrenia tended to move from neutral expressions to uncommon expressions (e.g., fear, surprise), whereas participants diagnosed with other psychoses (e.g., mood disorders with psychosis) moved toward expressions of sadness. This method has broad applications to the study of normal and altered expressions of emotion and can be integrated with telemedicine to improve psychiatric assessment and treatment.

Deep learning reveals what facial expressions mean to people in different cultures

Cross-cultural studies of the meaning of facial expressions have largely focused on judgments of small sets of stereotypical images by small numbers of people. Here, we used large-scale data collection and machine learning to map what facial expressions convey in six countries. Using a mimicry paradigm, 5,833 participants formed facial expressions found in 4,659 naturalistic images, resulting in 423,193 participant-generated facial expressions. In their own language, participants also rated each expression in terms of 48 emotions and mental states. A deep neural network tasked with predicting the culture-specific meanings people attributed to facial movements while ignoring physical appearance and context discovered 28 distinct dimensions of facial expression, with 21 dimensions showing strong evidence of universality and the remainder showing varying degrees of cultural specificity. These results capture the underlying dimensions of the meanings of facial expressions within and across cultures in unprecedented detail.

Perceived Relational Support Is Associated With Everyday Positive, But Not Negative, Affectivity in a U.S. Sample

Research suggests that perceived social support bolsters emotional well-being. We tested whether perceived support from friends, family, and spouses/partners was associated with reduced negative and greater positive affectivity (i.e., everyday affective baseline), and whether perceived strain in these relationships had opposite effects, accounting for age and relevant covariates. Using data from the third waves of the Midlife in the United States survey and National Study of Daily Experience (n = 1,124), we found negative affectivity was not tied to relational support nor strain, but instead was associated positively with neuroticism and negatively with conscientiousness. In contrast, positive affectivity was related positively to support from friends and family, conscientiousness, and extroversion, and negatively to strain among partners and neuroticism. Exploratory analyses within second-wave Midlife in Japan data (n = 657) suggest patterns for future cross-cultural study. Some relationship dynamics may vary, but perceived support might enhance emotional well-being by bolstering positive, rather than mitigating negative, emotionality.

Cognitive fossils: using cultural artifacts to reconstruct psychological changes throughout history

Psychology is crucial for understanding human history. When aggregated, changes in the psychology of individuals - in the intensity of social trust, parental care, or intellectual curiosity - can lead to important changes in institutions, social norms, and cultures. However, studying the role of psychology in shaping human history has been hindered by the difficulty of documenting the psychological traits of people who are no longer alive. Recent developments in psychology suggest that cultural artifacts reflect in part the psychological traits of the individuals who produced or consumed them. Cultural artifacts can thus serve as 'cognitive fossils' - physical imprints of the psychological traits of long-dead people. We review the range of materials available to cognitive and behavioral scientists, and discuss the methods that can be used to recover and quantify changes in psychological traits throughout history.

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

This paper provides a comprehensive overview of affective computing systems for facial expression recognition (FER) research in naturalistic contexts. The first section presents an updated account of user-friendly FER toolboxes incorporating state-of-the-art deep learning models and elaborates on their neural architectures, datasets, and performances across domains. These sophisticated FER toolboxes can robustly address a variety of challenges encountered in the wild such as variations in illumination and head pose, which may otherwise impact recognition accuracy. The second section of this paper discusses multimodal large language models (MLLMs) and their potential applications in affective science. MLLMs exhibit human-level capabilities for FER and enable the quantification of various contextual variables to provide context-aware emotion inferences. These advancements have the potential to revolutionize current methodological approaches for studying the contextual influences on emotions, leading to the development of contextualized emotion models.

Emotional event perception is related to lexical complexity and emotion knowledge

Inferring emotion is a critical skill that supports social functioning. Emotion inferences are typically studied in simplistic paradigms by asking people to categorize isolated and static cues like frowning faces. Yet emotions are complex events that unfold over time. Here, across three samples (Study 1 N = 222; Study 2 N = 261; Study 3 N = 101), we present the Emotion Segmentation Paradigm to examine inferences about complex emotional events by extending cognitive paradigms examining event perception. Participants were asked to indicate when there were changes in the emotions of target individuals within continuous streams of activity in narrative film (Study 1) and documentary clips (Study 2, preregistered, and Study 3 test-retest sample). This Emotion Segmentation Paradigm revealed robust and reliable individual differences across multiple metrics. We also tested the constructionist prediction that emotion labels constrain emotion inference, which is traditionally studied by introducing emotion labels. We demonstrate that individual differences in active emotion vocabulary (i.e., readily accessible emotion words) correlate with emotion segmentation performance.

Py-Feat: Python Facial Expression Analysis Toolbox

Studying facial expressions is a notoriously difficult endeavor. Recent advances in the field of affective computing have yielded impressive progress in automatically detecting facial expressions from pictures and videos. However, much of this work has yet to be widely disseminated in social science domains such as psychology. Current state-of-the-art models require considerable domain expertise that is not traditionally incorporated into social science training programs. Furthermore, there is a notable absence of user-friendly and open-source software that provides a comprehensive set of tools and functions that support facial expression research. In this paper, we introduce Py-Feat, an open-source Python toolbox that provides support for detecting, preprocessing, analyzing, and visualizing facial expression data. Py-Feat makes it easy for domain experts to disseminate and benchmark computer vision models and also for end users to quickly process, analyze, and visualize face expression data. We hope this platform will facilitate increased use of facial expression data in human behavior research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-023-00191-4.

Emotion Regulation in Couples Across Adulthood

Intimate relationships are hotbeds of emotion. This article presents key findings and current directions in research on couples' emotion regulation across adulthood as a critical context in which older adults not only maintain functioning but may also outshine younger adults. First, I introduce key concepts, defining qualities (i.e., dynamic, coregulatory, bidirectional, bivalent), and measures (i.e., self-report versus performance-based) of couples' emotion regulation. Second, I highlight a socioemotional turn in our understanding of adult development with the advent of socioemotional selectivity theory. Third, I offer a life-span developmental perspective on emotion regulation in couples (i.e., across infancy, adolescence and young adulthood, midlife, and late life). Finally, I present the idea that emotion regulation may shift from "me to us" across adulthood and discuss how emotion regulation in couples may become more important, better, and increasingly consequential (e.g., for relationship outcomes, well-being, and health) with age. Ideas for future research are then discussed.

Basic Emotions or Constructed Emotions: Insights From Taking an Evolutionary Perspective

The ongoing debate between basic emotion theories (BETs) and the theory of constructed emotion (TCE) hampers progress in the field of emotion research. Providing a new perspective, here we aim to bring the theories closer together by dissecting them according to Tinbergen's four questions to clarify a focus on their evolutionary basis. On the basis of our review of the literature, we conclude that whereas BETs focus on the evolution question of Tinbergen, the TCE is more concerned with the causation of emotion. On the survival value of emotions both theories largely agree: to provide the best reaction in specific situations. Evidence is converging on the evolutionary history of emotions but is still limited for both theories-research within both frameworks focuses heavily on the causation. We conclude that BETs and the TCE explain two different phenomena: emotion and feeling. Therefore, they seem irreconcilable but possibly supplementary for explaining and investigating the evolution of emotion-especially considering their similar answer to the question of survival value. Last, this article further highlights the importance of carefully describing what aspect of emotion is being discussed or studied. Only then can evidence be interpreted to converge toward explaining emotion.

Infrequent facial expressions of emotion do not bias attention

Despite the obvious importance of facial expressions of emotion, most studies have found that they do not bias attention. A critical limitation, however, is that these studies generally present face distractors on all trials of the experiment. For other kinds of emotional stimuli, such as emotional scenes, infrequently presented stimuli elicit greater attentional bias than frequently presented stimuli, perhaps due to suppression or habituation. The goal of the current study then was to test whether such modulation of attentional bias by distractor frequency generalizes to facial expressions of emotion. In Experiment 1, both angry and happy faces were unable to bias attention, despite being infrequently presented. Even when the location of these face cues were more unpredictable-presented in one of two possible locations-still no attentional bias was observed (Experiment 2). Moreover, there was no bottom-up influence for angry and happy faces shown under high or low perceptual load (Experiment 3). We conclude that task-irrelevant posed facial expressions of emotion cannot bias attention even when presented infrequently.

Kin relationality and ecological belonging: a cultural psychology of Indigenous transcendence

In this article, we consider prosociality through the lens of an Indigenous "ethics of belonging" and its two constitutive concepts: kin relationality and ecological belonging. Kin relationality predicates that all living beings and phenomena share a familial identity of interdependence, mutuality, and organization. Within the value system of ecological belonging, an individual's identity is constituted in relation to the natural environment, centered on the sentiments of responsibility and reverence for Nature. We detail how Indigenous perspectives upon prosociality differ from Western scientific accounts in terms of the motives, scope, and rewards of altruistic action. Grounded in this understanding, we then profile three self-transcendent states, compassion, gratitude, and awe, and their similarities across Indigenous and Western approaches, and how kin relationality and ecological belonging give rise to cultural variations. We consider convergent insights across Indigenous and Western science concerning the role of ritual and narrative and the cultural cultivation of kin relationality and ecological belonging. We conclude by highlighting how these two core concepts might guide future inquiry in cultural psychology.

Advancing Naturalistic Affective Science with Deep Learning

People express their own emotions and perceive others' emotions via a variety of channels, including facial movements, body gestures, vocal prosody, and language. Studying these channels of affective behavior offers insight into both the experience and perception of emotion. Prior research has predominantly focused on studying individual channels of affective behavior in isolation using tightly controlled, non-naturalistic experiments. This approach limits our understanding of emotion in more naturalistic contexts where different channels of information tend to interact. Traditional methods struggle to address this limitation: manually annotating behavior is time-consuming, making it infeasible to do at large scale; manually selecting and manipulating stimuli based on hypotheses may neglect unanticipated features, potentially generating biased conclusions; and common linear modeling approaches cannot fully capture the complex, nonlinear, and interactive nature of real-life affective processes. In this methodology review, we describe how deep learning can be applied to address these challenges to advance a more naturalistic affective science. First, we describe current practices in affective research and explain why existing methods face challenges in revealing a more naturalistic understanding of emotion. Second, we introduce deep learning approaches and explain how they can be applied to tackle three main challenges: quantifying naturalistic behaviors, selecting and manipulating naturalistic stimuli, and modeling naturalistic affective processes. Finally, we describe the limitations of these deep learning methods, and how these limitations might be avoided or mitigated. By detailing the promise and the peril of deep learning, this review aims to pave the way for a more naturalistic affective science.

Neural evidence of face processing in social anxiety disorder: A systematic review with meta-analysis

Numerous previous studies have used event-related potentials (ERPs) to examine facial processing deficits in individuals with social anxiety disorder (SAD). However, researchers still need to determine whether the deficits are general or specific and what the dominant factors are behind different cognitive stages. Meta-analysis was performed to quantitatively identify face processing deficits in individuals with SAD. Ninety-seven results in 27 publications involving 1032 subjects were calculated using Hedges' g. The results suggest that the face itself elicits enlarged P1 amplitudes, threat-related facial expressions induce larger P2 amplitudes, and negative facial expressions lead to enhanced P3/LPP amplitudes in SAD individuals compared with controls. That is, there is face perception attentional bias in the early phase (P1), threat attentional bias in the mid-term phase (P2), and negative emotion attentional bias in the late phase (P3/LPP), which can be summarized into a three-phase SAD face processing deficit model. These findings provide an essential theoretical basis for cognitive behavioral therapy and have significant application value for the initial screening, intervention, and treatment of social anxiety.

The nature and neurobiology of fear and anxiety: State of the science and opportunities for accelerating discovery

Fear and anxiety play a central role in mammalian life, and there is considerable interest in clarifying their nature, identifying their biological underpinnings, and determining their consequences for health and disease. Here we provide a roundtable discussion on the nature and biological bases of fear- and anxiety-related states, traits, and disorders. The discussants include scientists familiar with a wide variety of populations and a broad spectrum of techniques. The goal of the roundtable was to take stock of the state of the science and provide a roadmap to the next generation of fear and anxiety research. Much of the discussion centered on the key challenges facing the field, the most fruitful avenues for future research, and emerging opportunities for accelerating discovery, with implications for scientists, funders, and other stakeholders. Understanding fear and anxiety is a matter of practical importance. Anxiety disorders are a leading burden on public health and existing treatments are far from curative, underscoring the urgency of developing a deeper understanding of the factors governing threat-related emotions.

The Meso-Expression Test (MET): A Novel Assessment of Emotion Perception

Emotion perception is a primary facet of Emotional Intelligence (EI) and the underpinning of interpersonal communication. In this study, we examined meso-expressions-the everyday, moderate-intensity emotions communicated through the face, voice, and body. We theoretically distinguished meso-expressions from other well-known emotion research paradigms (i.e., macro-expression and micro-expressions). In Study 1, we demonstrated that people can reliably discriminate between meso-expressions, and we created a corpus of 914 unique video displays of meso-expressions across a race- and gender-diverse set of expressors. In Study 2, we developed a novel video-based assessment of emotion perception ability: The Meso-Expression Test (MET). In this study, we found that the MET is psychometrically valid and demonstrated measurement equivalence across Asian, Black, Hispanic, and White perceiver groups and across men and women. In Study 3, we examined the construct validity of the MET and showed that it converged with other well-known measures of emotion perception and diverged from cognitive ability. Finally, in Study 4, we showed that the MET is positively related to important psychosocial outcomes, including social well-being, social connectedness, and empathic concern and is negatively related to alexithymia, stress, depression, anxiety, and adverse social interactions. We conclude with a discussion focused on the implications of our findings for EI ability research and the practical applications of the MET.

Comparative physiological anthropogeny: exploring molecular underpinnings of distinctly human phenotypes

Anthropogeny is a classic term encompassing transdisciplinary investigations of the origins of the human species. Comparative anthropogeny is a systematic comparison of humans and other living nonhuman hominids (so-called "great apes"), aiming to identify distinctly human features in health and disease, with the overall goal of explaining human origins. We begin with a historical perspective, briefly describing how the field progressed from the earliest evolutionary insights to the current emphasis on in-depth molecular and genomic investigations of "human-specific" biology and an increased appreciation for cultural impacts on human biology. While many such genetic differences between humans and other hominids have been revealed over the last two decades, this information remains insufficient to explain the most distinctive phenotypic traits distinguishing humans from other living hominids. Here we undertake a complementary approach of "comparative physiological anthropogeny," along the lines of the preclinical medical curriculum, i.e., beginning with anatomy and considering each physiological system and in each case considering genetic and molecular components that are relevant. What is ultimately needed is a systematic comparative approach at all levels from molecular to physiological to sociocultural, building networks of related information, drawing inferences, and generating testable hypotheses. The concluding section will touch on distinctive considerations in the study of human evolution, including the importance of gene-culture interactions.

Differences Between East Asians and Westerners in the Mental Representations and Visual Information Extraction Involved in the Decoding of Pain Facial Expression Intensity

Effectively communicating pain is crucial for human beings. Facial expressions are one of the most specific forms of behavior associated with pain, but the way culture shapes expectations about the intensity with which pain is typically facially conveyed, and the visual strategies deployed to decode pain intensity in facial expressions, is poorly understood. The present study used a data-driven approach to compare two cultures, namely East Asians and Westerners, with respect to their mental representations of pain facial expressions (experiment 1, N=60; experiment 2, N=74) and their visual information utilization during the discrimination of facial expressions of pain of different intensities (experiment 3; N=60). Results reveal that compared to Westerners, East Asians expect more intense pain expressions (experiments 1 and 2), need more signal, and do not rely as much as Westerners on core facial features of pain expressions to discriminate between pain intensities (experiment 3). Together, those findings suggest that cultural norms regarding socially accepted pain behaviors shape the expectations about pain facial expressions and decoding visual strategies. Furthermore, they highlight the complexity of emotional facial expressions and the importance of studying pain communication in multicultural settings.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-023-00186-1.

Visual perception of emotion cues in dogs: a critical review of methodologies

Comparative studies of human-dog cognition have grown exponentially since the 2000's, but the focus on how dogs look at us (as well as other dogs) as social partners is a more recent phenomenon despite its importance to human-dog interactions. Here, we briefly summarise the current state of research in visual perception of emotion cues in dogs and why this area is important; we then critically review its most commonly used methods, by discussing conceptual and methodological challenges and associated limitations in depth; finally, we suggest some possible solutions and recommend best practice for future research. Typically, most studies in this field have concentrated on facial emotional cues, with full body information rarely considered. There are many challenges in the way studies are conceptually designed (e.g., use of non-naturalistic stimuli) and the way researchers incorporate biases (e.g., anthropomorphism) into experimental designs, which may lead to problematic conclusions. However, technological and scientific advances offer the opportunity to gather much more valid, objective, and systematic data in this rapidly expanding field of study. Solving conceptual and methodological challenges in the field of emotion perception research in dogs will not only be beneficial in improving research in dog-human interactions, but also within the comparative psychology area, in which dogs are an important model species to study evolutionary processes.

Manipulating facial musculature with functional electrical stimulation as an intervention for major depressive disorder: a focused search of literature for a proposal

BACKGROUND

Major Depressive Disorder (MDD) is associated with interoceptive deficits expressed throughout the body, particularly the facial musculature. According to the facial feedback hypothesis, afferent feedback from the facial muscles suffices to alter the emotional experience. Thus, manipulating the facial muscles could provide a new "mind-body" intervention for MDD. This article provides a conceptual overview of functional electrical stimulation (FES), a novel neuromodulation-based treatment modality that can be potentially used in the treatment of disorders of disrupted brain connectivity, such as MDD.

METHODS

A focused literature search was performed for clinical studies of FES as a modulatory treatment for mood symptoms. The literature is reviewed in a narrative format, integrating theories of emotion, facial expression, and MDD.

RESULTS

A rich body of literature on FES supports the notion that peripheral muscle manipulation in patients with stroke or spinal cord injury may enhance central neuroplasticity, restoring lost sensorimotor function. These neuroplastic effects suggest that FES may be a promising innovative intervention for psychiatric disorders of disrupted brain connectivity, such as MDD. Recent pilot data on repetitive FES applied to the facial muscles in healthy participants and patients with MDD show early promise, suggesting that FES may attenuate the negative interoceptive bias associated with MDD by enhancing positive facial feedback. Neurobiologically, the amygdala and nodes of the emotion-to-motor transformation loop may serve as potential neural targets for facial FES in MDD, as they integrate proprioceptive and interoceptive inputs from muscles of facial expression and fine-tune their motor output in line with socio-emotional context.

CONCLUSIONS

Manipulating facial muscles may represent a mechanistically novel treatment strategy for MDD and other disorders of disrupted brain connectivity that is worthy of investigation in phase II/III trials.

Infant AFAR: Automated facial action recognition in infants

Automated detection of facial action units in infants is challenging. Infant faces have different proportions, less texture, fewer wrinkles and furrows, and unique facial actions relative to adults. For these and related reasons, action unit (AU) detectors that are trained on adult faces may generalize poorly to infant faces. To train and test AU detectors for infant faces, we trained convolutional neural networks (CNN) in adult video databases and fine-tuned these networks in two large, manually annotated, infant video databases that differ in context, head pose, illumination, video resolution, and infant age. AUs were those central to expression of positive and negative emotion. AU detectors trained in infants greatly outperformed ones trained previously in adults. Training AU detectors across infant databases afforded greater robustness to between-database differences than did training database specific AU detectors and outperformed previous state-of-the-art in infant AU detection. The resulting AU detection system, which we refer to as Infant AFAR (Automated Facial Action Recognition), is available to the research community for further testing and applications in infant emotion, social interaction, and related topics.

Evidence for cultural differences in affect during mother-infant interactions

Maternal care is considered a universal and even cross-species set of typical behaviors, which are necessary to determine the social development of children. In humans, most research on mother-infant bonding is based on Western cultures and conducted in European and American countries. Thus, it is still unknown which aspects of mother-infant behaviors are universal and which vary with culture. Here we test whether typical mother-infant behaviors of affect-communication and affect-regulation are equally represented during spontaneous interaction in Palestinian-Arab and Jewish cultures. 30 Palestinian-Arab and 43 Jewish mother-infant dyads were recruited and videotaped. Using AffectRegulation Coding System (ARCS), we behaviorally analyzed the second-by-second display of valence and arousal in each participant and calculated the dynamic patterns of affect co-regulation. The results show that Palestinian-Arab infants express more positive valence than Jewish infants and that Palestinian-Arab mothers express higher arousal compared to Jewish mothers. Moreover, we found culturally-distinct strategies to regulate the infant: increased arousal in Palestinian-Arab dyads and increased mutual affective match in Jewish dyads. Such cross-cultural differences in affect indicate that basic features of emotion that are often considered universal are differentially represented in different cultures. Affect communication and regulation patterns can be transmitted across generations in early-life socialization with caregivers.

Awe as a Pathway to Mental and Physical Health

How do experiences in nature or in spiritual contemplation or in being moved by music or with psychedelics promote mental and physical health? Our proposal in this article is awe. To make this argument, we first review recent advances in the scientific study of awe, an emotion often considered ineffable and beyond measurement. Awe engages five processes-shifts in neurophysiology, a diminished focus on the self, increased prosocial relationality, greater social integration, and a heightened sense of meaning-that benefit well-being. We then apply this model to illuminate how experiences of awe that arise in nature, spirituality, music, collective movement, and psychedelics strengthen the mind and body.

Testing, explaining, and exploring models of facial expressions of emotions

Models are the hallmark of mature scientific inquiry. In psychology, this maturity has been reached in a pervasive question-what models best represent facial expressions of emotion? Several hypotheses propose different combinations of facial movements [action units (AUs)] as best representing the six basic emotions and four conversational signals across cultures. We developed a new framework to formalize such hypotheses as predictive models, compare their ability to predict human emotion categorizations in Western and East Asian cultures, explain the causal role of individual AUs, and explore updated, culture-accented models that improve performance by reducing a prevalent Western bias. Our predictive models also provide a noise ceiling to inform the explanatory power and limitations of different factors (e.g., AUs and individual differences). Thus, our framework provides a new approach to test models of social signals, explain their predictive power, and explore their optimization, with direct implications for theory development.

Deep learning reveals what vocal bursts express in different cultures

Human social life is rich with sighs, chuckles, shrieks and other emotional vocalizations, called 'vocal bursts'. Nevertheless, the meaning of vocal bursts across cultures is only beginning to be understood. Here, we combined large-scale experimental data collection with deep learning to reveal the shared and culture-specific meanings of vocal bursts. A total of n = 4,031 participants in China, India, South Africa, the USA and Venezuela mimicked vocal bursts drawn from 2,756 seed recordings. Participants also judged the emotional meaning of each vocal burst. A deep neural network tasked with predicting the culture-specific meanings people attributed to vocal bursts while disregarding context and speaker identity discovered 24 acoustic dimensions, or kinds, of vocal expression with distinct emotion-related meanings. The meanings attributed to these complex vocal modulations were 79% preserved across the five countries and three languages. These results reveal the underlying dimensions of human emotional vocalization in remarkable detail.

The Average Facial Expressions: A Range of Motion Analysis for Different Sex and Age Groups

Facial expressions are ubiquitous in communication. Therefore, assessment of mimic function is essential in facial surgery, but no reference standards are currently available. This prospective study aims to create reference values of three-dimensional landmark displacement for different sex and age groups.

Methods

Three-dimensional photographs were taken from healthy subjects in rest, maximum closed smile, and pouting. Displacement for both exercises of perioral landmarks was analyzed with MATLAB as absolute displacement and as the ratio of mouth width. Additionally, displacement in three planes was analyzed for each landmark. Averages were calculated for both genders in four age groups: 4-8, 8-12, 12-16, and >16 years.

Results

In total, 328 subjects were included. Oral landmarks predominantly moved forward and backward for both exercises. Nasal landmarks predominantly moved vertically. Growing up, oral landmark displacement decreased for smiling, whereas nasal landmark displacement increased. For pouting, oral landmark displacement increased while growing up, whereas nasal landmark displacement decreased.

Conclusions

The present study creates reference values for movement of perioral structures for different sex and age groups, for two facial expressions. These data are of great value for the assessment of mimic function and give insight into the development of facial animation over time.

Neural correlates of emotional valence for faces and words

Stimuli with negative emotional valence are especially apt to influence perception and action because of their crucial role in survival, a property that may not be precisely mirrored by positive emotional stimuli of equal intensity. The aim of this study was to identify the neural circuits differentially coding for positive and negative valence in the implicit processing of facial expressions and words, which are among the main ways human beings use to express emotions. Thirty-six healthy subjects took part in an event-related fMRI experiment. We used an implicit emotional processing task with the visual presentation of negative, positive, and neutral faces and words, as primary stimuli. Dynamic Causal Modeling (DCM) of the fMRI data was used to test effective brain connectivity within two different anatomo-functional models, for the processing of words and faces, respectively. In our models, the only areas showing a significant differential response to negative and positive valence across both face and word stimuli were early visual cortices, with faces eliciting stronger activations. For faces, DCM revealed that this effect was mediated by a facilitation of activity in the amygdala by positive faces and in the fusiform face area by negative faces; for words, the effect was mainly imputable to a facilitation of activity in the primary visual cortex by positive words. These findings support a role of early sensory cortices in discriminating the emotional valence of both faces and words, where the effect may be mediated chiefly by the subcortical/limbic visual route for faces, and rely more on the direct thalamic pathway to primary visual cortex for words.

Automatic facial coding predicts self-report of emotion, advertisement and brand effects elicited by video commercials

Introduction

Consumers' emotional responses are the prime target for marketing commercials. Facial expressions provide information about a person's emotional state and technological advances have enabled machines to automatically decode them.

Method

With automatic facial coding we investigated the relationships between facial movements (i.e., action unit activity) and self-report of commercials advertisement emotion, advertisement and brand effects. Therefore, we recorded and analyzed the facial responses of 219 participants while they watched a broad array of video commercials.

Results

Facial expressions significantly predicted self-report of emotion as well as advertisement and brand effects. Interestingly, facial expressions had incremental value beyond self-report of emotion in the prediction of advertisement and brand effects. Hence, automatic facial coding appears to be useful as a non-verbal quantification of advertisement effects beyond self-report.

Discussion

This is the first study to measure a broad spectrum of automatically scored facial responses to video commercials. Automatic facial coding is a promising non-invasive and non-verbal method to measure emotional responses in marketing.

What is missing in the study of emotion expression?

While approaching celebrations for the 150 years of "The Expression of the Emotions in Man and Animals", scientists' conclusions on emotion expression are still debated. Emotion expression has been traditionally anchored to prototypical and mutually exclusive facial expressions (e.g., anger, disgust, fear, happiness, sadness, and surprise). However, people express emotions in nuanced patterns and - crucially - not everything is in the face. In recent decades considerable work has critiqued this classical view, calling for a more fluid and flexible approach that considers how humans dynamically perform genuine expressions with their bodies in context. A growing body of evidence suggests that each emotional display is a complex, multi-component, motoric event. The human face is never static, but continuously acts and reacts to internal and environmental stimuli, with the coordinated action of muscles throughout the body. Moreover, two anatomically and functionally different neural pathways sub-serve voluntary and involuntary expressions. An interesting implication is that we have distinct and independent pathways for genuine and posed facial expressions, and different combinations may occur across the vertical facial axis. Investigating the time course of these facial blends, which can be controlled consciously only in part, is recently providing a useful operational test for comparing the different predictions of various models on the lateralization of emotions. This concise review will identify shortcomings and new challenges regarding the study of emotion expressions at face, body, and contextual levels, eventually resulting in a theoretical and methodological shift in the study of emotions. We contend that the most feasible solution to address the complex world of emotion expression is defining a completely new and more complete approach to emotional investigation. This approach can potentially lead us to the roots of emotional display, and to the individual mechanisms underlying their expression (i.e., individual emotional signatures).

Linking Affective and Hearing Sciences-Affective Audiology

A growing number of health-related sciences, including audiology, have increasingly recognized the importance of affective phenomena. However, in audiology, affective phenomena are mostly studied as a consequence of hearing status. This review first addresses anatomical and functional bidirectional connections between auditory and affective systems that support a reciprocal affect-hearing relationship. We then postulate, by focusing on four practical examples (hearing public campaigns, hearing intervention uptake, thorough hearing evaluation, and tinnitus), that some important challenges in audiology are likely affect-related and that potential solutions could be developed by inspiration from affective science advances. We continue by introducing useful resources from affective science that could help audiology professionals learn about the wide range of affective constructs and integrate them into hearing research and clinical practice in structured and applicable ways. Six important considerations for good quality affective audiology research are summarized. We conclude that it is worthwhile and feasible to explore the explanatory power of emotions, feelings, motivations, attitudes, moods, and other affective processes in depth when trying to understand and predict how people with hearing difficulties perceive, react, and adapt to their environment.

The neurobiological basis of affect is consistent with psychological construction theory and shares a common neural basis across emotional categories

Affective experience colours everyday perception and cognition, yet its fundamental and neurobiological basis is poorly understood. The current debate essentially centers around the communalities and specificities across individuals, events, and emotional categories like anger, sadness, and happiness. Using fMRI during the experience of these emotions, we critically compare the two dominant conflicting theories on human affect. Basic emotion theory posits emotions as discrete universal entities generated by dedicated emotion category-specific neural circuits, while psychological construction theory claims emotional events as unique, idiosyncratic, and constructed by psychological primitives like core affect and conceptualization, which underlie each emotional event and operate in a predictive framework. Based on the findings of 8 a priori-defined model-specific prediction tests on the neural response amplitudes and patterns, we conclude that the neurobiological basis of affect is primarily characterized by idiosyncratic mechanisms and a common neural basis shared across emotion categories, consistent with psychological construction theory. The findings provide further insight into the organizational principles of the neural basis of affect and brain function in general. Future studies in clinical populations with affective symptoms may reveal the corresponding underlying neural changes from a psychological construction perspective.

Context reconsidered: Complex signal ensembles, relational meaning, and population thinking in psychological science

This article considers the status and study of "context" in psychological science through the lens of research on emotional expressions. The article begins by updating three well-trod methodological debates on the role of context in emotional expressions to reconsider several fundamental assumptions lurking within the field's dominant methodological tradition: namely, that certain expressive movements have biologically prepared, inherent emotional meanings that issue from singular, universal processes which are independent of but interact with contextual influences. The second part of this article considers the scientific opportunities that await if we set aside this traditional understanding of "context" as a moderator of signals with inherent psychological meaning and instead consider the possibility that psychological events emerge in ecosystems of signal ensembles, such that the psychological meaning of any individual signal is entirely relational. Such a fundamental shift has radical implications not only for the science of emotion but for psychological science more generally. It offers opportunities to improve the validity and trustworthiness of psychological science beyond what can be achieved with improvements to methodological rigor alone. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

How Are You Feeling? Interpretation of Emotions through Facial Expressions of People Wearing Different Personal Protective Equipment: An Observational Study

(1) Background: The perception of others’ emotions based on non-verbal cues, such as facial expressions, is fundamental for interpersonal communication and mutual support. Using personal protection equipment (PPE) in a work environment during the SAR-CoV-2 pandemic challenged health professionals’ ability to recognise emotions and expressions while wearing PPE. The working hypothesis of this study was that the increased limitation of facial visibility, due to the use of a personal protective device, would interfere with the perception of basic emotions in the participants. (2) Methods: Through a cross-sectional descriptive study, the present research aimed to analyse the identification of four basic emotions (happiness; sadness; fear/surprise; and disgust/anger) through three types of PPE (FFP2 respirator, protective overall and powered air-purifying respirator (PAPR)), by using 32 photographs. The study was conducted using volunteer participants who met the inclusion criteria (individuals older than 13 without cognitive limitations). Participants had to recognise the emotions of actors in photographs that were randomly displayed in an online form. (3) Results: In general, the 690 participants better recognised happiness and fear, independently of the PPE utilised. Women could better identify different emotions, along with university graduates and young and middle-aged adults. Emotional identification was at its worst when the participants wore protective overalls (5.42 ± 1.22), followed by the PAPR (5.83 ± 1.38); the best scores were obtained using the FFP2 masks (6.57 ± 1.20). Sadness was the least recognised emotion, regardless of age. (4) Conclusions: The personal protective devices interfere in the recognition of emotions, with the protective overalls having the greatest impact, and the FFP2 mask the least. The emotions that were best recognised were happiness and fear/surprise, while the least recognised emotion was sadness. Women were better at identifying emotions, as well as participants with higher education, and young and middle-aged adults.

Comunicación no verbal de emociones: variables sociodemográficas y ventaja endogrupal

En el campo de la comunicación no verbal de las emociones aún existe un debate en torno a la universalidad de las expresiones de emoción y el efecto que tiene la cultura en ellas. Actualmente existen dos teorías que tratan de explicar este fenómeno, la teoría neurocultural y la teoría de los dialectos. Ambas se enfocan en explicar la comunicación no verbal de emociones, pero la primera se centra en los aspectos universales, mientras que la segunda lo hace en la cultura. El objetivo del presente estudio fue indagar la ventaja endogrupal al interior de una cultura. Se diseñó un cuasiexperimento en el que se solicitó a 107 participantes que indicaran la emoción expresada en 42 estímulos en tres formatos de presentación distintos.  Los resultados indican la existencia de dicha ventaja en las mujeres y jóvenes. Los presentes resultados ilustran los efectos de la cultura en este fenómeno.

Visuospatial and Affective Perspective-Taking

Abstract. Perspective-taking is the ability to intuit another person’s mental state. Historically, cognitive and affective perspective-taking are distinguished from visuospatial perspective-taking because the content these processes operate on is too dissimilar. However, all three share functional similarities. Following recent research showing relations between cognitive and visuospatial perspective-taking, this article explores links between visuospatial and affective perspective-taking. Data of three preregistered experiments suggest that visuospatial perspective-taking does not improve emotion recognition speed and only slightly increases emotion recognition accuracy (Experiment 1), yet visuospatial perspective-taking increases the perceived intensity of emotional expressions (Experiment 2), as well as the emotional contagiousness of negative emotions (Experiment 3). The implications of these findings for content-based, cognitive, and functional taxonomies of perspective-taking and related processes are discussed.

Acquiring Complex Communicative Systems: Statistical Learning of Language and Emotion

During the early postnatal years, most infants rapidly learn to understand two naturally evolved communication systems: language and emotion. While these two domains include different types of content knowledge, it is possible that similar learning processes subserve their acquisition. In this review, we compare the learnable statistical regularities in language and emotion input. We then consider how domain-general learning abilities may underly the acquisition of language and emotion, and how this process may be constrained in each domain. This comparative developmental approach can advance our understanding of how humans learn to communicate with others.

Determination of “Neutral”–“Pain”, “Neutral”–“Pleasure”, and “Pleasure”–“Pain” Affective State Distances by Using AI Image Analysis of Facial Expressions

(1) Background: In addition to verbalizations, facial expressions advertise one’s affective state. There is an ongoing debate concerning the communicative value of the facial expressions of pain and of pleasure, and to what extent humans can distinguish between these. We introduce a novel method of analysis by replacing human ratings with outputs from image analysis software. (2) Methods: We use image analysis software to extract feature vectors of the facial expressions neutral, pain, and pleasure displayed by 20 actresses. We dimension-reduced these feature vectors, used singular value decomposition to eliminate noise, and then used hierarchical agglomerative clustering to detect patterns. (3) Results: The vector norms for pain–pleasure were rarely less than the distances pain–neutral and pleasure–neutral. The pain–pleasure distances were Weibull-distributed and noise contributed 10% to the signal. The noise-free distances clustered in four clusters and two isolates. (4) Conclusions: AI methods of image recognition are superior to human abilities in distinguishing between facial expressions of pain and pleasure. Statistical methods and hierarchical clustering offer possible explanations as to why humans fail. The reliability of commercial software, which attempts to identify facial expressions of affective states, can be improved by using the results of our analyses.

The cultural learning account of first impressions

Humans spontaneously attribute character traits to strangers based on their facial appearance. Although these 'first impressions' typically have no basis in reality, some authors have assumed that they have an innate origin. By contrast, the Trait Inference Mapping (TIM) account proposes that first impressions are products of culturally acquired associative mappings that allow activation to spread from representations of facial appearance to representations of trait profiles. According to TIM, cultural instruments, including propaganda, illustrated storybooks, art and iconography, ritual, film, and TV, expose many individuals within a community to common sources of correlated face-trait experience, yielding first impressions that are shared by many, but typically inaccurate. Here, we review emerging empirical findings, many of which accord with TIM, and argue that future work must distinguish first impressions based on invariant facial features (e.g., shape) from those based on facial behaviours (e.g., expressions).

Posterior-prefrontal and medial orbitofrontal regions play crucial roles in happiness and sadness recognition

•

Brain areas underlying trade-off relations between emotions were identified.

•

Damage to the PPF area reduces accuracy of happiness recognition.

•

Damage to the PPF increases accuracy of sadness recognition.

•

A similar tendency was observed in orbitofrontal regions for sadness recognition.

•

Only a deficit in sadness, but not happiness, persisted in the chronic phase.

The core brain regions responsible for basic human emotions are not yet fully understood. We investigated the key areas responsible for emotion recognition of facial expressions of happiness and sadness using data obtained from patients who underwent local brain resection. A total of 44 patients with right cerebral hemispheric brain tumors and 33 healthy volunteers were enrolled and subjected to a facial expression recognition test. Voxel-based lesion-symptom mapping was performed to investigate the relationship between the accuracy of emotion recognition and the resected regions. Consequently, trade-off relationships were discovered: the posterior-prefrontal region was related to a low score of happiness recognition and a high score of sadness recognition (disorder-of-happiness group), whereas the medial orbitofrontal region was related to a low score of sadness recognition and a high score of happiness recognition (disorder-of-sadness group). The emotion recognition score in both the happiness and sadness disorder groups was significantly lower than that in the control group (p = 0.0009 and p = 0.021, respectively). Interestingly, the deficit in happiness recognition was temporary, whereas the deficit in sadness recognition persisted during the chronic phase. Using graph theoretical analysis, we identified structural connectivity between the posterior-prefrontal and medial orbitofrontal regions. When either of these regions was damaged, the tract volume connecting them was significantly reduced (p = 0.013). These results indicate that the posterior-prefrontal and medial orbitofrontal regions may be crucial for maintaining a balance between happiness and sadness recognition in humans. Investigating the clinical impact of certain area resections using lesion studies combined with connectivity analysis is a useful neuroimaging method for understanding neural networks.