Sixteen facial expressions occur in similar contexts worldwide

How emotions, relationships, and culture constitute each other: advances in social functionalist theory

Social Functionalist Theory (SFT) emerged 20 years ago to orient emotion science to the social nature of emotion. Here we expand upon SFT and make the case for how emotions, relationships, and culture constitute one another. First, we posit that emotions enable the individual to meet six "relational needs" within social interactions: security, commitment, status, trust, fairness, and belongingness. Building upon this new theorising, we detail four principles concerning emotional experience, cognition, expression, and the cultural archiving of emotion. We conclude by considering the bidirectional influences between culture, relationships, and emotion, outlining areas of future inquiry.

The Stressful Experience of Goal Orientations Under Frustration: Evidence Using Physiological Means

The purpose of the present study was to test the hypothesis that goal orientation is associated with divergent forms of emotional reactivity under frustration. Goal orientations were assessed using bifurcations of performance goals described earlier. Physiological stress levels were measured via a blood volume pulse analysis after individuals were subjected to a computerized Stroop task using a malfunctioning mouse to induce enhanced frustration. The results indicated that performance-avoidance goals were associated with the highest levels of emotional reactivity, with normative outcome goals being significantly more detrimental than ability goals. We concluded that the motivation to avoid failure or to outperform others is the most detrimental determinant of stress and needs to be avoided by all means. Instead, it is suggested that educators emphasize performance using personal best outcomes or by valuing engagement, deep processing and task completion.

Evidence for similar conceptual progress across diverse cultures in children’s understanding of emotion

Research with adults has increasingly moved beyond the focus on a small set of allegedly basic emotions, each associated with a signature facial expression. That expansion has been accompanied by a greater emphasis on the potential variability of emotion concepts across different cultural settings. In this conceptual review of children’s understanding of emotion, we argue that it is also important in developmental research to look beyond the small set of emotions associated with distinctive facial expressions. At the same time, we caution against any premature rejection of a universalist approach to children’s understanding of emotion. We review three different lines of evidence in support of this stance: (1) children’s ability to appropriately cite situational elicitors for emotions beyond the basic set; (2) their developing understanding of the relations between emotions and other mental processes; and (3) their realization that a person’s facially expressed emotion may not indicate their felt emotion. In each of these three domains, we target studies that have included children from a variety of cultures to assess how far they respond similarly or differently. We conclude that there is robust evidence for similar conceptual progress in children’s understanding of emotion across a range of cultural settings.

Long-term ecological assessment of intracranial electrophysiology synchronized to behavioral markers in obsessive-compulsive disorder

Detection of neural signatures related to pathological behavioral states could enable adaptive deep brain stimulation (DBS), a potential strategy for improving efficacy of DBS for neurological and psychiatric disorders. This approach requires identifying neural biomarkers of relevant behavioral states, a task best performed in ecologically valid environments. Here, in human participants with obsessive-compulsive disorder (OCD) implanted with recording-capable DBS devices, we synchronized chronic ventral striatum local field potentials with relevant, disease-specific behaviors. We captured over 1,000 h of local field potentials in the clinic and at home during unstructured activity, as well as during DBS and exposure therapy. The wide range of symptom severity over which the data were captured allowed us to identify candidate neural biomarkers of OCD symptom intensity. This work demonstrates the feasibility and utility of capturing chronic intracranial electrophysiology during daily symptom fluctuations to enable neural biomarker identification, a prerequisite for future development of adaptive DBS for OCD and other psychiatric disorders.

Clinician Facial Expression of Emotion Corresponds with Patient Mindset

BACKGROUND

Mental health has a notable and perhaps underappreciated relationship with symptom intensity related to musculoskeletal pathophysiology. Tools for increasing awareness of mental health opportunities may help musculoskeletal specialists identify and address psychological distress and unhealthy misconceptions with greater confidence. One such type of technology-software that identifies emotions by analyzing facial expressions-could be developed as a clinician-awareness tool. A first step in this endeavor is to conduct a pilot study to assess the ability to measure patient mental health through specialist facial expressions.

QUESTIONS/PURPOSES

(1) Does quantification of clinician emotion using facial recognition software correlate with patient psychological distress and unhealthy misconceptions? (2) Is there a correlation between clinician facial expressions of emotions and a validated measure of the quality of the patient-clinician relationship?

METHODS

In a cross-sectional pilot study, between April 2019 and July 2019, we made video recordings of the clinician's face during 34 initial musculoskeletal specialist outpatient evaluations. There were 16 men and 18 women, all fluent and literate in English, with a mean age of 43 ± 15 years. Enrollment was performed according to available personnel, equipment, and room availability. We did not track declines, but there were only a few. Video recordings were analyzed using facial-emotional recognition software, measuring the proportion of time spent by clinicians expressing measured emotions during a consultation. After the visit, patients completed a demographic questionnaire and measures of health anxiety (the Short Health Anxiety Inventory), fear of painful movement (the Tampa Scale for Kinesiophobia), catastrophic or worst-case thinking about pain (the Pain Catastrophizing Scale), symptoms of depression (the Patient Health Questionnaire), and the patient's perception of the quality of their relationship with the clinician (Patient-Doctor Relationship Questionnaire).

RESULTS

Clinician facial expressions consistent with happiness were associated with less patient health anxiety (r = -0.59; p < 0.001) and less catastrophic thinking (r = -0.37; p = 0.03). Lower levels of clinician expressions consistent with sadness were associated with less health anxiety (r = 0.36; p = 0.04), fewer symptoms of generalized anxiety (r = 0.36; p = 0.03), and less catastrophic thinking (r = 0.33; p = 0.05). Less time expressing anger was associated with greater health anxiety (r = -0.37; p = 0.03), greater symptoms of anxiety (r = -0.46; p < 0.01), more catastrophic thinking (r = -0.38; p = 0.03), and greater symptoms of depression (r = -0.42; p = 0.01). More time expressing surprise was associated with less health anxiety (r = -0.44; p < 0.01) and symptoms of depression (r = -0.52; p < 0.01). More time expressing fear was associated with less kinesiophobia (r = -0.35; p = 0.04). More time expressing disgust was associated with less catastrophic thinking (r = -0.37; p = 0.03) and less health anxiety (GAD-2; r = -0.42; p = 0.02) and symptoms of depression (r = -0.44; p < 0.01). There was no association between a clinicians' facial expression of emotions and patient experience with patient-clinician interactions.

CONCLUSION

The ability to measure a patient's mindset on the clinician's face confirms that clinicians are registering the psychological aspects of illness, whether they are consciously aware of them or not. Future research involving larger cohorts of patients, mapping clinician-patient interactions during consultation, and more sophisticated capture of nonverbal and verbal cues, including a broader range of emotional expressions, may help translate this innovation from the research setting to clinical practice.

CLINICAL RELEVANCE

Tools for measuring emotion through facial recognition could be used to train clinicians to become aware of the psychological aspects of health and to coach clinicians on effective communication strategies both for gentle reorientation of common misconceptions as well as for appropriate and timely diagnosis and treatment of psychological distress.

Happy Cow or Thinking Pig? WUR Wolf—Facial Coding Platform for Measuring Emotions in Farm Animals

Emotions play an indicative and informative role in the investigation of farm animal behaviors. Systems that respond and can measure emotions provide a natural user interface in enabling the digitalization of animal welfare platforms. The faces of farm animals can be one of the richest channels for expressing emotions. WUR Wolf (Wageningen University & Research: Wolf Mascot), a real-time facial recognition platform that can automatically code the emotions of farm animals, is presented in this study. The developed Python-based algorithms detect and track the facial features of cows and pigs, analyze the appearance, ear postures, and eye white regions, and correlate these with the mental/emotional states of the farm animals. The system is trained on a dataset of facial features of images of farm animals collected in over six farms and has been optimized to operate with an average accuracy of 85%. From these, the emotional states of animals in real time are determined. The software detects 13 facial actions and an inferred nine emotional states, including whether the animal is aggressive, calm, or neutral. A real-time emotion recognition system based on YoloV3, a Faster YoloV4-based facial detection platform and an ensemble Convolutional Neural Networks (RCNN) is presented. Detecting facial features of farm animals simultaneously in real time enables many new interfaces for automated decision-making tools for livestock farmers. Emotion sensing offers a vast potential for improving animal welfare and animal–human interactions.

Research on Emotion Recognition and Dementias: Foundations and Prospects

BACKGROUND

The study of emotion recognition could be crucial for detecting alterations in certain cognitive areas or as an early sign of neurological disorders.

OBJECTIVE

The main objective of the study is to characterize research development on emotion recognition, identifying the intellectual structure that supports this area of knowledge, and the main lines of research attracting investigators' interest.

METHODS

We identified publications on emotion recognition and dementia included in the Web of Science Core Collection, analyzing the scientific output and main disciplines involved in generating knowledge in the area. A co-citation analysis and an analysis of the bibliographic coupling between the retrieved documents elucidated the thematic orientations of the research and the reference works that constitute the foundation for development in the field.

RESULTS

A total of 345 documents, with 24,282 bibliographic references between them, were included. This is an emerging research area, attracting the interest of investigators in Neurosciences, Psychology, Clinical Neurology, and Psychiatry, among other disciplines. Four prominent topic areas were identified, linked to frontotemporal dementia, autism spectrum disorders, Alzheimer's disease, and Parkinson's and Huntington disease. Many recent papers focus on the detection of mild cognitive impairment.

CONCLUSION

Impaired emotion recognition may be a key sign facilitating the diagnosis and early treatment of different neurodegenerative diseases as well as for triggering the necessary provision of social and family support, explaining the growing research interest in this area.

Reading emotions, reading people: Emotion perception and inferences drawn from perceived emotions

Emotional expressions play an important role in coordinating social interaction. We review research on two critical processes that underlie such coordination: (1) perceiving emotions from emotion expressions and (2) drawing inferences from perceived emotions. Broad evidence indicates that (a) observers can accurately perceive emotions from a person's facial, bodily, vocal, verbal, and symbolic expressions and that such emotion perception is further informed by contextual information. Moreover, (b) observers draw consequential and contextualized inferences from these perceived emotions about the expresser, the situation, and the self. Thus, emotion expressions enable coordinated action by providing information that facilitates adaptive behavioral responses. We recommend that future studies investigate how people integrate information from different expressive modalities and how this affects consequential inferences.

Assessing Automated Facial Action Unit Detection Systems for Analyzing Cross-Domain Facial Expression Databases

In the field of affective computing, achieving accurate automatic detection of facial movements is an important issue, and great progress has already been made. However, a systematic evaluation of systems that now have access to the dynamic facial database remains an unmet need. This study compared the performance of three systems (FaceReader, OpenFace, AFARtoolbox) that detect each facial movement corresponding to an action unit (AU) derived from the Facial Action Coding System. All machines could detect the presence of AUs from the dynamic facial database at a level above chance. Moreover, OpenFace and AFAR provided higher area under the receiver operating characteristic curve values compared to FaceReader. In addition, several confusion biases of facial components (e.g., AU12 and AU14) were observed to be related to each automated AU detection system and the static mode was superior to dynamic mode for analyzing the posed facial database. These findings demonstrate the features of prediction patterns for each system and provide guidance for research on facial expressions.

An evaluation of the reading the mind in the eyes test's psychometric properties and scores in South Africa-cultural implications

The 'Reading the Mind in the Eyes' test (RMET) has been translated and tested in many cultural settings. Results indicate that items show variability in meeting the original psychometric testing criteria. Individuals from non-Western cultures score differently on the RMET. As such, questions arise as to the cross-cultural validity of the RMET. This study tested the English version of the RMET, that consists almost exclusively of White faces, at a large South African university to determine its validity in a culturally diverse context. A total of 443 students from a range of different demographic backgrounds completed the instrument. Students were selected using simple random sampling. 30 out of the 36 items continued to show satisfactory psychometric properties. Further evidence shows significant differences based on race and home language in both overall scores and item level scores. Black race and African home language respondents show lower RMET scores and different item level perspectives on certain mental states. The current RMET is not inclusive. It requires stimuli reflecting more races and cultures. This lack of diversity is likely to be influencing and biasing results and psychometric properties. The continued exclusion of racial stimuli such as Black race is also promoting a systemic discriminatory instrument. These results have cultural implications for how we interpret and use the RMET.

Are Emotions Natural Kinds After All? Rethinking the Issue of Response Coherence

The synchronized co-activation of multiple responses-motivational, behavioral, and physiological-has been taken as a defining feature of emotion. Such response coherence has been observed inconsistently however, and this has led some to view emotion programs as lacking biological reality. Yet, response coherence is not always expected or desirable if an emotion program is to carry out its adaptive function. Rather, the hallmark of emotion is the capacity to orchestrate multiple mechanisms adaptively-responses will co-activate in stereotypical fashion or not depending on how the emotion orchestrator interacts with the situation. Nevertheless, might responses cohere in the general case where input variables are specified minimally? Here we focus on shame as a case study. We measure participants' responses regarding each of 27 socially devalued actions and personal characteristics. We observe internal and external coherence: The intensities of felt shame and of various motivations of shame (hiding, lying, destroying evidence, and threatening witnesses) vary in proportion (i) to one another, and (ii) to the degree to which audiences devalue the disgraced individual-the threat shame defends against. These responses cohere both within and between the United States and India. Further, alternative explanations involving the low-level variable of arousal do not seem to account for these results, suggesting that coherence is imparted by a shame system. These findings indicate that coherence can be observed at multiple levels and raise the possibility that emotion programs orchestrate responses, even in those situations where coherence is low.

Semantic Space Theory: A Computational Approach to Emotion

Within affective science, the central line of inquiry, animated by basic emotion theory and constructivist accounts, has been the search for one-to-one mappings between six emotions and their subjective experiences, prototypical expressions, and underlying brain states. We offer an alternative perspective: semantic space theory. This computational approach uses wide-ranging naturalistic stimuli and open-ended statistical techniques to capture systematic variation in emotion-related behaviors. Upwards of 25 distinct varieties of emotional experience have distinct profiles of associated antecedents and expressions. These emotions are high-dimensional, categorical, and often blended. This approach also reveals that specific emotions, more than valence, organize emotional experience, expression, and neural processing. Overall, moving beyond traditional models to study broader semantic spaces of emotion can enrich our understanding of human experience.

Tracking historical changes in trustworthiness using machine learning analyses of facial cues in paintings

Social trust is linked to a host of positive societal outcomes, including improved economic performance, lower crime rates and more inclusive institutions. Yet, the origins of trust remain elusive, partly because social trust is difficult to document in time. Building on recent advances in social cognition, we design an algorithm to automatically estimate ratings of perceived trustworthiness evaluations from specific facial cues (such as muscle contractions associated with smiling) detected in European portraits in large historical databases. We used this measure as a proxy of social trust in history. Our results show that estimated levels of perceived trustworthiness in portraits increased over the period 1500–2000. Further analyses suggest that this rise of perceived trustworthiness is associated with increased living standards.

Quantifying how social trust evolved throughout history can help us understand the long-run dynamics of our societies. Here, the authors show an increase in displays of trustworthiness, using a face processing algorithm on early to modern European portraits.