Universal facial expressions uncovered in art of the ancient Americas: A computational approach

Emotional palette: a computational mapping of aesthetic experiences evoked by visual art

Despite the evolutionary history and cultural significance of visual art, the structure of aesthetic experiences it evokes has only attracted recent scientific attention. What kinds of experience does visual art evoke? Guided by Semantic Space Theory, we identify the concepts that most precisely describe people's aesthetic experiences using new computational techniques. Participants viewed 1457 artworks sampled from diverse cultural and historical traditions and reported on the emotions they felt and their perceived artwork qualities. Results show that aesthetic experiences are high-dimensional, comprising 25 categories of feeling states. Extending well beyond hedonism and broad evaluative judgments (e.g., pleasant/unpleasant), aesthetic experiences involve emotions of daily social living (e.g., "sad", "joy"), the imagination (e.g., "psychedelic", "mysterious"), profundity (e.g., "disgust", "awe"), and perceptual qualities attributed to the artwork (e.g., "whimsical", "disorienting"). Aesthetic emotions and perceptual qualities jointly predict viewers' liking of the artworks, indicating that we conceptualize aesthetic experiences in terms of the emotions we feel but also the qualities we perceive in the artwork. Aesthetic experiences are often mixed and lie along continuous gradients between categories rather than within discrete clusters. Our collection of artworks is visualized within an interactive map ( https://barradeau.com/2021/emotions-map/ ), revealing the high-dimensional space of aesthetic experiences associated with visual art.

How emotion is experienced and expressed in multiple cultures: a large-scale experiment across North America, Europe, and Japan

Core to understanding emotion are subjective experiences and their expression in facial behavior. Past studies have largely focused on six emotions and prototypical facial poses, reflecting limitations in scale and narrow assumptions about the variety of emotions and their patterns of expression. We examine 45,231 facial reactions to 2,185 evocative videos, largely in North America, Europe, and Japan, collecting participants' self-reported experiences in English or Japanese and manual and automated annotations of facial movement. Guided by Semantic Space Theory, we uncover 21 dimensions of emotion in the self-reported experiences of participants in Japan, the United States, and Western Europe, and considerable cross-cultural similarities in experience. Facial expressions predict at least 12 dimensions of experience, despite massive individual differences in experience. We find considerable cross-cultural convergence in the facial actions involved in the expression of emotion, and culture-specific display tendencies-many facial movements differ in intensity in Japan compared to the U.S./Canada and Europe but represent similar experiences. These results quantitatively detail that people in dramatically different cultures experience and express emotion in a high-dimensional, categorical, and similar but complex fashion.

Facial icons as indexes of emotions and intentions

Various objects and artifacts incorporate representations of faces, encompassing artworks like portraits, as well as ethnographic or industrial artifacts such as masks or humanoid robots. These representations exhibit diverse degrees of human-likeness, serving different functions and objectives. Despite these variations, they share common features, particularly facial attributes that serve as building blocks for facial expressions-an effective means of communicating emotions. To provide a unified conceptualization for this broad spectrum of face representations, we propose the term "facial icons" drawing upon Peirce's semiotic concepts. Additionally, based on these semiotic principles, we posit that facial icons function as indexes of emotions and intentions, and introduce a significant anthropological theory aligning with our proposition. Subsequently, we support our assertions by examining processes related to face and facial expression perception, as well as sensorimotor simulation processes involved in discerning others' mental states, including emotions. Our argumentation integrates cognitive and experimental evidence, reinforcing the pivotal role of facial icons in conveying mental states.

Deep learning reveals what facial expressions mean to people in different cultures

Cross-cultural studies of the meaning of facial expressions have largely focused on judgments of small sets of stereotypical images by small numbers of people. Here, we used large-scale data collection and machine learning to map what facial expressions convey in six countries. Using a mimicry paradigm, 5,833 participants formed facial expressions found in 4,659 naturalistic images, resulting in 423,193 participant-generated facial expressions. In their own language, participants also rated each expression in terms of 48 emotions and mental states. A deep neural network tasked with predicting the culture-specific meanings people attributed to facial movements while ignoring physical appearance and context discovered 28 distinct dimensions of facial expression, with 21 dimensions showing strong evidence of universality and the remainder showing varying degrees of cultural specificity. These results capture the underlying dimensions of the meanings of facial expressions within and across cultures in unprecedented detail.

Universal interpretations of vocal music

Despite the variability of music across cultures, some types of human songs share acoustic characteristics. For example, dance songs tend to be loud and rhythmic, and lullabies tend to be quiet and melodious. Human perceptual sensitivity to the behavioral contexts of songs, based on these musical features, suggests that basic properties of music are mutually intelligible, independent of linguistic or cultural content. Whether these effects reflect universal interpretations of vocal music, however, is unclear because prior studies focus almost exclusively on English-speaking participants, a group that is not representative of humans. Here, we report shared intuitions concerning the behavioral contexts of unfamiliar songs produced in unfamiliar languages, in participants living in Internet-connected industrialized societies (n = 5,516 native speakers of 28 languages) or smaller-scale societies with limited access to global media (n = 116 native speakers of three non-English languages). Participants listened to songs randomly selected from a representative sample of human vocal music, originally used in four behavioral contexts, and rated the degree to which they believed the song was used for each context. Listeners in both industrialized and smaller-scale societies inferred the contexts of dance songs, lullabies, and healing songs, but not love songs. Within and across cohorts, inferences were mutually consistent. Further, increased linguistic or geographical proximity between listeners and singers only minimally increased the accuracy of the inferences. These results demonstrate that the behavioral contexts of three common forms of music are mutually intelligible cross-culturally and imply that musical diversity, shaped by cultural evolution, is nonetheless grounded in some universal perceptual phenomena.

Beyond Nature Versus Nurture: the Emergence of Emotion

Affective science is stuck in a version of the nature-versus-nurture debate, with theorists arguing whether emotions are evolved adaptations or psychological constructions. We do not see these as mutually exclusive options. Many adaptive behaviors that humans have evolved to be good at, such as walking, emerge during development - not according to a genetically dictated program, but through interactions between the affordances of the body, brain, and environment. We suggest emotions are the same. As developing humans acquire increasingly complex goals and learn optimal strategies for pursuing those goals, they are inevitably pulled to particular brain-body-behavior states that maximize outcomes and self-reinforce via positive feedback loops. We call these recurring, self-organized states emotions. Emotions display many of the hallmark features of self-organized attractor states, such as hysteresis (prior events influence the current state), degeneracy (many configurations of the underlying variables can produce the same global state), and stability. Because most bodily, neural, and environmental affordances are shared by all humans - we all have cardiovascular systems, cerebral cortices, and caregivers who raised us - similar emotion states emerge in all of us. This perspective helps reconcile ideas that, at first glance, seem contradictory, such as emotion universality and neural degeneracy.

The nature and neurobiology of fear and anxiety: State of the science and opportunities for accelerating discovery

Fear and anxiety play a central role in mammalian life, and there is considerable interest in clarifying their nature, identifying their biological underpinnings, and determining their consequences for health and disease. Here we provide a roundtable discussion on the nature and biological bases of fear- and anxiety-related states, traits, and disorders. The discussants include scientists familiar with a wide variety of populations and a broad spectrum of techniques. The goal of the roundtable was to take stock of the state of the science and provide a roadmap to the next generation of fear and anxiety research. Much of the discussion centered on the key challenges facing the field, the most fruitful avenues for future research, and emerging opportunities for accelerating discovery, with implications for scientists, funders, and other stakeholders. Understanding fear and anxiety is a matter of practical importance. Anxiety disorders are a leading burden on public health and existing treatments are far from curative, underscoring the urgency of developing a deeper understanding of the factors governing threat-related emotions.

Deep learning reveals what vocal bursts express in different cultures

Human social life is rich with sighs, chuckles, shrieks and other emotional vocalizations, called 'vocal bursts'. Nevertheless, the meaning of vocal bursts across cultures is only beginning to be understood. Here, we combined large-scale experimental data collection with deep learning to reveal the shared and culture-specific meanings of vocal bursts. A total of n = 4,031 participants in China, India, South Africa, the USA and Venezuela mimicked vocal bursts drawn from 2,756 seed recordings. Participants also judged the emotional meaning of each vocal burst. A deep neural network tasked with predicting the culture-specific meanings people attributed to vocal bursts while disregarding context and speaker identity discovered 24 acoustic dimensions, or kinds, of vocal expression with distinct emotion-related meanings. The meanings attributed to these complex vocal modulations were 79% preserved across the five countries and three languages. These results reveal the underlying dimensions of human emotional vocalization in remarkable detail.

How to Develop Reliable Instruments to Measure the Cultural Evolution of Preferences and Feelings in History?

While we cannot directly measure the psychological preferences of individuals, and the moral, emotional, and cognitive tendencies of people from the past, we can use cultural artifacts as a window to the zeitgeist of societies in particular historical periods. At present, an increasing number of digitized texts spanning several centuries is available for a computerized analysis. In addition, developments form historical economics have enabled increasingly precise estimations of sociodemographic realities from the past. Crossing these datasets offer a powerful tool to test how the environment changes psychology and vice versa. However, designing the appropriate proxies of relevant psychological constructs is not trivial. The gold standard to measure psychological constructs in modern texts - Linguistic Inquiry and Word Count (LIWC) - has been validated by psychometric experimentation with modern participants. However, as a tool to investigate the psychology of the past, the LIWC is limited in two main aspects: (1) it does not cover the entire range of relevant psychological dimensions and (2) the meaning, spelling, and pragmatic use of certain words depend on the historical period from which the fiction work is sampled. These LIWC limitations make the design of custom tools inevitable. However, without psychometric validation, there is uncertainty regarding what exactly is being measured. To overcome these pitfalls, we suggest several internal and external validation procedures, to be conducted prior to diachronic analyses. First, the semantic adequacy of search terms in bags-of-words approaches should be verified by training semantic vector spaces with the historical text corpus using tools like word2vec. Second, we propose factor analyses to evaluate the internal consistency between distinct bag-of-words proxying the same underlying psychological construct. Third, these proxies can be externally validated using prior knowledge on the differences between genres or other literary dimensions. Finally, while LIWC is limited in the analysis of historical documents, it can be used as a sanity check for external validation of custom measures. This procedure allows a robust estimation of psychological constructs and how they change throughout history. Together with historical economics, it also increases our power in testing the relationship between environmental change and the expression of psychological traits from the past.

Demographic and sociocultural predictors of face image satisfaction: The U.S. Body Project I

Despite substantial literature surrounding how people process and perceive faces, there is very little research investigating how people evaluate their own faces. We examined how gender, body mass, race, age, and sexual orientation were linked to people's satisfaction with the appearance of their eyes, nose, facial shape, and face overall among 11,620 adults recruited via Mechanical Turk. Most people mostly or definitely agreed they were happy with their facial appearance. There were notable racial differences, with Asian participants tending to report greater dissatisfaction. For example, only 66% of Asian women and 60% of Asian men mostly or definitely agreed that they were happy with the appearance of their eyes, which was lower than other racial groups. BMI and age were not strongly associated with face satisfaction. Sexual minority men were less satisfied than heterosexual men. About one in four gay and bisexual men, compared to only one in seven heterosexual men, reported dissatisfaction with their overall facial appearance. Men and women with poorer face image engaged in more appearance surveillance, more strongly internalized the thin-ideal, and perceived stronger sociocultural pressures from peers, parents, and media. The current study highlights important sociocultural and demographic factors tied to poorer face image.

Emotion Perception Rules Abide by Cultural Display Rules

The current study compared emotion perception in two cultures where display rules for emotion expression deviate. In Experiment 1, participants from America and Korea played a repeated prisoner's dilemma game with a counterpart, who was, in actuality, a programmed defector. Emotion expressions were exchanged via emoticons at the end of every round. After winning more points by defecting, the counterpart sent either a matching emoticon (a joyful face) or a mismatching emoticon (a regretful face). The results showed that Americans in the matching condition were more likely to defect, or to punish, compared to those in the mismatching condition, suggesting that more weight was given to their counterpart's joyful expression. This difference was smaller for Koreans, suggesting a higher disregard for the outward expression. In a second, supplementary experiment, we found that Korean participants were more likely to cooperate in the mismatching or regretful condition, when they thought their counterpart was a Westerner. Overall, our data suggest that emotion perception rules abide by the display rules of one's culture but are also influenced by the counterpart's culture.

An Online Experiment During COVID-19: Testing the Influences of Autonomy Support Toward Emotions and Academic Persistence

Students’ academic persistence is a critical component of effective online learning. Promoting students’ academic persistence could potentially alleviate learning loss or drop-out, especially during challenging time like the COVID-19 pandemic. Previous research indicated that different emotions and autonomy support could all influence students’ academic persistence. However, few studies examined the multidimensionality of persistence using an experimental design with students’ real-time emotions. Using an experimental design and the Contain Intelligent Facial Expression Recognition System (CIFERS), this research explored the dynamic associations among real-time emotions (joy and anxiety), autonomy support (having choice and no choice), self-perceived persistence, self-reliance persistence, and help-seeking persistence. 177 college students participated in this study online via Zoom during COVID-19 university closure. The results revealed that having choice and high intensity of joy could promote students’ self-reliance persistence, but not help-seeking persistence. Interestingly, students who perceived themselves as more persistent experienced more joy during experiment. The theoretical and practical implications on facilitating students’ academic persistence were discussed.

From Text to Thought: How Analyzing Language Can Advance Psychological Science

Humans have been using language for millennia but have only just begun to scratch the surface of what natural language can reveal about the mind. Here we propose that language offers a unique window into psychology. After briefly summarizing the legacy of language analyses in psychological science, we show how methodological advances have made these analyses more feasible and insightful than ever before. In particular, we describe how two forms of language analysis—natural-language processing and comparative linguistics—are contributing to how we understand topics as diverse as emotion, creativity, and religion and overcoming obstacles related to statistical power and culturally diverse samples. We summarize resources for learning both of these methods and highlight the best way to combine language analysis with more traditional psychological paradigms. Applying language analysis to large-scale and cross-cultural datasets promises to provide major breakthroughs in psychological science.

Are Emotions Natural Kinds After All? Rethinking the Issue of Response Coherence

The synchronized co-activation of multiple responses-motivational, behavioral, and physiological-has been taken as a defining feature of emotion. Such response coherence has been observed inconsistently however, and this has led some to view emotion programs as lacking biological reality. Yet, response coherence is not always expected or desirable if an emotion program is to carry out its adaptive function. Rather, the hallmark of emotion is the capacity to orchestrate multiple mechanisms adaptively-responses will co-activate in stereotypical fashion or not depending on how the emotion orchestrator interacts with the situation. Nevertheless, might responses cohere in the general case where input variables are specified minimally? Here we focus on shame as a case study. We measure participants' responses regarding each of 27 socially devalued actions and personal characteristics. We observe internal and external coherence: The intensities of felt shame and of various motivations of shame (hiding, lying, destroying evidence, and threatening witnesses) vary in proportion (i) to one another, and (ii) to the degree to which audiences devalue the disgraced individual-the threat shame defends against. These responses cohere both within and between the United States and India. Further, alternative explanations involving the low-level variable of arousal do not seem to account for these results, suggesting that coherence is imparted by a shame system. These findings indicate that coherence can be observed at multiple levels and raise the possibility that emotion programs orchestrate responses, even in those situations where coherence is low.

Semantic Space Theory: A Computational Approach to Emotion

Within affective science, the central line of inquiry, animated by basic emotion theory and constructivist accounts, has been the search for one-to-one mappings between six emotions and their subjective experiences, prototypical expressions, and underlying brain states. We offer an alternative perspective: semantic space theory. This computational approach uses wide-ranging naturalistic stimuli and open-ended statistical techniques to capture systematic variation in emotion-related behaviors. Upwards of 25 distinct varieties of emotional experience have distinct profiles of associated antecedents and expressions. These emotions are high-dimensional, categorical, and often blended. This approach also reveals that specific emotions, more than valence, organize emotional experience, expression, and neural processing. Overall, moving beyond traditional models to study broader semantic spaces of emotion can enrich our understanding of human experience.

Tracking historical changes in trustworthiness using machine learning analyses of facial cues in paintings

Social trust is linked to a host of positive societal outcomes, including improved economic performance, lower crime rates and more inclusive institutions. Yet, the origins of trust remain elusive, partly because social trust is difficult to document in time. Building on recent advances in social cognition, we design an algorithm to automatically estimate ratings of perceived trustworthiness evaluations from specific facial cues (such as muscle contractions associated with smiling) detected in European portraits in large historical databases. We used this measure as a proxy of social trust in history. Our results show that estimated levels of perceived trustworthiness in portraits increased over the period 1500–2000. Further analyses suggest that this rise of perceived trustworthiness is associated with increased living standards.

Quantifying how social trust evolved throughout history can help us understand the long-run dynamics of our societies. Here, the authors show an increase in displays of trustworthiness, using a face processing algorithm on early to modern European portraits.