1
|
Han Y, Adolphs R. A shared structure for emotion experiences from narratives, videos, and everyday life. iScience 2024; 27:110378. [PMID: 39100924 PMCID: PMC11296042 DOI: 10.1016/j.isci.2024.110378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 05/03/2024] [Accepted: 06/24/2024] [Indexed: 08/06/2024] Open
Abstract
Our knowledge of the diversity and psychological organization of emotion experiences is based primarily on studies that used a single type of stimulus with an often limited set of rating scales and analyses. Here we take a comprehensive data-driven approach. We surveyed 1,000+ participants on a diverse set of ratings of emotion experiences to a validated set of ca. 150 text narratives, a validated set of ca. 1,000 videos, and over 10,000 personal experiences sampled longitudinally in everyday life, permitting a unique comparison. All three types of emotion experiences were characterized by similar dimensional spaces that included valence and arousal, as well as dimensions related to generalizability. Emotion experiences were distributed along continuous gradients, with no clear clusters even for the so-called basic emotions. Individual differences in personality traits were associated with differences in everyday emotion experiences but not with emotions evoked by narratives or videos.
Collapse
Affiliation(s)
- Yanting Han
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA 91125, USA
| | - The COVID-Dynamic Team
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA 91125, USA
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| | - Ralph Adolphs
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA 91125, USA
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| |
Collapse
|
2
|
Xiao C, Liu J. Semantic effects on the perception of emotional prosody in native and non-native Chinese speakers. Cogn Emot 2024:1-11. [PMID: 38973172 DOI: 10.1080/02699931.2024.2371088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Accepted: 06/16/2024] [Indexed: 07/09/2024]
Abstract
While previous research has found an in-group advantage (IGA) favouring native speakers in emotional prosody perception over non-native speakers, the effects of semantics on emotional prosody perception remain unclear. This study investigated the effects of semantics on emotional prosody perception in Chinese words and sentences for native and non-native Chinese speakers. The critical manipulation was the congruence of prosodic (positive, negative) and semantic (positive, negative, and neutral) valence. Participants listened to a series of audio clips and judged whether the emotional prosody was positive or negative for each utterance. The results revealed an IGA effect: native speakers perceived emotional prosody more accurately and quickly than non-native speakers in Chinese words and sentences. Furthermore, a semantic congruence effect was observed in Chinese words, where both native and non-native speakers recognised emotional prosody more accurately in the semantic-prosody congruent condition than in the incongruent condition. However, in Chinese sentences, this congruence effect was only present for non-native speakers. Additionally, the IGA effect and semantic congruence effect on emotional prosody perception were influenced by prosody valence. These findings illuminate the role of semantics in emotional prosody perception, highlighting perceptual differences between native and non-native Chinese speakers.
Collapse
Affiliation(s)
- Cheng Xiao
- Linguistics Program, University of South Carolina, Columbia, SC, USA
| | - Jiang Liu
- Linguistics Program, University of South Carolina, Columbia, SC, USA
- Department of Language, Literatures and Cultures, University of South Carolina, Columbia, SC, USA
| |
Collapse
|
3
|
Cowen AS, Brooks JA, Prasad G, Tanaka M, Kamitani Y, Kirilyuk V, Somandepalli K, Jou B, Schroff F, Adam H, Sauter D, Fang X, Manokara K, Tzirakis P, Oh M, Keltner D. How emotion is experienced and expressed in multiple cultures: a large-scale experiment across North America, Europe, and Japan. Front Psychol 2024; 15:1350631. [PMID: 38966733 PMCID: PMC11223574 DOI: 10.3389/fpsyg.2024.1350631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Accepted: 03/04/2024] [Indexed: 07/06/2024] Open
Abstract
Core to understanding emotion are subjective experiences and their expression in facial behavior. Past studies have largely focused on six emotions and prototypical facial poses, reflecting limitations in scale and narrow assumptions about the variety of emotions and their patterns of expression. We examine 45,231 facial reactions to 2,185 evocative videos, largely in North America, Europe, and Japan, collecting participants' self-reported experiences in English or Japanese and manual and automated annotations of facial movement. Guided by Semantic Space Theory, we uncover 21 dimensions of emotion in the self-reported experiences of participants in Japan, the United States, and Western Europe, and considerable cross-cultural similarities in experience. Facial expressions predict at least 12 dimensions of experience, despite massive individual differences in experience. We find considerable cross-cultural convergence in the facial actions involved in the expression of emotion, and culture-specific display tendencies-many facial movements differ in intensity in Japan compared to the U.S./Canada and Europe but represent similar experiences. These results quantitatively detail that people in dramatically different cultures experience and express emotion in a high-dimensional, categorical, and similar but complex fashion.
Collapse
Affiliation(s)
- Alan S. Cowen
- Hume AI, New York, NY, United States
- Department of Psychology, University of California, Berkeley, Berkeley, CA, United States
| | - Jeffrey A. Brooks
- Hume AI, New York, NY, United States
- Department of Psychology, University of California, Berkeley, Berkeley, CA, United States
| | | | - Misato Tanaka
- Advanced Telecommunications Research Institute, Kyoto, Japan
- Graduate School of Informatics, Kyoto University, Kyoto, Japan
| | - Yukiyasu Kamitani
- Advanced Telecommunications Research Institute, Kyoto, Japan
- Graduate School of Informatics, Kyoto University, Kyoto, Japan
| | | | - Krishna Somandepalli
- Google Research, Mountain View, CA, United States
- Department of Electrical Engineering, University of Southern California, Los Angeles, CA, United States
| | - Brendan Jou
- Google Research, Mountain View, CA, United States
| | | | - Hartwig Adam
- Google Research, Mountain View, CA, United States
| | - Disa Sauter
- Faculty of Social and Behavioural Sciences, University of Amsterdam, Amsterdam, Netherlands
| | - Xia Fang
- Zhejiang University, Zhejiang, China
| | - Kunalan Manokara
- Faculty of Social and Behavioural Sciences, University of Amsterdam, Amsterdam, Netherlands
| | | | - Moses Oh
- Hume AI, New York, NY, United States
| | - Dacher Keltner
- Hume AI, New York, NY, United States
- Department of Psychology, University of California, Berkeley, Berkeley, CA, United States
| |
Collapse
|
4
|
Brooks JA, Kim L, Opara M, Keltner D, Fang X, Monroy M, Corona R, Tzirakis P, Baird A, Metrick J, Taddesse N, Zegeye K, Cowen AS. Deep learning reveals what facial expressions mean to people in different cultures. iScience 2024; 27:109175. [PMID: 38433918 PMCID: PMC10906517 DOI: 10.1016/j.isci.2024.109175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2023] [Revised: 09/05/2023] [Accepted: 02/06/2024] [Indexed: 03/05/2024] Open
Abstract
Cross-cultural studies of the meaning of facial expressions have largely focused on judgments of small sets of stereotypical images by small numbers of people. Here, we used large-scale data collection and machine learning to map what facial expressions convey in six countries. Using a mimicry paradigm, 5,833 participants formed facial expressions found in 4,659 naturalistic images, resulting in 423,193 participant-generated facial expressions. In their own language, participants also rated each expression in terms of 48 emotions and mental states. A deep neural network tasked with predicting the culture-specific meanings people attributed to facial movements while ignoring physical appearance and context discovered 28 distinct dimensions of facial expression, with 21 dimensions showing strong evidence of universality and the remainder showing varying degrees of cultural specificity. These results capture the underlying dimensions of the meanings of facial expressions within and across cultures in unprecedented detail.
Collapse
Affiliation(s)
- Jeffrey A. Brooks
- Research Division, Hume AI, New York, NY 10010, USA
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Lauren Kim
- Research Division, Hume AI, New York, NY 10010, USA
| | | | - Dacher Keltner
- Research Division, Hume AI, New York, NY 10010, USA
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Xia Fang
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, Zhejiang, China
| | - Maria Monroy
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Rebecca Corona
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| | | | - Alice Baird
- Research Division, Hume AI, New York, NY 10010, USA
| | | | | | | | - Alan S. Cowen
- Research Division, Hume AI, New York, NY 10010, USA
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| |
Collapse
|
5
|
Larrouy-Maestri P, Poeppel D, Pell MD. The Sound of Emotional Prosody: Nearly 3 Decades of Research and Future Directions. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2024:17456916231217722. [PMID: 38232303 DOI: 10.1177/17456916231217722] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/19/2024]
Abstract
Emotional voices attract considerable attention. A search on any browser using "emotional prosody" as a key phrase leads to more than a million entries. Such interest is evident in the scientific literature as well; readers are reminded in the introductory paragraphs of countless articles of the great importance of prosody and that listeners easily infer the emotional state of speakers through acoustic information. However, despite decades of research on this topic and important achievements, the mapping between acoustics and emotional states is still unclear. In this article, we chart the rich literature on emotional prosody for both newcomers to the field and researchers seeking updates. We also summarize problems revealed by a sample of the literature of the last decades and propose concrete research directions for addressing them, ultimately to satisfy the need for more mechanistic knowledge of emotional prosody.
Collapse
Affiliation(s)
- Pauline Larrouy-Maestri
- Max Planck Institute for Empirical Aesthetics, Frankfurt, Germany
- School of Communication Sciences and Disorders, McGill University
- Max Planck-NYU Center for Language, Music, and Emotion, New York, New York
| | - David Poeppel
- Max Planck-NYU Center for Language, Music, and Emotion, New York, New York
- Department of Psychology and Center for Neural Science, New York University
- Ernst Strüngmann Institute for Neuroscience, Frankfurt, Germany
| | - Marc D Pell
- School of Communication Sciences and Disorders, McGill University
- Centre for Research on Brain, Language, and Music, Montreal, Quebec, Canada
| |
Collapse
|
6
|
Tsikandilakis M, Bali P. Learning emotional dialects: A British population study of cross-cultural communication. Perception 2023; 52:812-843. [PMID: 37796849 PMCID: PMC10634218 DOI: 10.1177/03010066231204180] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Accepted: 09/12/2023] [Indexed: 10/07/2023]
Abstract
The aim of the current research was to explore whether we can improve the recognition of cross-cultural freely-expressed emotional faces in British participants. We tested several methods for improving the recognition of freely-expressed emotional faces, such as different methods for presenting other-culture expressions of emotion from individuals from Chile, New Zealand and Singapore in two experimental stages. In the first experimental stage, in phase one, participants were asked to identify the emotion of cross-cultural freely-expressed faces. In the second phase, different cohorts were presented with interactive side-by-side, back-to-back and dynamic morphing of cross-cultural freely-expressed emotional faces, and control conditions. In the final phase, we repeated phase one using novel stimuli. We found that all non-control conditions led to recognition improvements. Morphing was the most effective condition for improving the recognition of cross-cultural emotional faces. In the second experimental stage, we presented morphing to different cohorts including own-to-other and other-to-own freely-expressed cross-cultural emotional faces and neutral-to-emotional and emotional-to-neutral other-culture freely-expressed emotional faces. All conditions led to recognition improvements and the presentation of freely-expressed own-to-other cultural-emotional faces provided the most effective learning. These findings suggest that training can improve the recognition of cross-cultural freely-expressed emotional expressions.
Collapse
|
7
|
Paletz SBF, Golonka EM, Pandža NB, Stanton G, Ryan D, Adams N, Rytting CA, Murauskaite EE, Buntain C, Johns MA, Bradley P. Social media emotions annotation guide (SMEmo): Development and initial validity. Behav Res Methods 2023:10.3758/s13428-023-02195-1. [PMID: 37697206 DOI: 10.3758/s13428-023-02195-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/10/2023] [Indexed: 09/13/2023]
Abstract
The proper measurement of emotion is vital to understanding the relationship between emotional expression in social media and other factors, such as online information sharing. This work develops a standardized annotation scheme for quantifying emotions in social media using recent emotion theory and research. Human annotators assessed both social media posts and their own reactions to the posts' content on scales of 0 to 100 for each of 20 (Study 1) and 23 (Study 2) emotions. For Study 1, we analyzed English-language posts from Twitter (N = 244) and YouTube (N = 50). Associations between emotion ratings and text-based measures (LIWC, VADER, EmoLex, NRC-EIL, Emotionality) demonstrated convergent and discriminant validity. In Study 2, we tested an expanded version of the scheme in-country, in-language, on Polish (N = 3648) and Lithuanian (N = 1934) multimedia Facebook posts. While the correlations were lower than with English, patterns of convergent and discriminant validity with EmoLex and NRC-EIL still held. Coder reliability was strong across samples, with intraclass correlations of .80 or higher for 10 different emotions in Study 1 and 16 different emotions in Study 2. This research improves the measurement of emotions in social media to include more dimensions, multimedia, and context compared to prior schemes.
Collapse
Affiliation(s)
- Susannah B F Paletz
- College of Information Studies, University of Maryland, College Park, MD, USA.
| | - Ewa M Golonka
- Applied Research Laboratory for Intelligence and Security (ARLIS), University of Maryland, College Park, MD, USA
| | - Nick B Pandža
- Applied Research Laboratory for Intelligence and Security (ARLIS), University of Maryland, College Park, MD, USA
- Program in Second Language Acquisition, University of Maryland, College Park, MD, USA
| | - Grace Stanton
- Department of Criminology, University of Maryland, College Park, MD, USA
| | - David Ryan
- Feminist, Gender, and Sexuality Studies, Stanford University, Stanford, CA, USA
| | - Nikki Adams
- Applied Research Laboratory for Intelligence and Security (ARLIS), University of Maryland, College Park, MD, USA
| | - C Anton Rytting
- Applied Research Laboratory for Intelligence and Security (ARLIS), University of Maryland, College Park, MD, USA
| | | | - Cody Buntain
- College of Information Studies, University of Maryland, College Park, MD, USA
| | - Michael A Johns
- Applied Research Laboratory for Intelligence and Security (ARLIS), University of Maryland, College Park, MD, USA
| | - Petra Bradley
- Applied Research Laboratory for Intelligence and Security (ARLIS), University of Maryland, College Park, MD, USA
| |
Collapse
|
8
|
Grogans SE, Bliss-Moreau E, Buss KA, Clark LA, Fox AS, Keltner D, Cowen AS, Kim JJ, Kragel PA, MacLeod C, Mobbs D, Naragon-Gainey K, Fullana MA, Shackman AJ. The nature and neurobiology of fear and anxiety: State of the science and opportunities for accelerating discovery. Neurosci Biobehav Rev 2023; 151:105237. [PMID: 37209932 PMCID: PMC10330657 DOI: 10.1016/j.neubiorev.2023.105237] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 05/11/2023] [Accepted: 05/13/2023] [Indexed: 05/22/2023]
Abstract
Fear and anxiety play a central role in mammalian life, and there is considerable interest in clarifying their nature, identifying their biological underpinnings, and determining their consequences for health and disease. Here we provide a roundtable discussion on the nature and biological bases of fear- and anxiety-related states, traits, and disorders. The discussants include scientists familiar with a wide variety of populations and a broad spectrum of techniques. The goal of the roundtable was to take stock of the state of the science and provide a roadmap to the next generation of fear and anxiety research. Much of the discussion centered on the key challenges facing the field, the most fruitful avenues for future research, and emerging opportunities for accelerating discovery, with implications for scientists, funders, and other stakeholders. Understanding fear and anxiety is a matter of practical importance. Anxiety disorders are a leading burden on public health and existing treatments are far from curative, underscoring the urgency of developing a deeper understanding of the factors governing threat-related emotions.
Collapse
Affiliation(s)
- Shannon E Grogans
- Department of Psychology, University of Maryland, College Park, MD 20742, USA
| | - Eliza Bliss-Moreau
- Department of Psychology, University of California, Davis, CA 95616, USA; California National Primate Research Center, University of California, Davis, CA 95616, USA
| | - Kristin A Buss
- Department of Psychology, The Pennsylvania State University, University Park, PA 16802 USA
| | - Lee Anna Clark
- Department of Psychology, University of Notre Dame, Notre Dame, IN 46556, USA
| | - Andrew S Fox
- Department of Psychology, University of California, Davis, CA 95616, USA; California National Primate Research Center, University of California, Davis, CA 95616, USA
| | - Dacher Keltner
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| | | | - Jeansok J Kim
- Department of Psychology, University of Washington, Seattle, WA 98195, USA
| | - Philip A Kragel
- Department of Psychology, Emory University, Atlanta, GA 30322, USA
| | - Colin MacLeod
- Centre for the Advancement of Research on Emotion, School of Psychological Science, The University of Western Australia, Perth, WA 6009, Australia
| | - Dean Mobbs
- Department of Humanities and Social Sciences, California Institute of Technology, Pasadena, California 91125, USA; Computation and Neural Systems Program, California Institute of Technology, Pasadena, CA 91125, USA
| | - Kristin Naragon-Gainey
- School of Psychological Science, University of Western Australia, Perth, WA 6009, Australia
| | - Miquel A Fullana
- Adult Psychiatry and Psychology Department, Institute of Neurosciences, Hospital Clinic, Barcelona, Spain; Imaging of Mood, and Anxiety-Related Disorders Group, Institut d'Investigacions Biomèdiques August Pi i Sunyer, CIBERSAM, University of Barcelona, Barcelona, Spain
| | - Alexander J Shackman
- Department of Psychology, University of Maryland, College Park, MD 20742, USA; Neuroscience and Cognitive Science Program, University of Maryland, College Park, MD 20742, USA; Maryland Neuroimaging Center, University of Maryland, College Park, MD 20742, USA.
| |
Collapse
|
9
|
Wu Y, Ying H. The background assimilation effect: Facial emotional perception is affected by surrounding stimuli. Iperception 2023; 14:20416695231190254. [PMID: 37654695 PMCID: PMC10467198 DOI: 10.1177/20416695231190254] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 07/10/2023] [Indexed: 09/02/2023] Open
Abstract
The perception of facial emotion is not only determined by the physical features of the face itself but also be influenced by the emotional information of the background or surrounding information. However, the details of such effect are not fully understood. Here, the authors tested the perceived emotion of a target face surrounded by stimuli with different levels of emotional valence. In Experiment 1, four types of objects were divided into three groups (negative, unpleasant flowers and unpleasant animals; mildly negative (neutral), houses; positive, pleasant flowers). In Experiment 2, three groups of surrounding faces with different social-emotional valence (negative, neutral, and positive) were formed with the memory of affective personal knowledge. The data from two experiments showed that the perception of facial emotion can be influenced and modulated by the emotional valence of the surrounding stimuli, which can be explained by assimilation: the positive stimuli increased the valence of a target face, while the negative stimuli comparatively decreased it. Furthermore, the neutral stimuli also increased the valence of the target, which could be explained by the social positive effect. Therefore, the process of assimilation is likely to be a high-level emotional cognition rather than a low-level visual perception. The results of this study may help us better understand face perception in realistic scenarios.
Collapse
Affiliation(s)
- Yujie Wu
- Department of Psychology, Soochow University, Suzhou, China
| | - Haojiang Ying
- Department of Psychology, Soochow University, Suzhou, China
| |
Collapse
|
10
|
Liu J, Hilton CB, Bergelson E, Mehr SA. Language experience predicts music processing in a half-million speakers of fifty-four languages. Curr Biol 2023; 33:1916-1925.e4. [PMID: 37105166 PMCID: PMC10306420 DOI: 10.1016/j.cub.2023.03.067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Revised: 02/08/2023] [Accepted: 03/23/2023] [Indexed: 04/29/2023]
Abstract
Tonal languages differ from other languages in their use of pitch (tones) to distinguish words. Lifelong experience speaking and hearing tonal languages has been argued to shape auditory processing in ways that generalize beyond the perception of linguistic pitch to the perception of pitch in other domains like music. We conducted a meta-analysis of prior studies testing this idea, finding moderate evidence supporting it. But prior studies were limited by mostly small sample sizes representing a small number of languages and countries, making it challenging to disentangle the effects of linguistic experience from variability in music training, cultural differences, and other potential confounds. To address these issues, we used web-based citizen science to assess music perception skill on a global scale in 34,034 native speakers of 19 tonal languages (e.g., Mandarin, Yoruba). We compared their performance to 459,066 native speakers of other languages, including 6 pitch-accented (e.g., Japanese) and 29 non-tonal languages (e.g., Hungarian). Whether or not participants had taken music lessons, native speakers of all 19 tonal languages had an improved ability to discriminate musical melodies on average, relative to speakers of non-tonal languages. But this improvement came with a trade-off: tonal language speakers were also worse at processing the musical beat. The results, which held across native speakers of many diverse languages and were robust to geographic and demographic variation, demonstrate that linguistic experience shapes music perception, with implications for relations between music, language, and culture in the human mind.
Collapse
Affiliation(s)
- Jingxuan Liu
- Columbia Business School, Columbia University, 665 W 130th Street, New York, NY 10027, USA; Department of Psychology & Neuroscience, Duke University, 417 Chapel Drive, Durham, NC 27708, USA.
| | - Courtney B Hilton
- Yale Child Study Center, Yale University, 300 George Street #900, New Haven, CT 06511, USA; School of Psychology, University of Auckland, 23 Symonds Street, Auckland 1010, New Zealand.
| | - Elika Bergelson
- Department of Psychology & Neuroscience, Duke University, 417 Chapel Drive, Durham, NC 27708, USA
| | - Samuel A Mehr
- Yale Child Study Center, Yale University, 300 George Street #900, New Haven, CT 06511, USA; School of Psychology, University of Auckland, 23 Symonds Street, Auckland 1010, New Zealand.
| |
Collapse
|
11
|
van Rijn P, Larrouy-Maestri P. Modelling individual and cross-cultural variation in the mapping of emotions to speech prosody. Nat Hum Behav 2023; 7:386-396. [PMID: 36646838 PMCID: PMC10038802 DOI: 10.1038/s41562-022-01505-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Accepted: 11/28/2022] [Indexed: 01/18/2023]
Abstract
The existence of a mapping between emotions and speech prosody is commonly assumed. We propose a Bayesian modelling framework to analyse this mapping. Our models are fitted to a large collection of intended emotional prosody, yielding more than 3,000 minutes of recordings. Our descriptive study reveals that the mapping within corpora is relatively constant, whereas the mapping varies across corpora. To account for this heterogeneity, we fit a series of increasingly complex models. Model comparison reveals that models taking into account mapping differences across countries, languages, sexes and individuals outperform models that only assume a global mapping. Further analysis shows that differences across individuals, cultures and sexes contribute more to the model prediction than a shared global mapping. Our models, which can be explored in an online interactive visualization, offer a description of the mapping between acoustic features and emotions in prosody.
Collapse
Affiliation(s)
- Pol van Rijn
- Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany.
| | - Pauline Larrouy-Maestri
- Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
- Max Planck-NYU Center for Language, Music, and Emotion, New York, NY, USA
| |
Collapse
|
12
|
Brooks JA, Tzirakis P, Baird A, Kim L, Opara M, Fang X, Keltner D, Monroy M, Corona R, Metrick J, Cowen AS. Deep learning reveals what vocal bursts express in different cultures. Nat Hum Behav 2023; 7:240-250. [PMID: 36577898 DOI: 10.1038/s41562-022-01489-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2022] [Accepted: 10/26/2022] [Indexed: 12/29/2022]
Abstract
Human social life is rich with sighs, chuckles, shrieks and other emotional vocalizations, called 'vocal bursts'. Nevertheless, the meaning of vocal bursts across cultures is only beginning to be understood. Here, we combined large-scale experimental data collection with deep learning to reveal the shared and culture-specific meanings of vocal bursts. A total of n = 4,031 participants in China, India, South Africa, the USA and Venezuela mimicked vocal bursts drawn from 2,756 seed recordings. Participants also judged the emotional meaning of each vocal burst. A deep neural network tasked with predicting the culture-specific meanings people attributed to vocal bursts while disregarding context and speaker identity discovered 24 acoustic dimensions, or kinds, of vocal expression with distinct emotion-related meanings. The meanings attributed to these complex vocal modulations were 79% preserved across the five countries and three languages. These results reveal the underlying dimensions of human emotional vocalization in remarkable detail.
Collapse
Affiliation(s)
- Jeffrey A Brooks
- Research Division, Hume AI, New York, NY, USA. .,University of California, Berkeley, Berkeley, CA, USA.
| | | | - Alice Baird
- Research Division, Hume AI, New York, NY, USA
| | - Lauren Kim
- Research Division, Hume AI, New York, NY, USA
| | | | - Xia Fang
- Zhejiang University, Hangzhou, China
| | - Dacher Keltner
- Research Division, Hume AI, New York, NY, USA.,University of California, Berkeley, Berkeley, CA, USA
| | - Maria Monroy
- University of California, Berkeley, Berkeley, CA, USA
| | | | | | - Alan S Cowen
- Research Division, Hume AI, New York, NY, USA. .,University of California, Berkeley, Berkeley, CA, USA.
| |
Collapse
|
13
|
Parada-Cabaleiro E, Batliner A, Schmitt M, Schedl M, Costantini G, Schuller B. Perception and classification of emotions in nonsense speech: Humans versus machines. PLoS One 2023; 18:e0281079. [PMID: 36716307 PMCID: PMC9886254 DOI: 10.1371/journal.pone.0281079] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Accepted: 01/13/2023] [Indexed: 02/01/2023] Open
Abstract
This article contributes to a more adequate modelling of emotions encoded in speech, by addressing four fallacies prevalent in traditional affective computing: First, studies concentrate on few emotions and disregard all other ones ('closed world'). Second, studies use clean (lab) data or real-life ones but do not compare clean and noisy data in a comparable setting ('clean world'). Third, machine learning approaches need large amounts of data; however, their performance has not yet been assessed by systematically comparing different approaches and different sizes of databases ('small world'). Fourth, although human annotations of emotion constitute the basis for automatic classification, human perception and machine classification have not yet been compared on a strict basis ('one world'). Finally, we deal with the intrinsic ambiguities of emotions by interpreting the confusions between categories ('fuzzy world'). We use acted nonsense speech from the GEMEP corpus, emotional 'distractors' as categories not entailed in the test set, real-life noises that mask the clear recordings, and different sizes of the training set for machine learning. We show that machine learning based on state-of-the-art feature representations (wav2vec2) is able to mirror the main emotional categories ('pillars') present in perceptual emotional constellations even in degradated acoustic conditions.
Collapse
Affiliation(s)
- Emilia Parada-Cabaleiro
- Institute of Computational Perception, Johannes Kepler University Linz, Linz, Austria
- Human-centered AI Group, Linz Institute of Technology (LIT), Linz, Austria
- Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany
- * E-mail:
| | - Anton Batliner
- Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany
| | - Maximilian Schmitt
- Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany
| | - Markus Schedl
- Institute of Computational Perception, Johannes Kepler University Linz, Linz, Austria
- Human-centered AI Group, Linz Institute of Technology (LIT), Linz, Austria
| | - Giovanni Costantini
- Department of Electronic Engineering, University of Rome Tor Vergata, Rome, Italy
| | - Björn Schuller
- Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany
- GLAM—Group on Language, Audio & Music, Imperial College London, London, United Kindom
| |
Collapse
|
14
|
Searching, Navigating, and Recommending Movies through Emotions: A Scoping Review. HUMAN BEHAVIOR AND EMERGING TECHNOLOGIES 2022. [DOI: 10.1155/2022/7831013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/04/2022]
Abstract
Movies offer viewers a broad range of emotional experiences, providing entertainment, and meaning. Following the PRISMA-ScR guidelines, we reviewed the literature on digital systems designed to help users search and browse movie libraries and offer recommendations based on emotional content. Our search yielded 83 eligible documents (published between 2000 and 2021). We identified 22 case studies, 34 empirical studies, 26 proof of concept, and one theoretical paper. User transactions (e.g., ratings, tags) were the preferred source of information. The documents examined approached emotions from both a categorical (
) and dimensional (
) perspectives, and nine documents offer a combination of both approaches. Although there are several authors mentioned, the references used are frequently dated, and 12 documents do not mention author or model used. We identified 61 words related to emotion or affect. Documents presented on average 1.36 positive terms and 2.64 negative terms. Sentiment analysis (
) is frequently used for emotion identification, followed by subjective evaluations (
), movie low-level audio and visual features (n = 11), and face recognition technologies (
). We discuss limitations and offer a brief review of current emotion models and research.
Collapse
|
15
|
Barrett LF. Context reconsidered: Complex signal ensembles, relational meaning, and population thinking in psychological science. AMERICAN PSYCHOLOGIST 2022; 77:894-920. [PMID: 36409120 PMCID: PMC9683522 DOI: 10.1037/amp0001054] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/26/2023]
Abstract
This article considers the status and study of "context" in psychological science through the lens of research on emotional expressions. The article begins by updating three well-trod methodological debates on the role of context in emotional expressions to reconsider several fundamental assumptions lurking within the field's dominant methodological tradition: namely, that certain expressive movements have biologically prepared, inherent emotional meanings that issue from singular, universal processes which are independent of but interact with contextual influences. The second part of this article considers the scientific opportunities that await if we set aside this traditional understanding of "context" as a moderator of signals with inherent psychological meaning and instead consider the possibility that psychological events emerge in ecosystems of signal ensembles, such that the psychological meaning of any individual signal is entirely relational. Such a fundamental shift has radical implications not only for the science of emotion but for psychological science more generally. It offers opportunities to improve the validity and trustworthiness of psychological science beyond what can be achieved with improvements to methodological rigor alone. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Collapse
|
16
|
Shakuf V, Ben-David B, Wegner TGG, Wesseling PBC, Mentzel M, Defren S, Allen SEM, Lachmann T. Processing emotional prosody in a foreign language: the case of German and Hebrew. JOURNAL OF CULTURAL COGNITIVE SCIENCE 2022; 6:251-268. [PMID: 35996660 PMCID: PMC9386669 DOI: 10.1007/s41809-022-00107-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2022] [Revised: 06/13/2022] [Accepted: 07/16/2022] [Indexed: 11/09/2022]
Abstract
This study investigated the universality of emotional prosody in perception of discrete emotions when semantics is not available. In two experiments the perception of emotional prosody in Hebrew and German by listeners who speak one of the languages but not the other was investigated. Having a parallel tool in both languages allowed to conduct controlled comparisons. In Experiment 1, 39 native German speakers with no knowledge of Hebrew and 80 native Israeli speakers rated Hebrew sentences spoken with four different emotional prosodies (anger, fear, happiness, sadness) or neutral. The Hebrew version of the Test for Rating of Emotions in Speech (T-RES) was used for this purpose. Ratings indicated participants’ agreement on how much the sentence conveyed each of four discrete emotions (anger, fear, happiness and sadness). In Experient 2, 30 native speakers of German, and 24 Israeli native speakers of Hebrew who had no knowledge of German rated sentences of the German version of the T-RES. Based only on the prosody, German-speaking participants were able to accurately identify the emotions in the Hebrew sentences and Hebrew-speaking participants were able to identify the emotions in the German sentences. In both experiments ratings between the groups were similar. These findings show that individuals are able to identify emotions in a foreign language even if they do not have access to semantics. This ability goes beyond identification of target emotion; similarities between languages exist even for “wrong” perception. This adds to accumulating evidence in the literature on the universality of emotional prosody.
Collapse
|
17
|
Su W, Sun X, Guo X, Zhang W, Li G. An analysis of awe evoked by COVID-19 on green purchasing behavior: A dual-path effect of approach-avoidance motivation. Front Psychol 2022; 13:952485. [PMID: 36033010 PMCID: PMC9407682 DOI: 10.3389/fpsyg.2022.952485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Accepted: 07/08/2022] [Indexed: 11/13/2022] Open
Abstract
The spread of the COVID-19 virus shows that it is time to re-emphasize the ethical attitude of “awe of others, awe of nature, and awe of life.” It once again reveals the importance of green development. In this study, we introduce awe into the context of COVID-19 and construct an “emotion-motivation-behavior” framework, aiming to explore the relationship between the epidemic and green purchasing behavior from a psychological perspective. Study 1 demonstrates the effect of awe on green purchasing and examines the mediating role of the motivation perspective, to reveal the potential different path. Specifically, prosocial motivation mediates the effect of positive awe evoked by COVID-19 on green purchasing; risk avoidance motivation mediates the effect of negative awe evoked by COVID-19 on green purchasing. Study 2 examined the moderating effect of self-construal. These findings have important management implications for enterprises to correctly use emotional guidance strategies and promote green marketing practices during the COVID-19.
Collapse
Affiliation(s)
- Weihuan Su
- School of Management, Wuhan University of Technology, Wuhan, China
| | - Xixiang Sun
- School of Management, Wuhan University of Technology, Wuhan, China
| | - Xiaodong Guo
- School of Management, Wuhan University of Technology, Wuhan, China
- *Correspondence: Xiaodong Guo
| | - Wei Zhang
- School of Safety Science and Emergency Management, Wuhan University of Technology, Wuhan, China
- Wei Zhang
| | - Gen Li
- School of Management, Wuhan University of Technology, Wuhan, China
| |
Collapse
|
18
|
Lau JCY, Patel S, Kang X, Nayar K, Martin GE, Choy J, Wong PCM, Losh M. Cross-linguistic patterns of speech prosodic differences in autism: A machine learning study. PLoS One 2022; 17:e0269637. [PMID: 35675372 PMCID: PMC9176813 DOI: 10.1371/journal.pone.0269637] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Accepted: 05/24/2022] [Indexed: 11/19/2022] Open
Abstract
Differences in speech prosody are a widely observed feature of Autism Spectrum Disorder (ASD). However, it is unclear how prosodic differences in ASD manifest across different languages that demonstrate cross-linguistic variability in prosody. Using a supervised machine-learning analytic approach, we examined acoustic features relevant to rhythmic and intonational aspects of prosody derived from narrative samples elicited in English and Cantonese, two typologically and prosodically distinct languages. Our models revealed successful classification of ASD diagnosis using rhythm-relative features within and across both languages. Classification with intonation-relevant features was significant for English but not Cantonese. Results highlight differences in rhythm as a key prosodic feature impacted in ASD, and also demonstrate important variability in other prosodic properties that appear to be modulated by language-specific differences, such as intonation.
Collapse
Affiliation(s)
- Joseph C. Y. Lau
- Roxelyn and Richard Pepper Department of Communication Sciences and Disorders, Northwestern University, Evanston, Illinois, United States of America
| | - Shivani Patel
- Roxelyn and Richard Pepper Department of Communication Sciences and Disorders, Northwestern University, Evanston, Illinois, United States of America
| | - Xin Kang
- Department of Linguistics and Modern Languages, The Chinese University of Hong Kong, Hong Kong S.A.R., China
- Brain and Mind Institute, The Chinese University of Hong Kong, Hong Kong S.A.R., China
- Research Centre for Language, Cognition and Language Application, Chongqing University, Chongqing, China
- School of Foreign Languages and Cultures, Chongqing University, Chongqing, China
| | - Kritika Nayar
- Roxelyn and Richard Pepper Department of Communication Sciences and Disorders, Northwestern University, Evanston, Illinois, United States of America
| | - Gary E. Martin
- Department of Communication Sciences and Disorders, St. John’s University, Staten Island, New York, United States of America
| | - Jason Choy
- Department of Linguistics and Modern Languages, The Chinese University of Hong Kong, Hong Kong S.A.R., China
| | - Patrick C. M. Wong
- Department of Linguistics and Modern Languages, The Chinese University of Hong Kong, Hong Kong S.A.R., China
- Brain and Mind Institute, The Chinese University of Hong Kong, Hong Kong S.A.R., China
| | - Molly Losh
- Roxelyn and Richard Pepper Department of Communication Sciences and Disorders, Northwestern University, Evanston, Illinois, United States of America
| |
Collapse
|
19
|
Patlar Akbulut F. Hybrid deep convolutional model-based emotion recognition using multiple physiological signals. Comput Methods Biomech Biomed Engin 2022; 25:1678-1690. [PMID: 35107402 DOI: 10.1080/10255842.2022.2032682] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
Emotion recognition has become increasingly utilized in the medical, advertising, and military domains. Recognizing the cues of emotion from human behaviors or physiological responses is encouraging for the research community. However, extracting true characteristics from sensor data to understand emotions can be challenging due to the complex nature of these signals. Therefore, advanced feature engineering techniques are required for accurate signal recognition. This study presents a hybrid affective model that employs a transfer learning approach for emotion classification using large-frame sensor signals which employ a genuine dataset of signal fusion gathered from 30 participants using wearable sensor systems interconnected with mobile devices. The proposed approach implements several learning algorithms such as Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Recurrent Neural Network (RNN), and several other shallow methods on the sensor input to handle the requirements for the traditional feature extraction process. The findings reveal that the use of deep learning methods is satisfactory in affect recognition when a great number of frames is employed, and the proposed hybrid deep model outperforms traditional neural network (overall accuracy of 54%) and deep learning approaches (overall accuracy of 76%), with an average classification accuracy of 93%. This hybrid deep model also has a higher accuracy than our previously proposed statistical autoregressive hidden Markov model (AR-HMM) approach, with 88.6% accuracy. Accuracy assessment was performed by means of several statistics measures (accuracy, precision, recall, F-measure, and RMSE).
Collapse
Affiliation(s)
- Fatma Patlar Akbulut
- Department of Computer Engineering, Istanbul Kültür University, Istanbul, Turkey
| |
Collapse
|
20
|
Fernández Carbonell M, Boman M, Laukka P. Comparing supervised and unsupervised approaches to multimodal emotion recognition. PeerJ Comput Sci 2021; 7:e804. [PMID: 35036530 PMCID: PMC8725659 DOI: 10.7717/peerj-cs.804] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2021] [Accepted: 11/11/2021] [Indexed: 06/14/2023]
Abstract
We investigated emotion classification from brief video recordings from the GEMEP database wherein actors portrayed 18 emotions. Vocal features consisted of acoustic parameters related to frequency, intensity, spectral distribution, and durations. Facial features consisted of facial action units. We first performed a series of person-independent supervised classification experiments. Best performance (AUC = 0.88) was obtained by merging the output from the best unimodal vocal (Elastic Net, AUC = 0.82) and facial (Random Forest, AUC = 0.80) classifiers using a late fusion approach and the product rule method. All 18 emotions were recognized with above-chance recall, although recognition rates varied widely across emotions (e.g., high for amusement, anger, and disgust; and low for shame). Multimodal feature patterns for each emotion are described in terms of the vocal and facial features that contributed most to classifier performance. Next, a series of exploratory unsupervised classification experiments were performed to gain more insight into how emotion expressions are organized. Solutions from traditional clustering techniques were interpreted using decision trees in order to explore which features underlie clustering. Another approach utilized various dimensionality reduction techniques paired with inspection of data visualizations. Unsupervised methods did not cluster stimuli in terms of emotion categories, but several explanatory patterns were observed. Some could be interpreted in terms of valence and arousal, but actor and gender specific aspects also contributed to clustering. Identifying explanatory patterns holds great potential as a meta-heuristic when unsupervised methods are used in complex classification tasks.
Collapse
Affiliation(s)
- Marcos Fernández Carbonell
- Department of Software and Computer Systems, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden
| | - Magnus Boman
- Department of Software and Computer Systems, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden
- Department of Learning, Informatics, Management and Ethics (LIME), Karolinska Institutet, Stockholm, Sweden
| | - Petri Laukka
- Department of Psychology, Stockholm University, Stockholm, Sweden
| |
Collapse
|
21
|
Mexican Emotional Speech Database Based on Semantic, Frequency, Familiarity, Concreteness, and Cultural Shaping of Affective Prosody. DATA 2021. [DOI: 10.3390/data6120130] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023] Open
Abstract
In this paper, the Mexican Emotional Speech Database (MESD) that contains single-word emotional utterances for anger, disgust, fear, happiness, neutral and sadness with adult (male and female) and child voices is described. To validate the emotional prosody of the uttered words, a cubic Support Vector Machines classifier was trained on the basis of prosodic, spectral and voice quality features for each case study: (1) male adult, (2) female adult and (3) child. In addition, cultural, semantic, and linguistic shaping of emotional expression was assessed by statistical analysis. This study was registered at BioMed Central and is part of the implementation of a published study protocol. Mean emotional classification accuracies yielded 93.3%, 89.4% and 83.3% for male, female and child utterances respectively. Statistical analysis emphasized the shaping of emotional prosodies by semantic and linguistic features. A cultural variation in emotional expression was highlighted by comparing the MESD with the INTERFACE for Castilian Spanish database. The MESD provides reliable content for linguistic emotional prosody shaped by the Mexican cultural environment. In order to facilitate further investigations, a corpus controlled for linguistic features and emotional semantics, as well as one containing words repeated across voices and emotions are provided. The MESD is made freely available.
Collapse
|
22
|
Dunbar RIM, Robledo JP, Tamarit I, Cross I, Smith E. Nonverbal Auditory Cues Allow Relationship Quality to be Inferred During Conversations. JOURNAL OF NONVERBAL BEHAVIOR 2021; 46:1-18. [PMID: 35250136 PMCID: PMC8881250 DOI: 10.1007/s10919-021-00386-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/16/2021] [Indexed: 11/30/2022]
Abstract
The claim that nonverbal cues provide more information than the linguistic content of a conversational exchange (the Mehrabian Conjecture) has been widely cited and equally widely disputed, mainly on methodological grounds. Most studies that have tested the Conjecture have used individual words or short phrases spoken by actors imitating emotions. While cue recognition is certainly important, speech evolved to manage interactions and relationships rather than simple information exchange. In a cross-cultural design, we tested participants’ ability to identify the quality of the interaction (rapport) in naturalistic third party conversations in their own and a less familiar language, using full auditory content versus audio clips whose verbal content has been digitally altered to differing extents. We found that, using nonverbal content alone, people are 75–90% as accurate as they are with full audio cues in identifying positive vs negative relationships, and 45–53% as accurate in identifying eight different relationship types. The results broadly support Mehrabian’s claim that a significant amount of information about others’ social relationships is conveyed in the nonverbal component of speech.
Collapse
Affiliation(s)
- R. I. M. Dunbar
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory Quarter, Anna Watts Building, Oxford, OX2 6GG UK
| | - Juan-Pablo Robledo
- Centre for Music and Science, Faculty of Music, University of Cambridge, 11 West Road, Cambridge, CB3 9DP UK
- Laboratoire Interpsy, Campus Lettres et Sciences Humaines et Sociales, Université de Lorraine, 23, Bd Albert 1er, 54015 Nancy cedex, France
- Millennium Institute for Caregiving Research (MICARE), Santiago, Chile
| | - Ignacio Tamarit
- Grupo Interdisciplinar de Sistemas Complejos, Departamento de Matemáticas, Universidad Carlos III de Madrid, 28911 Leganés, Madrid, Spain
| | - Ian Cross
- Centre for Music and Science, Faculty of Music, University of Cambridge, 11 West Road, Cambridge, CB3 9DP UK
| | - Emma Smith
- Wysing Arts Centre, Fox Road, Bourn, Cambridge, CB23 2TX UK
| |
Collapse
|
23
|
The representational dynamics of perceived voice emotions evolve from categories to dimensions. Nat Hum Behav 2021; 5:1203-1213. [PMID: 33707658 PMCID: PMC7611700 DOI: 10.1038/s41562-021-01073-0] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2020] [Accepted: 02/08/2021] [Indexed: 01/31/2023]
Abstract
Long-standing affective science theories conceive the perception of emotional stimuli either as discrete categories (for example, an angry voice) or continuous dimensional attributes (for example, an intense and negative vocal emotion). Which position provides a better account is still widely debated. Here we contrast the positions to account for acoustics-independent perceptual and cerebral representational geometry of perceived voice emotions. We combined multimodal imaging of the cerebral response to heard vocal stimuli (using functional magnetic resonance imaging and magneto-encephalography) with post-scanning behavioural assessment of voice emotion perception. By using representational similarity analysis, we find that categories prevail in perceptual and early (less than 200 ms) frontotemporal cerebral representational geometries and that dimensions impinge predominantly on a later limbic-temporal network (at 240 ms and after 500 ms). These results reconcile the two opposing views by reframing the perception of emotions as the interplay of cerebral networks with different representational dynamics that emphasize either categories or dimensions.
Collapse
|
24
|
Lin Y, Ding H, Zhang Y. Gender Differences in Identifying Facial, Prosodic, and Semantic Emotions Show Category- and Channel-Specific Effects Mediated by Encoder's Gender. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2021; 64:2941-2955. [PMID: 34310173 DOI: 10.1044/2021_jslhr-20-00553] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Purpose The nature of gender differences in emotion processing has remained unclear due to the discrepancies in existing literature. This study examined the modulatory effects of emotion categories and communication channels on gender differences in verbal and nonverbal emotion perception. Method Eighty-eight participants (43 females and 45 males) were asked to identify three basic emotions (i.e., happiness, sadness, and anger) and neutrality encoded by female or male actors from verbal (i.e., semantic) or nonverbal (i.e., facial and prosodic) channels. Results While women showed an overall advantage in performance, their superiority was dependent on specific types of emotion and channel. Specifically, women outperformed men in regard to two basic emotions (happiness and sadness) in the nonverbal channels and only the anger category with verbal content. Conversely, men did better for the anger category in the nonverbal channels and for the other two emotions (happiness and sadness) in verbal content. There was an emotion- and channel-specific interaction effect between the two types of gender differences, with male subjects showing higher sensitivity to sad faces and prosody portrayed by the female encoders. Conclusion These findings reveal explicit emotion processing as a highly dynamic complex process with significant gender differences tied to specific emotion categories and communication channels. Supplemental Material https://doi.org/10.23641/asha.15032583.
Collapse
Affiliation(s)
- Yi Lin
- Speech-Language-Hearing Center, School of Foreign Languages, Shanghai Jiao Tong University, Shanghai, China
| | - Hongwei Ding
- Speech-Language-Hearing Center, School of Foreign Languages, Shanghai Jiao Tong University, Shanghai, China
| | - Yang Zhang
- Department of Speech-Language-Hearing Sciences & Center for Neurobehavioral Development, University of Minnesota Twin Cities, Minneapolis
| |
Collapse
|
25
|
Elfenbein HA, Laukka P, Althoff J, Chui W, Iraki FK, Rockstuhl T, Thingujam NS. What Do We Hear in the Voice? An Open-Ended Judgment Study of Emotional Speech Prosody. PERSONALITY AND SOCIAL PSYCHOLOGY BULLETIN 2021; 48:1087-1104. [PMID: 34296644 DOI: 10.1177/01461672211029786] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
The current study investigated what can be understood from another person's tone of voice. Participants from five English-speaking nations (Australia, India, Kenya, Singapore, and the United States) listened to vocal expressions of nine positive and nine negative affective states recorded by actors from their own nation. In response, they wrote open-ended judgments of what they believed the actor was trying to express. Responses cut across the chronological emotion process and included descriptions of situations, cognitive appraisals, feeling states, physiological arousal, expressive behaviors, emotion regulation, and attempts at social influence. Accuracy in terms of emotion categories was overall modest, whereas accuracy in terms of valence and arousal was more substantial. Coding participants' 57,380 responses yielded a taxonomy of 56 categories, which included affective states as well as person descriptors, communication behaviors, and abnormal states. Open-ended responses thus reveal a wide range of ways in which people spontaneously perceive the intent behind emotional speech prosody.
Collapse
|
26
|
von Suchodoletz A, Hepach R. Cultural values shape the expression of self-evaluative social emotions. Sci Rep 2021; 11:13169. [PMID: 34162979 PMCID: PMC8222260 DOI: 10.1038/s41598-021-92652-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Accepted: 06/11/2021] [Indexed: 11/17/2022] Open
Abstract
Social emotions are key to everyday social life and therefore shaped by cultural values in their expression. Prior research has focused on facial expressions of emotions. What is less clear, however, is the extent to which cultural values shape other modalities of emotional expression. In the present study, we applied a novel paradigm using depth sensor imaging technology to capture changes in participants’ body posture in real time. We aimed to (1) identify the nuances in the postural expression that are thought to characterize social emotions and (2) assess how individual differences in cultural values impact the postural expression of emotions. Participants in two separate studies were 132 undergraduate college students whose upper-body postural expansion was recorded after they recalled emotion episodes. Positive emotions elevated participants’ upper-body posture whereas negative emotions resulted in lowered upper-body posture. The effects on changes in upper-body posture were moderated by participants’ self-ratings of the vertical and horizontal dimensions of individualism and collectivism. The findings provide initial evidence of the nuances in the way cultural values influence the postural expression of emotions.
Collapse
Affiliation(s)
- Antje von Suchodoletz
- Department of Psychology, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, UAE.
| | | |
Collapse
|
27
|
Sznycer D, Cohen AS. Are Emotions Natural Kinds After All? Rethinking the Issue of Response Coherence. EVOLUTIONARY PSYCHOLOGY 2021; 19:14747049211016009. [PMID: 34060370 PMCID: PMC10355299 DOI: 10.1177/14747049211016009] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Accepted: 04/20/2021] [Indexed: 11/16/2022] Open
Abstract
The synchronized co-activation of multiple responses-motivational, behavioral, and physiological-has been taken as a defining feature of emotion. Such response coherence has been observed inconsistently however, and this has led some to view emotion programs as lacking biological reality. Yet, response coherence is not always expected or desirable if an emotion program is to carry out its adaptive function. Rather, the hallmark of emotion is the capacity to orchestrate multiple mechanisms adaptively-responses will co-activate in stereotypical fashion or not depending on how the emotion orchestrator interacts with the situation. Nevertheless, might responses cohere in the general case where input variables are specified minimally? Here we focus on shame as a case study. We measure participants' responses regarding each of 27 socially devalued actions and personal characteristics. We observe internal and external coherence: The intensities of felt shame and of various motivations of shame (hiding, lying, destroying evidence, and threatening witnesses) vary in proportion (i) to one another, and (ii) to the degree to which audiences devalue the disgraced individual-the threat shame defends against. These responses cohere both within and between the United States and India. Further, alternative explanations involving the low-level variable of arousal do not seem to account for these results, suggesting that coherence is imparted by a shame system. These findings indicate that coherence can be observed at multiple levels and raise the possibility that emotion programs orchestrate responses, even in those situations where coherence is low.
Collapse
Affiliation(s)
- Daniel Sznycer
- Department of Psychology, University of Montreal, QC, Canada
| | | |
Collapse
|
28
|
Engelberg JWM, Schwartz JW, Gouzoules H. The emotional canvas of human screams: patterns and acoustic cues in the perceptual categorization of a basic call type. PeerJ 2021; 9:e10990. [PMID: 33854835 PMCID: PMC7953872 DOI: 10.7717/peerj.10990] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2020] [Accepted: 02/01/2021] [Indexed: 11/20/2022] Open
Abstract
Screams occur across taxonomically widespread species, typically in antipredator situations, and are strikingly similar acoustically, but in nonhuman primates, they have taken on acoustically varied forms in association with more contextually complex functions related to agonistic recruitment. Humans scream in an even broader range of contexts, but the extent to which acoustic variation allows listeners to perceive different emotional meanings remains unknown. We investigated how listeners responded to 30 contextually diverse human screams on six different emotion prompts as well as how selected acoustic cues predicted these responses. We found that acoustic variation in screams was associated with the perception of different emotions from these calls. Emotion ratings generally fell along two dimensions: one contrasting perceived anger, frustration, and pain with surprise and happiness, roughly associated with call duration and roughness, and one related to perceived fear, associated with call fundamental frequency. Listeners were more likely to rate screams highly in emotion prompts matching the source context, suggesting that some screams conveyed information about emotional context, but it is noteworthy that the analysis of screams from happiness contexts (n = 11 screams) revealed that they more often yielded higher ratings of fear. We discuss the implications of these findings for the role and evolution of nonlinguistic vocalizations in human communication, including consideration of how the expanded diversity in calls such as human screams might represent a derived function of language.
Collapse
Affiliation(s)
| | - Jay W. Schwartz
- Department of Psychology, Emory University, Atlanta, GA, USA
- Psychological Sciences Department, Western Oregon University, Monmouth, OR, USA
| | | |
Collapse
|
29
|
Bertoux M, Duclos H, Caillaud M, Segobin S, Merck C, de La Sayette V, Belliard S, Desgranges B, Eustache F, Laisney M. When affect overlaps with concept: emotion recognition in semantic variant of primary progressive aphasia. Brain 2021; 143:3850-3864. [PMID: 33221846 PMCID: PMC7805810 DOI: 10.1093/brain/awaa313] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2020] [Revised: 07/20/2020] [Accepted: 07/31/2020] [Indexed: 02/06/2023] Open
Abstract
The most recent theories of emotions have postulated that their expression and recognition depend on acquired conceptual knowledge. In other words, the conceptual knowledge derived from prior experiences guide our ability to make sense of such emotions. However, clear evidence is still lacking to contradict more traditional theories, considering emotions as innate, distinct and universal physiological states. In addition, whether valence processing (i.e. recognition of the pleasant/unpleasant character of emotions) also relies on semantic knowledge is yet to be determined. To investigate the contribution of semantic knowledge to facial emotion recognition and valence processing, we conducted a behavioural and neuroimaging study in 20 controls and 16 patients with the semantic variant of primary progressive aphasia, a neurodegenerative disease that is prototypical of semantic memory impairment, and in which an emotion recognition deficit has already been described. We assessed participants’ knowledge of emotion concepts and recognition of 10 basic (e.g. anger) or self-conscious (e.g. embarrassment) facial emotional expressions presented both statically (images) and dynamically (videos). All participants also underwent a brain MRI. Group comparisons revealed deficits in both emotion concept knowledge and emotion recognition in patients, independently of type of emotion and presentation. These measures were significantly correlated with each other in patients and with semantic fluency in patients and controls. Neuroimaging analyses showed that both emotion recognition and emotion conceptual knowledge were correlated with reduced grey matter density in similar areas within frontal ventral, temporal, insular and striatal regions, together with white fibre degeneration in tracts connecting frontal regions with each other as well as with temporal regions. We then performed a qualitative analysis of responses made during the facial emotion recognition task, by delineating valence errors (when one emotion was mistaken for another of a different valence), from other errors made during the emotion recognition test. We found that patients made more valence errors. The number of valence errors correlated with emotion conceptual knowledge as well as with reduced grey matter volume in brain regions already retrieved to correlate with this score. Specificity analyses allowed us to conclude that this cognitive relationship and anatomical overlap were not mediated by a general effect of disease severity. Our findings suggest that semantic knowledge guides the recognition of emotions and is also involved in valence processing. Our study supports a constructionist view of emotion recognition and valence processing, and could help to refine current theories on the interweaving of semantic knowledge and emotion processing.
Collapse
Affiliation(s)
- Maxime Bertoux
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France.,Univ. Lille, Inserm, CHU Lille, UMRS1172, Lille Neurosciences & Cognition Institute, F-59000 Lille, France
| | - Harmony Duclos
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France.,CRP-CPO, Picardy Jules Verne University, Amiens, France
| | - Marie Caillaud
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France
| | - Shailendra Segobin
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France
| | - Catherine Merck
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France.,Neurology Department, Pontchaillou University Hospital, Rennes, France
| | - Vincent de La Sayette
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France.,Neurology Department, Caen University Hospital, Caen, France
| | - Serge Belliard
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France.,Neurology Department, Pontchaillou University Hospital, Rennes, France
| | - Béatrice Desgranges
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France
| | - Francis Eustache
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France
| | - Mickaël Laisney
- Neuropsychology and Imaging of Human Memory research unit, Caen-Normandy University-PSL Research University-EPHE-INSERM-Caen University Hospital, UMRS1077, GIP Cyceron, Caen, France
| |
Collapse
|
30
|
Cowen AS, Keltner D, Schroff F, Jou B, Adam H, Prasad G. Sixteen facial expressions occur in similar contexts worldwide. Nature 2021; 589:251-257. [PMID: 33328631 DOI: 10.1038/s41586-020-3037-7] [Citation(s) in RCA: 59] [Impact Index Per Article: 19.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2020] [Accepted: 10/30/2020] [Indexed: 01/29/2023]
Abstract
Understanding the degree to which human facial expressions co-vary with specific social contexts across cultures is central to the theory that emotions enable adaptive responses to important challenges and opportunities1-6. Concrete evidence linking social context to specific facial expressions is sparse and is largely based on survey-based approaches, which are often constrained by language and small sample sizes7-13. Here, by applying machine-learning methods to real-world, dynamic behaviour, we ascertain whether naturalistic social contexts (for example, weddings or sporting competitions) are associated with specific facial expressions14 across different cultures. In two experiments using deep neural networks, we examined the extent to which 16 types of facial expression occurred systematically in thousands of contexts in 6 million videos from 144 countries. We found that each kind of facial expression had distinct associations with a set of contexts that were 70% preserved across 12 world regions. Consistent with these associations, regions varied in how frequently different facial expressions were produced as a function of which contexts were most salient. Our results reveal fine-grained patterns in human facial expressions that are preserved across the modern world.
Collapse
Affiliation(s)
- Alan S Cowen
- Department of Psychology, University of California Berkeley, Berkeley, CA, USA. .,Google Research, Mountain View, CA, USA.
| | - Dacher Keltner
- Department of Psychology, University of California Berkeley, Berkeley, CA, USA
| | | | | | | | | |
Collapse
|
31
|
Chen Z, Whitney D. Inferential affective tracking reveals the remarkable speed of context-based emotion perception. Cognition 2020; 208:104549. [PMID: 33340812 DOI: 10.1016/j.cognition.2020.104549] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2020] [Revised: 12/08/2020] [Accepted: 12/09/2020] [Indexed: 10/22/2022]
Abstract
Understanding the emotional states of others is important for social functioning. Recent studies show that context plays an essential role in emotion recognition. However, it remains unclear whether emotion inference from visual scene context is as efficient as emotion recognition from faces. Here, we measured the speed of context-based emotion perception, using Inferential Affective Tracking (IAT) with naturalistic and dynamic videos. Using cross-correlation analyses, we found that inferring affect based on visual context alone is just as fast as tracking affect with all available information including face and body. We further demonstrated that this approach has high precision and sensitivity to sub-second lags. Our results suggest that emotion recognition from dynamic contextual information might be automatic and immediate. Seemingly complex context-based emotion perception is far more efficient than previously assumed.
Collapse
Affiliation(s)
- Zhimin Chen
- Department of Psychology, University of California, Berkeley, CA 94720, United States of America.
| | - David Whitney
- Department of Psychology, University of California, Berkeley, CA 94720, United States of America; Vision Science Program, University of California, Berkeley, CA 94720, United States of America; Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, United States of America
| |
Collapse
|
32
|
Semantic Space Theory: A Computational Approach to Emotion. Trends Cogn Sci 2020; 25:124-136. [PMID: 33349547 DOI: 10.1016/j.tics.2020.11.004] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2020] [Revised: 11/09/2020] [Accepted: 11/13/2020] [Indexed: 12/30/2022]
Abstract
Within affective science, the central line of inquiry, animated by basic emotion theory and constructivist accounts, has been the search for one-to-one mappings between six emotions and their subjective experiences, prototypical expressions, and underlying brain states. We offer an alternative perspective: semantic space theory. This computational approach uses wide-ranging naturalistic stimuli and open-ended statistical techniques to capture systematic variation in emotion-related behaviors. Upwards of 25 distinct varieties of emotional experience have distinct profiles of associated antecedents and expressions. These emotions are high-dimensional, categorical, and often blended. This approach also reveals that specific emotions, more than valence, organize emotional experience, expression, and neural processing. Overall, moving beyond traditional models to study broader semantic spaces of emotion can enrich our understanding of human experience.
Collapse
|
33
|
Cowen AS, Keltner D. Universal facial expressions uncovered in art of the ancient Americas: A computational approach. SCIENCE ADVANCES 2020; 6:eabb1005. [PMID: 32875109 PMCID: PMC7438103 DOI: 10.1126/sciadv.abb1005] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2020] [Accepted: 07/08/2020] [Indexed: 05/08/2023]
Abstract
Central to the study of emotion is evidence concerning its universality, particularly the degree to which emotional expressions are similar across cultures. Here, we present an approach to studying the universality of emotional expression that rules out cultural contact and circumvents potential biases in survey-based methods: A computational analysis of apparent facial expressions portrayed in artwork created by members of cultures isolated from Western civilization. Using data-driven methods, we find that facial expressions depicted in 63 sculptures from the ancient Americas tend to accord with Western expectations for emotions that unfold in specific social contexts. Ancient American sculptures tend to portray at least five facial expressions in contexts predicted by Westerners, including "pain" in torture, "determination"/"strain" in heavy lifting, "anger" in combat, "elation" in social touch, and "sadness" in defeat-supporting the universality of these expressions.
Collapse
Affiliation(s)
- Alan S. Cowen
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Dacher Keltner
- Department of Psychology, University of California, Berkeley, Berkeley, CA 94720, USA
| |
Collapse
|
34
|
Horikawa T, Cowen AS, Keltner D, Kamitani Y. The Neural Representation of Visually Evoked Emotion Is High-Dimensional, Categorical, and Distributed across Transmodal Brain Regions. iScience 2020; 23:101060. [PMID: 32353765 PMCID: PMC7191651 DOI: 10.1016/j.isci.2020.101060] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2019] [Revised: 03/11/2020] [Accepted: 04/09/2020] [Indexed: 12/12/2022] Open
Abstract
Central to our subjective lives is the experience of different emotions. Recent behavioral work mapping emotional responses to 2,185 videos found that people experience upward of 27 distinct emotions occupying a high-dimensional space, and that emotion categories, more so than affective dimensions (e.g., valence), organize self-reports of subjective experience. Here, we sought to identify the neural substrates of this high-dimensional space of emotional experience using fMRI responses to all 2,185 videos. Our analyses demonstrated that (1) dozens of video-evoked emotions were accurately predicted from fMRI patterns in multiple brain regions with different regional configurations for individual emotions; (2) emotion categories better predicted cortical and subcortical responses than affective dimensions, outperforming visual and semantic covariates in transmodal regions; and (3) emotion-related fMRI responses had a cluster-like organization efficiently characterized by distinct categories. These results support an emerging theory of the high-dimensional emotion space, illuminating its neural foundations distributed across transmodal regions.
Collapse
Affiliation(s)
- Tomoyasu Horikawa
- Department of Neuroinformatics, ATR Computational Neuroscience Laboratories, Hikaridai, Seika, Soraku, Kyoto, 619-0288, Japan.
| | - Alan S Cowen
- Department of Psychology, University of California, Berkeley, CA 94720-1500, USA
| | - Dacher Keltner
- Department of Psychology, University of California, Berkeley, CA 94720-1500, USA
| | - Yukiyasu Kamitani
- Department of Neuroinformatics, ATR Computational Neuroscience Laboratories, Hikaridai, Seika, Soraku, Kyoto, 619-0288, Japan; Graduate School of Informatics, Kyoto University, Yoshida-honmachi, Sakyo-ku, Kyoto, 606-8501, Japan.
| |
Collapse
|
35
|
Reply to Bowling: How specific emotions are primary in subjective experience. Proc Natl Acad Sci U S A 2020; 117:9694-9695. [DOI: 10.1073/pnas.2003626117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
36
|
Torre I, Goslin J, White L. If your device could smile: People trust happy-sounding artificial agents more. COMPUTERS IN HUMAN BEHAVIOR 2020. [DOI: 10.1016/j.chb.2019.106215] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
37
|
Cowen AS, Fang X, Sauter D, Keltner D. What music makes us feel: At least 13 dimensions organize subjective experiences associated with music across different cultures. Proc Natl Acad Sci U S A 2020; 117:1924-1934. [PMID: 31907316 PMCID: PMC6995018 DOI: 10.1073/pnas.1910704117] [Citation(s) in RCA: 60] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022] Open
Abstract
What is the nature of the feelings evoked by music? We investigated how people represent the subjective experiences associated with Western and Chinese music and the form in which these representational processes are preserved across different cultural groups. US (n = 1,591) and Chinese (n = 1,258) participants listened to 2,168 music samples and reported on the specific feelings (e.g., "angry," "dreamy") or broad affective features (e.g., valence, arousal) that they made individuals feel. Using large-scale statistical tools, we uncovered 13 distinct types of subjective experience associated with music in both cultures. Specific feelings such as "triumphant" were better preserved across the 2 cultures than levels of valence and arousal, contrasting with theoretical claims that valence and arousal are building blocks of subjective experience. This held true even for music selected on the basis of its valence and arousal levels and for traditional Chinese music. Furthermore, the feelings associated with music were found to occupy continuous gradients, contradicting discrete emotion theories. Our findings, visualized within an interactive map (https://www.ocf.berkeley.edu/∼acowen/music.html) reveal a complex, high-dimensional space of subjective experience associated with music in multiple cultures. These findings can inform inquiries ranging from the etiology of affective disorders to the neurological basis of emotion.
Collapse
Affiliation(s)
- Alan S Cowen
- Department of Psychology, University of California, Berkeley, CA 94720;
| | - Xia Fang
- Department of Psychology, University of Amsterdam, 1001 NK Amsterdam, The Netherlands
- Department of Psychology, York University, Toronto, ON M3J 1P3, Canada
| | - Disa Sauter
- Department of Psychology, University of Amsterdam, 1001 NK Amsterdam, The Netherlands
| | - Dacher Keltner
- Department of Psychology, University of California, Berkeley, CA 94720
| |
Collapse
|
38
|
Haider F, Koutsombogera M, Conlan O, Vogel C, Campbell N, Luz S. An Active Data Representation of Videos for Automatic Scoring of Oral Presentation Delivery Skills and Feedback Generation. FRONTIERS IN COMPUTER SCIENCE 2020. [DOI: 10.3389/fcomp.2020.00001] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/29/2023] Open
|
39
|
Affiliation(s)
| | - Yukiko UCHIDA
- Kyoto University
- 2019–20 Berggruen Fellow at the Center for Advanced Study in the Behavioral Sciences, Stanford University
| |
Collapse
|
40
|
Berrios R. What Is Complex/Emotional About Emotional Complexity? Front Psychol 2019; 10:1606. [PMID: 31354593 PMCID: PMC6639786 DOI: 10.3389/fpsyg.2019.01606] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2018] [Accepted: 06/26/2019] [Indexed: 12/14/2022] Open
Abstract
Affective experiences can fluctuate, be combined, and fused, resulting in various phenomena labeled as being emotionally complex. Despite the lack of a common theoretical framework, several phenomena including mixed emotions, emodiversity, meta-emotions, awe, among several others, have been defined as being emotionally complex. In this conceptual analysis, I aim to integrate the diversity of emotional complexity by describing various phenomena associated with this construct. This integration offers a more comprehensive panorama of the current usage of the concept of emotional complexity compared to previous attempts to consolidate the field. Furthermore, this conceptual analysis intends to disentangle the emotional fingerprints of emotional complexity. In particular, I present evidence and arguments showing that complex emotions can be characterized as having specific facial expressions, appraisals, and functional significance. Finally, I suggest that it is possible to describe emotional complexity using concepts and properties from the complex systems theory. Concepts such as the hierarchical organization of the affect system and emergent self-organization are used to explain current evidence on emotional complexity. I explain that applying complex systems theory to emotional complexity is not only theoretically convenient, but that complex systems theory also serves to advance new forms to conceptualize the affect system. The current conceptual analysis can help to organize current research and theory in order to encourage new research endeavors in the field of emotional complexity and acknowledge the importance of emotional complexity in models of affect, for which I suggest some specific guidelines.
Collapse
Affiliation(s)
- Raul Berrios
- Department of Management, Universidad de Santiago de Chile, Santiago, Chile
| |
Collapse
|
41
|
Cowen A, Sauter D, Tracy JL, Keltner D. Mapping the Passions: Toward a High-Dimensional Taxonomy of Emotional Experience and Expression. Psychol Sci Public Interest 2019; 20:69-90. [PMID: 31313637 PMCID: PMC6675572 DOI: 10.1177/1529100619850176] [Citation(s) in RCA: 46] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
What would a comprehensive atlas of human emotions include? For 50 years, scientists have sought to map emotion-related experience, expression, physiology, and recognition in terms of the "basic six"-anger, disgust, fear, happiness, sadness, and surprise. Claims about the relationships between these six emotions and prototypical facial configurations have provided the basis for a long-standing debate over the diagnostic value of expression (for review and latest installment in this debate, see Barrett et al., p. 1). Building on recent empirical findings and methodologies, we offer an alternative conceptual and methodological approach that reveals a richer taxonomy of emotion. Dozens of distinct varieties of emotion are reliably distinguished by language, evoked in distinct circumstances, and perceived in distinct expressions of the face, body, and voice. Traditional models-both the basic six and affective-circumplex model (valence and arousal)-capture a fraction of the systematic variability in emotional response. In contrast, emotion-related responses (e.g., the smile of embarrassment, triumphant postures, sympathetic vocalizations, blends of distinct expressions) can be explained by richer models of emotion. Given these developments, we discuss why tests of a basic-six model of emotion are not tests of the diagnostic value of facial expression more generally. Determining the full extent of what facial expressions can tell us, marginally and in conjunction with other behavioral and contextual cues, will require mapping the high-dimensional, continuous space of facial, bodily, and vocal signals onto richly multifaceted experiences using large-scale statistical modeling and machine-learning methods.
Collapse
Affiliation(s)
- Alan Cowen
- Department of Psychology, University of California, Berkeley
| | - Disa Sauter
- Faculty of Social and Behavioural Sciences, University of Amsterdam
| | | | - Dacher Keltner
- Department of Psychology, University of California, Berkeley
| |
Collapse
|
42
|
Cowen AS, Keltner D. What the face displays: Mapping 28 emotions conveyed by naturalistic expression. ACTA ACUST UNITED AC 2019; 75:349-364. [PMID: 31204816 DOI: 10.1037/amp0000488] [Citation(s) in RCA: 39] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
Abstract
What emotions do the face and body express? Guided by new conceptual and quantitative approaches (Cowen, Elfenbein, Laukka, & Keltner, 2018; Cowen & Keltner, 2017, 2018), we explore the taxonomy of emotion recognized in facial-bodily expression. Participants (N = 1,794; 940 female, ages 18-76 years) judged the emotions captured in 1,500 photographs of facial-bodily expression in terms of emotion categories, appraisals, free response, and ecological validity. We find that facial-bodily expressions can reliably signal at least 28 distinct categories of emotion that occur in everyday life. Emotion categories, more so than appraisals such as valence and arousal, organize emotion recognition. However, categories of emotion recognized in naturalistic facial and bodily behavior are not discrete but bridged by smooth gradients that correspond to continuous variations in meaning. Our results support a novel view that emotions occupy a high-dimensional space of categories bridged by smooth gradients of meaning. They offer an approximation of a taxonomy of facial-bodily expressions, visualized within an online interactive map. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Collapse
|