Unattended Emotional Prosody Affects Visual Processing of Facial Expressions in Mandarin-Speaking Chinese: A Comparison With English-Speaking Canadians

Instrumental music training relates to intensity assessment but not emotional prosody recognition in Mandarin

Building on research demonstrating the benefits of music training for emotional prosody recognition in nontonal languages, this study delves into its unexplored influence on tonal languages. In tonal languages, the acoustic similarity between lexical tones and music, along with the dual role of pitch in conveying lexical and affective meanings, create a unique interplay. We evaluated 72 participants, half of whom had extensive instrumental music training, with the other half serving as demographically matched controls. All participants completed an online test consisting of 210 Chinese pseudosentences, each designed to express one of five emotions: happiness, sadness, fear, anger, or neutrality. Our robust statistical analyses, which included effect size estimates and Bayesian factors, revealed that music and nonmusic groups exhibit similar abilities in identifying the emotional prosody of various emotions. However, the music group attributed higher intensity ratings to emotional prosodies of happiness, fear, and anger compared to the nonmusic group. These findings suggest that while instrumental music training is not related to emotional prosody recognition, it does appear to be related to perceived emotional intensity. This dissociation between emotion recognition and intensity evaluation adds a new piece to the puzzle of the complex relationship between music training and emotion perception in tonal languages.

Cultural background and input familiarity influence multisensory emotion perception

OBJECTIVES

During multisensory emotion perception, the attention devoted to the visual versus the auditory modality (i.e., modality dominance) varies depending on the cultural background of the perceiver. In the present study, we examined (a) how cultural familiarity influences multisensory emotion perception in Eastern and Western cultures and (b) the underlying processes accounting for the cultural difference in modality dominance.

METHOD

Native Mandarin speakers from China and native English speakers from the United States were presented with audiovisual emotional stimuli from their own culture (i.e., familiar) and from a different culture (i.e., unfamiliar) and asked to evaluate the emotion from one of the two modalities. Across modalities, the emotions were either the same (i.e., congruent, happy face, and happy voice) or different (i.e., incongruent, happy face, and sad voice).

RESULTS

When the input was in a familiar cultural context, American participants were more influenced by the visual modality, while Chinese participants were more influenced by the auditory modality. While both groups integrated the incongruent emotion from the irrelevant modality, only the American group integrated the congruent emotion from the irrelevant modality. When the input was in a less familiar cultural context, both groups showed increased visual dominance, but only the Chinese group simultaneously showed decreased auditory dominance.

CONCLUSIONS

We conclude that cultural background and input familiarity interact to influence modality dominance during multisensory emotion perception. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Cultural differences in vocal expression analysis: Effects of task, language, and stimulus-related factors

Cultural context shapes the way that emotions are expressed and socially interpreted. Building on previous research looking at cultural differences in judgements of facial expressions, we examined how listeners recognize speech-embedded emotional expressions and make inferences about a speaker's feelings in relation to their vocal display. Canadian and Chinese participants categorized vocal expressions of emotions (anger, fear, happiness, sadness) expressed at different intensity levels in three languages (English, Mandarin, Hindi). In two additional tasks, participants rated the intensity of each emotional expression and the intensity of the speaker's feelings from the same stimuli. Each group was more accurate at recognizing emotions produced in their native language (in-group advantage). However, Canadian and Chinese participants both judged the speaker's feelings to be equivalent or more intense than their actual display (especially for highly aroused, negative emotions), suggesting that similar inference rules were applied to vocal expressions by the two cultures in this task. Our results provide new insights on how people categorize and interpret speech-embedded vocal expressions versus facial expressions and what cultural factors are at play.