Good vibrations: A review of vocal expressions of positive emotions

Self-Rated Confidence in Vocal Emotion Recognition Ability: The Role of Gender

PURPOSE

We studied the role of gender in metacognition of voice emotion recognition ability (ERA), reflected by self-rated confidence (SRC). To this end, we guided our study in two approaches: first, by examining the role of gender in voice ERA and SRC independently and second, by looking for gender effects on the ERA association with SRC.

METHOD

We asked 100 participants (50 men, 50 women) to interpret a set of vocal expressions portrayed by 30 actors (16 men, 14 women) as defined by their emotional meaning. Targets were 180 repetitive lexical sentences articulated in congruent emotional voices (anger, sadness, surprise, happiness, fear) and neutral expressions. Trial by trial, the participants were assigned retrospective SRC based on their emotional recognition performance.

RESULTS

A binomial generalized linear mixed model (GLMM) estimating ERA accuracy revealed a significant gender effect, with women encoders (speakers) yielding higher accuracy levels than men. There was no significant effect of the decoder's (listener's) gender. A second GLMM estimating SRC found a significant effect of encoder and decoder genders, with women outperforming men. Gamma correlations were significantly greater than zero for women and men decoders.

CONCLUSIONS

In spite of varying interpretations of gender in each independent rating (ERA and SRC), our results suggest that both men and women decoders were accurate in their metacognition regarding voice emotion recognition. Further research is needed to study how individuals of both genders use metacognitive knowledge in their emotional recognition and whether and how such knowledge contributes to effective social communication.

The Sound of Emotional Prosody: Nearly 3 Decades of Research and Future Directions

Emotional voices attract considerable attention. A search on any browser using "emotional prosody" as a key phrase leads to more than a million entries. Such interest is evident in the scientific literature as well; readers are reminded in the introductory paragraphs of countless articles of the great importance of prosody and that listeners easily infer the emotional state of speakers through acoustic information. However, despite decades of research on this topic and important achievements, the mapping between acoustics and emotional states is still unclear. In this article, we chart the rich literature on emotional prosody for both newcomers to the field and researchers seeking updates. We also summarize problems revealed by a sample of the literature of the last decades and propose concrete research directions for addressing them, ultimately to satisfy the need for more mechanistic knowledge of emotional prosody.

The Role of Hedonic Goal Pursuit in Self-Control and Self-Regulation: Is Pleasure the Problem or Part of the Solution?

This paper examines the role of hedonic goal pursuit in self-control and self-regulation. We argue that not all pursuit of immediate pleasure is problematic and that successful hedonic goal pursuit can be beneficial for long-term goal pursuit and for achieving positive self-regulatory outcomes, such as health and well-being. The following two key questions for future research are discussed: How can people's positive affective experiences during hedonic goal pursuit be enhanced, and how exactly do those affective experiences contribute to self-regulatory outcomes? We also call for an intercultural perspective linking hedonic goal pursuit to self-regulatory outcomes at different levels. We suggest that understanding the cognitive, motivational, and affective mechanisms at play can help individuals reap the benefits of successful hedonic goal pursuit. Considering those potential benefits, hedonic goal pursuit should be studied more systematically. To achieve this, we argue for a stronger integration of affective science and self-control research.

ADscreen: A speech processing-based screening system for automatic identification of patients with Alzheimer's disease and related dementia

Alzheimer's disease and related dementias (ADRD) present a looming public health crisis, affecting roughly 5 million people and 11 % of older adults in the United States. Despite nationwide efforts for timely diagnosis of patients with ADRD, >50 % of them are not diagnosed and unaware of their disease. To address this challenge, we developed ADscreen, an innovative speech-processing based ADRD screening algorithm for the protective identification of patients with ADRD. ADscreen consists of five major components: (i) noise reduction for reducing background noises from the audio-recorded patient speech, (ii) modeling the patient's ability in phonetic motor planning using acoustic parameters of the patient's voice, (iii) modeling the patient's ability in semantic and syntactic levels of language organization using linguistic parameters of the patient speech, (iv) extracting vocal and semantic psycholinguistic cues from the patient speech, and (v) building and evaluating the screening algorithm. To identify important speech parameters (features) associated with ADRD, we used the Joint Mutual Information Maximization (JMIM), an effective feature selection method for high dimensional, small sample size datasets. Modeling the relationship between speech parameters and the outcome variable (presence/absence of ADRD) was conducted using three different machine learning (ML) architectures with the capability of joining informative acoustic and linguistic with contextual word embedding vectors obtained from the DistilBERT (Bidirectional Encoder Representations from Transformers). We evaluated the performance of the ADscreen on an audio-recorded patients' speech (verbal description) for the Cookie-Theft picture description task, which is publicly available in the dementia databank. The joint fusion of acoustic and linguistic parameters with contextual word embedding vectors of DistilBERT achieved F1-score = 84.64 (standard deviation [std] = ±3.58) and AUC-ROC = 92.53 (std = ±3.34) for training dataset, and F1-score = 89.55 and AUC-ROC = 93.89 for the test dataset. In summary, ADscreen has a strong potential to be integrated with clinical workflow to address the need for an ADRD screening tool so that patients with cognitive impairment can receive appropriate and timely care.

The impacts of aging on the comprehension of affective prosody: A systematic review

Recent clinical reports have suggested a possible decline in the ability to understand emotions in speech (affective prosody comprehension) with aging. The present study aims to further examine the differences in performance between older and younger adults in terms of affective prosody comprehension. Following a recent cognitive model dividing affective prosody comprehension into perceptual and lexico-semantic components, a cognitive approach targeting these components was adopted. The influence of emotions' valence and category on aging performance was also investigated. A systematic review of the literature was carried out using six databases. Twenty-one articles, presenting 25 experiments, were included. All experiments analyzed affective prosody comprehension performance of older versus younger adults. The results confirmed that older adults' performance in identifying emotions in speech was reduced compared to younger adults. The results also brought out the fact that affective prosody comprehension abilities could be modulated by the emotion category but not by the emotional valence. Various theories account for this difference in performance, namely auditory perception, brain aging, and socioemotional selectivity theory suggesting that older people tend to neglect negative emotions. However, the explanation of the underlying deficits of the affective prosody decline is still limited.

Automated Vowel Articulation Analysis in Connected Speech Among Progressive Neurological Diseases, Dysarthria Types, and Dysarthria Severities

PURPOSE

Although articulatory impairment represents distinct speech characteristics in most neurological diseases affecting movement, methods allowing automated assessments of articulation deficits from the connected speech are scarce. This study aimed to design a fully automated method for analyzing dysarthria-related vowel articulation impairment and estimate its sensitivity in a broad range of neurological diseases and various types and severities of dysarthria.

METHOD

Unconstrained monologue and reading passages were acquired from 459 speakers, including 306 healthy controls and 153 neurological patients. The algorithm utilized a formant tracker in combination with a phoneme recognizer and subsequent signal processing analysis.

RESULTS

Articulatory undershoot of vowels was presented in a broad spectrum of progressive neurodegenerative diseases, including Parkinson's disease, progressive supranuclear palsy, multiple-system atrophy, Huntington's disease, essential tremor, cerebellar ataxia, multiple sclerosis, and amyotrophic lateral sclerosis, as well as in related dysarthria subtypes including hypokinetic, hyperkinetic, ataxic, spastic, flaccid, and their mixed variants. Formant ratios showed a higher sensitivity to vowel deficits than vowel space area. First formants of corner vowels were significantly lower for multiple-system atrophy than cerebellar ataxia. Second formants of vowels /a/ and /i/ were lower in ataxic compared to spastic dysarthria. Discriminant analysis showed a classification score of up to 41.0% for disease type, 39.3% for dysarthria type, and 49.2% for dysarthria severity. Algorithm accuracy reached an F-score of 0.77.

CONCLUSIONS

Distinctive vowel articulation alterations reflect underlying pathophysiology in neurological diseases. Objective acoustic analysis of vowel articulation has the potential to provide a universal method to screen motor speech disorders.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23681529.

Body exposure and vocal analysis: validation of fundamental frequency as a correlate of emotional arousal and valence

Introduction

Vocal analysis of fundamental frequency (f0) represents a suitable index to assess emotional activation. However, although f0 has often been used as an indicator of emotional arousal and different affective states, its psychometric properties are unclear. Specifically, there is uncertainty regarding the validity of the indices of f0_mean and f0_{variabilitymeasures} (f0_dispersion, f0_range, and f0_SD) and whether higher or lower f0 indices are associated with higher arousal in stressful situations. The present study therefore aimed to validate f0 as a marker of vocally encoded emotional arousal, valence, and body-related distress during body exposure as a psychological stressor.

Methods

N = 73 female participants first underwent a 3-min, non-activating neutral reference condition, followed by a 7-min activating body exposure condition. Participants completed questionnaires on affect (i.e., arousal, valence, body-related distress), and their voice data and heart rate (HR) were recorded continuously. Vocal analyses were performed using Praat, a program for extracting paralinguistic measures from spoken audio.

Results

The results revealed no effects for f0 and state body dissatisfaction or general affect. F0_mean correlated positively with self-reported arousal and negatively with valence, but was not correlated with HR_mean/maximum. No correlations with any measure were found for any f0_{variabililtymeasures}.

Discussion

Given the promising findings regarding f0_mean for arousal and valence and the inconclusive findings regarding f0 as a marker of general affect and body-related distress, it may be assumed that f0_mean represents a valid global marker of emotional arousal and valence rather than of concrete body-related distress. In view of the present findings regarding the validity of f0, it may be suggested that f0_mean, but not f0_{variabilitymeasures}, can be used to assess emotional arousal and valence in addition to self-report measures, which is less intrusive than conventional psychophysiological measures.

Acoustic Features Distinguishing Emotions in Swedish Speech

Few studies have examined which acoustic features of speech can be used to distinguish between different emotions, and how combinations of acoustic parameters contribute to identification of emotions. The aim of the present study was to investigate which acoustic parameters in Swedish speech are most important for differentiation between, and identification of, the emotions anger, fear, happiness, sadness, and surprise in Swedish sentences. One-way ANOVAs were used to compare acoustic parameters between the emotions and both simple and multiple logistic regression models were used to examine the contribution of different acoustic parameters to differentiation between emotions. Results showed differences between emotions for several acoustic parameters in Swedish speech: surprise was the most distinct emotion, with significant differences compared to the other emotions across a range of acoustic parameters, while anger and happiness did not differ from each other on any parameter. The logistic regression models showed that fear was the best-predicted emotion while happiness was most difficult to predict. Frequency- and spectral-balance-related parameters were best at predicting fear. Amplitude- and temporal-related parameters were most important for surprise, while a combination of frequency-, amplitude- and spectral balance-related parameters are important for sadness. Assuming that there are similarities between acoustic models and how listeners infer emotions in speech, results suggest that individuals with hearing loss, who lack abilities of frequency detection, may compared to normal hearing individuals have difficulties in identifying fear in Swedish speech. Since happiness and fear relied primarily on amplitude- and spectral-balance-related parameters, detection of them are probably facilitated more by hearing aid use.

The children's emotional speech recognition by adults: Cross-cultural study on Russian and Tamil language

The current study investigated the features of cross-cultural recognition of four basic emotions "joy-neutral (calm state)-sad-anger" in the spontaneous and acting speech of Indian and Russian children aged 8-12 years across Russian and Tamil languages. The research tasks were to examine the ability of Russian and Indian experts to recognize the state of Russian and Indian children by their speech, determine the acoustic features of correctly recognized speech samples, and specify the influence of the expert's language on the cross-cultural recognition of the emotional states of children. The study includes a perceptual auditory study by listeners and instrumental spectrographic analysis of child speech. Different accuracy and agreement between Russian and Indian experts were shown in recognizing the emotional states of Indian and Russian children by their speech, with more accurate recognition of the emotional state of children in their native language, in acting speech vs spontaneous speech. Both groups of experts recognize the state of anger via acting speech with the high agreement. The difference between the groups of experts was in the definition of joy, sadness, and neutral states depending on the test material with a different agreement. Speech signals with emphasized differences in acoustic patterns were more accurately classified by experts as belonging to emotions of different activation. The data showed that, despite the universality of basic emotions, on the one hand, the cultural environment affects their expression and perception, on the other hand, there are universal non-linguistic acoustic features of the voice that allow us to identify emotions via speech.

The Sound of Emotion: Pinpointing Emotional Voice Processing Via Frequency Tagging EEG

Successfully engaging in social communication requires efficient processing of subtle socio-communicative cues. Voices convey a wealth of social information, such as gender, identity, and the emotional state of the speaker. We tested whether our brain can systematically and automatically differentiate and track a periodic stream of emotional utterances among a series of neutral vocal utterances. We recorded frequency-tagged EEG responses of 20 neurotypical male adults while presenting streams of neutral utterances at a 4 Hz base rate, interleaved with emotional utterances every third stimulus, hence at a 1.333 Hz oddball frequency. Four emotions (happy, sad, angry, and fear) were presented as different conditions in different streams. To control the impact of low-level acoustic cues, we maximized variability among the stimuli and included a control condition with scrambled utterances. This scrambling preserves low-level acoustic characteristics but ensures that the emotional character is no longer recognizable. Results revealed significant oddball EEG responses for all conditions, indicating that every emotion category can be discriminated from the neutral stimuli, and every emotional oddball response was significantly higher than the response for the scrambled utterances. These findings demonstrate that emotion discrimination is fast, automatic, and is not merely driven by low-level perceptual features. Eventually, here, we present a new database for vocal emotion research with short emotional utterances (EVID) together with an innovative frequency-tagging EEG paradigm for implicit vocal emotion discrimination.

Clients' Facial Expressions of Self-Compassion, Self-Criticism, and Self-Protection in Emotion-Focused Therapy Videos

Clients' facial expressions allow psychotherapists to gather more information about clients' emotional processing. This study aims to examine and investigate the facial Action Units (AUs) of self-compassion, self-criticism, and self-protection within real Emotion-Focused Therapy (EFT) sessions. For this purpose, we used the facial analysis software iMotions. Twelve video sessions were selected for the analysis based on specific criteria. For self-compassion, the following AUs were significant: AUs 4 (brow furrow), 15 (lip corner depressor), and the AU12_smile (lip corner puller). For self-criticism, iMotions identified the AUs 2 (outer brow raise), AU1 (inner brow raise), AU7 (lid tighten), AU12_smirk (unilateral lip corner puller), and AU43 (eye closure). Self-protection was combined using the occurrence of AUs 1 and 4 and AU12_smirk. Moreover, the findings support the significance of discerning self-compassion and self-protection as two different concepts.

Cologne questionnaire on speechlessness: Development and validation

Speechlessness forms a psychological concept that describes non-speaking or silence in different situations. Speechlessness occurs in particular during emotional stress. The Cologne Questionnaire on Speechlessness (ger.: Kölner Fragebogen zur Sprachlosigkeit) is an instrument for measuring speechlessness as a function of emotional perception and processing in situations of emotional stress or existing emotional dysregulation. The questionnaire was developed in theoretical proximity to the constructs of alexithymia and expressive suppression. Item selection was performed on a first line sample of N = 307 individuals of a normal population. Acquisition of an exploratory model to classify the phenomenon was conducted within four samples in clinical and non-clinical settings. Validation of the factorial structure was performed using an overarching dataset (N = 1293) consisting of all samples. The results of a confirmatory factor analysis (CFA) indicated the best model fit (χ2 (df, 146) = 953.856; p < .001; Tucker-Lewis-Index = .891; Comparative Fit Index = .916; Root Mean Square Error of Approximation = .065; p < .001; N = 1293) with a four-factorial structure of the questionnaire. Both the overall acceptable validity and reliability recommend the application of KFS on individuals of the normal population as well as clinical subgroups. In addition, the questionnaire can also be used in the context of research on the regulation of emotions.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12144-022-04102-x.

Prosodic modulations in child-directed language and their impact on word learning

Child-directed language can support language learning, but how? we addressed two questions: (1) how caregivers prosodically modulated their speech as a function of word familiarity (known or unknown to the child) and accessibility of referent (visually present or absent from the immediate environment); (2) whether such modulations affect children's unknown word learning and vocabulary development. We used data from 38 English-speaking caregivers (from the ECOLANG corpus) talking about toys (both known and unknown to their children aged 3-4 years) both when the toys are present and when absent. We analyzed prosodic dimensions (i.e., speaking rate, pitch and intensity) of caregivers' productions of 6529 toy labels. We found that unknown labels were spoken with significantly slower speaking rate, wider pitch and intensity range than known labels, especially in the first mentions, suggesting that caregivers adjust their prosody based on children's lexical knowledge. Moreover, caregivers used slower speaking rate and larger intensity range to mark the first mentions of toys that were physically absent. After the first mentions, they talked about the referents louder with higher mean pitch when toys were present than when toys were absent. Crucially, caregivers' mean pitch of unknown words and the degree of mean pitch modulation for unknown words relative to known words (pitch ratio) predicted children's immediate word learning and vocabulary size 1 year later. In conclusion, caregivers modify their prosody when the learning situation is more demanding for children, and these helpful modulations assist children in word learning. RESEARCH HIGHLIGHTS: In naturalistic interactions, caregivers use slower speaking rate, wider pitch and intensity range when introducing new labels to 3-4-year-old children, especially in first mentions. Compared to when toys are present, caregivers speak more slowly with larger intensity range to mark the first mentions of toys that are physically absent. Mean pitch to mark word familiarity predicts children's immediate word learning and future vocabulary size.

Aprosodia Following Focal Brain Damage: What's Right and What's Left?

PURPOSE

Hemispheric specialization for the comprehension and expression of linguistic and emotional prosody is typically attributed to the right hemisphere. This study used techniques adapted from meta-analysis to critically examine the strength of existing evidence for hemispheric lateralization of prosody following brain damage.

METHOD

Twenty-one databases were searched for articles published from 1970 to 2020 addressing differences in prosody performance between groups defined by right hemisphere damage and left hemisphere damage. Hedges's g effect sizes were calculated for all possible prosody comparisons. Primary analyses summarize effects for four types: linguistic production, linguistic comprehension, emotion comprehension, and emotion production. Within each primary analysis, Hedges's g values were averaged across comparisons (usually from a single article) based on the same sample of individuals. Secondary analyses explore more specific classifications of comparisons.

RESULTS

Out of the 113 articles investigating comprehension and production of emotional and linguistic prosody, 62 were deemed appropriate for data extraction, but only 21 met inclusion criteria, passed quality reviews, and provided sufficient information for analysis. Evidence from this review illustrates the heterogeneity of research methods and results from studies that have investigated aprosodia. This review provides inconsistent support for selective contribution of the two cerebral hemispheres to prosody comprehension and production; however, the strongest finding suggests that right hemisphere lesions disrupt emotional prosody comprehension more than left hemisphere lesions.

CONCLUSION

This review highlights the impoverished nature of the existing literature; offers suggestions for future research; and highlights relevant clinical implications for the prognostication, evaluation, and treatment of aprosodia.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.20334987.

Machine Learning Algorithms for Detection and Classifications of Emotions in Contact Center Applications

Over the past few years, virtual assistant solutions used in Contact Center systems are gaining popularity. One of the main tasks of the virtual assistant is to recognize the intentions of the customer. It is important to note that quite often the actual intention expressed in a conversation is also directly influenced by the emotions that accompany that conversation. Unfortunately, scientific literature has not identified what specific types of emotions in Contact Center applications are relevant to the activities they perform. Therefore, the main objective of this work was to develop an Emotion Classification for Machine Detection of Affect-Tinged Conversational Contents dedicated directly to the Contact Center industry. In the conducted study, Contact Center voice and text channels were considered, taking into account the following families of emotions: anger, fear, happiness, sadness vs. affective neutrality of the statements. The obtained results confirmed the usefulness of the proposed classification—for the voice channel, the highest efficiency was obtained using the Convolutional Neural Network (accuracy, 67.5%; precision, 80.3; F1-Score, 74.5%), while for the text channel, the Support Vector Machine algorithm proved to be the most efficient (accuracy, 65.9%; precision, 58.5; F1-Score, 61.7%).

Influence of Multiple Music Styles and Composition Styles on College Students' Mental Health

The purpose is to reduce students' psychological pressure and improve their quality of study and life. Here, 400 language-impaired students in the public elective psychology course at Northwestern University and the undergraduate psychology class at Xi'an Foreign Studies University in the 2018-2019 academic year are randomly selected as the research objects for this music psychology experiment. The students were divided into different experimental groups using the Questionnaire Survey (QS) method to analyze the students' psychological reactions to Baroque, classical, and romantic music styles. Then, it further discusses the students' emotional response and audiovisual synaesthesia, as well as their recognition and choice of music style. The results show that there are significant differences in the same emotional response intensity of the subjects to different styles of music creation. The music expression is consistent with the actual feelings of the subjects. The tonality and color density of audiovisual synaesthesia vary with the style of music creation. Different music creation styles generate different associations in students' minds, thus showing different psychological reactions. The QS results indicate that soft and soothing music can relieve students' learning pressure most, while music with a strong sense of rhythm and vitality has no significant effect. Therefore, different music creation styles affect students' learning pressure differently. This work discusses the influence of different music creation styles on the mental health of contemporary college students and provides a reference for music therapy to relieve students' learning pressure in the future.

Vocal-Stress Diary: A Longitudinal Investigation of the Association of Everyday Work Stressors and Human Voice Features

The human voice conveys plenty of information about the speaker. A prevalent assumption is that stress-related changes in the human body affect speech production, thus affecting voice features. This suggests that voice data may be an easy-to-capture measure of everyday stress levels and can thus serve as a warning signal of stress-related health consequences. However, previous research is limited (i.e., has induced stress only through artificial tasks or has investigated only short-term or extreme stressors), leaving it open whether everyday work stressors are associated with voice features. Thus, our participants (111 adult working individuals) took part in a 1-week diary study (Sunday until Sunday), in which they provided voice messages and self-report data on daily work stressors. Results showed that work stressors were associated with voice features such as increased speech rate and voice intensity. We discuss theoretical, practical, and ethical implications regarding the voice as an indicator of psychological states.

Detecting emotion in speech expressing incongruent emotional cues through voice and content: investigation on dominant modality and language

This research investigated how we detect emotion in speech when the emotional cues in the sound of voice do not match the semantic content. It examined the dominance of the voice or semantics in the perception of emotion from incongruent speech and the influence of language on the interaction between the two modalities. Japanese participants heard a voice emoting anger, happiness or sadness while saying "I'm angry", "I'm pleased" or "I'm sad", which were in their native language, in their second language (English) and in unfamiliar languages (Khmer and Swedish). They reported how much they agree that the speaker is expressing each of the three emotions. Two experiments were conducted with different number of voice stimuli, and both found consistent results. Strong reliance on the voice was found for the speech in participants' second and unfamiliar languages but the dominance was weakened for the speech in their native language. Among the three emotions, voice was most important for perception of sadness. This research concludes that the impact of the emotional cues expressed by the voice and semantics varies depending on the expressed emotions and the language.

Superior Communication of Positive Emotions Through Nonverbal Vocalisations Compared to Speech Prosody

The human voice communicates emotion through two different types of vocalizations: nonverbal vocalizations (brief non-linguistic sounds like laughs) and speech prosody (tone of voice). Research examining recognizability of emotions from the voice has mostly focused on either nonverbal vocalizations or speech prosody, and included few categories of positive emotions. In two preregistered experiments, we compare human listeners’ (total n = 400) recognition performance for 22 positive emotions from nonverbal vocalizations (n = 880) to that from speech prosody (n = 880). The results show that listeners were more accurate in recognizing most positive emotions from nonverbal vocalizations compared to prosodic expressions. Furthermore, acoustic classification experiments with machine learning models demonstrated that positive emotions are expressed with more distinctive acoustic patterns for nonverbal vocalizations as compared to speech prosody. Overall, the results suggest that vocal expressions of positive emotions are communicated more successfully when expressed as nonverbal vocalizations compared to speech prosody.

External speech processing and auditory verbal hallucinations: A systematic review of functional neuroimaging studies

It has been documented that individuals who hear auditory verbal hallucinations (AVH) exhibit diminished capabilities in processing external speech. While functional neuroimaging studies have attempted to characterise the cortical regions and networks facilitating these deficits in a bid to understand AVH, considerable methodological heterogeneity has prevented a consensus being reached. The current systematic review investigated the neurobiological underpinnings of external speech processing deficits in voice-hearers in 38 studies published between January 1990 to June 2020. AVH-specific deviations in the activity and lateralisation of the temporal auditory regions were apparent when processing speech sounds, words and sentences. During active or affective listening tasks, functional connectivity changes arose within the language, limbic and default mode networks. However, poor study quality and lack of replicable results plague the field. A detailed list of recommendations has been provided to improve the quality of future research on this topic.

Affective Voice Interaction and Artificial Intelligence: A Research Study on the Acoustic Features of Gender and the Emotional States of the PAD Model

New types of artificial intelligence products are gradually transferring to voice interaction modes with the demand for intelligent products expanding from communication to recognizing users' emotions and instantaneous feedback. At present, affective acoustic models are constructed through deep learning and abstracted into a mathematical model, making computers learn from data and equipping them with prediction abilities. Although this method can result in accurate predictions, it has a limitation in that it lacks explanatory capability; there is an urgent need for an empirical study of the connection between acoustic features and psychology as the theoretical basis for the adjustment of model parameters. Accordingly, this study focuses on exploring the differences between seven major “acoustic features” and their physical characteristics during voice interaction with the recognition and expression of “gender” and “emotional states of the pleasure-arousal-dominance (PAD) model.” In this study, 31 females and 31 males aged between 21 and 60 were invited using the stratified random sampling method for the audio recording of different emotions. Subsequently, parameter values of acoustic features were extracted using Praat voice software. Finally, parameter values were analyzed using a Two-way ANOVA, mixed-design analysis in SPSS software. Results show that gender and emotional states of the PAD model vary among seven major acoustic features. Moreover, their difference values and rankings also vary. The research conclusions lay a theoretical foundation for AI emotional voice interaction and solve deep learning's current dilemma in emotional recognition and parameter optimization of the emotional synthesis model due to the lack of explanatory power.

The Voice Gives It Away

Abstract. Two experiments examined the impact of voice pitch on gender stereotyping. Participants listened to a text read by a female (Study 1; N = 171) or male (Study 2, N = 151) speaker, whose voice pitch was manipulated to be high or low. They rated the speaker on positive and negative facets of masculinity and femininity, competence, and likability. They also indicated their own gendered self-concept. High pitch was associated with the ascription of more feminine traits and greater likability. The high-pitch female speaker was rated as less competent, and the high-pitch male speaker was perceived as less masculine. Text content and participants’ gendered self-concept did not moderate the pitch effect. The findings underline the importance of voice pitch for impression formation.

Effects of aging on emotion recognition from dynamic multimodal expressions and vocalizations

Age-related differences in emotion recognition have predominantly been investigated using static pictures of facial expressions, and positive emotions beyond happiness have rarely been included. The current study instead used dynamic facial and vocal stimuli, and included a wider than usual range of positive emotions. In Task 1, younger and older adults were tested for their abilities to recognize 12 emotions from brief video recordings presented in visual, auditory, and multimodal blocks. Task 2 assessed recognition of 18 emotions conveyed by non-linguistic vocalizations (e.g., laughter, sobs, and sighs). Results from both tasks showed that younger adults had significantly higher overall recognition rates than older adults. In Task 1, significant group differences (younger > older) were only observed for the auditory block (across all emotions), and for expressions of anger, irritation, and relief (across all presentation blocks). In Task 2, significant group differences were observed for 6 out of 9 positive, and 8 out of 9 negative emotions. Overall, results indicate that recognition of both positive and negative emotions show age-related differences. This suggests that the age-related positivity effect in emotion recognition may become less evident when dynamic emotional stimuli are used and happiness is not the only positive emotion under study.

Beyond Correlation: Acoustic Transformation Methods for the Experimental Study of Emotional Voice and Speech

While acoustic analysis methods have become a commodity in voice emotion research, experiments that attempt not only to describe but to computationally manipulate expressive cues in emotional voice and speech have remained relatively rare. We give here a nontechnical overview of voice-transformation techniques from the audio signal-processing community that we believe are ripe for adoption in this context. We provide sound examples of what they can achieve, examples of experimental questions for which they can be used, and links to open-source implementations. We point at a number of methodological properties of these algorithms, such as being specific, parametric, exhaustive, and real-time, and describe the new possibilities that these open for the experimental study of the emotional voice.

Gradual positive and negative affect induction: The effect of verbalizing affective content

In this paper, we study the effect of verbalizing affective pictures on affective state and language production. Individuals describe (Study I: Spoken Descriptions of Pictures) or passively view (Study II: Passively Viewing Pictures) 40 pictures for the International Affective Picture System (IAPS) that gradually increase from neutral to either positive or negative content. We expected that both methods would result in successful affect induction, and that the effect would be stronger for verbally describing pictures than for passively viewing them. Results indicate that speakers indeed felt more negative after describing negative pictures, but that describing positive (compared to neutral) pictures did not result in a more positive state. Contrary to our hypothesis, no differences were found between describing and passively viewing the pictures. Furthermore, we analysed the verbal picture descriptions produced by participants on various dimensions. Results indicate that positive and negative pictures were indeed described with increasingly more affective language in the expected directions. In addition to informing our understanding of the relationship between (spoken) language production and affect, these results also potentially pave the way for a new method of affect induction that uses free expression.