Age-Related Differences in the Perception of Emotion in Spoken Language: The Relative Roles of Prosody and Semantics

Assessing the perception of emotional prosody in healthy ageing

BACKGROUND

Emotional prosody is the reflection of emotion types such as happiness, sadness, fear and anger in the speaker's tone of voice. Accurately perceiving, interpreting and expressing emotional prosody is an inseparable part of successful communication and social interaction. There are few studies on emotional prosody, which is crucial for communication, and the results of these studies have inconsistent information regarding age and gender.

AIMS

The primary aim of this study is to assess the perception of emotional prosody in healthy ageing. The other aim is to examine the effects of variables such as age, gender, language and neurocognitive capacity on the prediction of emotional prosody recognition skills.

METHODS AND PROCEDURES

Sixty-nine participants between the ages of 18-75 were included in the study. Participants were grouped as the young group aged 18-35 (n = 26), the middle-aged group aged 36-55 (n = 24) and the elderly group aged 56-75 (n = 19). Perceptual emotional prosody test, motor response time test, and neuropsychological test batteries were administered to the participants. Participants were asked to recognise the emotion in the sentences played on the computer. Natural (neutral, containing neither positive nor negative emotion), happy, angry, surprised and panic emotions were evaluated with sentences composed of pseudoword stimuli.

RESULTS AND OUTCOMES

It was observed that the elderly group performed worse in recognising angry, panic, natural and happy emotions and in total recognition, which gives the correct recognition performance in recognition of all emotions. There was no age-related difference in recognition of the emotion of surprise. The women were more successful in recognising angry, panic, happy and total emotions compared to men. Age and Motor Reaction Time Test scores were found to be significant predictors in the emotional response time regression model. Age, language, attention and gender variables were found to have a significant effect on the regression model created for the success of total recognition of emotions (p < 0.05).

CONCLUSIONS AND IMPLICATIONS

This was a novel study in which emotional prosody was assessed in the elderly by eliminating lexical-semantic cues related to emotional prosody and associating emotional prosody results with neuropsychiatric tests. All our findings revealed the importance of age for the perception of emotional prosody. In addition, the effects of cognitive functions such as attention, which decline with age, were found to be important. Therefore, it should not be forgotten that many factors contribute to the success of recognising emotional prosody correctly. In this context, clinicians should consider variables such as cognitive health and education when assessing the perception of emotional prosody in elderly individuals.

WHAT THIS PAPER ADDS

What is already known on the subject Most of the studies compare young and old groups, and these studies evaluate the perception of emotional prosody by using sentences formed by observing the speech sounds, syllables, words and grammar rules in the vocabulary of the language. It has been reported that the perception of emotional prosody is lower, mostly in the elderly group, but there is inconsistent information in terms of age and gender. What this paper adds to existing knowledge Perceptual Prosody Recognition was evaluated with an experimental design in which sentence structures consisting of lexemes were used as stimuli and neurocognitive tests were included, taking into account the phonological and syntactic rules of language. This study was a novel study in diagnosing emotional prosody in terms of comparing different age groups and determining the factors affecting multidimensional emotional prosody, including neuropsychiatric features. What are the clinical implications of this work? All our findings revealed the importance of age for the perception of emotional prosody. In addition, it was determined that the effects of cognitive functions such as attention were important with age.

Effects of age and hearing loss on speech emotion discrimination

Better communication with older people requires not only improving speech intelligibility but also understanding how well emotions can be conveyed and the effect of age and hearing loss (HL) on emotion perception. In this paper, emotion discrimination experiments were conducted using a vocal morphing method and an HL simulator in young normal hearing (YNH) and older participants. Speech sounds were morphed to represent intermediate emotions between all combinations of happiness, sadness, and anger. Discrimination performance was compared when the YNH listened to normal sounds, when the same YNH listened to HL simulated sounds, and when older people listened to the same normal sounds. The results showed that there was no significant difference between discrimination with and without HL simulation, suggesting that peripheral HL may not affect emotion perception. The discrimination performance of the older participants was significantly worse only for the anger-happiness pair than for the other emotion pairs and for the YNH. It was also found that the difficulty increases with age, not just with hearing level.

The Role of Early Intact Auditory Experience on the Perception of Spoken Emotions, Comparing Prelingual to Postlingual Cochlear Implant Users

OBJECTIVES

Cochlear implants (CI) are remarkably effective, but have limitations regarding the transformation of the spectro-temporal fine structures of speech. This may impair processing of spoken emotions, which involves the identification and integration of semantic and prosodic cues. Our previous study found spoken-emotions-processing differences between CI users with postlingual deafness (postlingual CI) and normal hearing (NH) matched controls (age range, 19 to 65 years). Postlingual CI users over-relied on semantic information in incongruent trials (prosody and semantics present different emotions), but rated congruent trials (same emotion) similarly to controls. Postlingual CI's intact early auditory experience may explain this pattern of results. The present study examined whether CI users without intact early auditory experience (prelingual CI) would generally perform worse on spoken emotion processing than NH and postlingual CI users, and whether CI use would affect prosodic processing in both CI groups. First, we compared prelingual CI users with their NH controls. Second, we compared the results of the present study to our previous study (Taitlebaum-Swead et al. 2022; postlingual CI).

DESIGN

Fifteen prelingual CI users and 15 NH controls (age range, 18 to 31 years) listened to spoken sentences composed of different combinations (congruent and incongruent) of three discrete emotions (anger, happiness, sadness) and neutrality (performance baseline), presented in prosodic and semantic channels (Test for Rating of Emotions in Speech paradigm). Listeners were asked to rate (six-point scale) the extent to which each of the predefined emotions was conveyed by the sentence as a whole (integration of prosody and semantics), or to focus only on one channel (rating the target emotion [RTE]) and ignore the other (selective attention). In addition, all participants performed standard tests of speech perception. Performance on the Test for Rating of Emotions in Speech was compared with the previous study (postlingual CI).

RESULTS

When asked to focus on one channel, semantics or prosody, both CI groups showed a decrease in prosodic RTE (compared with controls), but only the prelingual CI group showed a decrease in semantic RTE. When the task called for channel integration, both groups of CI users used semantic emotional information to a greater extent than their NH controls. Both groups of CI users rated sentences that did not present the target emotion higher than their NH controls, indicating some degree of confusion. However, only the prelingual CI group rated congruent sentences lower than their NH controls, suggesting reduced accumulation of information across channels. For prelingual CI users, individual differences in identification of monosyllabic words were significantly related to semantic identification and semantic-prosodic integration.

CONCLUSIONS

Taken together with our previous study, we found that the degradation of acoustic information by the CI impairs the processing of prosodic emotions, in both CI user groups. This distortion appears to lead CI users to over-rely on the semantic information when asked to integrate across channels. Early intact auditory exposure among CI users was found to be necessary for the effective identification of semantic emotions, as well as the accumulation of emotional information across the two channels. Results suggest that interventions for spoken-emotion processing should not ignore the onset of hearing loss.

"Love looks not with the eyes": supranormal processing of emotional speech in individuals with late-blindness versus preserved processing in individuals with congenital-blindness

Processing of emotional speech in the absence of visual information relies on two auditory channels: semantics and prosody. No study to date has investigated how blindness impacts this process. Two theories, Perceptual Deficit, and Sensory Compensation, yiled different expectations about the role of visual experience (or its lack thereof) in processing emotional speech. To test the effect of vision and early visual experience on processing of emotional speech, we compared individuals with congenital blindness (CB, n = 17), individuals with late blindness (LB, n = 15), and sighted controls (SC, n = 21) on identification and selective-attention of semantic and prosodic spoken-emotions. Results showed that individuals with blindness performed at least as well as SC, supporting Sensory Compensation and the role of cortical reorganisation. Individuals with LB outperformed individuals with CB, in accordance with Perceptual Deficit, supporting the role of early visual experience. The LB advantage was moderated by executive functions (working-memory). Namely, the advantage was erased for individuals with CB who showed higher levels of executive functions. Results suggest that vision is not necessary for processing of emotional speech, but early visual experience could improve it. The findings support a combination of the two aforementioned theories and reject a dichotomous view of deficiencies/enhancements of blindness.

Age-related differences in processing of emotions in speech disappear with babble noise in the background

Older adults process emotional speech differently than young adults, relying less on prosody (tone) relative to semantics (words). This study aimed to elucidate the mechanisms underlying these age-related differences via an emotional speech-in-noise test. A sample of 51 young and 47 older adults rated spoken sentences with emotional content on both prosody and semantics, presented on the background of wideband speech-spectrum noise (sensory interference) or on the background of multi-talker babble (sensory/cognitive interference). The presence of wideband noise eliminated age-related differences in semantics but not in prosody when processing emotional speech. Conversely, the presence of babble resulted in the elimination of age-related differences across all measures. The results suggest that both sensory and cognitive-linguistic factors contribute to age-related changes in emotional speech processing. Because real world conditions typically involve noisy background, our results highlight the importance of testing under such conditions.

Self-Rated Confidence in Vocal Emotion Recognition Ability: The Role of Gender

PURPOSE

We studied the role of gender in metacognition of voice emotion recognition ability (ERA), reflected by self-rated confidence (SRC). To this end, we guided our study in two approaches: first, by examining the role of gender in voice ERA and SRC independently and second, by looking for gender effects on the ERA association with SRC.

METHOD

We asked 100 participants (50 men, 50 women) to interpret a set of vocal expressions portrayed by 30 actors (16 men, 14 women) as defined by their emotional meaning. Targets were 180 repetitive lexical sentences articulated in congruent emotional voices (anger, sadness, surprise, happiness, fear) and neutral expressions. Trial by trial, the participants were assigned retrospective SRC based on their emotional recognition performance.

RESULTS

A binomial generalized linear mixed model (GLMM) estimating ERA accuracy revealed a significant gender effect, with women encoders (speakers) yielding higher accuracy levels than men. There was no significant effect of the decoder's (listener's) gender. A second GLMM estimating SRC found a significant effect of encoder and decoder genders, with women outperforming men. Gamma correlations were significantly greater than zero for women and men decoders.

CONCLUSIONS

In spite of varying interpretations of gender in each independent rating (ERA and SRC), our results suggest that both men and women decoders were accurate in their metacognition regarding voice emotion recognition. Further research is needed to study how individuals of both genders use metacognitive knowledge in their emotional recognition and whether and how such knowledge contributes to effective social communication.

Evaluating the Relative Perceptual Salience of Linguistic and Emotional Prosody in Quiet and Noisy Contexts

How people recognize linguistic and emotional prosody in different listening conditions is essential for understanding the complex interplay between social context, cognition, and communication. The perception of both lexical tones and emotional prosody depends on prosodic features including pitch, intensity, duration, and voice quality. However, it is unclear which aspect of prosody is perceptually more salient and resistant to noise. This study aimed to investigate the relative perceptual salience of emotional prosody and lexical tone recognition in quiet and in the presence of multi-talker babble noise. Forty young adults randomly sampled from a pool of native Mandarin Chinese with normal hearing listened to monosyllables either with or without background babble noise and completed two identification tasks, one for emotion recognition and the other for lexical tone recognition. Accuracy and speed were recorded and analyzed using generalized linear mixed-effects models. Compared with emotional prosody, lexical tones were more perceptually salient in multi-talker babble noise. Native Mandarin Chinese participants identified lexical tones more accurately and quickly than vocal emotions at the same signal-to-noise ratio. Acoustic and cognitive dissimilarities between linguistic prosody and emotional prosody may have led to the phenomenon, which calls for further explorations into the underlying psychobiological and neurophysiological mechanisms.

The impacts of aging on the comprehension of affective prosody: A systematic review

Recent clinical reports have suggested a possible decline in the ability to understand emotions in speech (affective prosody comprehension) with aging. The present study aims to further examine the differences in performance between older and younger adults in terms of affective prosody comprehension. Following a recent cognitive model dividing affective prosody comprehension into perceptual and lexico-semantic components, a cognitive approach targeting these components was adopted. The influence of emotions' valence and category on aging performance was also investigated. A systematic review of the literature was carried out using six databases. Twenty-one articles, presenting 25 experiments, were included. All experiments analyzed affective prosody comprehension performance of older versus younger adults. The results confirmed that older adults' performance in identifying emotions in speech was reduced compared to younger adults. The results also brought out the fact that affective prosody comprehension abilities could be modulated by the emotion category but not by the emotional valence. Various theories account for this difference in performance, namely auditory perception, brain aging, and socioemotional selectivity theory suggesting that older people tend to neglect negative emotions. However, the explanation of the underlying deficits of the affective prosody decline is still limited.

Recognition of Sentences With Complex Syntax in Speech Babble by Adolescents With Normal Hearing or Cochlear Implants

PURPOSE

General language abilities of children with cochlear implants have been thoroughly investigated, especially at young ages, but far less is known about how well they process language in real-world settings, especially in higher grades. This study addressed this gap in knowledge by examining recognition of sentences with complex syntactic structures in backgrounds of speech babble by adolescents with cochlear implants, and peers with normal hearing.

DESIGN

Two experiments were conducted. First, new materials were developed using young adults with normal hearing as the normative sample, creating a corpus of sentences with controlled, but complex syntactic structures presented in three kinds of babble that varied in voice gender and number of talkers. Second, recognition by adolescents with normal hearing or cochlear implants was examined for these new materials and for sentence materials used with these adolescents at younger ages. Analyses addressed three objectives: (1) to assess the stability of speech recognition across a multiyear age range, (2) to evaluate speech recognition of sentences with complex syntax in babble, and (3) to explore how bottom-up and top-down mechanisms account for performance under these conditions.

RESULTS

Results showed: (1) Recognition was stable across the ages of 10-14 years for both groups. (2) Adolescents with normal hearing performed similarly to young adults with normal hearing, showing effects of syntactic complexity and background babble; adolescents with cochlear implants showed poorer recognition overall, and diminished effects of both factors. (3) Top-down language and working memory primarily explained recognition for adolescents with normal hearing, but the bottom-up process of perceptual organization primarily explained recognition for adolescents with cochlear implants.

CONCLUSIONS

Comprehension of language in real-world settings relies on different mechanisms for adolescents with cochlear implants than for adolescents with normal hearing. A novel finding was that perceptual organization is a critical factor.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21965228.

Response to McKenzie et al. 2021: Keep It Simple; Young Adults With Autism Spectrum Disorder Without Intellectual Disability Can Process Basic Emotions

We recently read the interesting and informative paper entitled "Empathic accuracy and cognitive and affective empathy in young adults with and without autism spectrum disorder" (McKenzie et al. in Journal of Autism and Developmental Disorders 52: 1-15, 2021). This paper expands recent findings from our lab (Ben-David in Journal of Autism and Developmental Disorders 50: 741-756, 2020a; International Journal of Audiology 60: 319-321, 2020b) and a recent theoretical framework (Icht et al. in Autism Research 14: 1948-1964, 2021) that may suggest a new purview for McKenzie et al.'s results. Namely, these papers suggest that young adults with autism spectrum disorder without intellectual disability can successfully recruit their cognitive abilities to distinguish between different simple spoken emotions, but may still face difficulties processing complex, subtle emotions. McKenzie et al. (Journal of Autism and Developmental Disorders 52: 1-15, 2021) extended these findings to the processing of emotions in video clips, with both visual and auditory information.

Older adults' emotion recognition: No auditory-visual benefit for less clear expressions

The ability to recognise emotion from faces or voices appears to decline with advancing age. However, some studies have shown that emotion recognition of auditory-visual (AV) expressions is largely unaffected by age, i.e., older adults get a larger benefit from AV presentation than younger adults resulting in similar AV recognition levels. An issue with these studies is that they used well-recognised emotional expressions that are unlikely to generalise to real-life settings. To examine if an AV emotion recognition benefit generalizes across well and less well recognised stimuli, we conducted an emotion recognition study using expressions that had clear or unclear emotion information for both modalities, or clear visual, but unclear auditory information. Older (n = 30) and younger (n = 30) participants were tested on stimuli of anger, happiness, sadness, surprise, and disgust (expressed in spoken sentences) in auditory-only (AO), visual-only (VO), or AV format. Participants were required to respond by choosing one of 5 emotion options. Younger adults were more accurate in recognising emotions than older adults except for clear VO expressions. Younger adults showed an AV benefit even when unimodal recognition was poor. No such AV benefit was found for older adults; indeed, AV was worse than VO recognition when AO recognition was poor. Analyses of confusion responses indicated that older adults generated more confusion responses that were common between AO and VO conditions, than younger adults. We propose that older adults' poorer AV performance may be due to a combination of weak auditory emotion recognition and response uncertainty that resulted in a higher cognitive load.

Processing emotional prosody in a foreign language: the case of German and Hebrew

This study investigated the universality of emotional prosody in perception of discrete emotions when semantics is not available. In two experiments the perception of emotional prosody in Hebrew and German by listeners who speak one of the languages but not the other was investigated. Having a parallel tool in both languages allowed to conduct controlled comparisons. In Experiment 1, 39 native German speakers with no knowledge of Hebrew and 80 native Israeli speakers rated Hebrew sentences spoken with four different emotional prosodies (anger, fear, happiness, sadness) or neutral. The Hebrew version of the Test for Rating of Emotions in Speech (T-RES) was used for this purpose. Ratings indicated participants’ agreement on how much the sentence conveyed each of four discrete emotions (anger, fear, happiness and sadness). In Experient 2, 30 native speakers of German, and 24 Israeli native speakers of Hebrew who had no knowledge of German rated sentences of the German version of the T-RES. Based only on the prosody, German-speaking participants were able to accurately identify the emotions in the Hebrew sentences and Hebrew-speaking participants were able to identify the emotions in the German sentences. In both experiments ratings between the groups were similar. These findings show that individuals are able to identify emotions in a foreign language even if they do not have access to semantics. This ability goes beyond identification of target emotion; similarities between languages exist even for “wrong” perception. This adds to accumulating evidence in the literature on the universality of emotional prosody.

Sentence vs. Word Perception by Young Healthy Females: Toward a Better Understanding of Emotion in Spoken Language

Expression and perception of emotions by voice are fundamental for basic mental health stability. Since different languages interpret results differently, studies should be guided by the relationship between speech complexity and the emotional perception. The aim of our study was therefore to analyze the efficiency of speech stimuli, word vs. sentence, as it relates to the accuracy of four different categories of emotions: anger, sadness, happiness, and neutrality. To this end, a total of 2,235 audio clips were presented to 49 females, native Hebrew speakers, aged 20–30 years (M = 23.7; SD = 2.13). Participants were asked to judge audio utterances according to one of four emotional categories: anger, sadness, happiness, and neutrality. Simulated voice samples were consisting of words and meaningful sentences, provided by 15 healthy young females Hebrew native speakers. Generally, word vs. sentence was not originally accepted as a means of emotional recognition of voice; However, introducing a variety of speech utterances revealed a different perception. Thus, the emotional conveyance provided new, even higher precision to our findings: Anger emotions produced a higher impact to the single word (χ2 = 10.21, p &lt; 0.01) as opposed to the sentence, while sadness was identified more accurately with a sentence (χ2 = 3.83, p = 0.05). Our findings resulted in a better understanding of how speech types can interpret perception, as a part of mental health.

Age-Related Changes in the Perception of Emotions in Speech: Assessing Thresholds of Prosody and Semantics Recognition in Noise for Young and Older Adults

Older adults process emotions in speech differently than do young adults. However, it is unclear whether these age-related changes impact all speech channels to the same extent, and whether they originate from a sensory or a cognitive source. The current study adopted a psychophysical approach to directly compare young and older adults’ sensory thresholds for emotion recognition in two channels of spoken-emotions: prosody (tone) and semantics (words). A total of 29 young adults and 26 older adults listened to 50 spoken sentences presenting different combinations of emotions across prosody and semantics. They were asked to recognize the prosodic or semantic emotion, in separate tasks. Sentences were presented on the background of speech-spectrum noise ranging from SNR of −15 dB (difficult) to +5 dB (easy). Individual recognition thresholds were calculated (by fitting psychometric functions) separately for prosodic and semantic recognition. Results indicated that: (1). recognition thresholds were better for young over older adults, suggesting an age-related general decrease across channels; (2). recognition thresholds were better for prosody over semantics, suggesting a prosodic advantage; (3). importantly, the prosodic advantage in thresholds did not differ between age groups (thus a sensory source for age-related differences in spoken-emotions processing was not supported); and (4). larger failures of selective attention were found for older adults than for young adults, indicating that older adults experienced larger difficulties in inhibiting irrelevant information. Taken together, results do not support a sole sensory source, but rather an interplay of cognitive and sensory sources for age-related differences in spoken-emotions processing.

A Cross-Linguistic Validation of the Test for Rating Emotions in Speech: Acoustic Analyses of Emotional Sentences in English, German, and Hebrew

PURPOSE

The Test for Rating Emotions in Speech (T-RES) has been developed in order to assess the processing of emotions in spoken language. In this tool, spoken sentences, which are composed of emotional content (anger, happiness, sadness, and neutral) in both semantics and prosody in different combinations, are rated by listeners. To date, English, German, and Hebrew versions have been developed, as well as online versions, iT-RES, to adapt to COVID-19 social restrictions. Since the perception of spoken emotions may be affected by linguistic (and cultural) variables, it is important to compare the acoustic characteristics of the stimuli within and between languages. The goal of the current report was to provide cross-linguistic acoustic validation of the T-RES.

METHOD

T-RES sentences in the aforementioned languages were acoustically analyzed in terms of mean F0, F0 range, and speech rate to obtain profiles of acoustic parameters for different emotions.

RESULTS

Significant within-language discriminability of prosodic emotions was found, for both mean F0 and speech rate. Similarly, these measures were associated with comparable patterns of prosodic emotions for each of the tested languages and emotional ratings.

CONCLUSIONS

The results demonstrate the lack of dependence of prosody and semantics within the T-RES stimuli. These findings illustrate the listeners' ability to clearly distinguish between the different prosodic emotions in each language, providing a cross-linguistic validation of the T-RES and iT-RES.

Synchrony in Old Age: Playing the Mirror Game Improves Cognitive Performance

OBJECTIVES

Studies have shown that synchronized motion between people positively affects a range of emotional and social functions. The mirror-game is a synchrony-based paradigm, common to theater, performance arts, and therapy, which includes dyadic synchronized motion, playfulness, and spontaneity. The goal of the current study is to examine the effects of the mirror-game on subjective and cognitive indices in late life.

METHODS

Thirty-four older adults (aged 71-98) participated in a within-group study design. Participants conducted two sessions of 9-minute movement activities: the mirror-game and the control condition - a physical exercise class. Several measures were taken before and after experimental sessions to assess socio-emotional and attentional functions.

RESULTS

The mirror-game enhanced performance on the attention sub-scale and led to faster detections of spoken words in noise. Further, it enhanced perceived partner responsiveness and led to an increase in positive reported experience.

CONCLUSIONS

Our preliminary findings suggest that the mirror-game, rather than the exercise class, may have an immediate impact on mood and some attentional functions.

CLINICAL IMPLICATIONS

The mirror-game is a novel intervention, with potential benefits of social-emotional and cognitive functioning, which can be easily implemented into the daily routine care of older adults. Future studies should explore the effect of the mirror-game on additional cognitive and socio-emotional aspects.

Safe and Sound: The Effects of Experimentally Priming the Sense of Attachment Security on Pure-Tone Audiometric Thresholds Among Young and Older Adults

Attachment security has consistently been found to correlate with relaxed exploration, openness, and mindful attention to incoming information. The present studies explored whether contextually infusing a sense of attachment security (security priming) can improve hearing in young and older adults. In Study 1, participants (29 young, 30 older) performed a standardized pure-tone audiometric-thresholds test twice. In the security-priming condition, a picture of a participant's security-enhancing figure was presented throughout the task. In the control condition, a picture of an unknown person (matched in sex, age, and facial expression) was used as a neutral prime. Study 2 (14 young, 14 older) was almost identical, except that it was preregistered and the neutral prime was a circle. In both studies, participants performed better (had lower hearing thresholds) in the security-priming condition. The current study is the first to show that attachment security improves sensory perception, and these results have meaningful implications for theory and clinical hearing tests.

More Than Words: the Relative Roles of Prosody and Semantics in the Perception of Emotions in Spoken Language by Postlingual Cochlear Implant Users

OBJECTIVES

The processing of emotional speech calls for the perception and integration of semantic and prosodic cues. Although cochlear implants allow for significant auditory improvements, they are limited in the transmission of spectro-temporal fine-structure information that may not support the processing of voice pitch cues. The goal of the current study is to compare the performance of postlingual cochlear implant (CI) users and a matched control group on perception, selective attention, and integration of emotional semantics and prosody.

DESIGN

Fifteen CI users and 15 normal hearing (NH) peers (age range, 18-65 years) 1istened to spoken sentences composed of different combinations of four discrete emotions (anger, happiness, sadness, and neutrality) presented in prosodic and semantic channels-T-RES: Test for Rating Emotions in Speech. In three separate tasks, listeners were asked to attend to the sentence as a whole, thus integrating both speech channels (integration), or to focus on one channel only (rating of target emotion) and ignore the other (selective attention). Their task was to rate how much they agreed that the sentence conveyed each of the predefined emotions. In addition, all participants performed standard tests of speech perception.

RESULTS

When asked to focus on one channel, semantics or prosody, both groups rated emotions similarly with comparable levels of selective attention. When the task was called for channel integration, group differences were found. CI users appeared to use semantic emotional information more than did their NH peers. CI users assigned higher ratings than did their NH peers to sentences that did not present the target emotion, indicating some degree of confusion. In addition, for CI users, individual differences in speech comprehension over the phone and identification of intonation were significantly related to emotional semantic and prosodic ratings, respectively.

CONCLUSIONS

CI users and NH controls did not differ in perception of prosodic and semantic emotions and in auditory selective attention. However, when the task called for integration of prosody and semantics, CI users overused the semantic information (as compared with NH). We suggest that as CI users adopt diverse cue weighting strategies with device experience, their weighting of prosody and semantics differs from those used by NH. Finally, CI users may benefit from rehabilitation strategies that strengthen perception of prosodic information to better understand emotional speech.

Processing of Spoken Emotions in Schizophrenia: Forensic and Non-forensic Patients Differ in Emotional Identification and Integration but Not in Selective Attention

Patients with schizophrenia (PwS) typically demonstrate deficits in visual processing of emotions. Less is known about auditory processing of spoken-emotions, as conveyed by the prosodic (tone) and semantics (words) channels. In a previous study, forensic PwS (who committed violent offenses) identified spoken-emotions and integrated the emotional information from both channels similarly to controls. However, their performance indicated larger failures of selective-attention, and lower discrimination between spoken-emotions, than controls. Given that forensic schizophrenia represents a special subgroup, the current study compared forensic and non-forensic PwS. Forty-five PwS listened to sentences conveying four basic emotions presented in semantic or prosodic channels, in different combinations. They were asked to rate how much they agreed that the sentences conveyed a predefined emotion, focusing on one channel or on the sentence as a whole. Their performance was compared to that of 21 forensic PwS (previous study). The two groups did not differ in selective-attention. However, better emotional identification and discrimination, as well as better channel integration were found for the forensic PwS. Results have several clinical implications: difficulties in spoken-emotions processing might not necessarily relate to schizophrenia; attentional deficits might not be a risk factor for aggression in schizophrenia; and forensic schizophrenia might have unique characteristics as related to spoken-emotions processing (motivation, stimulation).

Unisensory and Multisensory Stroop Effects Modulate Gender Differences in Verbal and Nonverbal Emotion Perception

Purpose This study aimed to examine the Stroop effects of verbal and nonverbal cues and their relative impacts on gender differences in unisensory and multisensory emotion perception. Method Experiment 1 investigated how well 88 normal Chinese adults (43 women and 45 men) could identify emotions conveyed through face, prosody and semantics as three independent channels. Experiments 2 and 3 further explored gender differences during multisensory integration of emotion through a cross-channel (prosody-semantics) and a cross-modal (face-prosody-semantics) Stroop task, respectively, in which 78 participants (41 women and 37 men) were asked to selectively attend to one of the two or three communication channels. Results The integration of accuracy and reaction time data indicated that paralinguistic cues (i.e., face and prosody) of emotions were consistently more salient than linguistic ones (i.e., semantics) throughout the study. Additionally, women demonstrated advantages in processing all three types of emotional signals in the unisensory task, but only preserved their strengths in paralinguistic processing and showed greater Stroop effects of nonverbal cues on verbal ones during multisensory perception. Conclusions These findings demonstrate clear gender differences in verbal and nonverbal emotion perception that are modulated by sensory channels, which have important theoretical and practical implications. Supplemental Material https://doi.org/10.23641/asha.16435599.

Keep it simple: Identification of basic versus complex emotions in spoken language in individuals with autism spectrum disorder without intellectual disability: A meta-analysis study

Daily functioning involves identifying emotions in spoken language, a fundamental aspect of social interactions. To date, there is inconsistent evidence in the literature on whether individuals with autism spectrum disorder without intellectual disability (ASD-without-ID) experience difficulties in identification of spoken emotions. We conducted a meta-analysis (literature search following the PRISMA guidelines), with 26 data sets (taken from 23 peer-reviewed journal articles) comparing individuals with ASD-without-ID (N = 614) and typically-developed (TD) controls (N = 640), from nine countries and in seven languages (published until February 2020). In our analyses there was no sufficient evidence to suggest that individuals with HF-ASD differ from matched controls in the identification of simple prosodic emotions (e.g., sadness, happiness). However, individuals with ASD-without-ID were found to perform significantly worse than controls in identification of complex prosodic emotions (e.g., envy and boredom). The level of the semantic content of the stimuli presented (e.g., sentences vs. strings of digits) was not found to have an impact on the results. In conclusion, the difference in findings between simple and complex emotions calls for a new-look on emotion processing in ASD-without-ID. Intervention programs may rely on the intact abilities of individuals with ASD-without-ID to process simple emotions and target improved performance with complex emotions. LAY SUMMARY: Individuals with autism spectrum disorder without intellectual disability (ASD-without-ID) do not differ from matched controls in the identification of simple prosodic emotions (e.g., sadness, happiness). However, they were found to perform significantly worse than controls in the identification of complex prosodic emotions (e.g., envy, boredom). This was found in a meta-analysis of 26 data sets with 1254 participants from nine countries and in seven languages. Intervention programs may rely on the intact abilities of individuals with ASD-without-ID to process simple emotions.

Weighting of Prosodic and Lexical-Semantic Cues for Emotion Identification in Spectrally Degraded Speech and With Cochlear Implants

OBJECTIVES

Normally-hearing (NH) listeners rely more on prosodic cues than on lexical-semantic cues for emotion perception in speech. In everyday spoken communication, the ability to decipher conflicting information between prosodic and lexical-semantic cues to emotion can be important: for example, in identifying sarcasm or irony. Speech degradation in cochlear implants (CIs) can be sufficiently overcome to identify lexical-semantic cues, but the distortion of voice pitch cues makes it particularly challenging to hear prosody with CIs. The purpose of this study was to examine changes in relative reliance on prosodic and lexical-semantic cues in NH adults listening to spectrally degraded speech and adult CI users. We hypothesized that, compared with NH counterparts, CI users would show increased reliance on lexical-semantic cues and reduced reliance on prosodic cues for emotion perception. We predicted that NH listeners would show a similar pattern when listening to CI-simulated versions of emotional speech.

DESIGN

Sixteen NH adults and 8 postlingually deafened adult CI users participated in the study. Sentences were created to convey five lexical-semantic emotions (angry, happy, neutral, sad, and scared), with five sentences expressing each category of emotion. Each of these 25 sentences was then recorded with the 5 (angry, happy, neutral, sad, and scared) prosodic emotions by 2 adult female talkers. The resulting stimulus set included 125 recordings (25 Sentences × 5 Prosodic Emotions) per talker, of which 25 were congruent (consistent lexical-semantic and prosodic cues to emotion) and the remaining 100 were incongruent (conflicting lexical-semantic and prosodic cues to emotion). The recordings were processed to have 3 levels of spectral degradation: full-spectrum, CI-simulated (noise-vocoded) to have 8 channels and 16 channels of spectral information, respectively. Twenty-five recordings (one sentence per lexical-semantic emotion recorded in all five prosodies) were used for a practice run in the full-spectrum condition. The remaining 100 recordings were used as test stimuli. For each talker and condition of spectral degradation, listeners indicated the emotion associated with each recording in a single-interval, five-alternative forced-choice task. The responses were scored as proportion correct, where "correct" responses corresponded to the lexical-semantic emotion. CI users heard only the full-spectrum condition.

RESULTS

The results showed a significant interaction between hearing status (NH, CI) and congruency in identifying the lexical-semantic emotion associated with the stimuli. This interaction was as predicted, that is, CI users showed increased reliance on lexical-semantic cues in the incongruent conditions, while NH listeners showed increased reliance on the prosodic cues in the incongruent conditions. As predicted, NH listeners showed increased reliance on lexical-semantic cues to emotion when the stimuli were spectrally degraded.

CONCLUSIONS

The present study confirmed previous findings of prosodic dominance for emotion perception by NH listeners in the full-spectrum condition. Further, novel findings with CI patients and NH listeners in the CI-simulated conditions showed reduced reliance on prosodic cues and increased reliance on lexical-semantic cues to emotion. These results have implications for CI listeners' ability to perceive conflicts between prosodic and lexical-semantic cues, with repercussions for their identification of sarcasm and humor. Understanding instances of sarcasm or humor can impact a person's ability to develop relationships, follow conversation, understand vocal emotion and intended message of a speaker, following jokes, and everyday communication in general.

Nonliteral language processing across the lifespan

Studies investigating the effects of aging on nonliteral language processing have mainly focused on one sensory modality, for example written vignettes. In the current study, we used a video-based task to examine the effect of healthy aging on social communication perception using a novel database called RISC (Relation Inference in Social Communication). By means of an online recruitment platform, we asked young, middle-aged, and older adults between the ages of 18 and 76 (N = 100) to evaluate videos of actors using different forms of literal and nonliteral language, such as sarcasm or teasing. The participants' task was to infer the speakers' belief and the speakers' intention. Older participants demonstrated lower accuracy in discriminating nonliteral from literal interactions compared to younger and middle-aged groups. When evaluating speaker intentions, older adults judged sarcasm as friendlier compared to literal negative utterances. We also found that the older the participant, the more difficulty they have identifying teasing as insincere. Our results expand on age-related similarities and differences in evaluating speaker intentions and demonstrate the practicality of the RISC database for studying nonliteral language across the lifespan.

Focusing on Emotional and Social Intelligence Stimulation of People With Dementia by Playing a Serious Game—Proof of Concept Study

Context: Dementia is one of the top five chronic diseases, which has an overwhelming impact on patients' life quality, family, and caregivers. Currently, research relating to people with dementia (PwD) focuses on the deterioration of cognitive abilities. A more innovative approach, and the one taken by this paper, is focusing on methods to maintain and improve functionality, communication and quality of life of PwD by building on remaining capacities in the yet unexplored domain of emotional and social intelligence (ESI). The use of serious games for PwD (SG4D) aimed at building social and emotional capacity is a budding field of research.

Objectives: Proof of concept that the, low cost, easy to deploy SG4D, called “My Brain Works” (MBW), co-designed with PwD, enhances ESI, based on the Bar-On ESI model.

Methods: 27 PwD, clients at MELABEV dementia day center, participated in a mixed methods 12 weeks pilot, proof of concept study using a tablet SG4D co-designed with PwD. Quantitative performance data was collected automatically by the tablet during game sessions. In this paper we focus on the analysis of the qualitative and quantitative data related to ESI, observed by 10 different researchers, during each game session.

Results: Quantitative data revealed: both the PwD with high and low MoCA scores had similar average ESI scores. Qualitative analysis revealed that the PwD demonstrated 9 sub-components of the Bar-On ESI Model.

Conclusion: While there is no drug to stop cognitive decline associated with dementia, interventions related to ESI, on the other hand, may improve functioning and quality of life. Despite declines in cognitive abilities, our study shows that a tablet based SG4D can stimulate their ESI and evoke responses in self-awareness, empathy, social and communication capacities. Using SG4D to exercise and maintain social skills is an area that may be promising in the future and may help counter the negative effects of social isolation and loneliness. Such games, while not focusing on cognitive improvement, may also impact on cognitive functioning and help bridge the gap between caregiver and PwD. More research is needed with larger sample sizes.

Challenges and opportunities for telehealth assessment during COVID-19: iT-RES, adapting a remote version of the test for rating emotions in speech

OBJECTIVE

COVID-19 social isolation restrictions have accelerated the need to adapt clinical assessment tools to telemedicine. Remote adaptations are of special importance for populations at risk, e.g. older adults and individuals with chronic medical comorbidities. In response to this urgent clinical and scientific need, we describe a remote adaptation of the T-RES (Oron et al. 2020; IJA), designed to assess the complex processing of spoken emotions, based on identification and integration of the semantics and prosody of spoken sentences.

DESIGN

We present iT-RES, an online version of the speech-perception assessment tool, detailing the challenges considered and solution chosen when designing the telehealth tool. We show a preliminary validation of performance against the original lab-based T-RES.

STUDY SAMPLE

A between-participants design, within two groups of 78 young adults (T-RES, n = 39; iT-RES, n = 39).

RESULTS

i-TRES performance closely followed that of T-RES, with no group differences found in the main trends, identification of emotions, selective attention, and integration.

CONCLUSIONS

The design of iT-RES mapped the main challenges for remote auditory assessments, and solutions taken to address them. We hope that this will encourage further efforts for telehealth adaptations of clinical services, to meet the needs of special populations and avoid halting scientific research.

Processing of Emotions in Speech in Forensic Patients With Schizophrenia: Impairments in Identification, Selective Attention, and Integration of Speech Channels

Individuals with schizophrenia show deficits in recognition of emotions which may increase the risk of violence. This study explored how forensic patients with schizophrenia process spoken emotion by: (a) identifying emotions expressed in prosodic and semantic content separately, (b) selectively attending to one speech channel while ignoring the other, and (c) integrating the prosodic and the semantic channels, compared to non-clinical controls. Twenty-one forensic patients with schizophrenia and 21 matched controls listened to sentences conveying four emotions (anger, happiness, sadness, and neutrality) presented in semantic or prosodic channels, in different combinations. They were asked to rate how much they agreed that the sentences conveyed a predefined emotion, focusing on one channel or on the sentence as a whole. Forensic patients with schizophrenia performed with intact identification and integration of spoken emotions, but their ratings indicated reduced discrimination, larger failures of selective attention, and under-ratings of negative emotions, compared to controls. This finding doesn't support previous reports of an inclination to interpret social situations in a negative way among individuals with schizophrenia. Finally, current results may guide rehabilitation approaches matched to the pattern of auditory emotional processing presented by forensic patients with schizophrenia, improving social interactions and quality of life.