On the recognition of emotional vocal expressions: motivations for a holistic approach

Analysis on speech signal features of manic patients

Given the lack of effective biological markers for early diagnosis of bipolar mania, and the tendency for voice fluctuation during transition between mood states, this study aimed to investigate the speech features of manic patients to identify a potential set of biomarkers for diagnosis of bipolar mania. 30 manic patients and 30 healthy controls were recruited and their corresponding speech features were collected during natural dialogue using the Automatic Voice Collecting System. Bech-Rafaelsdn Mania Rating Scale (BRMS) and Clinical impression rating scale (CGI) were used to assess illness. The speech features were compared between two groups: mood group (mania vs remission) and bipolar group (manic patients vs healthy individuals). We found that the characteristic speech signals differed between mood groups and bipolar groups. The fourth formant (F4) and Linear Prediction Coefficient (LPC) (P < .05) were significantly differed when patients transmitted from manic to remission state. The first formant (F1), the second formant (F2), and LPC (P < .05) also played key roles in distinguishing between patients and healthy individuals. In addition, there was a significantly correlation between LPC and BRMS, indicating that LPC may play an important role in diagnosis of bipolar mania. In this study we traced speech features of bipolar mania during natural dialogue (conversation), which is an accessible approach in clinic practice. Such specific indicators may respectively serve as promising biomarkers for benefiting the diagnosis and clinical therapeutic evaluation of bipolar mania.

A psychometric investigation of "macroscopic" speech measures for clinical and psychological science

The analysis of vocal expression is a critical endeavor for psychological and clinical sciences and is an increasingly popular application for computer-human interfaces. Despite this, and despite advances in the efficiency, affordability, and sophistication of vocal analytic technologies, there is considerable variability across studies regarding what aspects of vocal expression are studied. Vocal signals can be quantified in a myriad of ways, and their underlying structure, at least with respect to "macroscopic" measures from extended speech, is presently unclear. To address this issue, we evaluated the psychometric properties-notably, the structural and construct validity-of a systematically defined set of global vocal features. Our analytic strategy focused on (a) identifying redundant variables among this set, (b) employing principal components analysis (PCA) to identify nonoverlapping domains of vocal expression, (c) examining the degrees to which the vocal variables are modulated as a function of changes in speech task, and (d) evaluating the relationship between the vocal variables and cognitive (i.e., verbal fluency) and clinical (i.e., depression, anxiety, and hostility) variables. Spontaneous speech samples from 11 independent studies of young adults (>60 s in length), employing one of three different speaking tasks, were examined (N = 1,350). Confounding variables (i.e., sex, ethnicity) were statistically controlled for. The PCA identified six distinct domains of vocal expression. Collectively, vocal expression (defined in terms of these domains) was modulated as a function of speech task and was related to the cognitive and clinical variables. These findings provide empirically grounded implications for the study of vocal expression in psychological and clinical sciences.

What is the Value of Embedding Artificial Emotional Prosody in Human-Computer Interactions? Implications for Theory and Design in Psychological Science

In computerized technology, artificial speech is becoming increasingly important, and is already used in ATMs, online gaming and healthcare contexts. However, today’s artificial speech typically sounds monotonous, a main reason for this being the lack of meaningful prosody. One particularly important function of prosody is to convey different emotions. This is because successful encoding and decoding of emotions is vital for effective social cognition, which is increasingly recognized in human–computer interaction contexts. Current attempts to artificially synthesize emotional prosody are much improved relative to early attempts, but there remains much work to be done due to methodological problems, lack of agreed acoustic correlates, and lack of theoretical grounding. If the addition of synthetic emotional prosody is not of sufficient quality, it may risk alienating users instead of enhancing their experience. So the value of embedding emotion cues in artificial speech may ultimately depend on the quality of the synthetic emotional prosody. However, early evidence on reactions to synthesized non-verbal cues in the facial modality bodes well. Attempts to implement the recognition of emotional prosody into artificial applications and interfaces have perhaps been met with greater success, but the ultimate test of synthetic emotional prosody will be to critically compare how people react to synthetic emotional prosody vs. natural emotional prosody, at the behavioral, socio-cognitive and neural levels.