Acoustic speech features in social comparison: how stress impacts the way you sound

Acoustic and prosodic speech features reflect physiological stress but not isolated negative affect: a multi-paradigm study on psychosocial stressors

Heterogeneity in speech under stress has been a recurring issue in stress research, potentially due to varied stress induction paradigms. This study investigated speech features in semi-guided speech following two distinct psychosocial stress paradigms (Cyberball and MIST) and their respective control conditions. Only negative affect increased during Cyberball, while self-reported stress, skin conductance response rate, and negative affect increased during MIST. Fundamental frequency (F0), speech rate, and jitter significantly changed during MIST, but not Cyberball; HNR and shimmer showed no expected changes. The results indicate that observed speech features are robust in semi-guided speech and sensitive to stressors eliciting additional physiological stress responses, not solely decreases in negative affect. These differences between stressors may explain literature heterogeneity. Our findings support the potential of speech as a stress level biomarker, especially when stress elicits physiological reactions, similar to other biomarkers. This highlights its promise as a tool for measuring stress in everyday settings, considering its affordability, non-intrusiveness, and ease of collection. Future research should test these results' robustness and specificity in naturalistic settings, such as freely spoken speech and noisy environments while exploring and validating a broader range of informative speech features in the context of stress.

Severity Classification Using Dynamic Time Warping-Based Voice Biomarkers for Patients With COVID-19: Feasibility Cross-Sectional Study

Background

In Japan, individuals with mild COVID-19 illness previously required to be monitored in designated areas and were hospitalized only if their condition worsened to moderate illness or worse. Daily monitoring using a pulse oximeter was a crucial indicator for hospitalization. However, a drastic increase in the number of patients resulted in a shortage of pulse oximeters for monitoring. Therefore, an alternative and cost-effective method for monitoring patients with mild illness was required. Previous studies have shown that voice biomarkers for Parkinson disease or Alzheimer disease are useful for classifying or monitoring symptoms; thus, we tried to adapt voice biomarkers for classifying the severity of COVID-19 using a dynamic time warping (DTW) algorithm where voice wavelets can be treated as 2D features; the differences between wavelet features are calculated as scores.

Objective

This feasibility study aimed to test whether DTW-based indices can generate voice biomarkers for a binary classification model using COVID-19 patients' voices to distinguish moderate illness from mild illness at a significant level.

Methods

We conducted a cross-sectional study using voice samples of COVID-19 patients. Three kinds of long vowels were processed into 10-cycle waveforms with standardized power and time axes. The DTW-based indices were generated by all pairs of waveforms and tested with the Mann-Whitney U test (α<.01) and verified with a linear discrimination analysis and confusion matrix to determine which indices were better for binary classification of disease severity. A binary classification model was generated based on a generalized linear model (GLM) using the most promising indices as predictors. The receiver operating characteristic curve/area under the curve (ROC/AUC) validated the model performance, and the confusion matrix calculated the model accuracy.

Results

Participants in this study (n=295) were infected with COVID-19 between June 2021 and March 2022, were aged 20 years or older, and recuperated in Kanagawa prefecture. Voice samples (n=110) were selected from the participants' attribution matrix based on age group, sex, time of infection, and whether they had mild illness (n=61) or moderate illness (n=49). The DTW-based variance indices were found to be significant (P<.001, except for 1 of 6 indices), with a balanced accuracy in the range between 79% and 88.6% for the /a/, /e/, and /u/ vowel sounds. The GLM achieved a high balance accuracy of 86.3% (for /a/), 80.2% (for /e/), and 88% (for /u/) and ROC/AUC of 94.8% (95% CI 90.6%-94.8%) for /a/, 86.5% (95% CI 79.8%-86.5%) for /e/, and 95.6% (95% CI 92.1%-95.6%) for /u/.

Conclusions

The proposed model can be a voice biomarker for an alternative and cost-effective method of monitoring the progress of COVID-19 patients in care.

Effects of Cognitive Load on Vocal Fold Vibratory Patterns in Bilingual Speakers of Low and High German

OBJECTIVES

Increased cognitive load has been observed to correlate with decreased vocal fold perturbation, reduced additive noise, increased periodicity, and a higher rate of vocal fold vibration. The aim of this study was to explore whether vocal fold vibratory patterns can serve as indicators of increased cognitive load in nonbalanced bilingual speakers when they use their weaker language.

STUDY DESIGN

This is a comparative experimental study with a within-speaker design.

METHODS

We recorded a total of 95 bilingual speakers of Low German (LG), which is an endangered language spoken in Northern Germany, and Standard High German (HG). Participants completed four tasks in both languages: engaging in free narration, describing a picture story, giving directions, and reading a narrative passage. For the last three tasks, the difficulty levels were varied. Measurements included jitter, shimmer, harmonics-to-noise ratio (HNR), cepstral peak prominence (CPP), the proportion of creak, pitch level, and pitch span. Changes in voice characteristics were examined both in terms of the participants' age and their language dominance. For the latter, we calculated a dominance score derived from age of acquisition, frequency of use, and self-perceived linguistic competence in the two languages.

RESULTS

Younger speakers showed a higher dominance of HG over LG, which decreased with age. Younger and more HG-dominant speakers exhibited lower jitter and shimmer, along with a higher HNR and a lower creak proportion in LG compared to HG. CPP and pitch level were higher in LG but showed little variation with age or language dominance. No clear effects on pitch span were observed. Overall, age was a slightly more reliable predictor than language dominance. Acoustic differences in voice quality were about equally detectable across the different speech tasks while varying difficulty levels had minimal impact.

CONCLUSIONS

The variation in vocal fold vibratory patterns suggests that younger and more HG-dominant speakers experienced greater cognitive load when speaking LG. Given that increased cognitive load may negatively impact language usage, voice analysis opens up new possibilities for evaluating the future prospects of endangered languages.

Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech

This article is devoted to solving the problem of converting sign language into a consistent text with intonation markup for subsequent voice synthesis of sign phrases by speech with intonation. The paper proposes an improved method of continuous recognition of sign language, the results of which are transmitted to a natural language processor based on analyzers of morphology, syntax, and semantics of the Kazakh language, including morphological inflection and the construction of an intonation model of simple sentences. This approach has significant practical and social significance, as it can lead to the development of technologies that will help people with disabilities to communicate and improve their quality of life. As a result of the cross-validation of the model, we obtained an average test accuracy of 0.97 and an average val_accuracy of 0.90 for model evaluation. We also identified 20 sentence structures of the Kazakh language with their intonational model.

Kindness Isn't Just about Being Nice: The Value Proposition of Kindness as Viewed through the Lens of Incivility in the Healthcare Workplace

The healthcare workplace is a high-stress environment. All stakeholders, including patients and providers, display evidence of that stress. High stress has several effects. Even acutely, stress can negatively affect cognitive function, worsening diagnostic acumen, decision-making, and problem-solving. It decreases helpfulness. As stress increases, it can progress to burnout and more severe mental health consequences, including depression and suicide. One of the consequences (and causes) of stress is incivility. Both patients and staff can manifest these unkind behaviors, which in turn have been shown to cause medical errors. The human cost of errors is enormous, reflected in thousands of lives impacted every year. The economic cost is also enormous, costing at least several billion dollars annually. The warrant for promoting kindness, therefore, is enormous. Kindness creates positive interpersonal connections, which, in turn, buffers stress and fosters resilience. Kindness, therefore, is not just a nice thing to do: it is critically important in the workplace. Ways to promote kindness, including leadership modeling positive behaviors as well as the deterrence of negative behaviors, are essential. A new approach using kindness media is described. It uplifts patients and staff, decreases irritation and stress, and increases happiness, calmness, and feeling connected to others.

Effects of acute psychosocial stress on source level EEG power and functional connectivity measures

The usage of EEG to uncover the influence of psychosocial stressors (PSSs) on neural activity has gained significant attention throughout recent years, but the results are often troubled by confounding stressor types. To investigate the effect of PSSs alone on neural activity, we employed a paradigm where participants are exposed to negative peer comparison as PSS, while other possible stressors are kept constant, and compared this with a condition where participants received neutral feedback. We analyzed commonly used sensor level EEG indices (frontal theta, alpha, and beta power) and further investigated whether source level power and functional connectivity (i.e., the temporal dependence between spatially seperated brain regions) measures, which have to our knowledge not yet been used, are more sensitive to PSSs than sensor level-derived EEG measures. Our results show that on sensor level, no significant frontal power changes are present (all p's > 0.16), indicating that sensor level frontal power measures are not sensitive enough to be affected by only PSSs. On source level, we find increased alpha power (indicative of decreased cortical activity) in the left- and right precuneus and right posterior cingulate cortex (all p's < 0.03) and increased functional connectivity between the left- and right precuneus (p < 0.001), indicating that acute, trial based PSSs lead to decreased precuneus/PCC activity, and possibly indicates a temporary disruption in the self-referential neural processes of an individual.

Speech as a Promising Biosignal in Precision Psychiatry

Health research and health care alike are presently based on infrequent assessments that provide an incomplete picture of clinical functioning. Consequently, opportunities to identify and prevent health events before they occur are missed. New health technologies are addressing these critical issues by enabling the continual monitoring of health-related processes using speech. These technologies are a great match for the healthcare environment because they make high-frequency assessments non-invasive and highly scalable. Indeed, existing tools can now extract a wide variety of health-relevant biosignals from smartphones by analyzing a person's voice and speech. These biosignals are linked to health-relevant biological pathways and have shown promise in detecting several disorders, including depression and schizophrenia. However, more research is needed to identify the speech signals that matter most, validate these signals against ground-truth outcomes, and translate these data into biomarkers and just-in-time adaptive interventions. We discuss these issues herein by describing how assessing everyday psychological stress through speech can help both researchers and health care providers monitor the impact that stress has on a wide variety of mental and physical health outcomes, such as self-harm, suicide, substance abuse, depression, and disease recurrence. If done appropriately and securely, speech is a novel digital biosignal that could play a key role in predicting high-priority clinical outcomes and delivering tailored interventions that help people when they need it most.