1
|
Yawer BA, Liss J, Berisha V. Reliability and validity of a widely-available AI tool for assessment of stress based on speech. Sci Rep 2023; 13:20224. [PMID: 37980431 PMCID: PMC10657363 DOI: 10.1038/s41598-023-47153-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Accepted: 11/09/2023] [Indexed: 11/20/2023] Open
Abstract
Cigna's online stress management toolkit includes an AI-based tool that purports to evaluate a person's psychological stress level based on analysis of their speech, the Cigna StressWaves Test (CSWT). In this study, we evaluate the claim that the CSWT is a "clinical grade" tool via an independent validation. The results suggest that the CSWT is not repeatable and has poor convergent validity; the public availability of the CSWT despite insufficient validation data highlights concerns regarding premature deployment of digital health tools for stress and anxiety management.
Collapse
|
2
|
Virk JS, Singh M, Singh M, Panjwani U, Ray K. A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents. SENSORS (BASEL, SWITZERLAND) 2023; 23:4129. [PMID: 37112470 PMCID: PMC10144633 DOI: 10.3390/s23084129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Revised: 04/16/2023] [Accepted: 04/18/2023] [Indexed: 06/19/2023]
Abstract
Sleep-deprived fatigued person is likely to commit more errors that may even prove to be fatal. Thus, it is necessary to recognize this fatigue. The novelty of the proposed research work for the detection of this fatigue is that it is nonintrusive and based on multimodal feature fusion. In the proposed methodology, fatigue is detected by obtaining features from four domains: visual images, thermal images, keystroke dynamics, and voice features. In the proposed methodology, the samples of a volunteer (subject) are obtained from all four domains for feature extraction, and empirical weights are assigned to the four different domains. Young, healthy volunteers (n = 60) between the age group of 20 to 30 years participated in the experimental study. Further, they abstained from the consumption of alcohol, caffeine, or other drugs impacting their sleep pattern during the study. Through this multimodal technique, appropriate weights are given to the features obtained from the four domains. The results are compared with k-nearest neighbors (kNN), support vector machines (SVM), random tree, random forest, and multilayer perceptron classifiers. The proposed nonintrusive technique has obtained an average detection accuracy of 93.33% in 3-fold cross-validation.
Collapse
Affiliation(s)
- Jitender Singh Virk
- EIE Department, Thapar Institute of Engineering and Technology, Patiala 147001, India
| | - Mandeep Singh
- EIE Department, Thapar Institute of Engineering and Technology, Patiala 147001, India
| | - Mandeep Singh
- EIE Department, Thapar Institute of Engineering and Technology, Patiala 147001, India
| | - Usha Panjwani
- DIPAS, Defence Research and Development Organisation, Delhi 110054, India
| | - Koushik Ray
- DIPAS, Defence Research and Development Organisation, Delhi 110054, India
| |
Collapse
|
3
|
An Efficient AP-ANN-Based Multimethod Fusion Model to Detect Stress through EEG Signal Analysis. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:7672297. [PMID: 36544857 PMCID: PMC9763020 DOI: 10.1155/2022/7672297] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/25/2022] [Revised: 09/30/2022] [Accepted: 10/31/2022] [Indexed: 12/14/2022]
Abstract
Stress is a universal emotion that every human experiences daily. Psychologists say stress may lead to heart attack, depression, hypertension, strokes, or even sudden death. Many technical explorations like stress detection through facial expression, speech, text, physical behaviors, etc., were explored, but no consensus has been reached on the best method. The advancement in biomedical engineering yielded a rapid development of electroencephalogram (EEG) signal analysis that has inspired the idea of a multimethod fusion approach for the first time which employs multiple techniques such as discrete wavelet transform (DWT) for de-noising, adaptive synthetic sampling (ADASYN) for class balancing, and affinity propagation (AP) as a stratified sampling model along with the artificial neural network (ANN) as the classifier model for human emotion classification. From the EEG recordings of the DEAP dataset, the artifacts are removed, the signal is decomposed using a DWT, and features are extracted and fused to form the feature vector. As the dataset is high-dimensional, feature selection is done and ADASYN is used to address the imbalance of classes resulting in large-scale data. The innovative idea of the proposed system is to perform sampling using affinity propagation as a stratified sampling-based clustering algorithm as it determines the number of representative samples automatically which makes it superior to the K-Means, K-Medoid, that requires the K-value. Those samples are used as inputs to various classification models, the comparison of the AP-ANN, AP-SVM, and AP-RF is done, and their most important five performance metrics such as accuracy, precision, recall, F1-score, and specificity were compared. From our experiment, the AP-ANN model provides better accuracy of 86.8% and greater precision of 85.7%, a higher F1 score of 84.9%, a recall rate of 84.1%, and a specificity value of 89.2% which altogether provides better results than the other existing algorithms.
Collapse
|
4
|
Schultz BG, Vogel AP. A Tutorial Review on Clinical Acoustic Markers in Speech Science. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2022; 65:3239-3263. [PMID: 36044888 DOI: 10.1044/2022_jslhr-21-00647] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
PURPOSE The human voice changes with the progression of neurological disease and the onset of diseases that affect articulators, often decreasing the effectiveness of communication. These changes can be objectively measured using signal processing techniques that extract acoustic features. When measuring acoustic features, there are often several steps and assumptions that might be known to experts in acoustics and phonetics, but are less transparent for other disciplines (e.g., clinical medicine, speech pathology, engineering, and data science). This tutorial describes these signal processing techniques, explicitly outlines the underlying steps for accurate measurement, and discusses the implications of clinical acoustic markers. CONCLUSIONS We establish a vocabulary using straightforward terms, provide visualizations to achieve common ground, and guide understanding for those outside the domains of acoustics and auditory signal processing. Where possible, we highlight the best practices for measuring clinical acoustic markers and suggest resources for obtaining and further understanding these measures.
Collapse
Affiliation(s)
- Benjamin Glenn Schultz
- Centre for Neuroscience of Speech, The University of Melbourne, Victoria, Australia
- Department of Audiology and Speech Pathology, The University of Melbourne, Victoria, Australia
| | - Adam P Vogel
- Centre for Neuroscience of Speech, The University of Melbourne, Victoria, Australia
- Department of Audiology and Speech Pathology, The University of Melbourne, Victoria, Australia
- Redenlab, Melbourne, Victoria, Australia
| |
Collapse
|
5
|
Comparison of Three Prototypes of PPG Sensors for Continual Real-Time Measurement in Weak Magnetic Field. SENSORS 2022; 22:s22103769. [PMID: 35632179 PMCID: PMC9144130 DOI: 10.3390/s22103769] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Revised: 05/11/2022] [Accepted: 05/14/2022] [Indexed: 12/04/2022]
Abstract
This paper is focused on investigation of three developed prototypes of sensors based on the photoplethysmography (PPG) principle for continual measurement of the PPG signal in the magnetic field environment with the inherent radiofrequency and electromagnetic disturbance. The tested prototypes differ in the used optical part of the PPG sensor and their working mode, control unit, power supply, and applied Bluetooth (BT) communication methods. The main aim of the current work was motivated by finding suitable and universal parameter settings for PPG signal real-time recording in different working mode conditions. Comparative measurements in laboratory conditions by certified commercial pulse oximeter and blood pressure monitor (BPM) devices show good stability and proper accuracy of finally determined heart rate values. The supplementary investigation certifies the necessity of the placement of the pressure cuff of the BPM device on the opposite arm than the tested PPG sensor. Measurement experiments inside the scanning area of the running weak field magnetic resonance scanner verify proper function and practical usability of sensed PPG signals for further processing and analysis in all three prototype cases. Additional testing shows that the BT transmission in the scanning area has no visible influence on the quality of the finally obtained scanner images.
Collapse
|
6
|
Suthar K, Yousefi Zowj F, Speights Atkins M, He QP. Feature engineering and machine learning for computer-assisted screening of children with speech disorders. PLOS DIGITAL HEALTH 2022; 1:e0000041. [PMID: 36812555 PMCID: PMC9931328 DOI: 10.1371/journal.pdig.0000041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/05/2022] [Accepted: 04/08/2022] [Indexed: 06/18/2023]
Abstract
Auditory perceptual analysis (APA) is the main method for clinical assessment of speech-language deficits, which are one of the most prevalent childhood disabilities. However, results from APA are susceptible to intra- and inter-rater variabilities. There are also other limitations of manual or hand transcription-based speech disorder diagnostic methods. There is increased interest in developing automated methods that quantify speech patterns for diagnosing speech disorders in children to address these limitations. Landmark (LM) analysis is an approach that characterizes acoustic events occurring due to sufficiently precise articulatory movements. This work investigates the utilization of LMs for automatic speech disorder detection in children. Besides the LM-based features that have been proposed in existing research, we propose a set of novel knowledge-based features that have not been proposed before. A systematic study and comparison of different linear and nonlinear machine learning classification techniques based on the raw features and the proposed features is conducted to assess the effectiveness of the novel features in classifying speech disorder patients from normal speakers.
Collapse
Affiliation(s)
- Kerul Suthar
- Department of Chemical Engineering, Auburn University, Auburn, Alabama, United States of America
| | - Farnaz Yousefi Zowj
- Department of Chemical Engineering, Auburn University, Auburn, Alabama, United States of America
| | - Marisha Speights Atkins
- Communication Sciences & Disorders, Northwestern University, Evanston, Illinois, United States of America
| | - Q. Peter He
- Department of Chemical Engineering, Auburn University, Auburn, Alabama, United States of America
| |
Collapse
|
7
|
Mann CC, Karsten AM. Assessment and Treatment of Prosody Behavior in Individuals with Level 1 Autism: A Review and Call for Research. Anal Verbal Behav 2021; 37:171-193. [PMID: 35141105 PMCID: PMC8789987 DOI: 10.1007/s40616-021-00154-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/24/2021] [Indexed: 01/19/2023] Open
Abstract
Differences in prosody behavior between individuals with autism spectrum disorder (ASD) and their typically developing peers have been considered a central feature of ASD since the earliest clinical descriptions of the disorder (e.g., Kanner, 1943/1973). Prosody includes pitch and volume among other dimensions of vocal-verbal behavior that discriminate responses of the listener; thus, people with ASD whose prosody has confusing or off-putting effects may have fewer social opportunities at work, at school, or in the community. The purpose of this review is to examine the state of the literature intervening on prosody with individuals with ASD and to provide recommendations for researchers who are interested in contributing to the scientific understanding of prosody.
Collapse
Affiliation(s)
- Charlotte C. Mann
- Department of Psychology, Western New England University, Springfield, MA, USA
- Department of Counseling and Applied Behavioral Studies, University of Saint Joseph, 1678 Asylum Ave, West Hartford, CT 06117 USA
| | - Amanda M. Karsten
- Department of Psychology, Western New England University, Springfield, MA, USA
| |
Collapse
|
8
|
Sahoo S, Dandapat S. Analyzing the vocal tract characteristics for out-of-breath speech. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2021; 150:1524. [PMID: 34470262 DOI: 10.1121/10.0005945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Accepted: 08/03/2021] [Indexed: 06/13/2023]
Abstract
In this work, vocal tract characteristic changes under the out-of-breath condition are explored. Speaking under the influence of physical exercise is called out-of-breath speech. The change in breathing pattern results in perceptual changes in the produced sound. For vocal tract, the first four formants show a lowering in their average frequency. The bandwidths BF1 and BF2 widen, whereas the other two get narrowed. The change in bandwidth is small for the last three. For a speaker, the change in frequency and bandwidth may not be uniform across formants. Subband analysis is carried out around formants for comparing the variation of the vocal tract with the source. A vocal tract adaptive empirical wavelet transform is used for extracting formant specific subbands from speech and source. The support vector machine performs the subband-based binary classification between the normal and out-of-breath speech. For all speakers, it shows an F1-score improvement of 4% over speech subbands. Similarly, a performance improvement of 5% can be seen for both male and female speakers. Furthermore, the misclassification amount is less for source compared to speech. These results suggest that physical exercise influences the source more than the vocal tract.
Collapse
Affiliation(s)
- Sibasis Sahoo
- Department of Electronics and Electrical Engineering, Indian Institute of Technology Guwahati, Guwahati 781039, India
| | - Samarendra Dandapat
- Department of Electronics and Electrical Engineering, Indian Institute of Technology Guwahati, Guwahati 781039, India
| |
Collapse
|
9
|
Pisanski K, Sorokowski P. Human Stress Detection: Cortisol Levels in Stressed Speakers Predict Voice-Based Judgments of Stress. Perception 2020; 50:80-87. [PMID: 33302780 DOI: 10.1177/0301006620978378] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Despite recent evidence of a positive relationship between cortisol levels and voice pitch in stressed speakers, the extent to which human listeners can reliably judge stress from the voice remains unknown. Here, we tested whether voice-based judgments of stress co-vary with the free cortisol levels and vocal parameters of speakers recorded in a real-life stressful situation (oral examination) and baseline (2 weeks prior). Hormone and acoustic analyses indicated elevated salivary cortisol levels and corresponding changes in voice pitch, vocal tract resonances (formants), and speed of speech during stress. In turn, listeners' stress ratings correlated significantly with speakers' cortisol levels. Higher pitched voices were consistently perceived as more stressed; however, the influence of formant frequencies, vocal perturbation and noise parameters on stress ratings varied across contexts, suggesting that listeners utilize different strategies when assessing calm versus stressed speech. These results indicate that nonverbal vocal cues can convey honest information about a speaker's underlying physiological level of stress that listeners can, to some extent, detect and utilize, while underscoring the necessity to control for individual differences in the biological stress response.
Collapse
Affiliation(s)
- Katarzyna Pisanski
- Institute of Psychology, University of Wroclaw, Poland; University of Lyon/Saint-Etienne, France.,Institute of Psychology, University of Wroclaw, Poland
| | | |
Collapse
|
10
|
Prasetio BH, Tamura H, Tanno K. Deep time-delay Markov network for prediction and modeling the stress and emotions state transition. Sci Rep 2020; 10:18071. [PMID: 33093631 PMCID: PMC7581816 DOI: 10.1038/s41598-020-75155-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2020] [Accepted: 10/12/2020] [Indexed: 11/09/2022] Open
Abstract
To recognize stress and emotion, most of the existing methods only observe and analyze speech patterns from present-time features. However, an emotion (especially for stress) can change because it was triggered by an event while speaking. To address this issue, we propose a novel method for predicting stress and emotions by analyzing prior emotional states. We named this method the deep time-delay Markov network (DTMN). Structurally, the proposed DTMN contains a hidden Markov model (HMM) and a time-delay neural network (TDNN). We evaluated the effectiveness of the proposed DTMN by comparing it with several state transition methods in predicting an emotional state from time-series (sequences) speech data of the SUSAS dataset. The experimental results show that the proposed DTMN can accurately predict present emotional states by outperforming the baseline systems in terms of the prediction error rate (PER). We then modeled the emotional state transition using a finite Markov chain based on the prediction result. We also conducted an ablation experiment to observe the effect of different HMM values and TDNN parameters on the prediction result and the computational training time of the proposed DTMN.
Collapse
Affiliation(s)
- Barlian Henryranu Prasetio
- Interdisciplinary Graduate School of Agriculture and Engineering, University of Miyazaki, Miyazaki, 889-2192, Japan.
| | - Hiroki Tamura
- Faculty of Engineering, University of Miyazaki, Miyazaki, 889-2192, Japan
| | - Koichi Tanno
- Faculty of Engineering, University of Miyazaki, Miyazaki, 889-2192, Japan
| |
Collapse
|
11
|
Reilly KJ. Vowel and Sibilant Production in Noise: Effects of Noise Frequency and Phonological Similarity. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2020; 63:1002-1017. [PMID: 32293944 PMCID: PMC7242990 DOI: 10.1044/2020_jslhr-19-00345] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/04/2019] [Revised: 11/20/2019] [Accepted: 12/30/2019] [Indexed: 06/11/2023]
Abstract
Purpose This study investigated vowel and sibilant productions in noise to determine whether responses to noise (a) are sensitive to the spectral characteristics of the noise signal and (b) are modulated by the contribution of vowel or sibilant contrasts to word discrimination. Method Vowel and sibilant productions were elicited during serial recall of three-word sequences that were produced in quiet or during exposure to speaker-specific noise signals. These signals either masked a speaker's productions of the sibilants /s/ and /ʃ/ or their productions of the vowels /a/ and /æ/. The contribution of the vowel and sibilant contrasts to word discrimination in a sequence was manipulated by varying the number of times that the target sibilant and vowel pairs occurred in the same word position in each sequence. Results Spectral noise effects were observed for both sibilants and vowels: Responses to noise were larger and/or involved to more acoustic features when the noise signal masked the acoustic characteristics of that phoneme class. Word discrimination effects were limited and consisted of only small increases in vowel duration. Interaction effects between noise and similarity indicated that the phonological similarity of sequences containing both sibilants and/or both vowels influenced articulation in ways not related to speech clarity. Conclusion The findings of this study indicate that sensorimotor control of speech exhibits some sensitivity to noise spectral characteristics. However, productions of sibilants and vowels were not sensitive to their importance in discriminating the words in a sequence. In addition, phonological similarity effects were observed that likely reflected processing demands related to the recall and sequencing of high-similarity words.
Collapse
Affiliation(s)
- Kevin J. Reilly
- Department of Audiology and Speech Pathology, College of Health Professions, University of Tennessee Health Science Center, Memphis
| |
Collapse
|
12
|
Slavich GM, Taylor S, Picard RW. Stress measurement using speech: Recent advancements, validation issues, and ethical and privacy considerations. Stress 2019; 22:408-413. [PMID: 30945584 PMCID: PMC7081839 DOI: 10.1080/10253890.2019.1584180] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/01/2023] Open
Abstract
Life stress is a well-established risk factor for a variety of mental and physical health problems, including anxiety disorders, depression, chronic pain, heart disease, asthma, autoimmune diseases, and neurodegenerative disorders. The purpose of this article is to describe emerging approaches for assessing stress using speech, which we do by reviewing the methodological advantages of these digital health tools, and the validation, ethical, and privacy issues raised by these technologies. As we describe, it is now possible to assess stress via the speech signal using smartphones and smart speakers that employ software programs and artificial intelligence to analyze several features of speech and speech acoustics, including pitch, jitter, energy, rate, and length and number of pauses. Because these digital devices are ubiquitous, we can now assess individuals' stress levels in real time in almost any natural environment in which people speak. These technologies thus have great potential for advancing digital health initiatives that involve continuously monitoring changes in psychosocial functioning and disease risk over time. However, speech-based indices of stress have yet to be well-validated against stress biomarkers (e.g., cortisol, cytokines) that predict disease risk. In addition, acquiring speech samples raises the possibility that conversations intended to be private could one day be made public; moreover, obtaining real-time psychosocial risk information prompts ethical questions regarding how these data should be used for medical, commercial, and personal purposes. Although assessing stress using speech thus has enormous potential, there are critical validation, privacy, and ethical issues that must be addressed.
Collapse
Affiliation(s)
- George M. Slavich
- Cousins Center for Psychoneuroimmunology and Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, CA 90095-7076, USA
| | - Sara Taylor
- Affective Computing Group, MIT Media Lab, Massachusetts Institute of Technology,Cambridge, MA02139, USA
| | - Rosalind W. Picard
- Affective Computing Group, MIT Media Lab, Massachusetts Institute of Technology,Cambridge, MA02139, USA
| |
Collapse
|
13
|
Van Puyvelde M, Neyt X, McGlone F, Pattyn N. Voice Stress Analysis: A New Framework for Voice and Effort in Human Performance. Front Psychol 2018; 9:1994. [PMID: 30515113 PMCID: PMC6255927 DOI: 10.3389/fpsyg.2018.01994] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2018] [Accepted: 09/28/2018] [Indexed: 11/24/2022] Open
Abstract
People rely on speech for communication, both in a personal and professional context, and often under different conditions of physical, cognitive and/or emotional load. Since vocalization is entirely integrated within both our central (CNS) and autonomic nervous system (ANS), a mounting number of studies have examined the relationship between voice output and the impact of stress. In the current paper, we will outline the different stages of voice output, i.e., breathing, phonation and resonance in relation to a neurovisceral integrated perspective on stress and human performance. In reviewing the function of these three stages of voice output, we will give an overview of the voice parameters encountered in studies on voice stress analysis (VSA) and review the impact of the different types of physiological, cognitive and/or emotional load. In the section "Discussion," with regard to physical load, a competition for ventilation processes required to speak and those to meet metabolic demand of exercised muscles is described. With regard to cognitive and emotional load, we will present the "Model for Voice and Effort" (MoVE) that comprises the integration of ongoing top-down and bottom-up activity under different types of load and combined patterns of voice output. In the MoVE, it is proposed that the fundamental frequency (F0) values as well as jitter give insight in bottom-up/arousal activity and the effort a subject is capable to generate but that its range and variance are related to ongoing top-down processes and the amount of control a subject can maintain. Within the MoVE, a key-role is given to the anterior cingulate cortex (ACC) which is known to be involved in both the equilibration between bottom-up arousal and top-down regulation and vocal activity. Moreover, the connectivity between the ACC and the nervus vagus (NV) is underlined as an indication of the importance of respiration. Since respiration is the driving force of both stress and voice production, it is hypothesized to be the missing-link in our understanding of the underlying mechanisms of the dynamic between speech and stress.
Collapse
Affiliation(s)
- Martine Van Puyvelde
- VIPER Research Unit, LIFE Department, Royal Military Academy, Brussels, Belgium
- Brain, Body and Cognition, Experimental and Applied Psychology, Department of Psychological and Educational Sciences, Vrije Universiteit Brussel, Brussels, Belgium
- Clinical and Lifespan Psychology, Department of Psychological and Educational Sciences, Vrije Universiteit Brussel, Brussels, Belgium
| | - Xavier Neyt
- VIPER Research Unit, LIFE Department, Royal Military Academy, Brussels, Belgium
| | - Francis McGlone
- School of Natural Sciences and Psychology, Faculty of Science, Liverpool John Moores University, Liverpool, United Kingdom
| | - Nathalie Pattyn
- VIPER Research Unit, LIFE Department, Royal Military Academy, Brussels, Belgium
- Brain, Body and Cognition, Experimental and Applied Psychology, Department of Psychological and Educational Sciences, Vrije Universiteit Brussel, Brussels, Belgium
- MFYS-BLITS, Department of Human Physiology, Vrije Universiteit Brussel, Brussels, Belgium
| |
Collapse
|
14
|
Young V, Rochon E, Mihailidis A. Exploratory analysis of real personal emergency response call conversations: considerations for personal emergency response spoken dialogue systems. J Neuroeng Rehabil 2016; 13:97. [PMID: 27842598 PMCID: PMC5109662 DOI: 10.1186/s12984-016-0207-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2015] [Accepted: 11/03/2016] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The purpose of this study was to derive data from real, recorded, personal emergency response call conversations to help improve the artificial intelligence and decision making capability of a spoken dialogue system in a smart personal emergency response system. The main study objectives were to: develop a model of personal emergency response; determine categories for the model's features; identify and calculate measures from call conversations (verbal ability, conversational structure, timing); and examine conversational patterns and relationships between measures and model features applicable for improving the system's ability to automatically identify call model categories and predict a target response. METHODS This study was exploratory and used mixed methods. Personal emergency response calls were pre-classified according to call model categories identified qualitatively from response call transcripts. The relationships between six verbal ability measures, three conversational structure measures, two timing measures and three independent factors: caller type, risk level, and speaker type, were examined statistically. RESULTS Emergency medical response services were the preferred response for the majority of medium and high risk calls for both caller types. Older adult callers mainly requested non-emergency medical service responders during medium risk situations. By measuring the number of spoken words-per-minute and turn-length-in-words for the first spoken utterance of a call, older adult and care provider callers could be identified with moderate accuracy. Average call taker response time was calculated using the number-of-speaker-turns and time-in-seconds measures. Care providers and older adults used different conversational strategies when responding to call takers. The words 'ambulance' and 'paramedic' may hold different latent connotations for different callers. CONCLUSIONS The data derived from the real personal emergency response recordings may help a spoken dialogue system classify incoming calls by caller type with moderate probability shortly after the initial caller utterance. Knowing the caller type, the target response for the call may be predicted with some degree of probability and the output dialogue could be tailored to this caller type. The average call taker response time measured from real calls may be used to limit the conversation length in a spoken dialogue system before defaulting to a live call taker.
Collapse
Affiliation(s)
- Victoria Young
- University Health Network - Toronto Rehabilitation Institute, Toronto, ON, Canada.
| | - Elizabeth Rochon
- University Health Network - Toronto Rehabilitation Institute, Toronto, ON, Canada.,Rehabilitation Sciences Institute, Department of Speech-Language Pathology, University of Toronto, Toronto, ON, Canada
| | - Alex Mihailidis
- Rehabilitation Sciences Institute, University of Toronto, Toronto, ON, Canada.,Institute of Biomaterials and Biomedical Engineering, University of Toronto, Toronto, ON, Canada.,University Health Network - Toronto Rehabilitation Institute, Toronto, ON, Canada.,Department of Occupational Science and Occupational Therapy, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
15
|
Heinrich C, Schiel F. The influence of alcoholic intoxication on the short-time energy function of speech. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2014; 135:2942-2951. [PMID: 24815274 DOI: 10.1121/1.4870705] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
This study investigates rhythmic features based on the short-time energy function of speech signals with the aim of finding robust, speaker-independent features that indicate speaker intoxication. Data from the German Alcohol Language Corpus, which comprises read, spontaneous, and command&control speech uttered by 162 speakers of both genders and various age groups when sober and intoxicated, were analyzed. Energy contours are compared directly (Root Mean Squared Error, statistical correlation, or the Euclidean distance in the spectral space of the contour) and by parameterization of the contour using the Discrete Cosine Transform (DCT) and the first and second moments of the lower DCT spectrum. Contours are also analyzed by Principal Components Analysis aiming at fundamental "eigen contour" changes that might encode intoxication. Energy contours differ significantly with intoxication in terms of distance measures, the second and fourth DCT coefficients, and the first and second moments of the lower DCT spectrum. Principal Components Analysis did not yield interpretable "eigen contours" that could be used in distinguishing intoxicated from sober contours.
Collapse
Affiliation(s)
- Christian Heinrich
- Institute of Phonetics and Speech Processing, Ludwig-Maximilians-Universität, Schellingstrasse 3, 80799, Munich, Germany
| | - Florian Schiel
- Institute of Phonetics and Speech Processing, Ludwig-Maximilians-Universität, Schellingstrasse 3, 80799, Munich, Germany
| |
Collapse
|
16
|
Sharma N, Gedeon T. Objective measures, sensors and computational techniques for stress recognition and classification: a survey. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2012; 108:1287-301. [PMID: 22921417 DOI: 10.1016/j.cmpb.2012.07.003] [Citation(s) in RCA: 87] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/02/2012] [Revised: 07/02/2012] [Accepted: 07/25/2012] [Indexed: 05/15/2023]
Abstract
Stress is a major growing concern in our day and age adversely impacting both individuals and society. Stress research has a wide range of benefits from improving personal operations, learning, and increasing work productivity to benefiting society - making it an interesting and socially beneficial area of research. This survey reviews sensors that have been used to measure stress and investigates techniques for modelling stress. It discusses non-invasive and unobtrusive sensors for measuring computed stress, a term we coin in the paper. Sensors that do not impede everyday activities that could be used by those who would like to monitor stress levels on a regular basis (e.g. vehicle drivers, patients with illnesses linked to stress) is the focus of the discussion. Computational techniques have the capacity to determine optimal sensor fusion and automate data analysis for stress recognition and classification. Several computational techniques have been developed to model stress based on techniques such as Bayesian networks, artificial neural networks, and support vector machines, which this survey investigates. The survey concludes with a summary and provides possible directions for further computational stress research.
Collapse
Affiliation(s)
- Nandita Sharma
- Information and Human Centred Computing Research Group, Research School of Computer Science, Australian National University, Canberra, ACT 0200, Australia.
| | | |
Collapse
|
17
|
Huttunen KH, Keränen HI, Pääkkönen RJ, Päivikki Eskelinen-Rönkä R, Leino TK. Effect of cognitive load on articulation rate and formant frequencies during simulator flights. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2011; 129:1580-1593. [PMID: 21428521 DOI: 10.1121/1.3543948] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
It was explored how three types of intensive cognitive load typical of military aviation (load on situation awareness, information processing, or decision-making) affect speech. The utterances of 13 male military pilots were recorded during simulated combat flights. Articulation rate was calculated from the speech samples, and the first formant (F1) and second formant (F2) were tracked from first-syllable short vowels in pre-defined phoneme environments. Articulation rate was found to correlate negatively (albeit with low coefficients) with loads on situation awareness and decision-making but not with changes in F1 or F2. Changes were seen in the spectrum of the vowels: mean F1 of front vowels usually increased and their mean F2 decreased as a function of cognitive load, and both F1 and F2 of back vowels increased. The strongest associations were seen between the three types of cognitive load and F1 and F2 changes in back vowels. Because fluent and clear radio speech communication is vital to safety in aviation and temporal and spectral changes may affect speech intelligibility, careful use of standard aviation phraseology and training in the production of clear speech during a high level of cognitive load are important measures that diminish the probability of possible misunderstandings.
Collapse
Affiliation(s)
- Kerttu H Huttunen
- Institute of Clinical Medicine/Department of Otorhinolaryngology, University of Oulu, P.O. Box 5000, Oulu, Finland.
| | | | | | | | | |
Collapse
|
18
|
|
19
|
Young V, Mihailidis A. Difficulties in Automatic Speech Recognition of Dysarthric Speakers and Implications for Speech-Based Applications Used by the Elderly: A Literature Review. Assist Technol 2010; 22:99-112; quiz 113-4. [DOI: 10.1080/10400435.2010.483646] [Citation(s) in RCA: 44] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022] Open
|