Stress measurement using speech: Recent advancements, validation issues, and ethical and privacy considerations

Sensing behavior change in chronic pain: a scoping review of sensor technology for use in daily life

ABSTRACT

Technology offers possibilities for quantification of behaviors and physiological changes of relevance to chronic pain, using wearable sensors and devices suitable for data collection in daily life contexts. We conducted a scoping review of wearable and passive sensor technologies that sample data of psychological interest in chronic pain, including in social situations. Sixty articles met our criteria from the 2783 citations retrieved from searching. Three-quarters of recruited people were with chronic pain, mostly musculoskeletal, and the remainder with acute or episodic pain; those with chronic pain had a mean age of 43 (few studies sampled adolescents or children) and 60% were women. Thirty-seven studies were performed in laboratory or clinical settings and the remainder in daily life settings. Most used only 1 type of technology, with 76 sensor types overall. The commonest was accelerometry (mainly used in daily life contexts), followed by motion capture (mainly in laboratory settings), with a smaller number collecting autonomic activity, vocal signals, or brain activity. Subjective self-report provided "ground truth" for pain, mood, and other variables, but often at a different timescale from the automatically collected data, and many studies reported weak relationships between technological data and relevant psychological constructs, for instance, between fear of movement and muscle activity. There was relatively little discussion of practical issues: frequency of sampling, missing data for human or technological reasons, and the users' experience, particularly when users did not receive data in any form. We conclude the review with some suggestions for content and process of future studies in this field.

Mapping the ethical landscape of digital biomarkers: A scoping review

In the evolving landscape of digital medicine, digital biomarkers have emerged as a transformative source of health data, positioning them as an indispensable element for the future of the discipline. This necessitates a comprehensive exploration of the ethical complexities and challenges intrinsic to this cutting-edge technology. To address this imperative, we conducted a scoping review, seeking to distill the scientific literature exploring the ethical dimensions of the use of digital biomarkers. By closely scrutinizing the literature, this review aims to bring to light the underlying ethical issues associated with the development and integration of digital biomarkers into medical practice.

Acoustic and prosodic speech features reflect physiological stress but not isolated negative affect: a multi-paradigm study on psychosocial stressors

Heterogeneity in speech under stress has been a recurring issue in stress research, potentially due to varied stress induction paradigms. This study investigated speech features in semi-guided speech following two distinct psychosocial stress paradigms (Cyberball and MIST) and their respective control conditions. Only negative affect increased during Cyberball, while self-reported stress, skin conductance response rate, and negative affect increased during MIST. Fundamental frequency (F0), speech rate, and jitter significantly changed during MIST, but not Cyberball; HNR and shimmer showed no expected changes. The results indicate that observed speech features are robust in semi-guided speech and sensitive to stressors eliciting additional physiological stress responses, not solely decreases in negative affect. These differences between stressors may explain literature heterogeneity. Our findings support the potential of speech as a stress level biomarker, especially when stress elicits physiological reactions, similar to other biomarkers. This highlights its promise as a tool for measuring stress in everyday settings, considering its affordability, non-intrusiveness, and ease of collection. Future research should test these results' robustness and specificity in naturalistic settings, such as freely spoken speech and noisy environments while exploring and validating a broader range of informative speech features in the context of stress.

Use of voice features from smartphones for monitoring depressive disorders: Scoping review

Object

This review evaluates the use of smartphone-based voice data for predicting and monitoring depression.

Methods

A scoping review was conducted, examining 14 studies from Medline, Scopus, and Web of Science (2010-2023) on voice data collection methods and the use of voice features for minitoring depression.

Results

Voice data, especially prosodic features like fundamental frequency and pitch, show promise for predicting depression, though their sole predictive power requires further validation. Integrating voice with multimodal sensor data has been shown to improve accuracy significantly.

Conclusion

Smartphone-based voice monitoring offers a promising, noninvasive, and cost-effective approach to depression management. The integration of machine learning with sensor data could significantly enhance mental health monitoring, necessitating further research and longitudinal studies for validation.

Reliability and validity of a widely-available AI tool for assessment of stress based on speech

Cigna's online stress management toolkit includes an AI-based tool that purports to evaluate a person's psychological stress level based on analysis of their speech, the Cigna StressWaves Test (CSWT). In this study, we evaluate the claim that the CSWT is a "clinical grade" tool via an independent validation. The results suggest that the CSWT is not repeatable and has poor convergent validity; the public availability of the CSWT despite insufficient validation data highlights concerns regarding premature deployment of digital health tools for stress and anxiety management.

Detecting Social Contexts from Mobile Sensing Indicators in Virtual Interactions with Socially Anxious Individuals

Mobile sensing is a ubiquitous and useful tool to make inferences about individuals' mental health based on physiology and behavior patterns. Along with sensing features directly associated with mental health, it can be valuable to detect different features of social contexts to learn about social interaction patterns over time and across different environments. This can provide insight into diverse communities' academic, work and social lives, and their social networks. We posit that passively detecting social contexts can be particularly useful for social anxiety research, as it may ultimately help identify changes in social anxiety status and patterns of social avoidance and withdrawal. To this end, we recruited a sample of highly socially anxious undergraduate students (N=46) to examine whether we could detect the presence of experimentally manipulated virtual social contexts via wristband sensors. Using a multitask machine learning pipeline, we leveraged passively sensed biobehavioral streams to detect contexts relevant to social anxiety, including (1) whether people were in a social situation, (2) size of the social group, (3) degree of social evaluation, and (4) phase of social situation (anticipating, actively experiencing, or had just participated in an experience). Results demonstrated the feasibility of detecting most virtual social contexts, with stronger predictive accuracy when detecting whether individuals were in a social situation or not and the phase of the situation, and weaker predictive accuracy when detecting the level of social evaluation. They also indicated that sensing streams are differentially important to prediction based on the context being predicted. Our findings also provide useful information regarding design elements relevant to passive context detection, including optimal sensing duration, the utility of different sensing modalities, and the need for personalization. We discuss implications of these findings for future work on context detection (e.g., just-in-time adaptive intervention development).

Manifestation of depression in speech overlaps with characteristics used to represent and recognize speaker identity

The sound of a person's voice is commonly used to identify the speaker. The sound of speech is also starting to be used to detect medical conditions, such as depression. It is not known whether the manifestations of depression in speech overlap with those used to identify the speaker. In this paper, we test the hypothesis that the representations of personal identity in speech, known as speaker embeddings, improve the detection of depression and estimation of depressive symptoms severity. We further examine whether changes in depression severity interfere with the recognition of speaker's identity. We extract speaker embeddings from models pre-trained on a large sample of speakers from the general population without information on depression diagnosis. We test these speaker embeddings for severity estimation in independent datasets consisting of clinical interviews (DAIC-WOZ), spontaneous speech (VocalMind), and longitudinal data (VocalMind). We also use the severity estimates to predict presence of depression. Speaker embeddings, combined with established acoustic features (OpenSMILE), predicted severity with root mean square error (RMSE) values of 6.01 and 6.28 in DAIC-WOZ and VocalMind datasets, respectively, lower than acoustic features alone or speaker embeddings alone. When used to detect depression, speaker embeddings showed higher balanced accuracy (BAc) and surpassed previous state-of-the-art performance in depression detection from speech, with BAc values of 66% and 64% in DAIC-WOZ and VocalMind datasets, respectively. Results from a subset of participants with repeated speech samples show that the speaker identification is affected by changes in depression severity. These results suggest that depression overlaps with personal identity in the acoustic space. While speaker embeddings improve depression detection and severity estimation, deterioration or improvement in mood may interfere with speaker verification.

Speech as a Promising Biosignal in Precision Psychiatry

Health research and health care alike are presently based on infrequent assessments that provide an incomplete picture of clinical functioning. Consequently, opportunities to identify and prevent health events before they occur are missed. New health technologies are addressing these critical issues by enabling the continual monitoring of health-related processes using speech. These technologies are a great match for the healthcare environment because they make high-frequency assessments non-invasive and highly scalable. Indeed, existing tools can now extract a wide variety of health-relevant biosignals from smartphones by analyzing a person's voice and speech. These biosignals are linked to health-relevant biological pathways and have shown promise in detecting several disorders, including depression and schizophrenia. However, more research is needed to identify the speech signals that matter most, validate these signals against ground-truth outcomes, and translate these data into biomarkers and just-in-time adaptive interventions. We discuss these issues herein by describing how assessing everyday psychological stress through speech can help both researchers and health care providers monitor the impact that stress has on a wide variety of mental and physical health outcomes, such as self-harm, suicide, substance abuse, depression, and disease recurrence. If done appropriately and securely, speech is a novel digital biosignal that could play a key role in predicting high-priority clinical outcomes and delivering tailored interventions that help people when they need it most.

Mental Stress Assessment Using Ultra Short Term HRV Analysis Based on Non-Linear Method

Mental stress is on the rise as one of the major health problems in modern society. It is important to detect and manage mental stress to prevent various diseases caused by stress and to maintain a healthy life. The purpose of this paper is to present new heart rate variability (HRV) features based on empirical mode decomposition and to detect acute mental stress through short-term HRV (5 min) and ultra-short-term HRV (under 5 min) analysis. HRV signals were acquired from 74 young police officers using acute stressors, including the Trier Social Stress Test and horror movie viewing, and a total of 26 features, including the proposed IMF energy features and general HRV features, were extracted. A support vector machine (SVM) classification model is used to classify the stress and non-stress states through leave-one-subject-out cross-validation. The classification accuracies of short-term HRV and ultra-short-term HRV analysis are 86.5% and 90.5%, respectively. In the results of ultra-short-term HRV analysis using various time lengths, we suggest the optimal duration to detect mental stress, which can be applied to wearable devices or healthcare systems.

Vocal-Stress Diary: A Longitudinal Investigation of the Association of Everyday Work Stressors and Human Voice Features

The human voice conveys plenty of information about the speaker. A prevalent assumption is that stress-related changes in the human body affect speech production, thus affecting voice features. This suggests that voice data may be an easy-to-capture measure of everyday stress levels and can thus serve as a warning signal of stress-related health consequences. However, previous research is limited (i.e., has induced stress only through artificial tasks or has investigated only short-term or extreme stressors), leaving it open whether everyday work stressors are associated with voice features. Thus, our participants (111 adult working individuals) took part in a 1-week diary study (Sunday until Sunday), in which they provided voice messages and self-report data on daily work stressors. Results showed that work stressors were associated with voice features such as increased speech rate and voice intensity. We discuss theoretical, practical, and ethical implications regarding the voice as an indicator of psychological states.

What Are We Measuring When We Evaluate Digital Interventions for Improving Lifestyle? A Scoping Meta-Review

Background: Lifestyle Medicine (LM) aims to address six main behavioral domains: diet/nutrition, substance use (SU), physical activity (PA), social relationships, stress management, and sleep. Digital Health Interventions (DHIs) have been used to improve these domains. However, there is no consensus on how to measure lifestyle and its intermediate outcomes aside from measuring each behavior separately. We aimed to describe (1) the most frequent lifestyle domains addressed by DHIs, (2) the most frequent outcomes used to measure lifestyle changes, and (3) the most frequent DHI delivery methods. Methods: We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA-ScR) Extension for Scoping Reviews. A literature search was conducted using MEDLINE, Cochrane Library, EMBASE, and Web of Science for publications since 2010. We included systematic reviews and meta-analyses of clinical trials using DHI to promote health, behavioral, or lifestyle change. Results: Overall, 954 records were identified, and 72 systematic reviews were included. Of those, 35 conducted meta-analyses, 58 addressed diet/nutrition, and 60 focused on PA. Only one systematic review evaluated all six lifestyle domains simultaneously; 1 systematic review evaluated five lifestyle domains; 5 systematic reviews evaluated 4 lifestyle domains; 14 systematic reviews evaluated 3 lifestyle domains; and the remaining 52 systematic reviews evaluated only one or two domains. The most frequently evaluated domains were diet/nutrition and PA. The most frequent DHI delivery methods were smartphone apps and websites. Discussion: The concept of lifestyle is still unclear and fragmented, making it hard to evaluate the complex interconnections of unhealthy behaviors, and their impact on health. Clarifying this concept, refining its operationalization, and defining the reporting guidelines should be considered as the current research priorities. DHIs have the potential to improve lifestyle at primary, secondary, and tertiary levels of prevention-but most of them are targeting clinical populations. Although important advances have been made to evaluate DHIs, some of their characteristics, such as the rate at which they become obsolete, will require innovative research designs to evaluate long-term outcomes in health.

Monitoring the effects of therapeutic interventions in depression through self-assessments

The treatment of major psychiatric disorders is an arduous and thorny path for the patients concerned, characterized by polypharmacy, massive adverse side effects, modest prospects of success, and constantly declining response rates. The more important is the early detection of psychiatric disorders prior to the development of clinically relevant symptoms, so that people can benefit from early interventions. A well-proven approach to monitoring mental health relies on voice analysis. This method has been successfully used with psychiatric patients to 'objectively' document the progress of improvement or the onset of relapse. The studies with psychiatric patients over 2-4 weeks demonstrated that daily voice assessments have a notable therapeutic effect in themselves. Therefore, daily voice assessments appear to be a lowthreshold form of therapeutic means that may be realized through self-assessments. To evaluate performance and reliability of this approach, we have carried out a longitudinal study on 82 university students in 3 different countries with daily assessments over 2 weeks. The sample included 41 males (mean age 24.2±3.83 years) and 41 females (mean age 21.6±2.05 years). Unlike other research in the field, this study was not concerned with the classification of individuals in terms of diagnostic categories. The focus lay on the monitoring aspect and the extent to which the effects of therapeutic interventions or of behavioural changes are visible in the results of self-assessment voice analyses. The test persons showed an over-proportionally good adherence to the daily voice analysis scheme. The accumulated data were of generally high quality: sufficiently high signal levels, a very limited number of movement artifacts, and little to no interfering background noise. The method was sufficiently sensitive to detect: i) habituation effects when test persons became used to the daily procedure; and ii) short-term fluctuations that exceeded prespecified thresholds and reached significance. Results are directly interpretable and provide information about what is going well, what is going less well, and where there is a need for action. The proposed self-assessment approach was found to be well-suited to serve as a health-monitoring tool for subjects with an elevated vulnerability to psychiatric disorders or to stress-induced mental health problems. Daily voice assessments are in fact a low-threshold form of therapeutic means that can be realized through selfassessments, that requires only little effort, can be carried out in the test person's own home, and has the potential to strengthen resilience and to induce positive behavioural changes.

Using mobile sensing data to assess stress: Associations with perceived and lifetime stress, mental health, sleep, and inflammation

Background

Although stress is a risk factor for mental and physical health problems, it can be difficult to assess, especially on a continual, non-invasive basis. Mobile sensing data, which are continuously collected from naturalistic smartphone use, may estimate exposure to acute and chronic stressors that have health-damaging effects. This initial validation study validated a mobile-sensing collection tool against assessments of perceived and lifetime stress, mental health, sleep duration, and inflammation.

Methods

Participants were 25 well-characterized healthy young adults (M_age = 20.64 years, SD = 2.74; 13 men, 12 women). We collected affective text language use with a custom smartphone keyboard. We assessed participants’ perceived and lifetime stress, depression and anxiety levels, sleep duration, and basal inflammatory activity (i.e. salivary C-reactive protein and interleukin-1β).

Results

Three measures of affective language (i.e. total positive words, total negative words, and total affective words) were strongly associated with lifetime stress exposure, and total negative words typed was related to fewer hours slept (all large effect sizes: r = 0.50 – 0.78). Total positive words, total negative words, and total affective words typed were also associated with higher perceived stress and lower salivary C-reactive protein levels (medium effect sizes; r = 0.22 – 0.32).

Conclusions

Data from this initial longitudinal validation study suggest that total and affective text use may be useful mobile sensing measures insofar as they are associated with several other stress, mental health, behavioral, and biological outcomes. This tool may thus help identify individuals at increased risk for stress-related health problems.

Digital Resilience Biomarkers for Personalized Health Maintenance and Disease Prevention

Health maintenance and disease prevention strategies become increasingly prioritized with increasing health and economic burden of chronic, lifestyle-related diseases. A key element in these strategies is the empowerment of individuals to control their health. Self-measurement plays an essential role in achieving such empowerment. Digital measurements have the advantage of being measured non-invasively, passively, continuously, and in a real-world context. An important question is whether such measurement can sensitively measure subtle disbalances in the progression toward disease, as well as the subtle effects of, for example, nutritional improvement. The concept of resilience biomarkers, defined as the dynamic evaluation of the biological response to an external challenge, has been identified as a viable strategy to measure these subtle effects. In this review, we explore the potential of integrating this concept with digital physiological measurements to come to digital resilience biomarkers. Additionally, we discuss the potential of wearable, non-invasive, and continuous measurement of molecular biomarkers. These types of innovative measurements may, in the future, also serve as a digital resilience biomarker to provide even more insight into the personal biological dynamics of an individual. Altogether, digital resilience biomarkers are envisioned to allow for the measurement of subtle effects of health maintenance and disease prevention strategies in a real-world context and thereby give personalized feedback to improve health.

Speech as an indicator for psychosocial stress: A network analytic approach

Recently, the possibilities of detecting psychosocial stress from speech have been discussed. Yet, there are mixed effects and a current lack of clarity in relations and directions for parameters derived from stressed speech. The aim of the current study is – in a controlled psychosocial stress induction experiment – to apply network modeling to (1) look into the unique associations between specific speech parameters, comparing speech networks containing fundamental frequency (F0), jitter, mean voiced segment length, and Harmonics-to-Noise Ratio (HNR) pre- and post-stress induction, and (2) examine how changes pre- versus post-stress induction (i.e., change network) in each of the parameters are related to changes in self-reported negative affect. Results show that the network of speech parameters is similar after versus before the stress induction, with a central role of HNR, which shows that the complex interplay and unique associations between each of the used speech parameters is not impacted by psychosocial stress (aim 1). Moreover, we found a change network (consisting of pre-post stress difference values) with changes in jitter being positively related to changes in self-reported negative affect (aim 2). These findings illustrate – for the first time in a well-controlled but ecologically valid setting – the complex relations between different speech parameters in the context of psychosocial stress. Longitudinal and experimental studies are required to further investigate these relationships and to test whether the identified paths in the networks are indicative of causal relationships.

A New Characterization of Mental Health Disorders Using Digital Behavioral Data: Evidence from Major Depressive Disorder

Mental health disorders are ambiguously defined and diagnosed. The established diagnosis technique, which is based on structured interviews, questionnaires and data subjectively reported by the patients themselves, leaves the mental health field behind other medical areas. We support these statements with examples from major depressive disorder (MDD). The National Institute of Mental Health (NIMH) launched the Research Domain Criteria (RDoC) project in 2009 as a new framework to investigate psychiatric pathologies from a multidisciplinary point of view. This is a good step in the right direction. Contemporary psychiatry considers mental illnesses as diseases that manifest in the mind and arise from the brain, expressed as a behavioral condition; therefore, we claim that these syndromes should be characterized primarily using behavioral characteristics. We suggest the use of smartphones and wearable devices to passively collect quantified behavioral data from patients by utilizing digital biomarkers of mental disorder symptoms. Various digital biomarkers of MDD symptoms have already been detected, and apps for collecting this longitudinal behavioral data have already been developed. This quantified data can be used to determine a patient’s diagnosis and personalized treatment, and thereby minimize the diagnosis rate of comorbidities. As there is a wide spectrum of human behavior, such a fluidic and personalized approach is essential.

The Concept of Advanced Multi-Sensor Monitoring of Human Stress

Many people live under stressful conditions which has an adverse effect on their health. Human stress, especially long-term one, can lead to a serious illness. Therefore, monitoring of human stress influence can be very useful. We can monitor stress in strictly controlled laboratory conditions, but it is time-consuming and does not capture reactions, on everyday stressors or in natural environment using wearable sensors, but with limited accuracy. Therefore, we began to analyze the current state of promising wearable stress-meters and the latest advances in the record of related physiological variables. Based on these results, we present the concept of an accurate, reliable and easier to use telemedicine device for long-term monitoring of people in a real life. In our concept, we ratify with two synchronized devices, one on the finger and the second on the chest. The results will be obtained from several physiological variables including electrodermal activity, heart rate and respiration, body temperature, blood pressure and others. All these variables will be measured using a coherent multi-sensors device. Our goal is to show possibilities and trends towards the production of new telemedicine equipment and thus, opening the door to a widespread application of human stress-meters.

Human stress classification during public speaking using physiological signals

Public speaking is a common type of social evaluative situation and a significant amount of the population feel uneasy with it. It is of utmost importance to detect public speaking stress so that appropriate action can be taken to minimize its impacts on human health. In this study, a multimodal human stress classification scheme in response to real-life public speaking activity is proposed. Electroencephalography (EEG), galvanic skin response (GSR), and photoplethysmography (PPG) signals of forty participants are acquired in rest-state and during public speaking activity to divide data into a stressed and non-stressed group. Frequency domain features from EEG and time-domain features from GSR and PPG signals are extracted. The selected set of features from all modalities are fused to classify the stress into two classes. Classification is performed via a leave-one-out cross-validation scheme by using five different classifiers. The highest accuracy of 96.25% is achieved using a support vector machine classifier with radial basis function. Statistical analysis is performed to examine the significance of EEG, GSR, and PPG signals between the two phases of the experiment. Statistical significance is found in certain EEG frequency bands as well as GSR and PPG data recorded before and after public speaking supporting the fact that brain activity, skin conductance, and blood volumetric flow are credible measures of human stress during public speaking activity.

Acoustic analysis of surgeons’ voices to assess change in the stress response during surgical in situ simulation

Introduction

Stress may serve as an adjunct (challenge) or hindrance (threat) to the learning process. Determining the effect of an individual’s response to situational demands in either a real or simulated situation may enable optimisation of the learning environment. Studies of acoustic analysis suggest that mean fundamental frequency and formant frequencies of voice vary with an individual’s response during stressful events. This hypothesis is reviewed within the otolaryngology (ORL) simulation environment to assess whether acoustic analysis could be used as a tool to determine participants’ stress response and cognitive load in medical simulation. Such an assessment could lead to optimisation of the learning environment.

Methodology

ORL simulation scenarios were performed to teach the participants teamwork and refine clinical skills. Each was performed in an actual operating room (OR) environment (in situ) with a multidisciplinary team consisting of ORL surgeons, OR nurses and anaesthesiologists. Ten of the scenarios were led by an ORL attending and ten were led by an ORL fellow. The vocal communication of each of the 20 individual leaders was analysed using a long-term pitch analysis PRAAT software (autocorrelation method) to obtain mean fundamental frequency (F0) and first four formant frequencies (F1, F2, F3 and F4). In reviewing individual scenarios, each leader’s voice was analysed during a non-stressful environment (WHO sign-out procedure) and compared with their voice during a stressful portion of the scenario (responding to deteriorating oxygen saturations in the manikin).

Results

The mean unstressed F0 for the male voice was 161.4 Hz and for the female voice was 217.9 Hz. The mean fundamental frequency of speech in the ORL fellow (lead surgeon) group increased by 34.5 Hz between the scenario’s baseline and stressful portions. This was significantly different to the mean change of −0.5 Hz noted in the attending group (p=0.01). No changes were seen in F1, F2, F3 or F4.

Conclusions

This study demonstrates a method of acoustic analysis of the voices of participants taking part in medical simulations. It suggests acoustic analysis of participants may offer a simple, non-invasive, non-intrusive adjunct in evaluating and titrating the stress response during simulation.

Linking physical and social environments with mental health in old age: a multisensor approach for continuous real-life ecological and emotional assessment

Background

Urban stress is mentioned as a plausible mechanism leading to chronic stress, which is a risk factor of depression. Yet, an accurate assessment of urban stressors in environmental epidemiology requires new methods. This article discusses methods for the sensor-based continuous assesment of geographic environments, stress and depressive symptoms in older age. We report protocols of the promoting mental well-being and healthy ageing in cities (MINDMAP) and Healthy Aging and Networks in Cities (HANC) studies nested in the RECORD Cohort as a background for a broad discussion about the theoretical foundation and monitoring tools of mobile sensing research in older age. Specifically, these studies allow one to compare how older people with and without depression perceive, navigate and use their environment; and how the built environments, networks of social contacts, and spatial mobility patterns influence the mental health of older people.

Methods

Our research protocol combines (1) Global Positioning System (GPS) and accelerometer tracking and a GPS-based mobility survey to assess participants’ mobility patterns, activity patterns and environmental exposures; (2) proximity detection to assess whether household members are close to each other; (3) ecological momentary assessment to track momentary mood and stress and environmental perceptions; and (4) electrodermal activity for the tentative prediction of stress. Data will be compared within individuals (at different times) and between persons with and without depressive symptoms.

Conclusion

The development of mobile sensing and survey technologies opens an avenue to improve understanding of the role of momentary stressors and resourcing features of residential and non-residential environments for older populations’ mental health. However, validation, privacy and ethical aspects are important issues to consider.

Application Experiences Using IoT Devices in Education

The Internet of Things (IoT) is becoming a regular part of our lives. The devices can be used in many sectors, such as education and in the learning process. The article describes the possibilities of using commonly available devices such as smart wristbands (watches) and eye tracking technology, i.e., using existing technical solutions and methods that rely on the application of sensors while maintaining non-invasiveness. By comparing the data from these devices, we observed how the students’ attention affects their results. We looked for a correlation between eye tracking, heart rate, and student attention and how it all impacts their learning outcomes. We evaluate the obtained data in order to determine whether there is a degree of dependence between concentration and heart rate of students.

Using AI to predict service agent stress from emotion patterns in service interactions

Purpose

A vast body of literature has documented the negative consequences of stress on employee performance and well-being. These deleterious effects are particularly pronounced for service agents who need to constantly endure and manage customer emotions. The purpose of this paper is to introduce and describe a deep learning model to predict in real-time service agent stress from emotion patterns in voice-to-voice service interactions.

Design/methodology/approach

A deep learning model was developed to identify emotion patterns in call center interactions based on 363 recorded service interactions, subdivided in 27,889 manually expert-labeled three-second audio snippets. In a second step, the deep learning model was deployed in a call center for a period of one month to be further trained by the data collected from 40 service agents in another 4,672 service interactions.

Findings

The deep learning emotion classifier reached a balanced accuracy of 68% in predicting discrete emotions in service interactions. Integrating this model in a binary classification model, it was able to predict service agent stress with a balanced accuracy of 80%.

Practical implications

Service managers can benefit from employing the deep learning model to continuously and unobtrusively monitor the stress level of their service agents with numerous practical applications, including real-time early warning systems for service agents, customized training and automatically linking stress to customer-related outcomes.

Originality/value

The present study is the first to document an artificial intelligence (AI)-based model that is able to identify emotions in natural (i.e. nonstaged) interactions. It is further a pioneer in developing a smart emotion-based stress measure for service agents. Finally, the study contributes to the literature on the role of emotions in service interactions and employee stress.

The Stress and Adversity Inventory for Adults (Adult STRAIN) in Brazilian Portuguese: Initial Validation and Links With Executive Function, Sleep, and Mental and Physical Health

It has been widely hypothesized that stressors occurring over the lifespan exert a cumulative impact on health, but little work has directly tested these theories given the difficulty associated with measuring cumulative stress exposure over the lifespan. We addressed this issue in Brazil by translating the Stress and Adversity Inventory for Adults (Adult STRAIN) into Brazilian Portuguese. We then examined the instrument's usability and acceptability; concurrent, discriminant, predictive, and incremental validity; and test-retest reliability. Participants were 330 Brazilian adults (238 women; M age = 32.16; range: 18-76 years old) who completed the Adult STRAIN in Brazilian Portuguese, Childhood Trauma Questionnaire-Short Form (CTQ-SF), and Perceived Stress Scale (PSS). They also completed measures of socioeconomic status, personality, social desirability, negative affect, physical and mental health complaints, sleep quality, executive function, and doctor-diagnosed general health problems and autoimmune disorders. The STRAIN exhibited excellent usability and acceptability and was completed in 16 min and 27 s, on average. It showed good concurrent validity relative to the CTQ-SF and PSS (rs ≥ 0.377) and good discriminant validity, both with and without adjusting for covariates. In addition, the STRAIN significantly predicted all of the health outcomes assessed except for executive function and explained substantial variance in these outcomes over and above the CTQ-SF, PSS, and covariates assessed. Finally, the test-retest reliability indices for total lifetime stressor count and severity were outstanding (r icc = 0.936 and 0.953, respectively, over M = 34.86 days). The Adult STRAIN in Brazilian Portuguese thus exhibits excellent usability and acceptability, good concurrent and discriminant validity, strong predictive and incremental validity across a variety of health outcomes, and outstanding test-retest reliability. We therefore conclude that the STRAIN is a practical, valid, and reliable instrument for researchers and clinicians looking to efficiently assess cumulative lifetime stress exposure in Brazilian Portuguese.

Unlocking stress and forecasting its consequences with digital technology

Chronic stress is a major underlying origin of the top leading causes of death, globally. Yet, the mechanistic explanation of the association between stress and disease is poorly understood. This stems from the inability to adequately measure stress in its naturally occurring state and the extreme heterogeneity by inter and intraindividual characteristics. The growth and availability of digital technologies involving wearable devices and mobile phone apps afford the opportunity to dramatically improve measurement of the biological stress response in real time. In parallel, the advancement and capabilities of artificial intelligence (AI) and machine learning could discern heterogeneous, multidimensional information from individual signs of stress, and possibly inform how these signs forecast the downstream consequences of stress in the form of end-organ damage. The marriage of these tools could dramatically enhance the field of stress research contributing to impactful and empowering interventions for individuals bridging knowledge to practice, and intervention to real-world use. Here we discuss this potential, anticipated challenges, and emerging opportunities.