1
|
Başaran OT, Can YS, André E, Ersoy C. Relieving the burden of intensive labeling for stress monitoring in the wild by using semi-supervised learning. Front Psychol 2024; 14:1293513. [PMID: 38250116 PMCID: PMC10797089 DOI: 10.3389/fpsyg.2023.1293513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 12/08/2023] [Indexed: 01/23/2024] Open
Abstract
Stress, a natural process affecting individuals' wellbeing, has a profound impact on overall quality of life. Researchers from diverse fields employ various technologies and methodologies to investigate it and alleviate the negative effects of this phenomenon. Wearable devices, such as smart bands, capture physiological data, including heart rate variability, motions, and electrodermal activity, enabling stress level monitoring through machine learning models. However, labeling data for model accuracy assessment poses a significant challenge in stress-related research due to incomplete or inaccurate labels provided by individuals in their daily lives. To address this labeling predicament, our study proposes implementing Semi-Supervised Learning (SSL) models. Through comparisons with deep learning-based supervised models and clustering-based unsupervised models, we evaluate the performance of our SSL models. Our experiments show that our SSL models achieve 77% accuracy with a classifier trained on an augmented dataset prepared using the label propagation (LP) algorithm. Additionally, our deep autoencoder network achieves 76% accuracy. These results highlight the superiority of SSL models over unsupervised learning techniques and their comparable performance to supervised learning models, even with limited labeled data. By relieving the burden of labeling in daily life stress recognition, our study advances stress-related research, recognizing stress as a natural process rather than a disease. This facilitates the development of more efficient and accurate stress monitoring methods in the wild.
Collapse
Affiliation(s)
- Osman Tugay Başaran
- Computer and Communication Systems (CCS) Labs, Telecommunication Networks Group (TKN), Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany
| | - Yekta Said Can
- Faculty of Applied Computer Science, Institute of Computer Science, Universität Augsburg, Augsburg, Germany
| | - Elisabeth André
- Faculty of Applied Computer Science, Institute of Computer Science, Universität Augsburg, Augsburg, Germany
| | - Cem Ersoy
- NETLAB Research Laboratory, Department of Computer Engineering, Bogazici University, Istanbul, Turkey
| |
Collapse
|
2
|
Staderini EM, Kambampati H, Ramakrishnaiah AK, Mugnaini S, Magrini A, Gentili S. A New Algorithm for Estimating a Noiseless, Evenly Sampled, Heart Rate Modulating Signal. Bioengineering (Basel) 2023; 10:bioengineering10050552. [PMID: 37237622 DOI: 10.3390/bioengineering10050552] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 04/27/2023] [Accepted: 04/29/2023] [Indexed: 05/28/2023] Open
Abstract
Heart rate variability (HRV) is commonly intended as the variation in the heart rate (HR), and it is evaluated in the time and frequency domains with various well-known methods. In the present paper, the heart rate is considered as a time domain signal, at first as an abstract model in which the HR is the instantaneous frequency of an otherwise periodic signal, such as with an electrocardiogram (ECG). In this model, the ECG is assumed to be a frequency modulated signal, or carrier signal, where HRV or HRV(t) is the time-domain signal which is frequency modulating the carrier ECG signal around its average frequency. Hence, an algorithm able to frequency demodulate the ECG signal to extract the signal HRV(t) is described, with possibly enough time resolution to analyse fast time-domain variations in the instantaneous HR. After exhaustive testing of the method on simulated frequency modulated sinusoidal signals, the new procedure is eventually applied on actual ECG tracings for preliminary nonclinical testing. The purpose of the work is to use this algorithm as a tool and a more reliable method for the assessment of heart rate before any further clinical or physiological analysis.
Collapse
Affiliation(s)
- Enrico M Staderini
- Healthy World Association Switzerland, 1400 Yverdon-les-Bains, Switzerland
| | - Harish Kambampati
- Doctorate School in Industrial Engineering, "Tor Vergata" University of Rome, 00133 Roma, Italy
| | - Amith K Ramakrishnaiah
- Department of Electronics Engineering, "Tor Vergata" University of Rome, 00133 Roma, Italy
| | - Stefano Mugnaini
- Occupational Medicine Section, Department of Biomedicine and Prevention, "Tor Vergata" University of Rome, 00133 Roma, Italy
| | - Andrea Magrini
- Occupational Medicine Section, Department of Biomedicine and Prevention, "Tor Vergata" University of Rome, 00133 Roma, Italy
| | - Sandro Gentili
- Occupational Medicine Section, Department of Biomedicine and Prevention, "Tor Vergata" University of Rome, 00133 Roma, Italy
| |
Collapse
|
3
|
Vos G, Trinh K, Sarnyai Z, Rahimi Azghadi M. Generalizable machine learning for stress monitoring from wearable devices: A systematic literature review. Int J Med Inform 2023; 173:105026. [PMID: 36893657 DOI: 10.1016/j.ijmedinf.2023.105026] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 02/21/2023] [Accepted: 02/23/2023] [Indexed: 03/06/2023]
Abstract
INTRODUCTION Wearable sensors have shown promise as a non-intrusive method for collecting biomarkers that may correlate with levels of elevated stress. Stressors cause a variety of biological responses, and these physiological reactions can be measured using biomarkers including Heart Rate Variability (HRV), Electrodermal Activity (EDA) and Heart Rate (HR) that represent the stress response from the Hypothalamic-Pituitary-Adrenal (HPA) axis, the Autonomic Nervous System (ANS), and the immune system. While Cortisol response magnitude remains the gold standard indicator for stress assessment [1], recent advances in wearable technologies have resulted in the availability of a number of consumer devices capable of recording HRV, EDA and HR sensor biomarkers, amongst other signals. At the same time, researchers have been applying machine learning techniques to the recorded biomarkers in order to build models that may be able to predict elevated levels of stress. OBJECTIVE The aim of this review is to provide an overview of machine learning techniques utilized in prior research with a specific focus on model generalization when using these public datasets as training data. We also shed light on the challenges and opportunities that machine learning-enabled stress monitoring and detection face. METHODS This study reviewed published works contributing and/or using public datasets designed for detecting stress and their associated machine learning methods. The electronic databases of Google Scholar, Crossref, DOAJ and PubMed were searched for relevant articles and a total of 33 articles were identified and included in the final analysis. The reviewed works were synthesized into three categories of publicly available stress datasets, machine learning techniques applied using those, and future research directions. For the machine learning studies reviewed, we provide an analysis of their approach to results validation and model generalization. The quality assessment of the included studies was conducted in accordance with the IJMEDI checklist [2]. RESULTS A number of public datasets were identified that are labeled for stress detection. These datasets were most commonly produced from sensor biomarker data recorded using the Empatica E4 device, a well-studied, medical-grade wrist-worn wearable that provides sensor biomarkers most notable to correlate with elevated levels of stress. Most of the reviewed datasets contain less than twenty-four hours of data, and the varied experimental conditions and labeling methodologies potentially limit their ability to generalize for unseen data. In addition, we discuss that previous works show shortcomings in areas such as their labeling protocols, lack of statistical power, validity of stress biomarkers, and model generalization ability. CONCLUSION Health tracking and monitoring using wearable devices is growing in popularity, while the generalization of existing machine learning models still requires further study, and research in this area will continue to provide improvements as newer and more substantial datasets become available.
Collapse
Affiliation(s)
- Gideon Vos
- College of Science and Engineering, James Cook University, James Cook Dr, Townsville, 4811, QLD, Australia
| | - Kelly Trinh
- College of Science and Engineering, James Cook University, James Cook Dr, Townsville, 4811, QLD, Australia
| | - Zoltan Sarnyai
- College of Public Health, Medical, and Vet Sciences, James Cook University, James Cook Dr, Townsville, 4811, QLD, Australia
| | - Mostafa Rahimi Azghadi
- College of Science and Engineering, James Cook University, James Cook Dr, Townsville, 4811, QLD, Australia.
| |
Collapse
|
4
|
Amidei A, Spinsante S, Iadarola G, Benatti S, Tramarin F, Pavan P, Rovati L. Driver Drowsiness Detection: A Machine Learning Approach on Skin Conductance. SENSORS (BASEL, SWITZERLAND) 2023; 23:4004. [PMID: 37112345 PMCID: PMC10143251 DOI: 10.3390/s23084004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 04/05/2023] [Accepted: 04/11/2023] [Indexed: 06/19/2023]
Abstract
The majority of car accidents worldwide are caused by drowsy drivers. Therefore, it is important to be able to detect when a driver is starting to feel drowsy in order to warn them before a serious accident occurs. Sometimes, drivers are not aware of their own drowsiness, but changes in their body signals can indicate that they are getting tired. Previous studies have used large and intrusive sensor systems that can be worn by the driver or placed in the vehicle to collect information about the driver's physical status from a variety of signals that are either physiological or vehicle-related. This study focuses on the use of a single wrist device that is comfortable for the driver to wear and appropriate signal processing to detect drowsiness by analyzing only the physiological skin conductance (SC) signal. To determine whether the driver is drowsy, the study tests three ensemble algorithms and finds that the Boosting algorithm is the most effective in detecting drowsiness with an accuracy of 89.4%. The results of this study show that it is possible to identify when a driver is drowsy using only signals from the skin on the wrist, and this encourages further research to develop a real-time warning system for early detection of drowsiness.
Collapse
Affiliation(s)
- Andrea Amidei
- Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Via Pietro Vivarelli 10, 41125 Modena, Italy; (A.A.)
| | - Susanna Spinsante
- Department of Information Engineering, Polytechnic University of Marche, 60131 Ancona, Italy
| | - Grazia Iadarola
- Department of Information Engineering, Polytechnic University of Marche, 60131 Ancona, Italy
| | - Simone Benatti
- Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Via Pietro Vivarelli 10, 41125 Modena, Italy; (A.A.)
| | - Federico Tramarin
- Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Via Pietro Vivarelli 10, 41125 Modena, Italy; (A.A.)
| | - Paolo Pavan
- Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Via Pietro Vivarelli 10, 41125 Modena, Italy; (A.A.)
| | - Luigi Rovati
- Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Via Pietro Vivarelli 10, 41125 Modena, Italy; (A.A.)
| |
Collapse
|
5
|
Benchekroun M, Velmovitsky PE, Istrate D, Zalc V, Morita PP, Lenne D. Cross Dataset Analysis for Generalizability of HRV-Based Stress Detection Models. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23041807. [PMID: 36850407 PMCID: PMC9960690 DOI: 10.3390/s23041807] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 01/31/2023] [Accepted: 02/01/2023] [Indexed: 06/12/2023]
Abstract
Stress is an increasingly prevalent mental health condition across the world. In Europe, for example, stress is considered one of the most common health problems, and over USD 300 billion are spent on stress treatments annually. Therefore, monitoring, identification and prevention of stress are of the utmost importance. While most stress monitoring is carried out through self-reporting, there are now several studies on stress detection from physiological signals using Artificial Intelligence algorithms. However, the generalizability of these models is only rarely discussed. The main goal of this work is to provide a monitoring proof-of-concept tool exploring the generalization capabilities of Heart Rate Variability-based machine learning models. To this end, two Machine Learning models are used, Logistic Regression and Random Forest to analyze and classify stress in two datasets differing in terms of protocol, stressors and recording devices. First, the models are evaluated using leave-one-subject-out cross-validation with train and test samples from the same dataset. Next, a cross-dataset validation of the models is performed, that is, leave-one-subject-out models trained on a Multi-modal Dataset for Real-time, Continuous Stress Detection from Physiological Signals dataset and validated using the University of Waterloo stress dataset. While both logistic regression and random forest models achieve good classification results in the independent dataset analysis, the random forest model demonstrates better generalization capabilities with a stable F1 score of 61%. This indicates that the random forest can be used to generalize HRV-based stress detection models, which can lead to better analyses in the mental health and medical research field through training and integrating different models.
Collapse
Affiliation(s)
- Mouna Benchekroun
- Biomechanics and Bioengineering Lab, University of Technology of Compiègne (UMR CNRS 7338), 60200 Compiègne, France
- Heudiasyc Lab (Heuristics and Diagnosis of Complex Systems), University of Technology of Compiègne (UMR CNRS 7338), 60200 Compiègne, France
| | - Pedro Elkind Velmovitsky
- School of Public Health and Health Systems, University of Waterloo, Waterloo, ON N2L 3G1, Canada
| | - Dan Istrate
- Biomechanics and Bioengineering Lab, University of Technology of Compiègne (UMR CNRS 7338), 60200 Compiègne, France
| | - Vincent Zalc
- Biomechanics and Bioengineering Lab, University of Technology of Compiègne (UMR CNRS 7338), 60200 Compiègne, France
| | - Plinio Pelegrini Morita
- School of Public Health and Health Systems, University of Waterloo, Waterloo, ON N2L 3G1, Canada
- Research Institute for Aging, University of Waterloo, Waterloo, ON N2J 0E2, Canada
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada
- Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, ON M5T 3M6, Canada
- Centre for Digital Therapeutics, Techna Institute, University Health Network, Toronto, ON M5G 2C4, Canada
| | - Dominique Lenne
- Heudiasyc Lab (Heuristics and Diagnosis of Complex Systems), University of Technology of Compiègne (UMR CNRS 7338), 60200 Compiègne, France
| |
Collapse
|
6
|
Whiston A, Igou ER, Fortune DG, Semkovska M. Examining Stress and Residual Symptoms in Remitted and Partially Remitted Depression Using a Wearable Electrodermal Activity Device: A Pilot Study. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE 2022; 11:96-106. [PMID: 36644642 PMCID: PMC9833495 DOI: 10.1109/jtehm.2022.3228483] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/16/2022] [Revised: 11/06/2022] [Accepted: 11/30/2022] [Indexed: 12/15/2022]
Abstract
Consistent evidence suggests residual symptoms and stress are the most reliable predictors of relapse in remitted depression. Prevailing methodologies often do not enable continuous real-time sampling of stress. Thus, little is known about day-to-day interactions between residual symptoms and stress in remitted depression. In preparation for a full-scale trial, this study aimed to pilot a wrist-worn wearable electrodermal activity monitor: ADI (Analog Devices, Inc.) Study Watch for assessing interactions between physiological stress and residual depressive symptoms following depression remission. 13 individuals remitted from major depression completed baseline, daily diary, and post-daily diary assessments. Self-reported stress and residual symptoms were measured at baseline and post-daily diary. Diary assessments required participants to wear ADI's Study Watch during waking hours and complete self-report questionnaires every evening over one week. Sleep problems, fatigue, energy loss, and agitation were the most frequently reported residual symptoms. Average skin conductance responses (SCRs) were 16.09 per-hour, with an average of 11.30 hours of wear time per-day. Increased residual symptoms were associated with enhanced self-reported stress on the same day. Increased SCRs on one day predicted increased residual symptoms on the next day. This study showed a wearable electrodermal activity device can be recommended for examining stress as a predictor of remitted depression. This study also provides preliminary work on relationships between residual symptoms and stress in remitted depression. Importantly, significant findings from the small sample of this pilot are preliminary with an aim to follow up with a 3-week full-scale study to draw conclusions about psychological processes explored. Clinical and Translational Impact Statemen-ADI's wearable electrodermal activity device enables a continuous measure of physiological stress for identifying its interactions with residual depressive symptoms following remission. This novel procedure is promising for future studies.
Collapse
Affiliation(s)
- Aoife Whiston
- Department of PsychologyUniversity of Limerick Limerick V94 T9PX Ireland
| | - Eric R Igou
- Department of PsychologyUniversity of Limerick Limerick V94 T9PX Ireland
| | - Donal G Fortune
- Department of PsychologyUniversity of Limerick Limerick V94 T9PX Ireland
| | - Maria Semkovska
- Department of PsychologyUniversity of Southern Denmark 5230 Odense Denmark
| |
Collapse
|
7
|
Roos AL, Goetz T, Krannich M, Donker M, Bieleke M, Caltabiano A, Mainhard T. Control, anxiety and test performance: Self-reported and physiological indicators of anxiety as mediators. BRITISH JOURNAL OF EDUCATIONAL PSYCHOLOGY 2022; 93 Suppl 1:72-89. [PMID: 35906734 DOI: 10.1111/bjep.12536] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Revised: 06/29/2022] [Accepted: 07/08/2022] [Indexed: 11/30/2022]
Abstract
BACKGROUND This study investigated the role of different test anxiety components (affective, cognitive, motivational and physiological) as mediators between control and performance as proposed by Pekrun's control-value theory (CVT). While all components were assessed via self-report, the physiological component was additionally assessed via electrodermal activity (EDA). AIMS We examined the relative impact of the self-reported anxiety components and EDA in this mediating mechanism to identify the most relevant assessment(s) (i.e., self-reported anxiety components and/or EDA) for predicting test performance. SAMPLE The study comprised 50 eighth graders. METHODS Data were collected during a mathematics test comprising six task blocks. State self-reports of control and anxiety components along with test performance and other test emotions were collected block-wise (i.e., repeated assessments within students). EDA was continuously recorded. RESULTS Consistent with CVT, intra-individual mediation analysis with multiple mediators revealed that higher control predicted lower anxiety (i.e., all self-reported components). Unexpectedly, higher control was associated with increased EDA. Follow-up analyses taking other test emotions into account suggested this might reflect positive activation. Correlations between EDA and control and self-reported anxiety components differed depending on which test emotion was dominant in each situation. Regarding test performance, only the cognitive component was a significant mediator and thus seems to play a pivotal role in the relationship between control and performance. CONCLUSIONS Distinguishing between anxiety components and including unbiased physiological measures improve our understanding of the mechanisms behind the relationship between test anxiety and performance. Higher physiological arousal may be a sign of anxiety but can also be a sign of positive activation. When aiming to reduce negative effects of anxiety on performance, targeting the cognitive component seems crucial. Implications of these findings for educational and psychological practice are discussed.
Collapse
Affiliation(s)
- Anna-Lena Roos
- University of Applied Sciences and Arts Northwestern Switzerland, FHNW, Olten, Switzerland
| | | | | | | | | | | | - Tim Mainhard
- Education and Child Studies, Leiden University, Leiden, The Netherlands
| |
Collapse
|
8
|
Kyamakya K, Al-Machot F, Haj Mosa A, Bouchachia H, Chedjou JC, Bagula A. Emotion and Stress Recognition Related Sensors and Machine Learning Technologies. SENSORS 2021; 21:s21072273. [PMID: 33804987 PMCID: PMC8037255 DOI: 10.3390/s21072273] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 03/11/2021] [Accepted: 03/15/2021] [Indexed: 12/23/2022]
Affiliation(s)
- Kyandoghere Kyamakya
- Institute for Smart Systems Technologies, Universitaet Klagenfurt, A9020 Klagenfurt, Austria; (A.H.M.); (J.C.C.)
- Correspondence:
| | - Fadi Al-Machot
- Department of Applied Informatics, Universitaet Klagenfurt, 9020 Klagenfurt, Austria;
| | - Ahmad Haj Mosa
- Institute for Smart Systems Technologies, Universitaet Klagenfurt, A9020 Klagenfurt, Austria; (A.H.M.); (J.C.C.)
| | - Hamid Bouchachia
- Machine Intelligence Group, Bournemouth University, Bournemouth BH12 5BB, UK;
| | - Jean Chamberlain Chedjou
- Institute for Smart Systems Technologies, Universitaet Klagenfurt, A9020 Klagenfurt, Austria; (A.H.M.); (J.C.C.)
| | - Antoine Bagula
- ISAT Laboratory, University of the Western Cape, 7535 Bellville, South Africa;
| |
Collapse
|
9
|
|
10
|
Larradet F, Niewiadomski R, Barresi G, Caldwell DG, Mattos LS. Toward Emotion Recognition From Physiological Signals in the Wild: Approaching the Methodological Issues in Real-Life Data Collection. Front Psychol 2020; 11:1111. [PMID: 32760305 PMCID: PMC7374761 DOI: 10.3389/fpsyg.2020.01111] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2019] [Accepted: 04/30/2020] [Indexed: 12/26/2022] Open
Abstract
Emotion, mood, and stress recognition (EMSR) has been studied in laboratory settings for decades. In particular, physiological signals are widely used to detect and classify affective states in lab conditions. However, physiological reactions to emotional stimuli have been found to differ in laboratory and natural settings. Thanks to recent technological progress (e.g., in wearables) the creation of EMSR systems for a large number of consumers during their everyday activities is increasingly possible. Therefore, datasets created in the wild are needed to insure the validity and the exploitability of EMSR models for real-life applications. In this paper, we initially present common techniques used in laboratory settings to induce emotions for the purpose of physiological dataset creation. Next, advantages and challenges of data collection in the wild are discussed. To assess the applicability of existing datasets to real-life applications, we propose a set of categories to guide and compare at a glance different methodologies used by researchers to collect such data. For this purpose, we also introduce a visual tool called Graphical Assessment of Real-life Application-Focused Emotional Dataset (GARAFED). In the last part of the paper, we apply the proposed tool to compare existing physiological datasets for EMSR in the wild and to show possible improvements and future directions of research. We wish for this paper and GARAFED to be used as guidelines for researchers and developers who aim at collecting affect-related data for real-life EMSR-based applications.
Collapse
Affiliation(s)
- Fanny Larradet
- Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Radoslaw Niewiadomski
- Contact Unit, Istituto Italiano di Tecnologia, Genoa, Italy.,Department of Psychology and Cognitive Science, University of Trento, Rovereto, Italy
| | - Giacinto Barresi
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy
| | | | | |
Collapse
|