Evaluation of an Integrated System of Wearable Physiological Sensors for Stress Monitoring in Working Environments by Using Biological Markers

Spatial-Frequency Characteristics of EEG Associated With the Mental Stress in Human-Machine Systems

Accurate assessment of user mental stress in human-machine system plays a crucial role in ensuring task performance and system safety. However, the underlying neural mechanisms of stress in human-machine tasks and assessment methods based on physiological indicators remain fundamental challenges. In this paper, we employ a virtual unmanned aerial vehicle (UAV) control experiment to explore the reorganization of functional brain network patterns under stress conditions. The results indicate enhanced functional connectivity in the frontal theta band and central beta band, as well as reduced functional connectivity in the left parieto-occipital alpha band, which is associated with increased mental stress. Evaluation of network metrics reveals that decreased global efficiency in the theta and beta bands is linked to elevated stress levels. Subsequently, inspired by the frequency-specific patterns in the stress brain network, a cross-band graph convolutional network (CBGCN) model is constructed for mental stress brain state recognition. The proposed method captures the spatial-frequency topological relationships of cross-band brain networks through multiple branches, with the aim of integrating complex dynamic patterns hidden in the brain network and learning discriminative cognitive features. Experimental results demonstrate that the neuro-inspired CBGCN model improves classification performance and enhances model interpretability. The study suggests that the proposed approach provides a potentially viable solution for recognizing stress states in human-machine system by using EEG signals.

Mental stress recognition on the fly using neuroplasticity spiking neural networks

Mental stress is found to be strongly connected with human cognition and wellbeing. As the complexities of human life increase, the effects of mental stress have impacted human health and cognitive performance across the globe. This highlights the need for effective non-invasive stress detection methods. In this work, we introduce a novel, artificial spiking neural network model called Online Neuroplasticity Spiking Neural Network (O-NSNN) that utilizes a repertoire of learning concepts inspired by the brain to classify mental stress using Electroencephalogram (EEG) data. These models are personalized and tested on EEG data recorded during sessions in which participants listen to different types of audio comments designed to induce acute stress. Our O-NSNN models learn on the fly producing an average accuracy of 90.76% (σ = 2.09) when classifying EEG signals of brain states associated with these audio comments. The brain-inspired nature of the individual models makes them robust and efficient and has the potential to be integrated into wearable technology. Furthermore, this article presents an exploratory analysis of trained O-NSNNs to discover links between perceived and acute mental stress. The O-NSNN algorithm proved to be better for personalized stress recognition in terms of accuracy, efficiency, and model interpretability.

A Study of Brain Function Characteristics of Service Members at High Risk for Accidents in the Military

Military accidents are often associated with stress and depressive psychological conditions among soldiers, and they often fail to adapt to military life. Therefore, this study analyzes whether there are differences in EEG and pulse wave indices between general soldiers and three groups of soldiers who have not adapted to military life and are at risk of accidents. Data collection was carried out using a questionnaire and a device that can measure EEG and pulse waves, and data analysis was performed using SPSS. The results showed that the concentration level and brain activity indices were higher in the general soldiers and the soldiers in the first stage of accident risk. The body stress index was higher for each stage of accident risk, and the physical vitality index was higher for general soldiers. Therefore, it can be seen that soldiers who have not adapted to military life and are at risk of accidents have somewhat lower concentration and brain activity than general soldiers, and have symptoms of stress and lethargy. The results of this study will contribute to reducing human accidents through EEG and pulse wave measurements not only in the military but also in occupations with a high risk of accidents such as construction.

Real-World Driver Stress Recognition and Diagnosis Based on Multimodal Deep Learning and Fuzzy EDAS Approaches

Mental stress is known as a prime factor in road crashes. The devastation of these crashes often results in damage to humans, vehicles, and infrastructure. Likewise, persistent mental stress could lead to the development of mental, cardiovascular, and abdominal disorders. Preceding research in this domain mostly focuses on feature engineering and conventional machine learning approaches. These approaches recognize different levels of stress based on handcrafted features extracted from various modalities including physiological, physical, and contextual data. Acquiring good quality features from these modalities using feature engineering is often a difficult job. Recent developments in the form of deep learning (DL) algorithms have relieved feature engineering by automatically extracting and learning resilient features. This paper proposes different CNN and CNN-LSTSM-based fusion models using physiological signals (SRAD dataset) and multimodal data (AffectiveROAD dataset) for the driver's two and three stress levels. The fuzzy EDAS (evaluation based on distance from average solution) approach is used to evaluate the performance of the proposed models based on different classification metrics (accuracy, recall, precision, F-score, and specificity). Fuzzy EDAS performance estimation shows that the proposed CNN and hybrid CNN-LSTM models achieved the first ranks based on the fusion of BH, E4-Left (E4-L), and E4-Right (E4-R). Results showed the significance of multimodal data for designing an accurate and trustworthy stress recognition diagnosing model for real-world driving conditions. The proposed model can also be used for the diagnosis of the stress level of a subject during other daily life activities.

Electrocardiogram Monitoring Wearable Devices and Artificial-Intelligence-Enabled Diagnostic Capabilities: A Review

Worldwide, population aging and unhealthy lifestyles have increased the incidence of high-risk health conditions such as cardiovascular diseases, sleep apnea, and other conditions. Recently, to facilitate early identification and diagnosis, efforts have been made in the research and development of new wearable devices to make them smaller, more comfortable, more accurate, and increasingly compatible with artificial intelligence technologies. These efforts can pave the way to the longer and continuous health monitoring of different biosignals, including the real-time detection of diseases, thus providing more timely and accurate predictions of health events that can drastically improve the healthcare management of patients. Most recent reviews focus on a specific category of disease, the use of artificial intelligence in 12-lead electrocardiograms, or on wearable technology. However, we present recent advances in the use of electrocardiogram signals acquired with wearable devices or from publicly available databases and the analysis of such signals with artificial intelligence methods to detect and predict diseases. As expected, most of the available research focuses on heart diseases, sleep apnea, and other emerging areas, such as mental stress. From a methodological point of view, although traditional statistical methods and machine learning are still widely used, we observe an increasing use of more advanced deep learning methods, specifically architectures that can handle the complexity of biosignal data. These deep learning methods typically include convolutional and recurrent neural networks. Moreover, when proposing new artificial intelligence methods, we observe that the prevalent choice is to use publicly available databases rather than collecting new data.

The Effect of Previous Exposure on Virtual Reality Induced Public Speaking Anxiety: A Physiological and Behavioral Study

Public speaking anxiety (PSA) is defined as a strong distress when performing a speech in front of an audience, causing impairment in terms of work possibilities and social relationships. Audience behavior and feedback received during a speech are a crucial variable to induce PSA, affecting performance and perception. In this study, two different virtual reality public speaking scenarios were developed to investigate the impact of positive (more assertive) versus negative (more hostile) audience behavior regarding perceived anxiety and physiological arousal during performance. Moreover, the presence of any carry-over effect based on first experiences (positive vs. negative) was investigated by using a within-between design. Both explicit (questionnaires) and implicit physiological measures (heart rate [HR]) were used to assess participants' experience. The results confirmed the influence of audience behavior on perceived anxiety. As expected, negative audience elicited greater anxiety and lower experience pleasantness. More interesting, the first experience influenced the perceived anxiety and arousal during performance, suggesting some sort of priming effect due to the valence of previous experience. In particular, starting with an encouraging feedback scenario did not increase the perceived anxiety and HR in front of a subsequent annoying audience. This modulation did not appear in the group who started with the annoying audience, which clearly reported higher HR and anxiety during the annoying exposure compared with the encouraging audience. These results are discussed considering previous evidence on the effect of feedback on performance. In addition, physiological results are interpreted considering the role of somatic marker theory in human performance.

Peripheral blood flow estimated by laser doppler flowmetry provides additional information about sleep state beyond that provided by pulse rate variability

Pulse rate variability (PRV), derived from Laser Doppler flowmetry (LDF) or photoplethysmography, has recently become widely used for sleep state assessment, although it cannot identify all the sleep stages. Peripheral blood flow (BF), also estimated by LDF, may be modulated by sleep stages; however, few studies have explored its potential for assessing sleep state. Thus, we aimed to investigate whether peripheral BF could provide information about sleep stages, and thus improve sleep state assessment. We performed electrocardiography and simultaneously recorded BF signals by LDF from the right-index finger and ear concha of 45 healthy participants (13 women; mean age, 22.5 ± 3.4 years) during one night of polysomnographic recording. Time- and frequency-domain parameters of peripheral BF, and time-domain, frequency-domain, and non-linear indices of PRV and heart rate variability (HRV) were calculated. Finger-BF parameters in the time and frequency domains provided information about different sleep stages, some of which (such as the difference between N1 and rapid eye movement sleep) were not revealed by finger-PRV. In addition, finger-PRV patterns and HRV patterns were similar for most parameters. Further, both finger- and ear-BF results showed 0.2-0.3 Hz oscillations that varied with sleep stages, with a significant increase in N3, suggesting a modulation of respiration within this frequency band. These results showed that peripheral BF could provide information for different sleep stages, some of which was complementary to the information provided by PRV. Furthermore, the combination of peripheral BF and PRV may be more advantageous than HRV alone in assessing sleep states and related autonomic nervous activity.

Gated Recurrent Unit Network for Psychological Stress Classification Using Electrocardiograms from Wearable Devices

In recent years, research on human psychological stress using wearable devices has gradually attracted attention. However, the physical and psychological differences among individuals and the high cost of data collection are the main challenges for further research on this problem. In this work, our aim is to build a model to detect subjects' psychological stress in different states through electrocardiogram (ECG) signals. Therefore, we design a VR high-altitude experiment to induce psychological stress for the subject to obtain the ECG signal dataset. In the experiment, participants wear smart ECG T-shirts with embedded sensors to complete different tasks so as to record their ECG signals synchronously. Considering the temporal continuity of individual psychological stress, a deep, gated recurrent unit (GRU) neural network is developed to capture the mapping relationship between subjects' ECG signals and stress in different states through heart rate variability features at different moments, so as to build a neural network model from the ECG signal to psychological stress detection. The experimental results show that compared with all comparison methods, our method has the best classification performance on the four stress states of resting, VR scene adaptation, VR task and recovery, and it can be a remote stress monitoring solution for some special industries.

Occupational Stress Monitoring Using Biomarkers and Smartwatches: A Systematic Review

This article presents a systematic review of the literature concerning scientific publications on wrist wearables that can help to identify stress levels. The study is part of a research project aimed at modeling a stress surveillance system and providing coping recommendations. The investigation followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. In total, 38 articles were selected for full reading, and 10 articles were selected owing to their alignment with the study proposal. The types of technologies used in the research stand out amongst our main results after analyzing the articles. It is noteworthy that stress assessments are still based on standardized questionnaires, completed by the participants. The main biomarkers collected by the devices used in the selected works included: heart rate variation, cortisol analysis, skin conductance, body temperature, and blood volume at the wrist. This study concludes that developing a wrist wearable for stress identification using physiological and chemical sensors is challenging but possible and applicable.

User Profiling to Enhance Clinical Assessment and Human-Robot Interaction: A Feasibility Study

Socially Assistive Robots (SARs) are designed to support us in our daily life as a companion, and assistance but also to support the caregivers' work. SARs should show personalized and human-like behavior to improve their acceptance and, consequently, their use. Additionally, they should be trustworthy by caregivers and professionals to be used as support for their work (e.g. objective assessment, decision support tools). In this context the aim of the paper is dual. Firstly, this paper aims to present and discuss the robot behavioral model based on sensing, perception, decision support, and interaction modules. The novel idea behind the proposed model is to extract and use the same multimodal features set for two purposes: (i) to profile the user, so to be used by the caregiver as a decision support tool for the assessment and monitoring of the patient; (ii) to fine-tune the human-robot interaction if they can be correlated to the social cues. Secondly, this paper aims to test in a real environment the proposed model using a SAR robot, namely ASTRO. Particularly, it measures the body posture, the gait cycle, and the handgrip strength during the walking support task. Those collected data were analyzed to assess the clinical profile and to fine-tune the physical interaction. Ten older people (65.2 ± 15.6 years) were enrolled for this study and were asked to walk with ASTRO at their normal speed for 10 m. The obtained results underline a good estimation (p < 0.05) of gait parameters, handgrip strength, and angular excursion of the torso with respect to most used instruments. Additionally, the sensory outputs were combined in the perceptual model to profile the user using non-classical and unsupervised techniques for dimensionality reduction namely T-distributed Stochastic Neighbor Embedding (t-SNE) and non-classic multidimensional scaling (nMDS). Indeed, these methods can group the participants according to their residual walking abilities.

A scoping review on monitoring mental health using smart wearable devices

With the continuous development of the times, social competition is becoming increasingly fierce, people are facing enormous pressure and mental health problems have become common. Long-term and persistent mental health problems can lead to severe mental disorders and even death in individuals. The real-time and accurate prediction of individual mental health has become an effective method to prevent the occurrence of mental health disorders. In recent years, smart wearable devices have been widely used for monitoring mental health and have played an important role. This paper provides a comprehensive review of the application fields, application mechanisms, common signals, common techniques and results of smart wearable devices for the detection of mental health problems, aiming to achieve more efficient and accurate prediction for individual mental health, and to achieve early identification, early prevention and early intervention to provide a reference for improving the level of individual mental health.

Symmetric Convolutional and Adversarial Neural Network Enables Improved Mental Stress Classification from EEG

Electroencephalography (EEG) is widely used for mental stress classification, but effective feature extraction and transfer across subjects remain challenging due to its variability. In this paper, a novel deep neural network combining convolutional neural network (CNN) and adversarial theory, named symmetric deep convolutional adversarial network (SDCAN), is proposed for stress classification based on EEG. The adversarial inference is introduced to automatically capture invariant and discriminative features from raw EEG, which aims to improve the classification accuracy and generalization ability across subjects. Experiments were conducted with 22 human subjects, where each participant's stress was induced by the Trier Social Stress Test paradigm while EEG was collected. Stress states were then calibrated into four or five stages according to the changing trend of salivary cortisol concentration. The results show that the proposed network achieves improved accuracies of 87.62% and 81.45% on the classification of four and five stages, respectively, compared to conventional CNN methods. Euclidean space data alignment approach (EA) was applied and the improved generalization ability of EA-SDCAN across subjects was also validated via the leave-one-subject-out-cross-validation, with the accuracies of four and five stages being 60.52% and 48.17%, respectively. These findings indicate that the proposed SDCAN network is more feasible and effective for classifying the stages of mental stress based on EEG compared with other conventional methods.

A smart computer mouse with biometric sensors for unobtrusive office work-related stress monitoring

Health disorders related to the prolonged exposure to stress are very common among office workers. The need for an automated and unobtrusive method of detecting and monitoring occupational stress is imperative and intensifies in the current conditions, where the pandemic COVID-19 causes changes in the working norms globally. In this study, we present a smart computer mouse with biometric sensors integrated in such a way that its structure and functionality remain unaffected. Photoplethysmography (PPG) signal is collected from user's thumb by a PPG sensor placed on the side wall of the mouse, while galvanic skin response (GSR) is measured from the palm through two electrodes placed on the top surface of the mouse. Biosignals are processed by a microcontroller and can be transferred wirelessly over Wi-Fi connection. Both the sensors and the microcontroller have been placed inside the mouse, enabling its plug and play use, without any additional equipment. The proposed module has been developed as part of a system that infers about the stress levels of office workers, based on their interactions with the computer and its peripheral devices.

MBioTracker: Multimodal Self-Aware Bio-Monitoring Wearable System for Online Workload Detection

Cognitive workload affects operators' performance principally in high-risk or time-demanding situations and when multitasking is required. An online cognitive workload monitoring system can provide valuable inputs to decision-making instances, such as the operator's state of mind and resulting performance. Therefore, it can allow potential adaptive support to the operator. This work presents a new design of a wearable embedded system for online cognitive workload monitoring. This new wearable system consists of, on the hardware side, a multi-channel physiological signals acquisition (respiration cycles, heart rate, skin temperature, and pulse waveform) and a low-power processing platform. Further, on the software side, our wearable embedded system includes a novel energy-aware bio-signal processing algorithm. We also use the concept of application self-awareness to enable energy-scalable embedded machine learning algorithms and methods for online subjects' cognitive workload monitoring. Our results show that this new wearable system can continuously monitor multiple bio-signals, compute their key features, and provide reliable detection of high and low cognitive workload levels with a time resolution of 1 minute and a battery lifetime of 14.58 h in our experimental conditions. It achieves a detection accuracy of 76.6% (2.6% lower than analogous offline computer-based analysis) with a sensitivity of 77.04% and a specificity of 81.75%, on a simulated drone rescue mission task. Moreover, by applying our self-aware monitoring to exploit different energy-scalable modes, we can increase battery lifetime by 51.6% (up to 22.11 hours) while incurring an insignificant accuracy loss of 1.07%.

CAFS: Cost-Aware Features Selection Method for Multimodal Stress Monitoring on Wearable Devices

OBJECTIVE

Today, stress monitoring on wearable devices is challenged by the tension between high-detection accuracy and battery lifetime driven by multimodal data acquisition and processing. Limited research has addressed the classification cost on multimodal wearable sensors, particularly when the features are cost-dependent. Thus, we design a Cost-Aware Feature Selection (CAFS) methodology that trades-off between prediction-power and energy-cost for multimodal stress monitoring.

METHODS

CAFS selects the most important features under different energy-constraints, which allows us to obtain energy-scalable stress monitoring models. We further propose a self-aware stress monitoring method that intelligently switches among the energy-scalable models, reducing energy consumption.

RESULTS

Using CAFS methodology on experimental data and simulation, we reduce the energy-cost of the stress model designed without energy constrains up to 94.37%. We obtain 90.98% and 95.74% as the best accuracy and confidence values, respectively, on unseen data, outperforming state-of-the-art studies. Analyzing our interpretable and energy-scalable models, we showed that simple models that use only heart rate (HR) or skin conductance level (SCL), confidently predict stress for HR >93.30 BPM and non-stress for SCL <6.42S, but, outside these values, a multimodal model using respiration and pulse waves features is needed for confident stress classification. Our self-aware stress monitoring proposal saves10x energy and provides 88.72% of ac-curacy on unseen data.

CONCLUSION

We propose a comprehensive solution for the design of cost-aware stress monitoring addressing the problem of selecting an optimal feature subset considering their cost-dependency and cost-constrains. Significant: Our design framework enables long-term, confident, and accurate stress monitoring on wearable devices.

A Review on Mental Stress Assessment Methods Using EEG Signals

Mental stress is one of the serious factors that lead to many health problems. Scientists and physicians have developed various tools to assess the level of mental stress in its early stages. Several neuroimaging tools have been proposed in the literature to assess mental stress in the workplace. Electroencephalogram (EEG) signal is one important candidate because it contains rich information about mental states and condition. In this paper, we review the existing EEG signal analysis methods on the assessment of mental stress. The review highlights the critical differences between the research findings and argues that variations of the data analysis methods contribute to several contradictory results. The variations in results could be due to various factors including lack of standardized protocol, the brain region of interest, stressor type, experiment duration, proper EEG processing, feature extraction mechanism, and type of classifier. Therefore, the significant part related to mental stress recognition is choosing the most appropriate features. In particular, a complex and diverse range of EEG features, including time-varying, functional, and dynamic brain connections, requires integration of various methods to understand their associations with mental stress. Accordingly, the review suggests fusing the cortical activations with the connectivity network measures and deep learning approaches to improve the accuracy of mental stress level assessment.

Real-Time EEG-Based Cognitive Workload Monitoring on Wearable Devices

OBJECTIVE

Cognitive workload monitoring (CWM) can enhance human-machine interaction by supporting task execution assistance considering the operators cognitive state. Therefore, we propose a machine learning design methodology and a data processing strategy to enable CWM on resource-constrained wearable devices.

METHODS

Our CWM solution is built upon edge computing on a simple wearable system, with only four peripheral channels of electroencephalography (EEG). We assess our solution on experimental data from 24 volunteers. Moreover, to overcome the system's memory constraints, we adopt an optimization strategy for model size reduction and a multi-batch data processing scheme for optimizing RAM memory footprint. Finally, we implement our data processing strategy on a state-of-the-art wearable platform and assess its execution and system battery life.

RESULTS

We achieve an accuracy of 74.5% and a 74.0% geometric mean between sensitivity and specificity for CWM classification on unseen data. Besides, the proposed model optimization strategy generates a 27.5x smaller model compared to the one generated with default parameters, and the multi-batch data processing scheme reduces RAM memory footprint by 14x compared to a single batch data processing. Finally, our algorithm uses only 1.28% of the available processing time, thus, allowing our system to achieve 28.5 hours of battery life.

CONCLUSION

We provide a reliable and optimized CWM solution using wearable devices, enabling more than a day of operation on a single battery charge.

SIGNIFICANCE

The proposed methodology enables real-time data processing on resource-constrained devices and supports real-time wearable monitoring based on EEG for applications as CWM in human-machine interaction.

The Concept of Advanced Multi-Sensor Monitoring of Human Stress

Many people live under stressful conditions which has an adverse effect on their health. Human stress, especially long-term one, can lead to a serious illness. Therefore, monitoring of human stress influence can be very useful. We can monitor stress in strictly controlled laboratory conditions, but it is time-consuming and does not capture reactions, on everyday stressors or in natural environment using wearable sensors, but with limited accuracy. Therefore, we began to analyze the current state of promising wearable stress-meters and the latest advances in the record of related physiological variables. Based on these results, we present the concept of an accurate, reliable and easier to use telemedicine device for long-term monitoring of people in a real life. In our concept, we ratify with two synchronized devices, one on the finger and the second on the chest. The results will be obtained from several physiological variables including electrodermal activity, heart rate and respiration, body temperature, blood pressure and others. All these variables will be measured using a coherent multi-sensors device. Our goal is to show possibilities and trends towards the production of new telemedicine equipment and thus, opening the door to a widespread application of human stress-meters.

Pain and Stress Detection Using Wearable Sensors and Devices-A Review

Pain is a subjective feeling; it is a sensation that every human being must have experienced all their life. Yet, its mechanism and the way to immune to it is still a question to be answered. This review presents the mechanism and correlation of pain and stress, their assessment and detection approach with medical devices and wearable sensors. Various physiological signals (i.e., heart activity, brain activity, muscle activity, electrodermal activity, respiratory, blood volume pulse, skin temperature) and behavioral signals are organized for wearables sensors detection. By reviewing the wearable sensors used in the healthcare domain, we hope to find a way for wearable healthcare-monitoring system to be applied on pain and stress detection. Since pain leads to multiple consequences or symptoms such as muscle tension and depression that are stress related, there is a chance to find a new approach for chronic pain detection using daily life sensors or devices. Then by integrating modern computing techniques, there is a chance to handle pain and stress management issue.

Measurement Device for Stress Level and Vital Sign Based on Sensor Fusion

Objectives

Medical health monitoring generally refers to two important aspects of health, namely, physical and mental health. Physical health can be measured through the basic parameters of normal values of vital signs, while mental health can be known from the prevalence of mental and emotional disorders, such as stress. Currently, the medical devices that are generally used to measure these two aspects of health are still separate, so they are less effective than they might be otherwise. To overcome this problem, we designed and realized a device that can measure stress levels through vital signs of the body, namely, heart rate, oxygen saturation, body temperature, and galvanic skin response (GSR).

Methods

The sensor fusion method is used to process data from multiple sensors, so the output that shows the stress level and health status of vital signs can be more accurate and precise.

Results

Based on the results of testing, this device is able to show the health status of vital signs and stress levels within ±20 seconds, with the accuracies of body temperature measurements, oxygen saturation, and GSR of 97.227%, 99.4%, and 98.6%, respectively.

Conclusions

A device for the measurement of stress levels and vital signs based on sensor fusion has been successfully designed and realized in accordance with the expected functions and specifications.

Wearable Devices for Ambulatory Cardiac Monitoring: JACC State-of-the-Art Review

Ambulatory monitoring devices are enabling a new paradigm of health care by collecting and analyzing long-term data for reliable diagnostics. These devices are becoming increasingly popular for continuous monitoring of cardiac diseases. Recent advancements have enabled solutions that are both affordable and reliable, allowing monitoring of vulnerable populations from the comfort of their homes. They provide early detection of important physiological events, leading to timely alerts for seeking medical attention. In this review, the authors aim to summarize the recent developments in the area of ambulatory and remote monitoring solutions for cardiac diagnostics. The authors cover solutions based on wearable devices, smartphones, and other ambulatory sensors. The authors also present an overview of the limitations of current technologies, their effectiveness, and their adoption in the general population, and discuss some of the recently proposed methods to overcome these challenges. Lastly, we discuss the possibilities opened by this new paradigm, for the future of health care and personalized medicine.

Wireless Sensors for Brain Activity—A Survey

Over the last decade, the area of electroencephalography (EEG) witnessed a progressive move from high-end large measurement devices, relying on accurate construction and providing high sensitivity, to miniature hardware, more specifically wireless wearable EEG devices. While accurate, traditional EEG systems need a complex structure and long periods of application time, unwittingly causing discomfort and distress on the users. Given their size and price, aside from their lower sensitivity and narrower spectrum band(s), wearable EEG devices may be used regularly by individuals for continuous collection of user data from non-medical environments. This allows their usage for diverse, nontraditional, non-medical applications, including cognition, BCI, education, and gaming. Given the reduced need for standardization or accuracy, the area remains a rather incipient one, mostly driven by the emergence of new devices that represent the critical link of the innovation chain. In this context, the aim of this study is to provide a holistic assessment of the consumer-grade EEG devices for cognition, BCI, education, and gaming, based on the existing products, the success of their underlying technologies, as benchmarked by the undertaken studies, and their integration with current applications across the four areas. Beyond establishing a reference point, this review also provides the critical and necessary systematic guidance for non-medical EEG research and development efforts at the start of their investigation.

Car Driver's Sympathetic Reaction Detection Through Electrodermal Activity and Electrocardiogram Measurements

OBJECTIVE

in this paper we propose a system to detect a subject's sympathetic reaction, which is related to unexpected or challenging events during a car drive.

METHODS

we use the Electrocardiogram (ECG) signal and the Skin Potential Response (SPR) signal, which has several advantages with respect to other Electrodermal (EDA) signals. We record one SPR signal for each hand, and use an algorithm that, selecting the smoother signal, is able to remove motion artifacts. We extract statistical features from the ECG and SPR signals in order to classify signal segments and identify the presence or absence of emotional events via a Supervised Learning Algorithm. The experiments were carried out in a company which specializes in driving simulator equipment, using a motorized platform and a driving simulator. Different subjects were tested with this setup, with different challenging events happening on predetermined locations on the track.

RESULTS

we obtain an Accuracy as high as 79.10% for signal blocks and as high as 91.27% for events.

CONCLUSION

results demonstrate the good performance of the presented system in detecting sympathetic reactions, and the effectiveness of the motion artifact removal procedure.

SIGNIFICANCE

our work demonstrates the possibility to classify the emotional state of the driver, using the ECG and EDA signals and a slightly invasive setup. In particular, the proposed use of SPR and of the motion artifact removal procedure are crucial for the effectiveness of the system.

Optimizing Probability Threshold of Convolution Neural Network to Improve HRV-based Acute Stress Detection Performance

As stress is linked to numerous emotional and physical conditions, its timely detection and proper management is important for our health. Convolution neural network (CNN) has been shown to be promising in stress detection because it could automatically capture the discriminant information regarding physiological change from heart rate variability (HRV), usually derived from electrocardiogram (ECG) signals. This study proposed a two-step training method to improve the acute stress detection performance through optimizing the probability threshold of a CNN. The results showed that the average error rate was significantly reduced from 17.3 ± 9.2% to 9.2 ± 5.7% after probability threshold optimization, and the classification results were more balanced between stress and rest data. This study presented a simple method to improve stress detection performance using CNN without additional data, rendering benefits for the practical application of HRV-based stress measurement.

StressFoot: Uncovering the Potential of the Foot for Acute Stress Sensing in Sitting Posture

Stress is a naturally occurring psychological response and identifiable by several body signs. We propose a novel way to discriminate acute stress and relaxation, using movement and posture characteristics of the foot. Based on data collected from 23 participants performing tasks that induced stress and relaxation, we developed several machine learning models to construct the validity of our method. We tested our models in another study with 11 additional participants. The results demonstrated replicability with an overall accuracy of 87%. To also demonstrate external validity, we conducted a field study with 10 participants, performing their usual everyday office tasks over a working day. The results showed substantial robustness. We describe ten significant features in detail to enable an easy replication of our models.

Cost-efficient and Custom Electrode-holder Assembly Infrastructure for EEG Recordings

Mobile electroencephalogram (EEG)-sensing technologies have rapidly progressed and made the access of neuroelectrical brain activity outside the laboratory in everyday life more realistic. However, most existing EEG headsets exhibit a fixed design, whereby its immobile montage in terms of electrode density and coverage inevitably poses a great challenge with applicability and generalizability to the fundamental study and application of the brain-computer interface (BCI). In this study, a cost-efficient, custom EEG-electrode holder infrastructure was designed through the assembly of primary components, including the sensor-positioning ring, inter-ring bridge, and bridge shield. It allows a user to (re)assemble a compact holder grid to accommodate a desired number of electrodes only to the regions of interest of the brain and iteratively adapt it to a given head size for optimal electrode-scalp contact and signal quality. This study empirically demonstrated its easy-to-fabricate nature by a low-end fused deposition modeling (FDM) 3D printer and proved its practicability of capturing event-related potential (ERP) and steady-state visual-evoked potential (SSVEP) signatures over 15 subjects. This paper highlights the possibilities for a cost-efficient electrode-holder assembly infrastructure with replaceable montage, flexibly retrofitted in an unlimited fashion, for an individual for distinctive fundamental EEG studies and BCI applications.

An Empirical Study Comparing Unobtrusive Physiological Sensors for Stress Detection in Computer Work

Several unobtrusive sensors have been tested in studies to capture physiological reactions to stress in workplace settings. Lab studies tend to focus on assessing sensors during a specific computer task, while in situ studies tend to offer a generalized view of sensors' efficacy for workplace stress monitoring, without discriminating different tasks. Given the variation in workplace computer activities, this study investigates the efficacy of unobtrusive sensors for stress measurement across a variety of tasks. We present a comparison of five physiological measurements obtained in a lab experiment, where participants completed six different computer tasks, while we measured their stress levels using a chest-band (ECG, respiration), a wristband (PPG and EDA), and an emerging thermal imaging method (perinasal perspiration). We found that thermal imaging can detect increased stress for most participants across all tasks, while wrist and chest sensors were less generalizable across tasks and participants. We summarize the costs and benefits of each sensor stream, and show how some computer use scenarios present usability and reliability challenges for stress monitoring with certain physiological sensors. We provide recommendations for researchers and system builders for measuring stress with physiological sensors during workplace computer use.

Deep ECG-Respiration Network (DeepER Net) for Recognizing Mental Stress

Unmanaged long-term mental stress in the workplace can lead to serious health problems and reduced productivity. To prevent this, it is important to recognize and relieve mental stress in a timely manner. Here, we propose a novel stress detection algorithm based on end-to-end deep learning using multiple physiological signals, such as electrocardiogram (ECG) and respiration (RESP) signal. To mimic workplace stress in our experiments, we used Stroop and math tasks as stressors, with each stressor being followed by a relaxation task. Herein, we recruited 18 subjects and measured both ECG and RESP signals using Zephyr BioHarness 3.0. After five-fold cross validation, the proposed network performed well, with an average accuracy of 83.9%, an average F1 score of 0.81, and an average area under the receiver operating characteristic (ROC) curve (AUC) of 0.92, demonstrating its superiority over conventional machine learning models. Furthermore, by visualizing the activation of the trained network’s neurons, we found that they were activated by specific ECG and RESP patterns. In conclusion, we successfully validated the feasibility of end-to-end deep learning using multiple physiological signals for recognition of mental stress in the workplace. We believe that this is a promising approach that will help to improve the quality of life of people suffering from long-term work-related mental stress.