Biosignal analysis to assess mental stress in automatic driving of trucks: palmar perspiration and masseter electromyography

Leveraging Wearable Sensors in Virtual Reality Driving Simulators: A Review of Techniques and Applications

Virtual reality (VR) driving simulators are very promising tools for driver assessment since they provide a controlled and adaptable setting for behavior analysis. At the same time, wearable sensor technology provides a well-suited and valuable approach to evaluating the behavior of drivers and their physiological or psychological state. This review paper investigates the potential of wearable sensors in VR driving simulators. Methods: A literature search was performed on four databases (Scopus, Web of Science, Science Direct, and IEEE Xplore) using appropriate search terms to retrieve scientific articles from a period of eleven years, from 2013 to 2023. Results: After removing duplicates and irrelevant papers, 44 studies were selected for analysis. Some important aspects were extracted and presented: the number of publications per year, countries of publication, the source of publications, study aims, characteristics of the participants, and types of wearable sensors. Moreover, an analysis and discussion of different aspects are provided. To improve car simulators that use virtual reality technologies and boost the effectiveness of particular driver training programs, data from the studies included in this systematic review and those scheduled for the upcoming years may be of interest.

Exploring the impact of human-robot interaction on workers' mental stress in collaborative assembly tasks

Advances in robotics have contributed to the prevalence of human-robot collaboration (HRC). However, working and interacting with collaborative robots in close proximity can be psychologically stressful. Therefore, understanding the impacts of human-robot interaction (HRI) on mental stress is crucial for enhancing workplace well-being. To this end, this study investigated how the HRI factors - presence, complexity, and modality - affect the psychological stress of workers. We employed both the NASA-Task Load Index for subjective assessment and physiological metrics including galvanic skin responses, electromyography, and heart rate for objective evaluation. An experimental setup was implemented in which human operators worked together with a collaborative robot on Lego assembly tasks, using different interaction paradigms including pressing buttons, showing hand gestures, and giving verbal commands. The results revealed that the introduction of interactions during HRC helped reduce mental stress and that complex interactions resulted in higher mental stress than simple interactions. Meanwhile, using hand gestures led to significantly higher mental stress than pressing buttons and verbal commands. The findings provided practical insights for mitigating mental stress in the workplace and promoting wellness in the era of HRC.

Noninvasive characterization of peripheral sympathetic activation across sensory stimuli using a peripheral arterial stiffness index

Introduction: The peripheral arterial stiffness index has been proposed and validated as a noninvasive measure quantifying stimulus intensity based on amplitude changes induced by sympathetic innervation of vascular tone. However, its temporal response characteristics remain unclear, thus hindering continuous and accurate monitoring of the dynamic process of sympathetic activation. This paper presents a study aimed at modeling the transient response of the index across sensory stimuli to characterize the corresponding peripheral sympathetic activation. Methods: The index was measured using a continuous arterial pressure monitor and a pulse oximeter during experiments with local pain and local cooling stimuli designed to elicit different patterns of sympathetic activation. The corresponding response of the index was modeled to clarify its transient response characteristics across stimuli. Results: The constructed transfer function accurately depicted the transient response of the index to local pain and local cooling stimuli (Fit percentage: 78.4% ± 11.00% and 79.92% ± 8.79%). Differences in dead time (1.17 ± 0.67 and 0.99 ± 0.56 s, p = 0.082), peak time (2.89 ± 0.81 and 2.64 ± 0.68 s, p = 0.006), and rise time (1.81 ± 0.50 and 1.65 ± 0.48 s, p = 0.020) revealed different response patterns of the index across stimuli. The index also accurately characterized similar vasomotor velocities at different normalized peak amplitudes (0.19 ± 0.16 and 0.16 ± 0.19 a.u., p = 0.007). Discussion: Our findings flesh out the characterization of peripheral arterial stiffness index responses to different sensory stimuli and demonstrate its validity in characterizing peripheral sympathetic activation. This study valorizes a noninvasive method to characterize peripheral sympathetic activation, with the potential to use this index to continuously and accurately track sympathetic activators.

Decreased nerve conduction velocity may be a predictor of fingertip dexterity and subjective complaints

We examined the causes of decreased fingertip dexterity in elderly individuals with an aim to improve their quality of life by improving their activities of daily living. We calculated nerve conduction velocity, absolute error during force adjustment tasks, and fingertip dexterity test scores for 30 young (21-34 years old) and 30 elderly (60-74 years old) participants to identify age-related changes. We also assessed subjective complaints of pain, motor function, and numbness. Motor nerve (young: 55.8 ± 3.7 m/s; elderly: 52.2 ± 5.0 m/s) and sensory nerve (young: 59.4 ± 3.4 m/s; elderly: 55.5 ± 5.3 m/s) conduction velocities decreased in an age-dependent manner. Moreover, the decrease of motor nerve conduction velocity was associated with decreased fingertip dexterity (objective index), while the decrease of sensory nerve conduction velocity was associated with subjective complaints of pain and motor function (subjective index).

Bispectral Index Alterations and Associations With Autonomic Changes During Hypnosis in Trauma Center Researchers: Formative Evaluation Study

BACKGROUND

Previous work performed by our group demonstrated that intermittent reductions in bispectral index (BIS) values were found during neurofeedback following mindfulness instructions. Hypnosis was induced to enhance reductions in BIS values.

OBJECTIVE

This study aims to assess physiologic relaxation and explore its associations with BIS values using autonomic monitoring.

METHODS

Each session consisted of reading a 4-minute baseline neutral script and playing an 18-minute hypnosis tape to 3 researchers involved in the BIS neurofeedback study. In addition to BIS monitoring, autonomic monitoring was performed, and this included measures of electromyography (EMG), skin temperature, skin conductance, respiratory rate, expired carbon dioxide, and heart rate variability. The resulting data were analyzed using two-tailed t tests, correlation analyses, and multivariate linear regression analyses.

RESULTS

We found that hypnosis was associated with reductions in BIS (P<.001), EMG (P<.001), respiratory rate (P<.001), skin conductance (P=.006), and very low frequency power (P=.04); it was also associated with increases in expired carbon dioxide (P<.001), skin temperature (P=.04), high frequency power (P<.001), and successive heart interbeat interval difference (P=.04) values. Decreased BIS values were associated with reduced EMG measures (R=0.76; P<.001), respiratory rate (R=0.35; P=.004), skin conductance (R=0.57; P<.001), and low frequency power (R=0.32; P=.01) and with increased high frequency power (R=-0.53; P<.001), successive heart interbeat interval difference (R=-0.32; P=.009), and heart interbeat interval SD (R=-0.26; P=.04) values.

CONCLUSIONS

Hypnosis appeared to induce mental and physical relaxation, enhance parasympathetic neural activation, and attenuate sympathetic nervous system activity, changes that were associated with BIS values. Findings from this preliminary formative evaluation suggest that the current hypnosis model may be useful for assessing autonomic physiological associations with changes in BIS values, thus motivating us to proceed with a larger investigation in trauma center nurses and physicians.

Car Driver's Sympathetic Reaction Detection Through Electrodermal Activity and Electrocardiogram Measurements

OBJECTIVE

in this paper we propose a system to detect a subject's sympathetic reaction, which is related to unexpected or challenging events during a car drive.

METHODS

we use the Electrocardiogram (ECG) signal and the Skin Potential Response (SPR) signal, which has several advantages with respect to other Electrodermal (EDA) signals. We record one SPR signal for each hand, and use an algorithm that, selecting the smoother signal, is able to remove motion artifacts. We extract statistical features from the ECG and SPR signals in order to classify signal segments and identify the presence or absence of emotional events via a Supervised Learning Algorithm. The experiments were carried out in a company which specializes in driving simulator equipment, using a motorized platform and a driving simulator. Different subjects were tested with this setup, with different challenging events happening on predetermined locations on the track.

RESULTS

we obtain an Accuracy as high as 79.10% for signal blocks and as high as 91.27% for events.

CONCLUSION

results demonstrate the good performance of the presented system in detecting sympathetic reactions, and the effectiveness of the motion artifact removal procedure.

SIGNIFICANCE

our work demonstrates the possibility to classify the emotional state of the driver, using the ECG and EDA signals and a slightly invasive setup. In particular, the proposed use of SPR and of the motion artifact removal procedure are crucial for the effectiveness of the system.

Evaluation of the effect of a newly developed steering unit with enhanced self-alignment and deadband on mental workload during driving of agricultural tractors

The aim of present study was to investigate the effect of a newly developed steering unit with enhanced self-alignment and deadband on mental workload (MW) during heavy vehicle operation. Fourteen participants performed two tasks consisting of a lane keeping and a double lane shift with two tractors equipped with 1) a conventional and 2) an enhanced steering system. Physiological measurements, i.e., electromyography, electrodermal activity and heart rate were recorded during the tasks. Furthermore, performance measurements and subjective perception of MW were collected. Present study demonstrated that participants perceived the enhanced steering system requiring less mental demands to operate. Participants improved their performance during the lane keeping task and tended to improve in the double lane shift task with the enhanced system. Physiological measurements did not reveal differences between the steering systems. This study highlighted the dissociation of subjective indices of mental workload from physiological indices in driving of heavy vehicles.

Effects of an Unexpected and Expected Event on Older Adults' Autonomic Arousal and Eye Fixations During Autonomous Driving

Driving cessation for some older adults can exacerbate physical, cognitive, and mental health challenges due to loss of independence and social isolation. Fully autonomous vehicles may offer an alternative transport solution, increasing social contact and encouraging independence. However, there are gaps in understanding the impact of older adults’ passive role on safe human–vehicle interaction, and on their well-being. 37 older adults (mean age ± SD = 68.35 ± 8.49 years) participated in an experiment where they experienced fully autonomous journeys consisting of a distinct stop (an unexpected event versus an expected event). The autonomous behavior of the vehicle was achieved using the Wizard of Oz approach. Subjective ratings of trust and reliability, and driver state monitoring including visual attention strategies (fixation duration and count) and physiological arousal (skin conductance and heart rate), were captured during the journeys. Results revealed that subjective trust and reliability ratings were high after journeys for both types of events. During an unexpected stop, overt visual attention was allocated toward the event, whereas during an expected stop, visual attention was directed toward the human–machine interface (HMI) and distributed across the central and peripheral driving environment. Elevated skin conductance level reflecting increased arousal persisted only after the unexpected event. These results suggest that safety-critical events occurring during passive fully automated driving may narrow visual attention and elevate arousal mechanisms. To improve in-vehicle user experience for older adults, a driver state monitoring system could examine such psychophysiological indices to evaluate functional state and well-being. This information could then be used to make informed decisions on vehicle behavior and offer reassurance during elevated arousal during unexpected events.

EEG based Classification of Long-term Stress Using Psychological Labeling

Stress research is a rapidly emerging area in the field of electroencephalography (EEG) signal processing. The use of EEG as an objective measure for cost effective and personalized stress management becomes important in situations like the nonavailability of mental health facilities. In this study, long-term stress was classified with machine learning algorithms using resting state EEG signal recordings. The labeling for the stress and control groups was performed using two currently accepted clinical practices: (i) the perceived stress scale score and (ii) expert evaluation. The frequency domain features were extracted from five-channel EEG recordings in addition to the frontal and temporal alpha and beta asymmetries. The alpha asymmetry was computed from four channels and used as a feature. Feature selection was also performed to identify statistically significant features for both stress and control groups (via t-test). We found that support vector machine was best suited to classify long-term human stress when used with alpha asymmetry as a feature. It was observed that the expert evaluation-based labeling method had improved the classification accuracy by up to 85.20%. Based on these results, it is concluded that alpha asymmetry may be used as a potential bio-marker for stress classification, when labels are assigned using expert evaluation.

The psychobiology of using automated driving systems: A systematic review and integrative model

Using vehicles with engaged automated driving systems (ADS) ('highly automated driving', HAD) will substantially impact on future society's mobility, yet the current understanding of human psychobiology related to HAD is still limited. Hence, we synthesized evidence on the psychobiology of subjects using HAD, informing an integrative model of the psychobiology of HAD, and providing guidance for reporting future research on this topic. We included (non-)randomized studies assessing human peripheral biology markers of in-vehicle-users in real or simulated driving environments, using vehicles with vs. without engaged ADS, published in English until April 2018. We systematically searched Web of Science, SCOPUS, and PubMed. The search consisted of a combination of terms describing HAD and psychobiological parameters. Risk of bias was assessed regarding randomization, blinding, incomplete outcome data, selective outcome reporting, and other potential causes. We extracted data using predefined data fields. Four out of five studies included in this review (N = 194 subjects) reported associations of use of vehicles with vs. without engaged ADS with various psychobiological parameters, including heart rate, respiratory sinus arrhythmia (RSA), indicators of electrodermal activity (EDA), and masseter electromyography (EMG). Heart rate tended to be reduced during HAD along with increased EDA and EMG, with no clear indication for changes in RSA. We cannot exclude substantial risk of bias, among others because the status of engagement of ADS was mostly non-randomized. Yet, findings suggest that HAD goes along with tractable changes in peripheral biology. Informed by the conceptual endophenotype approach (Hellhammer et al., 2018, Psychoneuroendocrinology), we propose the Embodied Driving (EMBODD) model that describes how HAD reshapes vehicle use experience, and highlight how to make future ADS equipped vehicles successful regarding user's health. Based on the review, we suggest reporting guidelines for future research on the psychobiology of HAD.

Portable System for Real-Time Detection of Stress Level

Currently, mental stress is a major problem in our society. It is related to a wide variety of diseases and is mainly caused by daily-life factors. The use of mobile technology for healthcare purposes has dramatically increased during the last few years. In particular, for out-of-lab stress detection, a considerable number of biosignal-based methods and systems have been proposed. However, these approaches have not matured yet into applications that are reliable and useful enough to significantly improve people’s quality of life. Further research is needed. In this paper, we propose a portable system for real-time detection of stress based on multiple biosignals such as electroencephalography, electrocardiography, electromyography, and galvanic skin response. In order to validate our system, we conducted a study using a previously published and well-established methodology. In our study, ten subjects were stressed and then relaxed while their biosignals were simultaneously recorded with the portable system. The results show that our system can classify three levels of stress (stress, relax, and neutral) with a resolution of a few seconds and 86% accuracy. This suggests that the proposed system could have a relevant impact on people’s lives. It can be used to prevent stress episodes in many situations of everyday life such as work, school, and home.

Human-Computer Interaction in Smart Environments

Here, we provide an overview of the content of the Special Issue on “Human-computer interaction in smart environments”. The aim of this Special Issue is to highlight technologies and solutions encompassing the use of mass-market sensors in current and emerging applications for interacting with Smart Environments. Selected papers address this topic by analyzing different interaction modalities, including hand/body gestures, face recognition, gaze/eye tracking, biosignal analysis, speech and activity recognition, and related issues.