Drowsiness detection using heart rate variability

Predicting Fatigue-Associated Aberrant Driving Behaviors Using a Dynamic Weighted Moving Average Model With a Long Short-Term Memory Network Based on Heart Rate Variability

OBJECTIVE

This study proposed a moving average (MA) approach to dynamically process heart rate variability (HRV) and developed aberrant driving behavior (ADB) prediction models by using long short-term memory (LSTM) networks.

BACKGROUND

Fatigue-associated ADBs have traffic safety implications. Numerous models to predict such acts based on physiological responses have been developed but are still in embryonic stages.

METHOD

This study recorded the data of 20 commercial bus drivers during their routine tasks on four consecutive days and subsequently asked them to complete questionnaires, including subjective sleep quality, driver behavior questionnaire and the Karolinska Sleepiness Scale. Driving behaviors and corresponding HRV were determined using a navigational mobile application and a wristwatch. The dynamic-weighted MA (DWMA) and exponential-weighted MA were used to process HRV in 5-min intervals. The data were independently separated for training and testing. Models were trained with 10-fold cross-validation strategy, their accuracies were evaluated, and Shapley additive explanation (SHAP) values were used to determine feature importance.

RESULTS

Significant increases in the standard deviation of NN intervals (SDNN), root mean square of successive heartbeat interval differences (RMSSD), and normalized spectrum of high frequency (nHF) were observed in the pre-event stage. The DWMA-based model exhibited the highest accuracy for both driver types (urban: 84.41%; highway: 80.56%). The SDNN, RMSSD, and nHF demonstrated relatively high SHAP values.

CONCLUSION

HRV metrics can serve as indicators of mental fatigue. DWMA-based LSTM could predict the occurrence of the level of fatigue associated with ADBs.

APPLICATION

The established models can be used in realistic driving scenarios.

Cardiac indices of driver fatigue across in-lab and on-road studies

Driver fatigue is a major contributor to road accidents. Therefore, driver assistance systems (DAS) that would monitor drivers' states may contribute to road safety. Such monitoring can potentially be achieved with input from ECG indices (e.g., heart rate). We reviewed the empirical literature on responses of cardiac measures to driver fatigue and on detecting fatigue with cardiac indices and classification algorithms. We used meta-analytical methods to explore the pooled effect sizes of different cardiac indices of fatigue, their heterogeneity, and the consistency of their responses across studies. Our large pool of studies (N = 39) allowed us to stratify the results across on-road and simulator studies. We found that despite the large heterogeneity of the effect sizes between the studies, many indices had significant pooled effect sizes across the studies, and more frequently across the on-road studies. We also found that most indices showed consistent responses across both on-road and simulator studies. Regarding the detection accuracy, we found that even on-road classification could have been as accurate as 70% with only 2-min of data. However, we could only find two on-road studies that employed fatigue classification algorithms. Overall, our findings are encouraging with respect to the prospect of using cardiac measures for detecting driver fatigue. Yet, to fully explore this possibility, there is a need for additional on-road studies that would employ a similar set of cardiac indices and detection algorithms, a unified definition of fatigue, and additional levels of fatigue than the two fatigue vs alert states.

Use of a wearable device to compare subjective and objective fatigue in lung cancer patients and cancer-free controls

PURPOSE

The study evaluates the use of heart rate variability (HRV), a measure of autonomic nervous system (ANS) modulation via wearable smart bands, to objectively assess cancer-related fatigue (CRF) levels. It aims to enhance understanding of fatigue by distinguishing between LF/HF ratios and LF/HF disorder ratios through HRV and photoplethysmography (PPG), identifying them as potential biomarkers.

METHODS

Seventy-one lung cancer patients and 75 non-cancer controls wore smart bands for one week. Fatigue was assessed using Brief Fatigue Inventory, alongside sleep quality and daily interference. HRV parameters were analyzed to compare groups.

RESULTS

Cancer patients showed higher fatigue and interference levels than controls (64.8% vs. 54.7%). Those with mild fatigue had elevated LF/HF disorder ratios during sleep (40% vs. 20%, P = 0.01), similar to those with moderate to severe fatigue (50% vs. 20%, P = 0.01), indicating more significant autonomic dysregulation. Notably, mild fatigue patients had higher mean LF/HF ratios than controls (1.9 ± 1.34 vs. 1.2 ± 0.6, P = 0.01), underscoring the potential of disorder ratios in signaling fatigue severity.

CONCLUSIONS

Utilizing wearable smart bands for HRV-based analysis is feasible for objectively assess CRF levels in cancer patients, especially during sleep. By distinguishing between LF/HF ratios and LF/HF disorder ratios, our findings suggest that wearable technology and detailed HRV analysis offer promising avenues for real-time fatigue monitoring. This approach has the potential to significantly improve cancer care by providing new methods for managing and intervening in CRF, particularly with a focus on autonomic dysregulation as a crucial factor.

Sleepiness should be reinvestigated through the lens of clinical neurophysiology: A mixed expertal and big-data Natural Language Processing approach

Historically, the field of sleep medicine has revolved around electrophysiological tools. However, the use of these tools as a neurophysiological method of investigation seems to be underrepresented today, from both international recommendations and sleep centers, in contrast to behavioral and psychometric tools. The aim of this article is to combine a data-driven approach and neurophysiological and sleep medicine expertise to confirm or refute the hypothesis that neurophysiology has declined in favor of behavioral or self-reported dimensions in sleep medicine for the investigation of sleepiness, despite the use of electrophysiological tools. Using Natural Language Processing methods, we analyzed the abstracts of the 18,370 articles indexed by PubMed containing the terms 'sleepiness' or 'sleepy' in the title, abstract, or keywords. For this purpose, we examined these abstracts using two methods: a lexical network, enabling the identification of concepts (neurophysiological or clinical) related to sleepiness in these articles and their interconnections; furthermore, we analyzed the temporal evolution of these concepts to extract historical trends. These results confirm the hypothesis that neurophysiology has declined in favor of behavioral or self-reported dimensions in sleep medicine for the investigation of sleepiness. In order to bring sleepiness measurements closer to brain functioning and to reintroduce neurophysiology into sleep medicine, we discuss two strategies: the first is reanalyzing electrophysiological signals collected during the standard sleep electrophysiological test; the second takes advantage of the current trend towards dimensional models of sleepiness to situate clinical neurophysiology at the heart of the redefinition of sleepiness.

A systematic review of physiological signals based driver drowsiness detection systems

Driving a vehicle is a complex, multidimensional, and potentially risky activity demanding full mobilization and utilization of physiological and cognitive abilities. Drowsiness, often caused by stress, fatigue, and illness declines cognitive capabilities that affect drivers' capability and cause many accidents. Drowsiness-related road accidents are associated with trauma, physical injuries, and fatalities, and often accompany economic loss. Drowsy-related crashes are most common in young people and night shift workers. Real-time and accurate driver drowsiness detection is necessary to bring down the drowsy driving accident rate. Many researchers endeavored for systems to detect drowsiness using different features related to vehicles, and drivers' behavior, as well as, physiological measures. Keeping in view the rising trend in the use of physiological measures, this study presents a comprehensive and systematic review of the recent techniques to detect driver drowsiness using physiological signals. Different sensors augmented with machine learning are utilized which subsequently yield better results. These techniques are analyzed with respect to several aspects such as data collection sensor, environment consideration like controlled or dynamic, experimental set up like real traffic or driving simulators, etc. Similarly, by investigating the type of sensors involved in experiments, this study discusses the advantages and disadvantages of existing studies and points out the research gaps. Perceptions and conceptions are made to provide future research directions for drowsiness detection techniques based on physiological signals.

Home-Based Dynamics of Sleepiness-Related Conditions Starting at Biological Evening and Later (Beyond Working)

Shift work requires round-the-clock readiness to perform professional duties, and the workers' performance highly depends on their sleepiness level, which can be underestimated during a shift. Various factors, including the time of day, can influence sleepiness in shift workers. The objective of this study was to explore the dynamics of sleepiness-related conditions assessed through heart rate variability analysis, starting from the biological evening and continuing in vivo (at home), without the need for artificial alertness support. The participants solely performed regular evening household duties. A total of 32 recordings were collected from the Subjective Sleepiness Dynamics Dataset for analysis. At 8:00 p.m. and every 30 min thereafter, the participants completed cyclic sleepiness scales (the KSS and the SSS) until the time they went to bed, while their heart rate was recorded. The results of the study indicated that during the biological evening, high sleepiness is associated with a 'stressed' condition characterized by higher sympathetic activation. Later on, it is associated with a 'drowsy' condition characterized by higher parasympathetic activation and a decline in heart rate variability. Our findings provide evidence that the type of condition experienced during high sleepiness depends on the biological time. This should be taken into account when managing work regimes in shift work and developing alertness detectors.

Cancer-related fatigue classification based on heart rate variability signals from wearables

Background

Cancer-related fatigue (CRF) is the most distressing side effect in cancer patients and affects the survival rate. However, most patients do not report their fatigue level. This study is aimed to develop an objective CRF assessment method based on heart rate variability (HRV).

Methods

In this study, patients with lung cancer who received chemotherapy or target therapy were enrolled. Patients wore wearable devices with photoplethysmography that regularly recorded HRV parameters for seven consecutive days and completed the Brief Fatigue Inventory (BFI) questionnaire. The collected parameters were divided into the active and sleep phase parameters to allow tracking of fatigue variation. Statistical analysis was used to identify correlations between fatigue scores and HRV parameters.

Findings

In this study, 60 patients with lung cancer were enrolled. The HRV parameters including the low-frequency/high-frequency (LF/HF) ratio and the LF/HF disorder ratio in the active phase and the sleep phase were extracted. A linear classifier with HRV-based cutoff points achieved correct classification rates of 73 and 88% for mild and moderate fatigue levels, respectively.

Conclusion

Fatigue was effectively identified, and the data were effectively classified using a 24-h HRV device. This objective fatigue monitoring method may enable clinicians to effectively handle fatigue problems.

On the potential of transauricular electrical stimulation to reduce visually induced motion sickness

Perturbations in the autonomic nervous system occur in individuals experiencing increasing levels of motion sickness. Here, we investigated the effects of transauricular electrical stimulation (tES) on autonomic function during visually induced motion sickness, through the analysis of spectral and time-frequency heart rate variability. To determine the efficacy of tES, we compared sham and tES conditions in a randomized, within-subjects, cross-over design in 14 healthy participants. We found that tES reduced motion sickness symptoms by significantly increasing normalized high-frequency (HF) power and decreasing both normalized low-frequency (LF) power and the power ratio of LF and HF components (LF/HF ratio). Furthermore, behavioral data recorded using the motion sickness assessment questionnaire (MSAQ) showed significant differences in decreased symptoms during tES compared to sham condition for the total MSAQ scores and, central and sopite categories of the MSAQ. Our preliminary findings suggest that by administering tES, parasympathetic modulation is increased, and autonomic imbalance induced by motion sickness is restored. This study provides first evidence that tES may have potential as a non-pharmacological neuromodulation tool to keep motion sickness at bay. Thus, these findings may have implications towards protecting people from becoming motion sick and possible accelerated recovery from the malady.

Detecting driver fatigue using heart rate variability: A systematic review

Driver fatigue detection systems have potential to improve road safety by preventing crashes and saving lives. Conventional driver monitoring systems based on driving performance and facial features may be challenged by the application of automated driving systems. This limitation could potentially be overcome by monitoring systems based on physiological measurements. Heart rate variability (HRV) is a physiological marker of interest for detecting driver fatigue that can be measured during real life driving. This systematic review investigates the relationship between HRV measures and driver fatigue, as well as the performance of HRV based fatigue detection systems. With the applied eligibility criteria, 18 articles were identified in this review. Inconsistent results can be found within the studies that investigated differences of HRV measures between alert and fatigued drivers. For studies that developed HRV based fatigue detection systems, the detection performance showed a large variation, where the detection accuracy ranged from 44% to 100%. The inconsistency and variation of the results can be caused by differences in several key aspects in the study designs. Progress in this field is needed to determine the relationship between HRV and different fatigue causal factors and its connection to driver performance. To be deployed, HRV-based fatigue detection systems need to be thoroughly tested in real life conditions with good coverage of relevant driving scenarios and a sufficient number of participants.

Smart Wearables for the Detection of Occupational Physical Fatigue: A Literature Review

Today's world is changing dramatically due to the influence of various factors. Whether due to the rapid development of technological tools, advances in telecommunication methods, global economic and social events, or other reasons, almost everything is changing. As a result, the concepts of a "job" or work have changed as well, with new work shifts being introduced and the office no longer being the only place where work is done. In addition, our non-stop active society has increased the stress and pressure at work, causing fatigue to spread worldwide and becoming a global problem. Moreover, it is medically proven that persistent fatigue is a cause of serious diseases and health problems. Therefore, monitoring and detecting fatigue in the workplace is essential to improve worker safety in the long term. In this paper, we provide an overview of the use of smart wearable devices to monitor and detect occupational physical fatigue. In addition, we present and discuss the challenges that hinder this field and highlight what can be done to advance the use of smart wearables in workplace fatigue detection.

Multi-Level Classification of Driver Drowsiness by Simultaneous Analysis of ECG and Respiration Signals Using Deep Neural Networks

The high number of fatal crashes caused by driver drowsiness highlights the need for developing reliable drowsiness detection methods. An ideal driver drowsiness detection system should estimate multiple levels of drowsiness accurately without intervening in the driving task. This paper proposes a multi-level drowsiness detection system by a deep neural network-based classification system using a combination of electrocardiogram and respiration signals. The proposed method is based on a combination of convolutional neural networks (CNNs) and long short-term memory (LSTM) networks for classifying drowsiness by concurrently using heart rate variability (HRV), power spectral density of HRV, and respiration rate signal as inputs. Two models, a CNN-based model and a hybrid CNN-LSTM-based model were used for multi-level classifications. The performance of the proposed method was evaluated on experimental data collected from 30 subjects in a simulated driving environment. The performance and the results of both models are presented and compared. The best performance for both three-level and five-level drowsiness classifications was achieved by the CNN-LSTM model. The results indicate that the three-level and five-level classifications of drowsiness can be achieved with 91 and 67% accuracy, respectively.

Assessment of Combination of Automated Pupillometry and Heart Rate Variability to Detect Driving Fatigue

Objectives

Approximately 20~30% of all traffic accidents are caused by fatigue driving. However, limited practicability remains a barrier for the real application of available techniques to detect driving fatigue. Use of pupillary light reflex (PLR) may be potentially effective for driving fatigue detection.

Methods

A 90 min monotonous simulated driving task was utilized to induce driving fatigue. During the task, PLR measurements were performed at baseline and at an interval of 30 min. Subjective rating scales, heart rate variability (HRV) were monitored simultaneously.

Results

Thirty-two healthy volunteers in China participated in our study. Based on the results of subjective evaluation and behavioral performances, driving fatigue was verified to be successfully induced by a simulated driving task. Significant variations of PLR and HRV parameters were observed, which also showed significant relevance with the change in Karolinska Sleepiness Scale at several timepoints (|r| = 0.55 ~ 0.72, P < 0.001). Furthermore, PLR variations had excellent ability to detect driving fatigue with high sensitivity and specificity, of which maximum constriction velocity variations achieved a sensitivity of 85.00% and specificity of 72.34% for driving fatigue detection, vs. 82.50 and 78.72% with a combination of HRV variations, a nonsignificant difference (AUC = 0.835, 0.872, P > 0.05).

Conclusions

Pupillary light reflex variation may be a potential indicator in the detection of driving fatigue, achieving a comparative performance compared with the combination with heart rate variability. Further work may be involved in developing a commercialized driving fatigue detection system based on pupillary parameters.

Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera

We demonstrate a Contactless Vital Sign Monitoring (CVSM) system and road-test the system for in-cabin driver monitoring using a near-infrared indirect Time-of-Flight (ToF) camera. The CVSM measures both heart rate (HR) and respiration rate (RR) by leveraging the simultaneously measured grayscale and depth information from a ToF camera. For a camera-based driver monitoring system (DMS), key challenges from varying background illumination and motion-induced artifacts need to be addressed. In this study, active illumination and depth-based motion compensation are used to mitigate these two challenges. For HR measurements, active illumination allows the system to work under various lighting conditions, while our depth-based motion compensation has the advantage of directly measuring the motion of the driver without making prior assumptions about the motion artifacts. In addition, we can extract RR directly from the chest wall motion, circumventing the challenge of acquiring RR from the near-infrared photoplethysmography (PPG) signal of low signal quality. We investigate the system’s performance in various scenarios, including monitoring both drivers and passengers while driving on highways and local roads. Our results show that our CVSM system is ambient light agnostic, and the success rates of HR measurements on the highway are 82% and 71.9% for the passenger and driver, respectively. At the same time, we show that the system can measure RR on users driving on a highway with a mean deviation of −1.4 breaths per minute (BPM). With reliable HR and RR measurement in the vehicle, the CVSM system could one day be a key enabler to sudden sickness or drowsiness detection in DMS.

Electrooculography and Tactile Perception Collaborative Interface for 3D Human-Machine Interaction

The human-machine interface (HMI) previously relied on a single perception interface that cannot realize three-dimensional (3D) interaction and convenient and accurate interaction in multiple scenes. Here, we propose a collaborative interface including electrooculography (EOG) and tactile perception for fast and accurate 3D human-machine interaction. The EOG signals are mainly used for fast, convenient, and contactless 2D (XY-axis) interaction, and the tactile sensing interface is mainly utilized for complex 2D movement control and Z-axis control in the 3D interaction. The honeycomb graphene electrodes for the EOG signal acquisition and tactile sensing array are prepared by a laser-induced process. Two pairs of ultrathin and breathable honeycomb graphene electrodes are attached around the eyes for monitoring nine different eye movements. A machine learning algorithm is designed to train and classify the nine different eye movements with an average prediction accuracy of 92.6%. Furthermore, an ultrathin (90 μm), stretchable (∼1000%), and flexible tactile sensing interface assembled by a pair of 4 × 4 planar electrode arrays is attached to the arm for 2D movement control and Z-axis interaction, which can realize single-point, multipoint and sliding touch functions. Consequently, the tactile sensing interface can achieve eight directions control and even more complex movement trajectory control. Meanwhile, the flexible and ultrathin tactile sensor exhibits an ultrahigh sensitivity of 1.428 kPa^-1 in the pressure range 0-300 Pa with long-term response stability and repeatability. Therefore, the collaboration between EOG and the tactile perception interface will play an important role in rapid and accurate 3D human-machine interaction.

A touch-based multimodal and cryptographic bio-human-machine interface

The awareness of the individuals’ biological status is critical for creating interactive environments. Accordingly, we devised a multimodal cryptographic bio-human–machine interface (CB-HMI), which seamlessly translates touch-based entries into encrypted biochemical, biophysical, and biometric indices (i.e., circulating biomarkers levels, heart rate, oxygen saturation level, and fingerprint pattern). As its central component, the CB-HMI features thin hydrogel-coated chemical sensors and a signal interpretation framework to access/interpret biochemical indices, bypassing the challenge of circulating analyte accessibility and the confounding effect of pressing force variability. Upgrading the surrounding objects with CB-HMI, we demonstrated new interactive solutions for driving safety and medication use, where the integrated CB-HMI uniquely enabled one-touch bioauthentication (based on the user’s biological state/identity), prior to rendering the intended services.

The awareness of individuals’ biological status is critical for creating interactive and adaptive environments that can actively assist the users to achieve optimal outcomes. Accordingly, specialized human–machine interfaces—equipped with bioperception and interpretation capabilities—are required. To this end, we devised a multimodal cryptographic bio-human–machine interface (CB-HMI), which seamlessly translates the user’s touch-based entries into encrypted biochemical, biophysical, and biometric indices. As its central component, the CB-HMI features thin hydrogel-coated chemical sensors and inference algorithms to noninvasively and inconspicuously acquire biochemical indices such as circulating molecules that partition onto the skin (here, ethanol and acetaminophen). Additionally, the CB-HMI hosts physical sensors and associated algorithms to simultaneously acquire the user’s heart rate, blood oxygen level, and fingerprint minutiae pattern. Supported by human subject studies, we demonstrated the CB-HMI’s capability in terms of acquiring physiologically relevant readouts of target bioindices, as well as user-identifying and biometrically encrypting/decrypting these indices in situ (leveraging the fingerprint feature). By upgrading the common surrounding objects with the CB-HMI, we created interactive solutions for driving safety and medication use. Specifically, we demonstrated a vehicle-activation system and a medication-dispensing system, where the integrated CB-HMI uniquely enabled user bioauthentication (on the basis of the user’s biological state and identity) prior to rendering the intended services. Harnessing the levels of bioperception achieved by the CB-HMI and other intelligent HMIs, we can equip our surroundings with a comprehensive and deep awareness of individuals’ psychophysiological state and needs.

Relationship between Subjective and Biological Responses to Comfortable and Uncomfortable Sounds

Various kinds of biological sensors are now embedded in wearable devices and data on human biological information have recently become more widespread. Among various environmental stressors, sound has emotional and biological impacts on humans, and it is worthwhile to investigate the relationship between the subjective impressions of and biological responses to such sounds. In this study, the relationship between subjective and biological responses to acoustic stimuli with two contrasting kinds of sounds, a murmuring river sound and white noise, was investigated. The subjective and biological responses were measured during the presentation of the sounds. Compared with the murmuring river sound, the white noise had a significantly decreased EEG-related index of α-EEG and HRV-related index of SD2/SD1. The correlation between each index of subjective and biological responses indicated that α-EEG was highly correlated with the results of subjective evaluation. However, based on a more detailed analysis with clustering, some subjects showed different biological responses in each trial since they felt the sound was powerful when listening to the murmuring river sound, as well as feeling that it was beautiful. It was suggested that biological responses to sound exposure may be affected by the impression of the sound, which varies by individual.

Study on the nonfatigue and fatigue states of orchard workers based on electrocardiogram signal analysis

In recent years, fatigue has become an important issue in modern life that cannot be ignored, especially in some special occupations. Agricultural workers are high-risk occupations that, under fatigue conditions over a long period, will cause health problems. In China, since very few studies have focused on the fatigue state of agricultural workers, we were interested in using electrocardiogram (ECG) signals to analyze the fatigue state of agricultural workers. Healthy agricultural workers were randomly recruited from hilly orchards in South China. Through the field experiment, 130 groups of 5-min interval ECG signals were collected, and we analyzed the ECG signal by HRV. The time domain (meanHR, meanRR, SDNN, RMSSD, SDSD, PNN20, PNN50 and CV), frequency domain (VLF percent, LF percent, HF percent, LF norm, HF norm and LF/HF) and nonlinear parameters (SD1, SD2, SD1/SD2 and sample entropy) were calculated and Spearman correlation coefficient analysis and Mann-Whitney U tests were performed on each parameter for further analysis. For all subjects, nine parameters were slightly correlated in nonfatigue and fatigue state. Six parameters were significantly increased and ten HRV parameters were significantly decreased compared the nonfatigue state. As for males, fifteen parameters were significantly different, and for females, eighteen parameters were significantly different. In addition, the probability density functions of SDNN, SDSD, VLF%, HFnorm and LF/HF were significantly different in nonfatigue and fatigue state for different genders, and the nonlinear parameters become more discrete compared the nonfatigue state. Finally, we obtained the most suitable parameters, which reflect the fatigue characteristics of orchard workers under different genders. The results have instructional significance for identifying fatigue in orchard workers and provide a convincing and valid reference for clinical diagnosis.

Driver drowsiness estimation using EEG signals with a dynamical encoder-decoder modeling framework

Drowsiness is a leading cause of accidents on the road as it negatively affects the driver’s ability to safely operate a vehicle. Neural activity recorded by EEG electrodes is a widely used physiological correlate of driver drowsiness. This paper presents a novel dynamical modeling solution to estimate the instantaneous level of the driver drowsiness using EEG signals, where the PERcentage of eyelid CLOSure (PERCLOS) is employed as the ground truth of driver drowsiness. Applying our proposed modeling framework, we find neural features present in EEG data that encode PERCLOS. In the decoding phase, we use a Bayesian filtering solution to estimate the PERCLOS level over time. A data set that comprises 18 driving tests, conducted by 13 drivers, has been used to investigate the performance of the proposed framework. The modeling performance in estimation of PERCLOS provides robust and repeatable results in tests with manual and automated driving modes by an average RMSE of 0.117 (at a PERCLOS range of 0 to 1) and average High Probability Density percentage of 62.5%. We further hypothesized that there are biomarkers that encode the PERCLOS across different driving tests and participants. Using this solution, we identified possible biomarkers such as Theta and Delta powers. Results show that about 73% and 66% of the Theta and Delta powers which are selected as biomarkers are increasing as PERCLOS grows during the driving test. We argue that the proposed method is a robust and reliable solution to estimate drowsiness in real-time which opens the door in utilizing EEG-based measures in driver drowsiness detection systems.

Drowsiness detection using portable wireless EEG

BACKGROUND AND OBJECTIVE

The ever-increasing fatality rate due to traffic and workplace accidents, resulting from drowsiness have been a persistent concern during the past years. An efficient technology capable of monitoring and detecting drowsiness can help to alleviate this concern and has potential applications in driver vigilance monitoring, vigilance monitoring in air traffic control rooms and other safety critical work places. In this paper, we present the feasibility of a wearable light weight wireless consumer grade Electroencephalogram (EEG)-based drowsiness detection.

METHODS

A set of informative features were extracted from short daytime nap EEG signals and their applicability in discriminating between alert and drowsy state was studied. We derived an optimal set of EEG features, that give maximum detection rate for the drowsy state. In addition, heart rate was also recorded concurrently with EEG and correlation between heart rate and the EEG features corresponding to drowsiness was also studied.

RESULTS

Using the selected features, the EEG data is shown to be capable of classifying alert and drowsy states with an accuracy of 78.3% using Support Vector Machine classifier employing cross subject validation. The feature selection results also revealed that, the EEG features extracted from the temporal electrodes are more significant for drowsiness detection than the features from frontal electrodes. In addition, EEG features extracted from the temporal electrodes yielded higher correlation coefficient with heart rate, which was in concordance with the feature selection results.

CONCLUSIONS

The results reveal that using the proposed drowsiness detection algorithm, it is possible to perform drowsiness detection using a single EEG electrode placed behind the ear.

Physiological signal-based drowsiness detection using machine learning: Singular and hybrid signal approaches

INTRODUCTION

Drowsiness is one of the main contributors to road-related crashes and fatalities worldwide. To address this pressing global issue, researchers are continuing to develop driver drowsiness detection systems that use a variety of measures. However, most research on drowsiness detection uses approaches based on a singular metric and, as a result, fail to attain satisfactory reliability and validity to be implemented in vehicles.

METHOD

This study examines the utility of drowsiness detection based on singular and a hybrid approach. This approach considered a range of metrics from three physiological signals - electroencephalography (EEG), electrooculography (EOG), and electrocardiography (ECG) - and used subjective sleepiness indices (assessed via the Karolinska Sleepiness Scale) as ground truth. The methodology consisted of signal recording with a psychomotor vigilance test (PVT), pre-processing, extracting, and determining the important features from the physiological signals for drowsiness detection. Finally, four supervised machine learning models were developed based on the subjective sleepiness responses using the extracted physiological features to detect drowsiness levels.

RESULTS

The results illustrate that the singular physiological measures show a specific performance metric pattern, with higher sensitivity and lower specificity or vice versa. In contrast, the hybrid biosignal-based models provide a better performance profile, reducing the disparity between the two metrics.

CONCLUSIONS

The outcome of the study indicates that the selected features provided higher performance in the hybrid approaches than the singular approaches, which could be useful for future research implications. Practical Applications: Use of a hybrid approach seems warranted to improve in-vehicle driver drowsiness detection system. Practical applications will need to consider factors such as intrusiveness, ergonomics, cost-effectiveness, and user-friendliness of any driver drowsiness detection system.

Driver Monitoring of Automated Vehicles by Classification of Driver Drowsiness Using a Deep Convolutional Neural Network Trained by Scalograms of ECG Signals

Driver drowsiness is one of the leading causes of traffic accidents. This paper proposes a new method for classifying driver drowsiness using deep convolution neural networks trained by wavelet scalogram images of electrocardiogram (ECG) signals. Three different classes were defined for drowsiness based on video observation of driving tests performed in a simulator for manual and automated modes. The Bayesian optimization method is employed to optimize the hyperparameters of the designed neural networks, such as the learning rate and the number of neurons in every layer. To assess the results of the deep network method, heart rate variability (HRV) data is derived from the ECG signals, some features are extracted from this data, and finally, random forest and k-nearest neighbors (KNN) classifiers are used as two traditional methods to classify the drowsiness levels. Results show that the trained deep network achieves balanced accuracies of about 77% and 79% in the manual and automated modes, respectively. However, the best obtained balanced accuracies using traditional methods are about 62% and 64%. We conclude that designed deep networks working with wavelet scalogram images of ECG signals significantly outperform KNN and random forest classifiers which are trained on HRV-based features.

Detecting Soldiers' Fatigue Using Eye-Tracking Glasses: Practical Field Applications and Research Opportunities

INTRODUCTION

Objectively determining soldiers' fatigue levels could help prevent injuries or accidents resulting from inattention or decreased alertness. Eye-tracking technologies, such as optical eye tracking (OET) and electrooculography (EOG), are often used to monitor fatigue. Eyeblinks-especially blink frequency and blink duration-are known as easily observable and valid biomarkers of fatigue. Currently, various eye trackers (i.e., eye-tracking glasses) are available on the market using either OET or EOG technologies. These wearable eye trackers offer several advantages, including unobtrusive functionality, practicality, and low costs. However, several challenges and limitations must be considered when implementing these technologies in the field to monitor fatigue levels. This review investigates the feasibility of eye tracking in the field focusing on the practical applications in military operational environments.

MATERIALS AND METHOD

This paper summarizes the existing literature about eyeblink dynamics and available wearable eye-tracking technologies, exposing challenges and limitations, as well as discussing practical recommendations on how to improve the feasibility of eye tracking in the field.

RESULTS

So far, no eye-tracking glasses can be recommended for use in a demanding work environment. First, eyeblink dynamics are influenced by multiple factors; therefore, environments, situations, and individual behavior must be taken into account. Second, the glasses' placement, sunlight, facial or body movements, vibrations, and sweat can drastically decrease measurement accuracy. The placement of the eye cameras for the OET and the placement of the electrodes for the EOG must be chosen consciously, the sampling rate must be minimal 200 Hz, and software and hardware must be robust to resist any factors influencing eye tracking.

CONCLUSION

Monitoring physiological and psychological readiness of soldiers, as well as other civil professionals that face higher risks when their attention is impaired or reduced, is necessary. However, improvements to eye-tracking devices' hardware, calibration method, sampling rate, and algorithm are needed in order to accurately monitor fatigue levels in the field.

Predicting Driver Stress Levels with a Sensor-Equipped Steering Wheel and a Quality-Aware Heart Rate Measurement Algorithm

Unobtrusive monitoring of driver mental states has been regarded as an important element in improving the safety of existing transportation systems. While many solutions exist relying on camera-based systems for e.g., drowsiness detection, these can be sensitive to varying lighting conditions and to driver facial accessories, such as eye/sunglasses. In this work, we evaluate the use of physiological signals derived from sensors embedded directly into the steering wheel. In particular, we are interested in monitoring driver stress levels. To achieve this goal, we first propose a modulation spectral signal representation to reliably extract electrocardiogram (ECG) signals from the steering wheel sensors, thus allowing for heart rate and heart rate variability features to be computed. When input to a simple logistic regression classifier, we show that up to 72% accuracy can be achieved when discriminating between stressful and non-stressful driving conditions. In particular, the proposed modulation spectral signal representation allows for direct quality assessment of the obtained heart rate information, thus can provide additional intelligence to autonomous driver monitoring systems.

Information fusion and multi-classifier system for miner fatigue recognition in plateau environments based on electrocardiography and electromyography signals

BACKGROUND AND OBJECTIVE

Human factors are important contributors to accidents, especially human error induced by fatigue. In this study, field tests and analyses were conducted on physiological indexes extracted from electrocardiography (ECG) and electromyography (EMG) signals in miners working under the extreme conditions of a plateau environment. To provide insights into models for fatigue classification and recognition based on machine learning, multi-modal feature information fusion and miner fatigue identification based on ECG and EMG signals as physiological indicators were studied.

METHODS

Fifty-five miners were randomly selected as field test subjects, and characteristic signals were extracted from 110 groups of ECG and EMG signals as the basic signals for fatigue analysis. We conducted principal component analysis (PCA) and grey relational analysis (GRA) on the measurement indicators. Support vector machine (SVM), random forest (RF) and extreme gradient boosting (XG-Boost) machine learning models were used for fatigue classification based on multi-modal information fusion. The area under the receiver operating characteristic (ROC) curve and the confusion matrix were used to evaluate the performance of the recognition models.

RESULTS

The ECG and EMG signals showed obvious changes with fatigue. The results of fatigue model identification showed that PCA feature fusion was superior to GRA feature fusion for all three machine learning approaches, and XG-Boost achieved the best performance, with a recognition accuracy of 89.47%, a sensitivity and specificity of 100%, and an AUC of 0.90. The SVM model also showed good recognition performance (89.47% accuracy, AUC=0.89). The worst performance was that of the RF model, with a recognition accuracy of only 78.95%.

CONCLUSIONS

This study shows that the physiological indexes of ECG and EMG exhibit obvious, regular changes with fatigue and that it is feasible to use SVM, RF and XG-Boost models for miner fatigue identification. The PCA fusion technique can improve the identification accuracy more than the GRA method. XG-Boost classification yields the best accuracy and robustness. This study can serve as a reference for clinical research on the identification of human fatigue at high altitudes and for the clinical study of acute mountain sickness and human acclimatization to high altitudes.

Trends in Heart-Rate Variability Signal Analysis

Heart rate variability (HRV) is the rate of variability between each heartbeat with respect to time. It is used to analyse the Autonomic Nervous System (ANS), a control system used to modulate the body's unconscious action such as cardiac function, respiration, digestion, blood pressure, urination, and dilation/constriction of the pupil. This review article presents a summary and analysis of various research works that analyzed HRV associated with morbidity, pain, drowsiness, stress and exercise through signal processing and machine learning methods. The points of emphasis with regards to HRV research as well as the gaps associated with processes which can be improved to enhance the quality of the research have been discussed meticulously. Restricting the physiological signals to Electrocardiogram (ECG), Electrodermal activity (EDA), photoplethysmography (PPG), and respiration (RESP) analysis resulted in 25 articles which examined the cause and effect of increased/reduced HRV. Reduced HRV was generally associated with increased morbidity and stress. High HRV normally indicated good health, and in some instances, it could signify clinical events of interest such as drowsiness. Effective analysis of HRV during ambulatory and motion situations such as exercise, video gaming, and driving could have a significant impact toward improving social well-being. Detection of HRV in motion is far from perfect, situations involving exercise or driving reported accuracy as high as 85% and as low as 59%. HRV detection in motion can be improved further by harnessing the advancements in machine learning techniques.

Designing a Glass Mounted Warning System to Prevent Drivers to Fall in Sleep Based on Neck Posture and Blinking Duration

Background:

In this study, an electronic system based on driver's neck position and blinking duration is designed to help prevent car crashed due to driver drowsiness. When a driver falls in sleep his/her head is felled down. Hence, driver's neck posture can be a good sign of sleep which is measured utilizing a two?dimensional accelerator. However, this sign is not enough because he/she may need to look down during a drive and alarming driver by every moving down of head can be annoying.

Methods:

Thus, in this system, we used blinking duration too. When a person is awake, blinks more frequently than when he is drowsy.

Result:

As a result, in this system, blinking is detected using an infrared transceiver and if both conditions, i.e., neck posture and blinking duration are showing signs of sleep mode, driver will be alarmed.

Conclusion:

In this study, it is designed 2D accelerometer and IR sensor based system to measure the driver's neck angle and detect driver's blinking to realize the drowsiness of vehicle drivers and alert them using these signs of drowsiness.

Survey and Synthesis of State of the Art in Driver Monitoring

Road vehicle accidents are mostly due to human errors, and many such accidents could be avoided by continuously monitoring the driver. Driver monitoring (DM) is a topic of growing interest in the automotive industry, and it will remain relevant for all vehicles that are not fully autonomous, and thus for decades for the average vehicle owner. The present paper focuses on the first step of DM, which consists of characterizing the state of the driver. Since DM will be increasingly linked to driving automation (DA), this paper presents a clear view of the role of DM at each of the six SAE levels of DA. This paper surveys the state of the art of DM, and then synthesizes it, providing a unique, structured, polychotomous view of the many characterization techniques of DM. Informed by the survey, the paper characterizes the driver state along the five main dimensions—called here “(sub)states”—of drowsiness, mental workload, distraction, emotions, and under the influence. The polychotomous view of DM is presented through a pair of interlocked tables that relate these states to their indicators (e.g., the eye-blink rate) and the sensors that can access each of these indicators (e.g., a camera). The tables factor in not only the effects linked directly to the driver, but also those linked to the (driven) vehicle and the (driving) environment. They show, at a glance, to concerned researchers, equipment providers, and vehicle manufacturers (1) most of the options they have to implement various forms of advanced DM systems, and (2) fruitful areas for further research and innovation.

Measurement of Cancer-Related Fatigue Based on Heart Rate Variability: Observational Study

BACKGROUND

Cancer-related fatigue is a serious side effect of cancer, and its treatment can disrupt the quality of life of patients. Clinically, the standard method for assessing cancer-related fatigue relies on subjective experience retrieved from patient self-reports, such as the Brief Fatigue Inventory (BFI). However, most patients do not self-report their fatigue levels.

OBJECTIVE

In this study, we aim to develop an objective cancer-related fatigue assessment method to track and monitor fatigue in patients with cancer.

METHODS

In total, 12 patients with lung cancer who were undergoing chemotherapy or targeted therapy were enrolled. We developed frequency-domain parameters of heart rate variability (HRV) and BFI based on a wearable-based HRV measurement system. All patients completed the BFI-Taiwan version questionnaire and wore the device for 7 consecutive days to record HRV parameters such as low frequency (LF), high frequency (HF), and LF-HF ratio (LF-HF). Statistical analysis was used to map the correlation between subjective fatigue and objective data.

RESULTS

A moderate positive correlation was observed between the average LF-HF ratio and BFI in the sleep phase (ρ=0.86). The mapped BFI score derived by the BFI mapping method could approximate the BFI from the patient self-report. The mean absolute error rate between the subjective and objective BFI scores was 3%.

CONCLUSIONS

LF-HF is highly correlated with the cancer-related fatigue experienced by patients with lung cancer undergoing chemotherapy or targeted therapy. Beyond revealing fatigue levels objectively, continuous HRV recordings through the photoplethysmography watch device and the defined parameters (LF-HF) can define the active phase and sleep phase in patients with lung cancer who undergo chemotherapy or targeted chemotherapy, allowing a deduction of their sleep patterns.

A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems

Detecting drowsiness in drivers, especially multi-level drowsiness, is a difficult problem that is often approached using neurophysiological signals as the basis for building a reliable system. In this context, electroencephalogram (EEG) signals are the most important source of data to achieve successful detection. In this paper, we first review EEG signal features used in the literature for a variety of tasks, then we focus on reviewing the applications of EEG features and deep learning approaches in driver drowsiness detection, and finally we discuss the open challenges and opportunities in improving driver drowsiness detection based on EEG. We show that the number of studies on driver drowsiness detection systems has increased in recent years and that future systems need to consider the wide variety of EEG signal features and deep learning approaches to increase the accuracy of detection.

Autoencoder-Based Extrasystole Detection and Modification of RRI Data for Precise Heart Rate Variability Analysis

Heart rate variability, which is the fluctuation of the R-R interval (RRI) in electrocardiograms (ECG), has been widely adopted for autonomous evaluation. Since the HRV features that are extracted from RRI data easily fluctuate when arrhythmia occurs, RRI data with arrhythmia need to be modified appropriately before HRV analysis. In this study, we consider two types of extrasystoles-premature ventricular contraction (PVC) and premature atrial contraction (PAC)-which are types of extrasystoles that occur every day, even in healthy persons who have no cardiovascular diseases. A unified framework for ectopic RRI detection and a modification algorithm that utilizes an autoencoder (AE) type of neural network is proposed. The proposed framework consists of extrasystole occurrence detection from the RRI data and modification, whose targets are PVC and PAC. The RRI data are monitored by means of the AE in real time in the detection phase, and a denoising autoencoder (DAE) modifies the ectopic RRI caused by the detected extrasystole. These are referred to as AE-based extrasystole detection (AED) and DAE-based extrasystole modification (DAEM), respectively. The proposed framework was applied to real RRI data with PVC and PAC. The result showed that AED achieved a sensitivity of 93% and a false positive rate of 0.08 times per hour. The root mean squared error of the modified RRI decreased to 31% in PVC and 73% in PAC from the original RRI data by DAEM. In addition, the proposed framework was validated through application to a clinical epileptic seizure problem, which showed that it correctly suppressed the false positives caused by PVC. Thus, the proposed framework can contribute to realizing accurate HRV-based health monitoring and medical sensing systems.

Psychophysiological data-driven multi-feature information fusion and recognition of miner fatigue in high-altitude and cold areas

Fatigue-induced human error is a leading cause of accidents. The purpose of this exploratory study in China was to perform field tests to measure fatigue psychophysiological parameters, such as electrocardiography (ECG), electromyography (EMG), pulse, blood pressure, reaction time and vital capacity (VC), in miners in high-altitude and cold areas and to perform multi-feature information fusion and fatigue identification. Forty-five miners were randomly selected as subjects for a field test, and feature signals were extracted from 90 psychophysiological features as basic signals for fatigue analysis. Fatigue sensitivity indices were obtained by Pearson correlation analysis, t-test and receiver operating characteristic (ROC) curve performance evaluation. The ECG time-domain, ECG frequency-domain, EMG, VC, systolic blood pressure (SBP), and pulse were significantly different after miner fatigue. The support vector machine (SVM) and random forest (RF) techniques were used to classify and identify fatigue by information fusion and factor combination. The optimal fatigue classification factors were ECG-FD (CV Accuracy = 85.0%) and EMG (CV Accuracy = 90.0%). The optimal combination of factors was ECG-TD + ECG-FD + EMG (CV accuracy = 80.0%). Furthermore, SVM machine learning had a good recognition effect. This study shows that SVM and RF can effectively identify miner fatigue based on fatigue-related factor combinations. ECG-FD and EMG are the best indicators of fatigue, and the best performance and robustness are obtained with three-factor combination classification. This study on miner fatigue identification provides a reference for research on clinical medicine and the identification of human fatigue under high-altitude, cold and low-oxygen conditions.

A Persistent Homology Approach to Heart Rate Variability Analysis With an Application to Sleep-Wake Classification

Persistent homology is a recently developed theory in the field of algebraic topology to study shapes of datasets. It is an effective data analysis tool that is robust to noise and has been widely applied. We demonstrate a general pipeline to apply persistent homology to study time series, particularly the instantaneous heart rate time series for the heart rate variability (HRV) analysis. The first step is capturing the shapes of time series from two different aspects—the persistent homologies and hence persistence diagrams of its sub-level set and Taken's lag map. Second, we propose a systematic and computationally efficient approach to summarize persistence diagrams, which we coined persistence statistics. To demonstrate our proposed method, we apply these tools to the HRV analysis and the sleep-wake, REM-NREM (rapid eyeball movement and non rapid eyeball movement) and sleep-REM-NREM classification problems. The proposed algorithm is evaluated on three different datasets via the cross-database validation scheme. The performance of our approach is better than the state-of-the-art algorithms, and the result is consistent throughout different datasets.

Driver sleepiness detection with deep neural networks using electrophysiological data

OBJECTIVE

The objective of this paper is to present a driver sleepiness detection model based on electrophysiological data and a neural network consisting of Convolutional Neural Networks and a Long Short Term Memory architecture.

APPROACH

The model was developed and evaluated on data from 12 different experiments with 269 drivers and 1187 driving sessions during daytime (low sleepiness condition) and night-time (high sleepiness condition), collected during naturalistic driving conditions on real roads in Sweden or in an advanced moving-base driving simulator. Electrooculographic and electroencephalographic time series data, split up in 16634 2.5-minute data segments was used as input to the deep neural network. This probably constitutes the largest labelled driver sleepiness dataset in the world. The model outputs a binary decision as alert (defined as ≤6 on the Karolinska Sleepiness Scale, KSS) or sleepy (KSS≥8) or a regression output corresponding to KSS ϵ [1-5,6,7,8,9].

MAIN RESULTS

The subject-independent mean absolute error (MAE) was 0.78. Binary classification accuracy for the regression model was 82.6% as compared to 82.0% for a model that was trained specifically for the binary classification task. Data from the eyes were more informative than data from the brain. A combined input improved performance for some models, but the gain was very limited.

SIGNIFICANCE

Improved classification results were achieved with the regression model compared to the classification model. This suggests that the implicit order of the KSS ratings, i.e. the progression from alert to sleepy, provides important information for robust modelling of driver sleepiness, and that class labels should not simply be aggregated into an alert and a sleepy class. Furthermore, the model consistently showed better results than a model trained on manually extracted features based on expert knowledge, indicating that the model can detect sleepiness that is not covered by traditional algorithms.

Sensitivity and specificity of the driver sleepiness detection methods using physiological signals: A systematic review

Driver sleepiness is a major contributor to road crashes. A system that monitors and warns the driver at a certain, critical level of arousal, could aid in reducing sleep-related crashes. To determine how driver sleepiness detection systems perform, a systematic review of the sensitivity and specificity outcomes was performed. In total, 21 studies were located that met inclusion criteria for the review. The range of sensitivity outcomes was between 39.0-98.8 % and between 73.0-98.9 % for specificity outcomes. There was considerable variation in the outcomes of the studies employing only one physiological measure (mono-signal approach), whereas, a poly-signal approach with multiple physiological signals resulted in more consistency with higher outcomes on both sensitivity and specificity metrics. Only six of the 21 studies had both sensitivity and specificity outcomes above 90.0 %, which included mono- and poly-signal approaches. Moreover, increases in the number of features used in the sleepiness detection system did not result in higher sensitivity and specificity outcomes. Overall, there was considerable variability between the studies reviewed, including measures of ground truth, the features employed and the machine learning approach of the systems. A critical need for progressing any system is a revalidation of the system on a new sample of users. These aspects indicate considerable progress is needed with physiological-based driver sleepiness systems before they are at a sufficient standard to be deployed on-road.

Stress level classification using statistical analysis of skin conductance signal while driving

Conventionally, multiple physiological signals are used in the field of stress realization. Although many studies have applied various methods in feature selection and classification, a desirable performance has not yet been achieved. This paper presents a novel method of stress level classification using physiological signals during the real-world driving task. Exploring the most reliable analysis method on a comprehensive physiological signal for stress realization has been commonly investigated in various studies. To obtain a high accuracy approach, a proper classification method should be applied to the most relevant physiological signal. In this study, we evaluate the feasibility and effectiveness of the analysis of variance (ANOVA) classifier learner on the single Galvanic Skin Response (GSR) signal. Three levels of stress are taken into account and two independent features including rising time and amplitude are extracted. These two features are extracted from foot and hand GSR signals in three different scenarios for the sake of training. The result indicates that the foot amplitude feature of the GSR signal solely is a reliable source of stress classification with an accuracy rate of 95.83% by applying the ANOVA approach. Accordingly, this methodology can substantially reduce the necessity of resorting to the high number of sensors and the corresponding computational burden associated with signal analysis. Besides, reducing the number of sensors during the measurement procedure would increase drivers’ safety by reducing the interference between human and measurement devices. In this study, the real data collected by Picard and his co-workers are used, available in the PHYSIONET database.

On Driver Behavior Recognition for Increased Safety: A Roadmap

Advanced Driver-Assistance Systems (ADASs) are used for increasing safety in the automotive domain, yet current ADASs notably operate without taking into account drivers’ states, e.g., whether she/he is emotionally apt to drive. In this paper, we first review the state-of-the-art of emotional and cognitive analysis for ADAS: we consider psychological models, the sensors needed for capturing physiological signals, and the typical algorithms used for human emotion classification. Our investigation highlights a lack of advanced Driver Monitoring Systems (DMSs) for ADASs, which could increase driving quality and security for both drivers and passengers. We then provide our view on a novel perception architecture for driver monitoring, built around the concept of Driver Complex State (DCS). DCS relies on multiple non-obtrusive sensors and Artificial Intelligence (AI) for uncovering the driver state and uses it to implement innovative Human–Machine Interface (HMI) functionalities. This concept will be implemented and validated in the recently EU-funded NextPerception project, which is briefly introduced.

Heart rate variability in healthy young adults exposed to global system for mobile communication (GSM) 900-MHz radiofrequency signal from mobile phones

Given the large number of mobile phone users and the increasing exposure to radiofrequency electromagnetic field (RF-EMF) worldwide, we aimed to study the effect of RF-EMF related to mobile phones on heart rate variability (HRV). Twenty-six healthy young adults participated in two experimental sessions with a double-blind, randomized and counter-balanced crossover design. During each session, participants were exposed for 26 min to a sham or real 900 MHz RF-EMF, generated by a commercial dual-band Global System for Mobile technology (GSM) mobile phone. We recorded an electrocardiogram at rest during the exposure. We evaluated HRV by time- and frequency-domain analysis. Evaluation of time-domain HRV parameters revealed a statistically significant increase of the standard deviation of interbeat intervals (SDNN) during the real exposure. Other time-domain parameters were not affected. Analysis in the frequency-domain demonstrated that total spectral power and low-frequency band (LF) absolute power were significantly increased during exposure (p = .046 and p = .043, respectively). However, other parameters were not affected. In conclusion, it seems that most HRV parameters were not affected by GSM signal exposure in our study. The weak effect observed on HRV frequency-domain is likely to represent a random occurrence rather than a real effect.

Sex Differences in Time-Domain and Frequency-Domain Heart Rate Variability Measures of Fatigued Drivers

The effects of fatigue on a driver's autonomic nervous system (ANS) were investigated through heart rate variability (HRV) measures considering the difference of sex. Electrocardiogram (ECG) data from 18 drivers were recorded during a simulator-based driving experiment. Thirteen short-term HRV measures were extracted through time-domain and frequency-domain methods. First, differences in HRV measures related to mental state (alert or fatigued) were analyzed in all subjects. Then, sex-specific changes between alert and fatigued states were investigated. Finally, sex differences between alert and fatigued states were compared. For all subjects, ten measures showed significant differences (Mann-Whitney U test, p < 0.01) between different mental states. In male and female drivers, eight and four measures, respectively, showed significant differences between different mental states. Six measures showed significant differences between males and females in an alert state, while ten measures showed significant sex differences in a fatigued state. In conclusion, fatigue impacts drivers' ANS activity, and this impact differs by sex; more differences exist between male and female drivers' ANS activity in a fatigued state than in an alert state.

Linear and nonlinear analyses of normal and fatigue heart rate variability signals for miners in high-altitude and cold areas

BACKGROUND AND OBJECTIVE

Fatigue is an important cause of operational errors, and human errors are the main cause of accidents. This study is an exploratory study in China. Field tests were conducted on heart rate variability (HRV) parameters and physiological indicators of fatigue among miners in high-altitude, cold and low-oxygen areas. This paper studies heart activity patterns during work fatigue in miners.

METHODS

Fatigue affects both the sympathetic and parasympathetic nervous systems, and it is expressed as an abnormal pattern of HRV parameters. Thirty miners were selected as subjects for a field test, and HRV was extracted from 60 groups of electrocardiography (ECG) datasets as basic signals for fatigue analysis. Then, we analyzed the HRV signals of the miners using linear (time domain and frequency domain) and nonlinear dynamics (Poincaré plot and sample entropy (SampEn)), and a Pearson's correlation coefficient analysis and t-tests were performed on the measured indices.

RESULTS

The results showed that the time-domain indices (SDNN, RMSSD, SDSD, pNN50, RRn, heart rate (HR), R-wave humps (RH)) and the coefficient of variation (CV)) and the frequency-domain indices (low frequency/high frequency (LF/HF), LFnorm and HFnorm) clearly changed after fatigue. These features were selected using a Poincaré plot, sample entropy, Pearson's correlation coefficient and a t-test for further analysis. The fatigue characteristics and sensitivity parameters of miners in a high-altitude, cold and hypoxic environment were obtained.

CONCLUSIONS

This study provides deep insight into the use of linear and nonlinear fatigue characteristics to effectively and reliably identify miner fatigue. Furthermore, the study provides a reference for clinical studies of acute mountain sickness in high-altitude, cold and hypoxic environments.

Nonintrusive Monitoring of Mental Fatigue Status Using Epidermal Electronic Systems and Machine-Learning Algorithms

Mental fatigue, characterized by subjective feelings of "tiredness" and "lack of energy", can degrade individual performance in a variety of situations, for example, in motor vehicle driving or while performing surgery. Thus, a method for nonintrusive monitoring of mental fatigue status is urgently needed. Recent research shows that physiological signal-based fatigue-classification methods using wearable electronics can be sufficiently accurate; by contrast, rigid, bulky devices constrain the behavior of those wearing them, potentially interfering with test signals. Recently, wearable electronics, such as epidermal electronics systems (EES) and electronic tattoos (E-tattoos), have been developed to meet the requirements for the comfortable measurement of various physiological signals. However, comfortable, effective, and nonintrusive monitoring of mental fatigue levels remains to be fulfilled. In this work, an EES is established to simultaneously detect multiple physiological signals in a comfortable and nonintrusive way. Machine-learning algorithms are employed to determine the mental fatigue levels and a predictive accuracy of up to 89% is achieved based on six different kinds of physiological features using decision tree algorithms. Furthermore, EES with the trained predictive model are applied to monitor in situ human mental fatigue levels when doing several routine research jobs, as well as the effect of relaxation methods in relieving fatigue.

Assessment of the Potential of Wrist-Worn Wearable Sensors for Driver Drowsiness Detection

Drowsy driving imposes a high safety risk. Current systems often use driving behavior parameters for driver drowsiness detection. The continuous driving automation reduces the availability of these parameters, therefore reducing the scope of such methods. Especially, techniques that include physiological measurements seem to be a promising alternative. However, in a dynamic environment such as driving, only non- or minimal intrusive methods are accepted, and vibrations from the roadbed could lead to degraded sensor technology. This work contributes to driver drowsiness detection with a machine learning approach applied solely to physiological data collected from a non-intrusive retrofittable system in the form of a wrist-worn wearable sensor. To check accuracy and feasibility, results are compared with reference data from a medical-grade ECG device. A user study with 30 participants in a high-fidelity driving simulator was conducted. Several machine learning algorithms for binary classification were applied in user-dependent and independent tests. Results provide evidence that the non-intrusive setting achieves a similar accuracy as compared to the medical-grade device, and high accuracies (>92%) could be achieved, especially in a user-dependent scenario. The proposed approach offers new possibilities for human-machine interaction in a car and especially for driver state monitoring in the field of automated driving.

Age-related differences in effects of non-driving related tasks on takeover performance in automated driving

INTRODUCTION

During SAE level 3 automated driving, the driver's role changes from active driver to fallback-ready driver. Drowsiness is one of the factors that may degrade driver's takeover performance. This study aimed to investigate effects of non-driving related tasks (NDRTs) to counter driver's drowsiness with a Level 3 system activated and to improve successive takeover performance in a critical situation. A special focus was placed on age-related differences in the effects.

METHOD

Participants of three age groups (younger, middle-aged, older) drove the Level 3 system implemented in a high-fidelity motion-based driving simulator for about 30 min under three experiment conditions: without NDRT, while watching a video clip, and while switching between watching a video clip and playing a game. The Karolinska Sleepiness Scale and eyeblink duration measured driver drowsiness. At the end of the drive, the drivers had to take over control of the vehicle and manually change the lane to avoid a collision. Reaction time and steering angle variability were measured to evaluate the two aspects of driving performance.

RESULTS

For younger drivers, both single and multiple NDRT engagements countered the development of driver drowsiness during automated driving, and their takeover performance was equivalent to or better than their performance without NDRT engagement. For older drivers, NDRT engagement did not affect the development of drowsiness but degraded takeover performance especially under the multiple NDRT engagement condition. The results for middle-aged drivers fell at an intermediate level between those for younger and older drivers. Practical Applications: The present findings do not support general recommendations of NDRT engagement to counter drowsiness during automated driving. This study is especially relevant to the automotive industry's search for options that will ensure the safest interfaces between human drivers and automation systems.

Heart Rate Variability for Driver Sleepiness Classification in Real Road Driving Conditions

Approximately 20-30% of all road fatalities are related to driver sleepiness. A long-lasting goal in driver state research has therefore been to develop a robust sleepiness detection system. Since the alertness level is reflected in autonomous nervous system activity, it has been suggested that various heart rate variability (HRV) metrics can be used as features for driver sleepiness classification. Since the heart rate is modulated by many different factors, and not just by sleepiness, it is relevant to question the high driver sleepiness classification accuracies that have occasionally been presented in the literature. The main objective of this paper is thus to test how well a sleepiness classification system based on HRV features really is. A unique data set with 86 drivers, obtained while driving on real roads in real traffic, both in alert and sleep deprived conditions, was used to train and test a support vector machine (SVM) classifier. Subjective ratings based on the Karolinska sleepiness scale (KSS) was used as ground truth to divide the data into three classes (alert, somewhat sleepy and severely sleepy). Even though nearly all the 24 investigated HRV metrics showed significant differences between sleepiness levels, the SVM results only reached a mean accuracy of 61 %, with the worst results originating from the severely sleepy cases. In summary, the high classification performance that may arise in studies with high experimental control could not be replicated under realistic driving conditions. Future works should focus on how various confounding factors should be accounted for when using HRV based metrics as input to a driver sleepiness detection system.

Feasibility of smart wearables for driver drowsiness detection and its potential among different age groups

Purpose

Drowsiness is a common cause of severe road accidents. Therefore, numerous drowsiness detection methods were developed and explored in recent years, especially concepts using physiological measurements achieved promising results. Nevertheless, existing systems have some limitations that hinder their use in vehicles. To overcome these limitations, this paper aims to investigate the development of a low-cost, non-invasive drowsiness detection system, using physiological signals obtained from conventional wearable devices.

Design/methodology/approach

Two simulator studies, the first study in a low-level driving simulator (N= 10) to check feasibility and efficiency, and the second study in a high-fidelity driving simulator (N= 30) including two age groups, were conducted. An algorithm was developed to extract features from the heart rate signals and a data set was created by labelling these features according to the identified driver state in the simulator study. Using this data set, binary classifiers were trained and tested using various machine learning algorithms.

Findings

The trained classifiers reached a classification accuracy of 99.9%, which is similar to the results obtained by the studies which used intrusive electrodes to detect ECG. The results revealed that heart rate patterns are sensitive to the drivers’ age, i.e. models trained with data from one age group are not efficient in detecting drowsiness for another age group, suggesting to develop universal driver models with data from different age groups combined with individual driver models.

Originality/value

This work investigated the feasibility of driver drowsiness detection by solely using physiological data from wrist-worn wearable devices, such as smartwatches or fitness trackers that are readily available in the consumer market. It was found that such devices are reliable in drowsiness detection.

Physiological Driver Monitoring Using Capacitively Coupled and Radar Sensors

Unobtrusive monitoring of drivers’ physiological parameters is a topic gaining interest, potentially allowing to improve the performance of safety systems to prevent accidents, as well as to improve the driver’s experience or provide health-related services. In this article, two unobtrusive sensing techniques are evaluated: capacitively coupled sensing of the electrocardiogram and respiration, and radar-based sensing of heartbeat and respiration. A challenge for use of these techniques in vehicles are the vibrations and other disturbances that occur in vehicles to which they are inherently more sensitive than contact-based sensors. In this work, optimized sensor architectures and signal processing techniques are proposed that significantly improve the robustness to artefacts. Experimental results, conducted under real driving conditions on public roads, demonstrate the feasibility of the proposed approach. R peak sensitivities and positive predictivities higher than 98% both in highway and city traffic, heart rate mean absolute error of 1.02 bpm resp. 2.06 bpm in highway and city traffic and individual beat R-R interval 95% percentile error within ±27.3 ms are demonstrated. The radar experimental results show that respiration can be measured while driving and heartbeat can be recovered from vibration noise using an accelerometer-based motion reduction algorithm.