Driver Fatigue Detection System Using Electroencephalography Signals Based on Combined Entropy Features

Real-time EEG-based detection of driving fatigue using a novel semi-dry electrode with self-replenishment of conductive fluid

A novel semi-dry electrode that can realize self-replenishment of conductive liquid is proposed in this study. Driving fatigue is detected by extracting the refined composite multiscale fluctuation dispersion entropy (RCMFDE) features in electroencephalogram (EEG) signals collected by this electrode. The results show that the new semi-dry electrode can automatically complete the conductive fluid supplement according to its own humidity conditions, which not only notably improves the effective working time, but also significantly reduces the skin impedance. By comparing with the classical entropy algorithms, the computational speed and the stability of the RCMFDE method are Substantially enhanced.

SFT-Net: A Network for Detecting Fatigue From EEG Signals by Combining 4D Feature Flow and Attention Mechanism

Fatigued driving is a leading cause of traffic accidents, and accurately predicting driver fatigue can significantly reduce their occurrence. However, modern fatigue detection models based on neural networks often face challenges such as poor interpretability and insufficient input feature dimensions. This article proposes a novel Spatial-Frequency-Temporal Network (SFT-Net) method for detecting driver fatigue using electroencephalogram (EEG) data. Our approach integrates EEG signals' spatial, frequency, and temporal information to improve recognition performance. We transform the differential entropy of five frequency bands of EEG signals into a 4D feature tensor to preserve these three types of information. An attention module is then used to recalibrate the spatial and frequency information of each input 4D feature tensor time slice. The output of this module is fed into a depthwise separable convolution (DSC) module, which extracts spatial and frequency features after attention fusion. Finally, long short-term memory (LSTM) is used to extract the temporal dependence of the sequence, and the final features are output through a linear layer. We validate the effectiveness of our model on the SEED-VIG dataset, and experimental results demonstrate that SFT-Net outperforms other popular models for EEG fatigue detection. Interpretability analysis supports the claim that our model has a certain level of interpretability. Our work addresses the challenge of detecting driver fatigue from EEG data and highlights the importance of integrating spatial, frequency, and temporal information.

A Brain-Controlled Quadruped Robot: A Proof-of-Concept Demonstration

Coupling brain-computer interfaces (BCIs) and robotic systems in the future can enable seamless personal assistant systems in everyday life, with the requests that can be performed in a discrete manner, using one's brain activity only. These types of systems might be of a particular interest for people with locked-in syndrome (LIS) or amyotrophic lateral sclerosis (ALS) because they can benefit from communicating with robotic assistants using brain sensing interfaces. In this proof-of-concept work, we explored how a wireless and wearable BCI device can control a quadruped robot-Boston Dynamics' Spot. The device measures the user's electroencephalography (EEG) and electrooculography (EOG) activity of the user from the electrodes embedded in the glasses' frame. The user responds to a series of questions with YES/NO answers by performing a brain-teaser activity of mental calculus. Each question-answer pair has a pre-configured set of actions for Spot. For instance, Spot was prompted to walk across a room, pick up an object, and retrieve it for the user (i.e., bring a bottle of water) when a sequence resolved to a YES response. Our system achieved at a success rate of 83.4%. To the best of our knowledge, this is the first integration of wireless, non-visual-based BCI systems with Spot in the context of personal assistant use cases. While this BCI quadruped robot system is an early prototype, future iterations may embody friendly and intuitive cues similar to regular service dogs. As such, this project aims to pave a path towards future developments in modern day personal assistant robots powered by wireless and wearable BCI systems in everyday living conditions.

Self-Attentive Channel-Connectivity Capsule Network for EEG-Based Driving Fatigue Detection

Deep neural networks have recently been successfully extended to EEG-based driving fatigue detection. Nevertheless, most existing models fail to reveal the intrinsic inter-channel relations that are known to be beneficial for EEG-based classification. Additionally, these models require substantial data for training, which is often impractical due to the high cost of data collection. To simultaneously address these two issues, we propose a Self-Attentive Channel-Connectivity Capsule Network (SACC-CapsNet) for EEG-based driving fatigue detection in this paper. SACC-CapsNet starts with a temporal-channel attention module to investigate the critical temporal information and important channels for driving fatigue detection, refining the input EEG signals. Subsequently, the refined EEG data are transformed into a channel covariance matrix to capture the inter-channel relations, followed by selective kernel attention to extract the highly discriminative channel-connectivity features. Finally, a capsule neural network is employed to effectively learn the relationships between connectivity features, which is more suitable for limited data. To confirm the effectiveness of SACC-CapsNet, we collected 24-channel EEG data from 31 subjects (mean age=23.13±2.68 years, male/female=18/13) in a simulated fatigue driving environment. Extensive experiments were conducted with the acquired data, and the comparison results show that our proposed model outperforms state-of-the-art methods. Additionally, the channel covariance matrix learned from SACC-CapsNet reveals that the frontal pole is most informative for detecting driving fatigue, followed by the parietal and central regions. Intriguingly, the temporal-channel attention module can enhance the significance of these critical regions, and the reconstructed channel covariance matrix generated by the decoder network of SACC-CapsNet can effectively preserve valuable information about them.

EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks

Driver fatigue detection is one of the essential tools to reduce accidents and improve traffic safety. Its main challenge lies in the problem of how to identify the driver's fatigue state accurately. Existing detection methods include yawning and blinking based on facial expressions and physiological signals. Still, lighting and the environment affect the detection results based on facial expressions. In contrast, the electroencephalographic (EEG) signal is a physiological signal that directly responds to the human mental state, thus reducing the impact on the detection results. This paper proposes a log-Mel spectrogram and Convolution Recurrent Neural Network (CRNN) model based on EEG to implement driver fatigue detection. This structure allows the advantages of the different networks to be exploited to overcome the disadvantages of using them individually. The process is as follows: first, the original EEG signal is subjected to a one-dimensional convolution method to achieve a Short Time Fourier Transform (STFT) and passed through a Mel filter bank to obtain a logarithmic Mel spectrogram, and then the resulting logarithmic Mel spectrogram is fed into a fatigue detection model to complete the fatigue detection task for the EEG signals. The fatigue detection model consists of a 6-layer convolutional neural network (CNN), bi-directional recurrent neural networks (Bi-RNNs), and a classifier. In the modeling phase, spectrogram features are transported to the 6-layer CNN to automatically learn high-level features, thereby extracting temporal features in the bi-directional RNN to obtain spectrogram-temporal information. Finally, the alert or fatigue state is obtained by a classifier consisting of a fully connected layer, a ReLU activation function, and a softmax function. Experiments were conducted on publicly available datasets in this study. The results show that the method can accurately distinguish between alert and fatigue states with high stability. In addition, the performance of four existing methods was compared with the results of the proposed method, all of which showed that the proposed method could achieve the best results so far.

Experimental Research on the Influence of Short-Term Noise Exposure on Miners’ Physiology

Coal mine noise affects human physiology, psychology, and behavior. It causes errors at work and increases accidents. In this study, we built a coal mine noise simulation experiment system. The system not only included an experimental environment simulation system and a physiological indicator test system, but it also added a miners’ working simulation system. This paper aimed to study the effect of different short-term (25 min) noise levels (60 dB, 70 dB, 80 dB, 90 dB, and 100 dB) on human physiology (skin conductivity and heart rate). Critical analysis showed that the stronger the noise intensity is, the shorter the contact time it takes for physiological indicators to present significant changes, and by setting different noises and measuring the skin conductivity and heart rate of human body, it was concluded that the noise level should be reduced to 90 dB to reduce accidents of miners.

Bimodal EEG-fNIRS in Neuroergonomics. Current Evidence and Prospects for Future Research

Neuroergonomics focuses on the brain signatures and associated mental states underlying behavior to design human-machine interfaces enhancing performance in the cognitive and physical domains. Brain imaging techniques such as functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) have been considered key methods for achieving this goal. Recent research stresses the value of combining EEG and fNIRS in improving these interface systems' mental state decoding abilities, but little is known about whether these improvements generalize over different paradigms and methodologies, nor about the potentialities for using these systems in the real world. We review 33 studies comparing mental state decoding accuracy between bimodal EEG-fNIRS and unimodal EEG and fNIRS in several subdomains of neuroergonomics. In light of these studies, we also consider the challenges of exploiting wearable versions of these systems in real-world contexts. Overall the studies reviewed suggest that bimodal EEG-fNIRS outperforms unimodal EEG or fNIRS despite major differences in their conceptual and methodological aspects. Much work however remains to be done to reach practical applications of bimodal EEG-fNIRS in naturalistic conditions. We consider these points to identify aspects of bimodal EEG-fNIRS research in which progress is expected or desired.

The Application of Electroencephalogram in Driving Safety: Current Status and Future Prospects

The driver is one of the most important factors in the safety of the transportation system. The driver's perceptual characteristics are closely related to driving behavior, while electroencephalogram (EEG) as the gold standard for evaluating human perception is non-deceptive. It is essential to study driving characteristics by analyzing the driver's brain activity pattern, effectively acquiring driver perceptual characteristics, creating a direct connection between the driver's brain and external devices, and realizing information interchange. This paper first introduces the theories related to EEG, then reviews the applications of EEG in scenarios such as fatigue driving, distracted driving, and emotional driving. The limitations of existing research have been identified and the prospect of EEG application in future brain-computer interface automotive assisted driving systems have been proposed. This review provides guidance for researchers to use EEG to improve driving safety. It also offers valuable suggestions for future research.

A Hybrid Approach for MS Diagnosis Through Nonlinear EEG Descriptors and Metaheuristic Optimized Classification Learning

Multiple sclerosis (MS), a disease of the central nervous system, affects the white matter of the brain. Neurologists interpret magnetic resonance images that are often complicated, time-consuming, and contradictory. Using EEG signals, this disease can be analyzed and diagnosed more accurately. However, it is imperative that MS be diagnosed by specialists using assistive technology. Until now, a few methods have been proposed in this field that are sometimes associated with different challenges in analysis. This paper presents a hybrid approach to MS diagnosis in order to decrease classification error rates. Using the proposed method, EEG descriptors in both the time and frequency domains are analyzed. After the feature extraction stage, a modified version of the ant colony optimization method (m-ACO) was used to select the appropriate subset of features. Then, the support vector machine is used to determine whether the disease exists. A metaheuristic algorithm adjusts the support vector machine's parameters to withstand overfitting challenges. Despite a limited number of input channels, significant classification accuracy has been achieved using wavelet analysis techniques, dividing all five subbands of EEG signals, signal windowing, and extracting efficient features from the data. Additionally, alpha, beta, and gamma bands of the signal are important, and the accuracy, sensitivity, and specificity levels are higher than 98.5%. Compared to similar MS diagnostic methods, the proposed method achieved significantly higher diagnostic accuracy. Application and implementation of this method can be effective in treating neurological diseases, including multiple sclerosis.

Classifying Driving Fatigue by Using EEG Signals

Fatigue driving is one of the main reasons for the occurrence of traffic accidents. Brain-computer interface, as a human-computer interaction method based on EEG signals, can communicate with the outside world and move freely through brain signals without relying on the peripheral neuromuscular system. In this paper, a simulation driving platform composed of driving simulation equipment and driving simulation software is used to simulate the real driving process. The EEG signals of the subjects are collected through simulated driving, and the EEG of five subjects is selected as the training sample, and the remaining one is the subject. As a test sample, perform feature extraction and classification experiments, select any set of normal signals and fatigue signals recorded in the driving fatigue experiment for data analysis, and then study the classification of driver fatigue levels. Experiments have proved that the PSO-H-ELM algorithm has only about 4% advantage compared with the average accuracy of the KNN algorithm and the SVM algorithm. The gap is not as big as expected, but as a new algorithm, it is applied to the detection of fatigue EEG. The two traditional algorithms are indeed more suitable. It shows that the driver fatigue level can be judged by detecting EEG, which will provide a basis for the development of on-board, real-time driving fatigue alarm devices. It will lay the foundation for traffic management departments to intervene in driving fatigue reasonably and provide a reliable basis for minimizing traffic accidents.

Research on Driving Fatigue Alleviation Using Interesting Auditory Stimulation Based on VMD-MMSE

The accurate detection and alleviation of driving fatigue are of great significance to traffic safety. In this study, we tried to apply the modified multi-scale entropy (MMSE) approach, based on variational mode decomposition (VMD), to driving fatigue detection. Firstly, the VMD was used to decompose EEG into multiple intrinsic mode functions (IMFs), then the best IMFs and scale factors were selected using the least square method (LSM). Finally, the MMSE features were extracted. Compared with the traditional sample entropy (SampEn), the VMD-MMSE method can identify the characteristics of driving fatigue more effectively. The VMD-MMSE characteristics combined with a subjective questionnaire (SQ) were used to analyze the change trends of driving fatigue under two driving modes: normal driving mode and interesting auditory stimulation mode. The results show that the interesting auditory stimulation method adopted in this paper can effectively relieve driving fatigue. In addition, the interesting auditory stimulation method, which simply involves playing interesting auditory information on the vehicle-mounted player, can effectively relieve driving fatigue. Compared with traditional driving fatigue-relieving methods, such as sleeping and drinking coffee, this interesting auditory stimulation method can relieve fatigue in real-time when the driver is driving normally.

Monitoring Gait Complexity as an Indicator for Running-Related Injury Risk in Collegiate Cross-Country Runners: A Proof-of-Concept Study

Dynamical systems theory suggests that studying the complexity of biological signals could lead to a single gait metric that reliably predicts risk of running-related injury (RRI). The purposes of this pilot study were to examine center of mass (COM) acceleration complexity at baseline, prior to RRI, and the change between timepoints between collegiate runners who developed RRI during a competitive season and those who remained uninjured, and to determine if complexity at these timepoints was associated with increased odds of RRI. Twenty-two collegiate runners from the same cross-country team wore a waist-mounted triaxial accelerometer (100 Hz) during easy-intensity runs throughout the competitive season. RRIs requiring medical attention were reported via an online survey. Control entropy was used to estimate the complexity of the resultant COM acceleration recorded during each run. Associations between complexity and RRI were assessed using a frequency-matching strategy where uninjured participants were paired with injured participants using complexity from the most time-proximal run prior to RRI. Seven runners sustained an RRI. No significant differences were observed between injured and uninjured groups for baseline complexity (p = 0.364, d = 0.405), pre-injury complexity (p = 0.258, d = 0.581), or change from baseline to pre-injury (p = 0.101, d = 0.963). There were no statistically significant associations found between complexity and RRI risk. Although no significant associations were found, the median effect from the models indicated that an increase in baseline complexity, pre-injury complexity, and change in complexity from baseline each corresponded to an increased odds of sustaining an RRI [baseline: odds ratio (OR) = 1.560, 95% CI = 0.587-4.143, p = 0.372; pre-injury: OR = 1.926, 95% CI: 0.689-5.382, p = 0.211; change from baseline: OR = 1.119; 95% CI: 0.839-1.491, p = 0.445). Despite non-significance and wide confidence intervals that included both positive and negative associations, the point estimates for >98% of the 10,000 frequency-case-control-matched model fits indicated that matching strategy did not influence the directionality of the association estimates between complexity and RRI risk (i.e., odds ratio >1.0). This pilot study demonstrates initial feasibility that additional research may support COM acceleration complexity as a useful single-metric monitoring system for RRI risk during real-world training. Follow-up work should assess longitudinal associations between gait complexity and running-related injury in larger cohorts.

Toward practical driving fatigue detection using three frontal EEG channels: a proof-of-concept study

Objective. Although various driving fatigue detection strategies have been introduced, the limited practicability is still an obstacle for the real application of these technologies. This study is based on the newly proposed non-hair-bearing (NHB) method to achieve practical driving fatigue detection with fewer channels from NHB areas and more efficient electroencephalogram (EEG) features.Approach. EEG data were recorded from 20 healthy subjects (15 males, age = 22.2 ± 3.2 years) in a 90 min simulated driving task using a remote wireless cap. Behaviorally, subjects demonstrated a salient fatigue effect, as reflected by a monotonic increase in reaction time. Using a sliding-window approach, we determined the vigilant and fatigued states at individual level to reduce the inter-subject differences in behavioral impairment and brain activity. Multiple EEG features, including power-spectrum density (PSD), functional connectivity (FC), and entropy, were estimated in a pairwise manner, which were set as input for fatigue classification.Main results. Intriguingly, this data-driven approach showed that the best classification performance was achieved using three EEG channel pairs located in the NHB area. The mixed features of the frontal NHB area lead to the high within-subject detection rate of driving fatigue (92.7% ± 0.92%) with satisfactory generalizability for fatigue classification across different subjects (77.13% ± 0.85%). Moreover, we found the most prominent contributing features were PSD of different frequency bands within the frontal NHB area and FC within the frontal NHB area and between frontal and parietal areas.Significance. In summary, the current work provided objective evidence to support the effectiveness of the NHB method and further improved the performance, thereby moving a step forward towards practical driving fatigue detection in real-world scenarios.

Driver sleepiness detection with deep neural networks using electrophysiological data

OBJECTIVE

The objective of this paper is to present a driver sleepiness detection model based on electrophysiological data and a neural network consisting of Convolutional Neural Networks and a Long Short Term Memory architecture.

APPROACH

The model was developed and evaluated on data from 12 different experiments with 269 drivers and 1187 driving sessions during daytime (low sleepiness condition) and night-time (high sleepiness condition), collected during naturalistic driving conditions on real roads in Sweden or in an advanced moving-base driving simulator. Electrooculographic and electroencephalographic time series data, split up in 16634 2.5-minute data segments was used as input to the deep neural network. This probably constitutes the largest labelled driver sleepiness dataset in the world. The model outputs a binary decision as alert (defined as ≤6 on the Karolinska Sleepiness Scale, KSS) or sleepy (KSS≥8) or a regression output corresponding to KSS ϵ [1-5,6,7,8,9].

MAIN RESULTS

The subject-independent mean absolute error (MAE) was 0.78. Binary classification accuracy for the regression model was 82.6% as compared to 82.0% for a model that was trained specifically for the binary classification task. Data from the eyes were more informative than data from the brain. A combined input improved performance for some models, but the gain was very limited.

SIGNIFICANCE

Improved classification results were achieved with the regression model compared to the classification model. This suggests that the implicit order of the KSS ratings, i.e. the progression from alert to sleepy, provides important information for robust modelling of driver sleepiness, and that class labels should not simply be aggregated into an alert and a sleepy class. Furthermore, the model consistently showed better results than a model trained on manually extracted features based on expert knowledge, indicating that the model can detect sleepiness that is not covered by traditional algorithms.

Drowsiness Detection Based on Intelligent Systems with Nonlinear Features for Optimal Placement of Encephalogram Electrodes on the Cerebral Area

Drowsiness while driving can lead to accidents that are related to the loss of perception during emergencies that harm the health. Among physiological signals, brain waves have been used as informative signals for the analyses of behavioral observations, steering information, and other biosignals during drowsiness. We inspected the machine learning methods for drowsiness detection based on brain signals with varying quantities of information. The results demonstrated that machine learning could be utilized to compensate for a lack of information and to account for individual differences. Cerebral area selection approaches to decide optimal measurement locations could be utilized to minimize the discomfort of participants. Although other statistics could provide additional information in further study, the optimized machine learning method could prevent the dangers of drowsiness while driving by considering a transitional state with nonlinear features. Because brain signals can be altered not only by mental fatigue but also by health status, the optimization analysis of the system hardware and software will be able to increase the power-efficiency and accessibility in acquiring brain waves for health enhancements in daily life.

Feature extraction of EEG signals based on functional data analysis and its application to recognition of driver fatigue state

OBJECTIVE

Our objective is to study how to obtain features which can reflect the continuity and internal dynamic changes of electroencephalography (EEG) signals and study an effective method for fatigued driving state recognition based on the obtained features.

APPROACH

A method of EEG signalfeature extraction based on functional data analysis is proposed. Combined with kernel principal component analysis method, the obtained features are applied to the recognition of driver fatigue state, and a corresponding recognition model of fatigued driving state is constructed.

MAIN RESULTS

The recognition model is tested on the real collected driver fatigue EEG signals by selecting a suitable classifier. The test results show that the proposed driver fatigue state recognition method has good recognition effect, especially on the classifier based on decision tree, with an average accuracy of 99.50%.

SIGNIFICANCE

The extracted features well reflect the continuityand internal dynamic changes of the EEG signals, and it is of great significance and application value to study an effective method of fatigued driver state recognition based on the features.

Detecting Unfavorable Driving States in Electroencephalography Based on a PCA Sample Entropy Feature and Multiple Classification Algorithms

Unfavorable driving states can cause a large number of vehicle crashes and are significant factors in leading to traffic accidents. Hence, the aim of this research is to design a robust system to detect unfavorable driving states based on sample entropy feature analysis and multiple classification algorithms. Multi-channel Electroencephalography (EEG) signals are recorded from 16 participants while performing two types of driving tasks. For the purpose of selecting optimal feature sets for classification, principal component analysis (PCA) is adopted for reducing dimensionality of feature sets. Multiple classification algorithms, namely, K nearest neighbor (KNN), decision tree (DT), support vector machine (SVM) and logistic regression (LR) are employed to improve the accuracy of unfavorable driving state detection. We use 10-fold cross-validation to assess the performance of the proposed systems. It is found that the proposed detection system, based on PCA features and the cubic SVM classification algorithm, shows robustness as it obtains the highest accuracy of 97.81%, sensitivity of 96.93%, specificity of 98.73% and precision of 98.75%. Experimental results show that the system we designed can effectively monitor unfavorable driving states.

Construction and Application of Functional Brain Network Based on Entropy

Functional brain network (FBN) is an intuitive expression of the dynamic neural activity interaction between different neurons, neuron clusters, or cerebral cortex regions. It can characterize the brain network topology and dynamic properties. The method of building an FBN to characterize the features of the brain network accurately and effectively is a challenging subject. Entropy can effectively describe the complexity, non-linearity, and uncertainty of electroencephalogram (EEG) signals. As a relatively new research direction, the research of the FBN construction method based on EEG data of fatigue driving has broad prospects. Therefore, it is of great significance to study the entropy-based FBN construction. We focus on selecting appropriate entropy features to characterize EEG signals and construct an FBN. On the real data set of fatigue driving, FBN models based on different entropies are constructed to identify the state of fatigue driving. Through analyzing network measurement indicators, the experiment shows that the FBN model based on fuzzy entropy can achieve excellent classification recognition rate and good classification stability. In addition, when compared with the other model based on the same data set, our model could obtain a higher accuracy and more stable classification results even if the length of the intercepted EEG signal is different.

Passive Brain-Computer Interfaces for Enhanced Human-Robot Interaction

Brain-computer interfaces (BCIs) have long been seen as control interfaces that translate changes in brain activity, produced either by means of a volitional modulation or in response to an external stimulation. However, recent trends in the BCI and neurofeedback research highlight passive monitoring of a user's brain activity in order to estimate cognitive load, attention level, perceived errors and emotions. Extraction of such higher order information from brain signals is seen as a gateway for facilitation of interaction between humans and intelligent systems. Particularly in the field of robotics, passive BCIs provide a promising channel for prediction of user's cognitive and affective state for development of a user-adaptive interaction. In this paper, we first illustrate the state of the art in passive BCI technology and then provide examples of BCI employment in human-robot interaction (HRI). We finally discuss the prospects and challenges in integration of passive BCIs in socially demanding HRI settings. This work intends to inform HRI community of the opportunities offered by passive BCI systems for enhancement of human-robot interaction while recognizing potential pitfalls.

Design of electroencephalogram authentication access control to smart car

In recent years, with the development of intelligent vehicles, the demand for security will be bigger and bigger. One of the most important solutions is the use of new biometric technology. At present, there are still some areas to be improved on biometric technology. For example, diseases will destroy some biological characteristics, some detection methods are too slow, many detection methods do not need living detection, and so on. Electroencephalogram (EEG) is a new biometric tool for living identification. In this Letter, a kind of identity authentication system based on the EEG signal is presented. The overall goal of this research is to design a new authentication method and develop the corresponding application. Therefore, the authors carried out a series of EEG experiments, and analysed and discussed the experimental results. Based on these results, they build and present an access control system based on the uniqueness of their EEG signals to be capable of authenticating access control to the car. The accuracy of the authentication system is >87.3%.

Partial directed coherence based graph convolutional neural networks for driving fatigue detection

The mental state of a driver can be accurately and reliably evaluated by detecting the driver's electroencephalogram (EEG) signals. However, traditional machine learning and deep learning methods focus on the single electrode feature analysis and ignore the functional connection of the brain. In addition, the recent brain function connection network method needs to manually extract substantial brain network features, which results in cumbersome operation. For this reason, this paper introduces graph convolution combined with brain function connection theory into the study of mental fatigue and proposes a method for driving fatigue detection based on the partial directed coherence graph convolutional neural network (PDC-GCNN), which can analyze the characteristics of single electrodes while automatically extracting the topological features of the brain network. We designed a fatigue driving simulation experiment and collected the EEG signals. In the present work, the PDC method constructs the adjacency matrix to describe the relationship between EEG channels, and the GCNN combines single-electrode local brain area information and brain area connection information to further improve the performance of detecting fatigue states. Based on the features of differential entropy (DE) and power spectral density (PSD), the average recognition accuracy of ten-fold cross validation is 84.32% and 83.84%, respectively. For further experiments on each subject, the average recognition results are 95.24%/5.10% (PSD) and 96.01%/3.81% (DE). This research can be embedded in the vehicle driving fatigue detection system, which has practical application value.

TF Entropy and RFE Based Diagnosis for Centrifugal Pumps Subject to the Limitation of Failure Samples

In practical engineering, the vibration-based fault diagnosis with few failure samples is gaining more and more attention from researchers, since it is generally hard to collect sufficient failure records of centrifugal pumps. In such circumstances, effective feature extraction becomes quite vital, since there may not be enough failure data to train an end-to-end classifier, like the deep neural network (DNN). Among the feature extraction, the entropy combined with signal decomposition algorithms is a powerful choice for fault diagnosis of rotating machinery, where the latter decomposes the non-stationary signal into multiple sequences and the former further measures their nonlinear characteristics. However, the existing entropy generally aims at processing the 1D sequence, which means that it cannot simultaneously extract the fault-related information from both the time and frequency domains. Once the sequence is not strictly stationary (hard to achieve in practices), the useful information will be inevitably lost due to the ignored domain, thus limiting its performance. To solve the above issue, a novel entropy method called time-frequency entropy (TfEn) is proposed to jointly measure the complexity and dynamic changes, by taking into account nonlinear behaviors of sequences from both dimensions of time and frequency, which can still fully extract the intrinsic fault features even if the sequence is not strictly stationary. Successively, in order to eliminate the redundant components and further improve the diagnostic accuracy, recursive feature elimination (RFE) is applied to select the optimal features, which has better interpretability and performance, with the help of the supervised embedding mechanism. To sum up, we propose a novel two-stage method to construct the fault representation for centrifugal pumps, which develops from the TfEn-based feature extraction and RFE-based feature selection. The experimental results using the real vibration data of centrifugal pumps show that, with extremely few failure samples, the proposed method respectively improves the average classification accuracy by 12.95% and 33.27%, compared with the mainstream entropy-based methods and the DNN-based ones, which reveals the advantage of our methodology.

Real-Time System for Driver Fatigue Detection Based on a Recurrent Neuronal Network

In recent years, the rise of car accident fatalities has grown significantly around the world. Hence, road security has become a global concern and a challenging problem that needs to be solved. The deaths caused by road accidents are still increasing and currently viewed as a significant general medical issue. The most recent developments have made in advancing knowledge and scientific capacities of vehicles, enabling them to see and examine street situations to counteract mishaps and secure travelers. Therefore, the analysis of driver's behaviors on the road has become one of the leading research subjects in recent years, particularly drowsiness, as it grants the most elevated factor of mishaps and is the primary source of death on roads. This paper presents a way to analyze and anticipate driver drowsiness by applying a Recurrent Neural Network over a sequence frame driver's face. We used a dataset to shape and approve our model and implemented repetitive neural network architecture multi-layer model-based 3D Convolutional Networks to detect driver drowsiness. After a training session, we obtained a promising accuracy that approaches a 92% acceptance rate, which made it possible to develop a real-time driver monitoring system to reduce road accidents.

Entropy-Based Effect Evaluation of Delineators in Tunnels on Drivers' Gaze Behavior

Driving safety in tunnels has always been an issue of great concern. Establishing delineators to improve drivers’ instantaneous cognition of the surrounding environment in tunnels can effectively enhance driver safety. Through a simulation study, this paper explored how delineators affect drivers’ gaze behavior (including fixation and scanpath) in tunnels. In addition to analyzing typical parameters, such as fixation position and fixation duration in areas of interest (AOIs), by modeling drivers’ switching process as Markov chains and calculating Shannon’s entropy of the fit Markov model, this paper quantified the complexity of individual switching patterns between AOIs under different delineator configurations and with different road alignments. A total of 25 subjects participated in this research. The results show that setting delineators in tunnels can attract drivers’ attention and make them focus on the pavement. When driving in tunnels equipped with delineators, especially tunnels with both wall delineators and pavement delineators, the participants exhibited a smaller transition entropy Ht and stationary entropy Hs, which can greatly reduce drivers’ visual fatigue. Compared with left curve and right curve, participants obtained higher Ht and Hs values in the straight section.

Passive Brain-Computer Interfaces for Enhanced Human-Robot Interaction

Brain-computer interfaces (BCIs) have long been seen as control interfaces that translate changes in brain activity, produced either by means of a volitional modulation or in response to an external stimulation. However, recent trends in the BCI and neurofeedback research highlight passive monitoring of a user's brain activity in order to estimate cognitive load, attention level, perceived errors and emotions. Extraction of such higher order information from brain signals is seen as a gateway for facilitation of interaction between humans and intelligent systems. Particularly in the field of robotics, passive BCIs provide a promising channel for prediction of user's cognitive and affective state for development of a user-adaptive interaction. In this paper, we first illustrate the state of the art in passive BCI technology and then provide examples of BCI employment in human-robot interaction (HRI). We finally discuss the prospects and challenges in integration of passive BCIs in socially demanding HRI settings. This work intends to inform HRI community of the opportunities offered by passive BCI systems for enhancement of human-robot interaction while recognizing potential pitfalls.

An iterative cross-subject negative-unlabeled learning algorithm for quantifying passive fatigue

OBJECTIVE

This paper proposes an iterative negative-unlabeled (NU) learning algorithm for cross-subject detection of passive fatigue from labelled alert (negative) and unlabeled driving EEG data.

APPROACH

Unlike other studies which used manual labeling of the fatigue state, the proposed algorithm (PA) first iteratively uses 29 subjects' alert data and unlabeled driving data to identify the most fatigued block of EEG data in each subject in a cross-subject manner. Subsequently, the PA computes subjects' driving fatigue score. Repeated measures correlations of the score to EEG band powers are then performed.

MAIN RESULTS

The PA yields an averaged accuracy of 93.77%  ±  8.15% across subjects in detecting fatigue, which is significantly better than the various baselines. The fatigue scores obtained are also significantly positively correlated with theta band power and negatively correlated with beta band power that are known to respectively increase and decrease in presence of passive fatigue. There is a strong negative correlation with alpha band power as well.

SIGNIFICANCE

The proposed iterative NU learning algorithm is capable of labelling and quantifying the most fatigued block in a cross-subject manner despite the lack of ground truth in the fatigue levels of unlabeled driving EEG data. Together with the significant correlations with theta, alpha and beta band power, the results show promise in the application of the proposed algorithm to detect fatigue from EEG.

A hybrid model for EEG-based gender recognition

The gender recognition is an important research field to study evidence regarding some personal characteristics in the information and data society. However, some current traditional methods such as vision and sound have been exposed their own security weaknesses. Recently, biometric gender recognition based on Electroencephalography (EEG) signals has been widely used in information safety and medical fields. It is necessary to explore potential of using EEG to present a more robust and accurate result with larger training data based on sophisticated machine learning approaches. In this contribution, we present an automated gender recognition system by a hybrid model based on EEG data of resting state from twenty-eight subjects. These data are useful and handy to get insights into assessing the differences in personal gender. For achieving a good performance and a strong robustness, the system develops a hybrid model of combining random forest and logistic regression, and employs four common entropy measures to analyze the non-stationary EEG signals. Result also suggests that the recognition performance achieve an improved progress with an accuracy of 0.9982 and AUC of 0.9926 based on a nested tenfold cross-validation loop, implying that show a significant potential applicability of the proposed approach and is capable of recognizing personal gender.

A Comprehensive Survey of Driving Monitoring and Assistance Systems

Improving a vehicle driver's performance decreases the damage caused by, and chances of, road accidents. In recent decades, engineers and researchers have proposed several strategies to model and improve driving monitoring and assistance systems (DMAS). This work presents a comprehensive survey of the literature related to driving processes, the main reasons for road accidents, the methods of their early detection, and state-of-the-art strategies developed to assist drivers for a safe and comfortable driving experience. The studies focused on the three main elements of the driving process, viz. driver, vehicle, and driving environment are analytically reviewed in this work, and a comprehensive framework of DMAS, major research areas, and their interaction is explored. A well-designed DMAS improves the driving experience by continuously monitoring the critical parameters associated with the driver, vehicle, and surroundings by acquiring and processing the data obtained from multiple sensors. A discussion on the challenges associated with the current and future DMAS and their potential solutions is also presented.

Abnormal Entropy Modulation of the EEG Signal in Patients With Schizophrenia During the Auditory Paired-Stimulus Paradigm

The complexity change in brain activity in schizophrenia is an interesting topic clinically. Schizophrenia patients exhibit abnormal task-related modulation of complexity, following entropy of electroencephalogram (EEG) analysis. However, complexity modulation in schizophrenia patients during the sensory gating (SG) task, remains unknown. In this study, the classical auditory paired-stimulus paradigm was introduced to investigate SG, and EEG data were recorded from 55 normal controls and 61 schizophrenia patients. Fuzzy entropy (FuzzyEn) was used to explore the complexity of brain activity under the conditions of baseline (BL) and the auditory paired-stimulus paradigm (S1 and S2). Generally, schizophrenia patients showed significantly higher FuzzyEn values in the frontal and occipital regions of interest (ROIs). Relative to the BL condition, the normalized values of FuzzyEn of normal controls were decreased greatly in condition S1 and showed less variance in condition S2. Schizophrenia patients showed a smaller decrease in the normalized values in condition S1. Moreover, schizophrenia patients showed significant diminution in the suppression ratios of FuzzyEn, attributed to the higher FuzzyEn values in condition S1. These results suggested that entropy modulation during the process of sensory information and SG was obvious in normal controls and significantly deficient in schizophrenia patients. Additionally, the FuzzyEn values measured in the frontal ROI were positively correlated with positive scores of Positive and Negative Syndrome Scale (PANSS), indicating that frontal entropy was a potential indicator in evaluating the clinical symptoms. However, negative associations were found between the FuzzyEn values of occipital ROIs and general and total scores of PANSS, likely reflecting the compensation effect in visual processing. Thus, our findings provided a deeper understanding of the deficits in sensory information processing and SG, which contribute to cognitive deficits and symptoms in patients with schizophrenia.

EEG classification of driver mental states by deep learning

Driver fatigue is attracting more and more attention, as it is the main cause of traffic accidents, which bring great harm to society and families. This paper proposes to use deep convolutional neural networks, and deep residual learning, to predict the mental states of drivers from electroencephalography (EEG) signals. Accordingly we have developed two mental state classification models called EEG-Conv and EEG-Conv-R. Tested on intra- and inter-subject, our results show that both models outperform the traditional LSTM- and SVM-based classifiers. Our major findings include (1) Both EEG-Conv and EEG-Conv-R yield very good classification performance for mental state prediction; (2) EEG-Conv-R is more suitable for inter-subject mental state prediction; (3) EEG-Conv-R converges more quickly than EEG-Conv. In summary, our proposed classifiers have better predictive power and are promising for application in practical brain-computer interaction .

Research on Recognition Method of Driving Fatigue State Based on Sample Entropy and Kernel Principal Component Analysis

In view of the nonlinear characteristics of electroencephalography (EEG) signals collected in the driving fatigue state recognition research and the issue that the recognition accuracy of the driving fatigue state recognition method based on EEG is still unsatisfactory, this paper proposes a driving fatigue recognition method based on sample entropy (SE) and kernel principal component analysis (KPCA), which combines the advantage of the high recognition accuracy of sample entropy and the advantages of KPCA in dimensionality reduction for nonlinear principal components and the strong non-linear processing capability. By using support vector machine (SVM) classifier, the proposed method (called SE_KPCA) is tested on the EEG data, and compared with those based on fuzzy entropy (FE), combination entropy (CE), three kinds of entropies including SE, FE and CE that merged with KPCA. Experiment results show that the method is effective.

Identifying the Causes of Drivers' Hazardous States Using Driver Characteristics, Vehicle Kinematics, and Physiological Measurements

Drivers’ hazardous physical and mental states (e.g., distraction, fatigue, stress, and high workload) have a major effect on driving performance and strongly contribute to 25–50% of all traffic accidents. They are caused by numerous factors, such as cell phone use or lack of sleep. However, while significant research has been done on detecting hazardous states, most studies have not tried to identify the causes of the hazardous states. Such information would be very useful, as it would allow intelligent vehicles to better respond to a detected hazardous state. Thus, this study examined whether the cause of a driver’s hazardous state can be automatically identified using a combination of driver characteristics, vehicle kinematics, and physiological measures. Twenty-one healthy participants took part in four 45-min sessions of simulated driving, of which they were mildly sleep-deprived for two sessions. Within each session, there were eight different scenarios with different weather (sunny or snowy), traffic density and cell phone usage (with or without cell phone). During each scenario, four physiological (respiration, electrocardiogram, skin conductance, and body temperature) and eight vehicle kinematics measures were monitored. Additionally, three self-reported driver characteristics were obtained: personality, stress level, and mood. Three feature sets were formed based on driver characteristics, vehicle kinematics, and physiological signals. All possible combinations of the three feature sets were used to classify sleep deprivation (drowsy vs. alert), traffic density (low vs. high), cell phone use, and weather conditions (foggy/snowy vs. sunny) with highest accuracies of 98.8%, 91.4%, 82.3%, and 71.5%, respectively. Vehicle kinematics were most useful for classification of weather and traffic density while physiology and driver characteristics were useful for classification of sleep deprivation and cell phone use. Furthermore, a second classification scheme was tested that also incorporates information about whether or not other causes of hazardous states are present, though this did not result in higher classification accuracy. In the future, these classifiers could be used to identify both the presence and cause of a driver’s hazardous state, which could serve as the basis for more intelligent intervention systems.

Identification of Auditory Object-Specific Attention from Single-Trial Electroencephalogram Signals via Entropy Measures and Machine Learning

Existing research has revealed that auditory attention can be tracked from ongoing electroencephalography (EEG) signals. The aim of this novel study was to investigate the identification of peoples’ attention to a specific auditory object from single-trial EEG signals via entropy measures and machine learning. Approximate entropy (ApEn), sample entropy (SampEn), composite multiscale entropy (CmpMSE) and fuzzy entropy (FuzzyEn) were used to extract the informative features of EEG signals under three kinds of auditory object-specific attention (Rest, Auditory Object1 Attention (AOA1) and Auditory Object2 Attention (AOA2)). The linear discriminant analysis and support vector machine (SVM), were used to construct two auditory attention classifiers. The statistical results of entropy measures indicated that there were significant differences in the values of ApEn, SampEn, CmpMSE and FuzzyEn between Rest, AOA1 and AOA2. For the SVM-based auditory attention classifier, the auditory object-specific attention of Rest, AOA1 and AOA2 could be identified from EEG signals using ApEn, SampEn, CmpMSE and FuzzyEn as features and the identification rates were significantly different from chance level. The optimal identification was achieved by the SVM-based auditory attention classifier using CmpMSE with the scale factor τ = 10. This study demonstrated a novel solution to identify the auditory object-specific attention from single-trial EEG signals without the need to access the auditory stimulus.

Analysis of Feature Fatigue EEG Signals Based on Wavelet Entropy

Fatigue driving is bringing more and more serious harm, but there are various reasons for fatigue driving, it is still difficult to test the driver’s fatigue. This paper defines a method to test driver’s fatigue based on the EEG, and different from other researches into fatigue driving, this paper mainly takes the fatigue features of EEG signals in fatigue state and uses wavelet entropy as the feature extraction method to analyze the features of wavelet entropy and spectral entropy features as well as the classification accuracy under the same classifier. The SVM is used to show the classifier’s results. The accuracy of the driver fatigue state monitoring using the wavelet entropy is 90.7%, which is higher than the use of spectral entropy as the characteristic accuracy rate of 81.3%. The results show that the frequency characteristics of EEG can be well applied to driving fatigue testing, but different frequency feature calculation methods will affect the classification accuracy.

Real-time eye tracking for the assessment of driver fatigue

Eye-tracking is an important approach to collect evidence regarding some participants' driving fatigue. In this contribution, the authors present a non-intrusive system for evaluating driver fatigue by tracking eye movement behaviours. A real-time eye-tracker was used to monitor participants' eye state for collecting eye-movement data. These data are useful to get insights into assessing participants' fatigue state during monotonous driving. Ten healthy subjects performed continuous simulated driving for 1-2 h with eye state monitoring on a driving simulator in this study, and these measured features of the fixation time and the pupil area were recorded via using eye movement tracking device. For achieving a good cost-performance ratio and fast computation time, the fuzzy K-nearest neighbour was employed to evaluate and analyse the influence of different participants on the variations in the fixation duration and pupil area of drivers. The findings of this study indicated that there are significant differences in domain value distribution of the pupil area under the condition with normal and fatigue driving state. Result also suggests that the recognition accuracy by jackknife validation reaches to about 89% in average, implying that show a significant potential of real-time applicability of the proposed approach and is capable of detecting driver fatigue.

Automated Detection of Driver Fatigue Based on AdaBoost Classifier with EEG Signals

Purpose: Driving fatigue has become one of the important causes of road accidents, there are many researches to analyze driver fatigue. EEG is becoming increasingly useful in the measuring fatigue state. Manual interpretation of EEG signals is impossible, so an effective method for automatic detection of EEG signals is crucial needed.

Method: In order to evaluate the complex, unstable, and non-linear characteristics of EEG signals, four feature sets were computed from EEG signals, in which fuzzy entropy (FE), sample entropy (SE), approximate Entropy (AE), spectral entropy (PE), and combined entropies (FE + SE + AE + PE) were included. All these feature sets were used as the input vectors of AdaBoost classifier, a boosting method which is fast and highly accurate. To assess our method, several experiments including parameter setting and classifier comparison were conducted on 28 subjects. For comparison, Decision Trees (DT), Support Vector Machine (SVM) and Naive Bayes (NB) classifiers are used.

Results: The proposed method (combination of FE and AdaBoost) yields superior performance than other schemes. Using FE feature extractor, AdaBoost achieves improved area (AUC) under the receiver operating curve of 0.994, error rate (ERR) of 0.024, Precision of 0.969, Recall of 0.984, F1 score of 0.976, and Matthews correlation coefficient (MCC) of 0.952, compared to SVM (ERR at 0.035, Precision of 0.957, Recall of 0.974, F1 score of 0.966, and MCC of 0.930 with AUC of 0.990), DT (ERR at 0.142, Precision of 0.857, Recall of 0.859, F1 score of 0.966, and MCC of 0.716 with AUC of 0.916) and NB (ERR at 0.405, Precision of 0.646, Recall of 0.434, F1 score of 0.519, and MCC of 0.203 with AUC of 0.606). It shows that the FE feature set and combined feature set outperform other feature sets. AdaBoost seems to have better robustness against changes of ratio of test samples for all samples and number of subjects, which might therefore aid in the real-time detection of driver fatigue through the classification of EEG signals.

Conclusion: By using combination of FE features and AdaBoost classifier to detect EEG-based driver fatigue, this paper ensured confidence in exploring the inherent physiological mechanisms and wearable application.

Noise Robustness Analysis of Performance for EEG-Based Driver Fatigue Detection Using Different Entropy Feature Sets

Driver fatigue is an important factor in traffic accidents, and the development of a detection system for driver fatigue is of great significance. To estimate and prevent driver fatigue, various classifiers based on electroencephalogram (EEG) signals have been developed; however, as EEG signals have inherent non-stationary characteristics, their detection performance is often deteriorated by background noise. To investigate the effects of noise on detection performance, simulated Gaussian noise, spike noise, and electromyogram (EMG) noise were added into a raw EEG signal. Four types of entropies, including sample entropy (SE), fuzzy entropy (FE), approximate entropy (AE), and spectral entropy (PE), were deployed for feature sets. Three base classifiers (K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Decision Tree (DT)) and two ensemble methods (Bootstrap Aggregating (Bagging) and Boosting) were employed and compared. Results showed that: (1) the simulated Gaussian noise and EMG noise had an impact on accuracy, while simulated spike noise did not, which is of great significance for the future application of driver fatigue detection; (2) the influence on noise performance was different based on each classifier, for example, the robust effect of classifier DT was the best and classifier SVM was the weakest; (3) the influence on noise performance was also different with each feature set where the robustness of feature set FE and the combined feature set were the best; and (4) while the Bagging method could not significantly improve performance against noise addition, the Boosting method may significantly improve performance against superimposed Gaussian and EMG noise. The entropy feature extraction method could not only identify driver fatigue, but also effectively resist noise, which is of great significance in future applications of an EEG-based driver fatigue detection system.