Fatigue driving recognition based on deep learning and graph neural network

Dynamical graph neural network with attention mechanism for epilepsy detection using single channel EEG

Epilepsy is a chronic brain disease, and identifying seizures based on electroencephalogram (EEG) signals would be conducive to implement interventions to help patients reduce impairment and improve quality of life. In this paper, we propose a classification algorithm to apply dynamical graph neural network with attention mechanism to single channel EEG signals. Empirical mode decomposition (EMD) are adopted to construct graphs and the optimal adjacency matrix is obtained by model optimization. A multilayer dynamic graph neural network with attention mechanism is proposed to learn more discriminative graph features. The MLP-pooling structure is proposed to fuse graph features. We performed 12 classification tasks on the epileptic EEG database of the University of Bonn, and experimental results showed that using 25 runs of ten-fold cross-validation produced the best classification results with an average of 99.83[Formula: see text] accuracy, 99.91[Formula: see text] specificity, 99.78[Formula: see text] sensitivity, 99.87[Formula: see text] precision, and 99.47[Formula: see text] [Formula: see text] score for the 12 classification tasks.

Mental fatigue assessment by an arbitrary channel EEG based on morphological features and LSTM-CNN

In order to achieve more sensitive mental fatigue assessment (MFA) based on an arbitrary channel EEG, this study proposed a series of feature extraction methods that combine mathematical morphology (MM), as well as an LSTM-CNN architecture. Firstly, 37 subjects had their resting-state EEGs collected at rested wakefulness (RW) and after 24 h of sleep deprivation (SD) using a 30-channel EEG acquisition device, the RW and SD groups were regarded as the negative and positive groups of mental fatigue, respectively, and the EEG collection were further categorized into two conditions: eye-opened state (EO) and eye-closed state (EC). Then, since MM can reflect the morphological characteristics of EEG rhythms and their potentials relatively independently of the time-frequency analysis and phase calculation, the MM methods were found to better reflect the mental fatigue after SD statistically, whether for single features (ANOVA: p<0.000001), multiple features (clustering by K-means, t-test: p<0.01), or time series feature spaces (calculating CD, t-test: p<0.01) of a single channel. Finally, the LSTM-CNN enhanced the generalization ability when dealing with different single-channel EEG by combining GRUs with convolutional layers: comparing the AUCs of different architectures for MFA based on an arbitrary channel, LSTM-CNN (0.992) > LSTM network (0.94) > CNN (0.831) > MLP (0.754). Moreover, the use of MM also improved the accuracy of analyzed architectures, and the true/false positive rate (TPR/FPR) of the LSTM-CNN architecture for MFA based on an arbitrary channel reached 97.024 %/3.497 %, which provided a feasible solution for the arbitrary channel EEG-based MFA.

Miner Fatigue Detection from Electroencephalogram-Based Relative Power Spectral Topography Using Convolutional Neural Network

Fatigue of miners is caused by intensive workloads, long working hours, and shift-work schedules. It is one of the major factors increasing the risk of safety problems and work mistakes. Examining the detection of miner fatigue is important because it can potentially prevent work accidents and improve working efficiency in underground coal mines. Many previous studies have introduced feature-based machine-learning methods to estimate miner fatigue. This work proposes a method that uses electroencephalogram (EEG) signals to generate topographic maps containing frequency and spatial information. It utilizes a convolutional neural network (CNN) to classify the normal state, critical state, and fatigue state of miners. The topographic maps are generated from the EEG signals and contrasted using power spectral density (PSD) and relative power spectral density (RPSD). These two feature extraction methods were applied to feature recognition and four representative deep-learning methods. The results showthat RPSD achieves better performance than PSD in classification accuracy with all deep-learning methods. The CNN achieved superior results to the other deep-learning methods, with an accuracy of 94.5%, precision of 97.0%, sensitivity of 94.8%, and F1 score of 96.3%. Our results also show that the RPSD-CNN method outperforms the current state of the art. Thus, this method might be a useful and effective miner fatigue detection tool for coal companies in the near future.

The architecture of functional brain network modulated by driving during adverse weather conditions

Traffic accidents caused by adverse weather conditions have attracted the attention of many countries. Previous studies have focused on the driver's response in a particular situation under foggy conditions, but little is known about the functional brain network (FBN) topology that is modulated by driving in foggy weather, especially when the vehicle encounters cars in the opposite lane. An experiment consisting of two driving tasks is designed and conducted using sixteen participants. Functional connectivity between all pairs of channels for multiple frequency bands is assessed using the phase-locking value (PLV). Based on this, a PLV-weighted network is subsequently generated. The clustering coefficient (C) and the characteristic path length (L) are adopted as measures for the graph analysis. Statistical analyses are performed on graph-derived metrics. The major finding is that the PLV is significantly increased in the delta, theta and beta frequency bands while driving in foggy weather. Additionally, for the brain network topology metric, compared with driving in clear weather, significant increases are observed (driving in foggy weather) in the clustering coefficient for alpha and beta frequency bands and the characteristic path length for all frequency bands considered in this work. Driving in foggy weather would regulate FBN reorganization in different frequency bands. Our findings also suggest that the effects of adverse weather conditions on functional brain networks with a trend toward a more economic but less efficient architecture. Graph theory analysis may be a beneficial tool to further understand the neural mechanisms of driving in adverse weather conditions, which in turn may help to reduce the occurrence of road traffic accidents to some extent.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09825-y.

A New Feature Analysis Approach to Selecting Channels of EEG for Fatigue Driving

Fatigued driving is a significant contributor to traffic accidents. There are some issues with common EEG data of 32 channels, 64 channels, and 128 channels, such as difficult acquisition, high data redundancy, and difficult practical application. A new channel selection method called ReliefF_SFS is proposed to address the problem of how to reduce the number of channels while maintaining classification accuracy. It combines the ReliefF algorithm and the sequential forward selection (SFS) algorithm. When only T6, O1, Oz, T4, P3, and FC3 are used, the classification accuracy under Theta_Std+FE combined with ReliefF_SFS achieves 99.45%. The strategy suggested in this paper not only ensures the recognition accuracy but also reduces the number of channels when compared to other models based on the same data set.

An Improved Gesture Segmentation Method for Gesture Recognition Based on CNN and YCbCr

With the continuous improvement of people’s requirements for interactive experience, gesture recognition is widely used as a basic human-computer interaction. However, due to the environment, light source, cover, and other factors, the diversity and complexity of gestures have a great impact on gesture recognition. In order to enhance the features of gesture recognition, firstly, the hand skin color is filtered through YCbCr color space to separate the gesture region to be recognized, and the Gaussian filter is used to process the noise of gesture edge; secondly, the morphological gray open operation is used to process the gesture data, the watershed algorithm based on marker is used to segment the gesture contour, and the eight-connected filling algorithm is used to enhance the gesture features; finally, the convolution neural network is used to recognize the gesture data set with fast convergence speed. The experimental results show that the proposed method can recognize all kinds of gestures quickly and accurately with an average recognition success rate of 96.46% and does not significantly increase the recognition time.