Rhythm-Dependent Multilayer Brain Network for the Detection of Driving Fatigue

Designing a practical fatigue detection system: A review on recent developments and challenges

INTRODUCTION

Fatigue is considered to have a life-threatening effect on human health and it has been an active field of research in different sectors. Deploying wearable physiological sensors helps to detect the level of fatigue objectively without any concern of bias in subjective assessment and interfering with work.

METHODS

This paper provides an in-depth review of fatigue detection approaches using physiological signals to pinpoint their main achievements, identify research gaps, and recommend avenues for future research. The review results are presented under three headings, including: signal modality, experimental environments, and fatigue detection models. Fatigue detection studies are first divided based on signal modality into uni-modal and multi-modal approaches. Then, the experimental environments utilized for fatigue data collection are critically analyzed. At the end, the machine learning models used for the classification of fatigue state are reviewed.

PRACTICAL APPLICATIONS

The directions for future research are provided based on critical analysis of past studies. Finally, the challenges of objective fatigue detection in the real-world scenario are discussed.

Automatically Extracting and Utilizing EEG Channel Importance Based on Graph Convolutional Network for Emotion Recognition

Graph convolutional network (GCN) based on the brain network has been widely used for EEG emotion recognition. However, most studies train their models directly without considering network dimensionality reduction beforehand. In fact, some nodes and edges are invalid information or even interference information for the current task. It is necessary to reduce the network dimension and extract the core network. To address the problem of extracting and utilizing the core network, a core network extraction model (CWGCN) based on channel weighting and graph convolutional network and a graph convolutional network model (CCSR-GCN) based on channel convolution and style-based recalibration for emotion recognition have been proposed. The CWGCN model automatically extracts the core network and the channel importance parameter in a data-driven manner. The CCSR-GCN model innovatively uses the output information of the CWGCN model to identify the emotion state. The experimental results on SEED show that: 1) the core network extraction can help improve the performance of the GCN model; 2) the models of CWGCN and CCSR-GCN achieve better results than the currently popular methods. The idea and its implementation in this paper provide a novel and successful perspective for the application of GCN in brain network analysis of other specific tasks.

EEG-based driving intuition and collision anticipation using joint temporal-frequency multi-layer dynamic brain network

Intuition plays a crucial role in human driving decision-making, and this rapid and unconscious cognitive process is essential for improving traffic safety. We used the first proposed multi-layer network analysis method, "Joint Temporal-Frequency Multi-layer Dynamic Brain Network" (JTF-MDBN), to study the EEG data from the initial and advanced phases of driving intuition training in the theta, alpha, and beta bands. Additionally, we conducted a comparative study between these two phases using multi-layer metrics as well as local and global metrics of single layers. The results show that brain region activity is more stable in the advanced phase of intuition training compared to the initial phase. Particularly in the alart state task, the JTF-MDBN demonstrated stronger connection strength. Multi-layer network analysis indicates that modularity is significantly higher for the non-alert state task than the alert state task in the alpha and beta bands. In the W4 time window (1 second before a collision), we identified significant features that can differentiate situations where a car collision is imminent from those where no collision occurs. Single-layer network analysis also revealed statistical differences in node strength and local efficiency for some EEG channels in the alpha and beta bands during the W4 and W5 time windows. Using these biomarkers to predict vehicle collision risk, the classification accuracy of a linear kernel SVM reached up to 87.5%, demonstrating the feasibility of predicting driving collisions through brain network biomarkers. These findings are important for the study of human intuition and the development of brain-computer interface-based intelligent driving hazard perception assistance systems.

CSF-GTNet: A Novel Multi-Dimensional Feature Fusion Network Based on Convnext-GeLU- BiLSTM for EEG-Signals-Enabled Fatigue Driving Detection

Electroencephalography (EEG) signal has been recognized as an effective fatigue detection method, which can intuitively reflect the drivers' mental state. However, the research on multi-dimensional features in existing work could be much better. The instability and complexity of EEG signals will increase the difficulty of extracting data features. More importantly, most current work only treats deep learning models as classifiers. They ignored the features of different subjects learned by the model. Aiming at the above problems, this paper proposes a novel multi-dimensional feature fusion network, CSF-GTNet, based on time and space-frequency domains for fatigue detection. Specifically, it comprises Gaussian Time Domain Network (GTNet) and Pure Convolutional Spatial Frequency Domain Network (CSFNet). The experimental results show that the proposed method effectively distinguishes between alert and fatigue states. The accuracy rates are 85.16% and 81.48% on the self-made and SEED-VIG datasets, respectively, which are higher than the state-of-the-art methods. Moreover, we analyze the contribution of each brain region for fatigue detection through the brain topology map. In addition, we explore the changing trend of each frequency band and the significance between different subjects in the alert state and fatigue state through the heat map. Our research can provide new ideas in brain fatigue research and play a specific role in promoting the development of this field.

Vigilance estimation using truncated l1 distance kernel-based sparse representation regression with physiological signals

BACKGROUND

With a large number of accidents caused by the decline in the vigilance of operators, finding effective automatic vigilance monitoring methods is a work of great significance in recent years. Based on physiological signals and machine learning algorithms, researchers have opened up a path for objective vigilance estimation.

METHODS

Sparse representation (SR)-based recognition algorithms with excellent performance and simple models are very promising approaches in this field. This paper aims to study the adaptability and performance improvement of truncated l1 distance (TL1) kernel on SR-based algorithm in the context of physiological signal vigilance estimation. Compared with the traditional radial basis function (RBF), the TL1 kernel has good adaptiveness to nonlinearity and is suitable for the discrimination of complex physiological signals. A recognition framework based on TL1 and SR theory is proposed. Firstly, the inseparable physiological features are mapped to the reproducing kernel Kreĭn space through the infinite-dimensional projection of the TL1 kernel. Then the obtained kernel matrix is converted into the symmetric positive definite matrix according to the eigenspectrum approaches. Finally, the final prediction result is obtained through the sparse representation regression process.

RESULTS

We verified the performance of the proposed framework on the popular SEED-VIG dataset containing physiological signals (electroencephalogram and electrooculogram) associated with vigilance. In the experimental results, the TL1 kernel is superior to the RBF kernel in both performance and kernel parameter stability.

CONCLUSIONS

This demonstrates the effectiveness of the TL1 kernel in distinguishing physiological signals and the excellent vigilance estimation capability of the proposed framework. Moreover, the contribution of our research motivates the development of physiological signal recognition based on kernel methods.

A regression model combined convolutional neural network and recurrent neural network for electroencephalogram-based cross-subject fatigue detection

Fatigue, one of the most important factors affecting road safety, has attracted many researchers' attention. Most existing fatigue detection methods are based on feature engineering and classification models. The feature engineering is greatly influenced by researchers' domain knowledge, which will lead to a poor performance in fatigue detection, especially in cross-subject experiment design. In addition, fatigue detection is often simplified as a classification problem of several discrete states. Models based on deep learning can realize automatic feature extraction without the limitation of researcher's domain knowledge. Therefore, this paper proposes a regression model combined convolutional neural network and recurrent neural network for electroencephalogram-based (EEG-based) cross-subject fatigue detection. At the same time, a twofold random-offset zero-overlapping sampling method is proposed to train a bigger model and reduce overfitting. Compared with existing results, the proposed method achieves a much better result of 0.94 correlation coefficient (COR) and 0.09 root mean square error (RMSE) in a within-subject experiment design. What is more, there is no misclassification between awake and drowsy states. For cross-subject experiment design, the COR and RMSE are 0.79 and 0.15, respectively, which are close to the existing within-subject results and better than similar cross-subject results. The cross-subject regression model is very important for fatigue detection application since the fatigue indication is more precise than several discrete states and no model calibration is required for a new user. The twofold random-offset zero-overlapping sampling method can also be used as a reference by other EEG-based deep learning research.

Characteristic analysis of epileptic brain network based on attention mechanism

Constructing an efficient and accurate epilepsy detection system is an urgent research task. In this paper, we developed an EEG-based multi-frequency multilayer brain network (MMBN) and an attentional mechanism based convolutional neural network (AM-CNN) model to study epilepsy detection. Specifically, based on the multi-frequency characteristics of the brain, we first use wavelet packet decomposition and reconstruction methods to divide the original EEG signals into eight frequency bands, and then construct MMBN through correlation analysis between brain regions, where each layer corresponds to a specific frequency band. The time, frequency and channel related information of EEG signals are mapped into the multilayer network topology. On this basis, a multi-branch AM-CNN model is designed, which completely matches the multilayer structure of the proposed brain network. The experimental results on public CHB-MIT datasets show that eight frequency bands divided in this work are all helpful for epilepsy detection, and the fusion of multi-frequency information can effectively decode the epileptic brain state, achieving accurate detection of epilepsy with an average accuracy of 99.75%, sensitivity of 99.43%, and specificity of 99.83%. All of these provide reliable technical solutions for EEG-based neurological disease detection, especially for epilepsy detection.

Community detection in multi-frequency EEG networks

Functional connectivity networks of the human brain are commonly studied using tools from complex network theory. Existing methods focus on functional connectivity within a single frequency band. However, it is well-known that higher order brain functions rely on the integration of information across oscillations at different frequencies. Therefore, there is a need to study these cross-frequency interactions. In this paper, we use multilayer networks to model functional connectivity across multiple frequencies, where each layer corresponds to a different frequency band. We then introduce the multilayer modularity metric to develop a multilayer community detection algorithm. The proposed approach is applied to electroencephalogram (EEG) data collected during a study of error monitoring in the human brain. The differences between the community structures within and across different frequency bands for two response types, i.e. error and correct, are studied. The results indicate that following an error response, the brain organizes itself to form communities across frequencies, in particular between theta and gamma bands while a similar cross-frequency community formation is not observed following the correct response.

T-A-MFFNet: Multi-feature fusion network for EEG analysis and driving fatigue detection based on time domain network and attention network

Driving fatigue detection based on EEG signals is a research hotspot in applying brain-computer interfaces. EEG signal is complex, unstable, and nonlinear. Most existing methods rarely analyze the data characteristics from multiple dimensions, so it takes work to analyze the data comprehensively. To analyze EEG signals more comprehensively, this paper evaluates a feature extraction strategy of EEG data based on differential entropy (DE). This method combines the characteristics of different frequency bands, extracts the frequency domain characteristics of EEG, and retains the spatial information between channels. This paper proposes a multi-feature fusion network (T-A-MFFNet) based on the time domain and attention network. The model is composed of a time domain network (TNet), channel attention network (CANet), spatial attention network (SANet), and multi-feature fusion network(MFFNet) based on a squeeze network. T-A-MFFNet aims to learn more valuable features from the input data to achieve good classification results. Specifically, the TNet network extracts high-level time series information from EEG data. CANet and SANet are used to fuse channel and spatial features. They use MFFNet to merge multi-dimensional features and realize classification. The validity of the model is verified on the SEED-VIG dataset. The experimental results show that the accuracy of the proposed method reaches 85.65 %, which is superior to the current popular model. The proposed method can learn more valuable information from EEG signals to improve the ability to identify fatigue status and promote the development of the research field of driving fatigue detection based on EEG signals.