Research on Channel Selection and Multi-Feature Fusion of EEG Signals for Mental Fatigue Detection

A novel deep-learning model based on τ-shaped convolutional network (τNet) with long short-term memory (LSTM) for physiological fatigue detection from EEG and EOG signals

In recent years, fatigue driving has become the main cause of traffic accidents, leading to increased attention towards fatigue detection systems. However, the pooling and strided convolutional operations in fatigue detection algorithm based on traditional deep learning methods may led to the loss of some useful information. This paper proposed a novel τ -shaped convolutional network ( τ Net ) aiming to address this issue. Unlike traditional network structures, τ Net incorporates the operations of upsampling features and concatenating high- and low-level features, enabling full utilization of useful information. Moreover, considering that the fatigue state is a mental state involving temporal evolution, we proposed the novel long short-term memory (LSTM)- τ -shaped convolutional network (LSTM- τ Net ), a parallel structure composed of LSTM and τ Net for fatigue detection, where τ Net extracts time-invariant features with location information, and LSTM extracts long temporal dependencies. We compared LSTM- τ Net with six competing methods based on two datasets. Results showed that the proposed algorithm achieved higher classification accuracy than the other methods, with 94.25% on EEG data (binary classification) and 82.19% on EOG data (triple classification). Additionally, the proposed algorithm exhibits low computational cost, good training stability, and robustness against insufficient training. Therefore, it is promising for further implementation of fatigue online detection systems.

Fractal Dimension as a discriminative feature for high accuracy classification in motor imagery EEG-based brain-computer interface

BACKGROUND AND OBJECTIVE

The brain-computer interface (BCI) technology acquires human brain electrical signals, which can be effectively and successfully used to control external devices, potentially supporting subjects suffering from motor impairments in the interaction with the environment. To this aim, BCI systems must correctly decode and interpret neurophysiological signals reflecting the intention of the subjects to move. Therefore, the accurate classification of single events in motor tasks represents a fundamental challenge in ensuring efficient communication and control between users and BCIs. Movement-associated changes in electroencephalographic (EEG) sensorimotor rhythms, such as event-related desynchronization (ERD), are well-known features of discriminating motor tasks. Fractal dimension (FD) can be used to evaluate the complexity and self-similarity in brain signals, potentially providing complementary information to frequency-based signal features.

METHODS

In the present work, we introduce FD as a novel feature for subject-independent event classification, and we test several machine learning (ML) models in behavioural tasks of motor imagery (MI) and motor execution (ME).

RESULTS

Our results show that FD improves the classification accuracy of ML compared to ERD. Furthermore, unilateral hand movements have higher classification accuracy than bilateral movements in both MI and ME tasks.

CONCLUSIONS

These results provide further insights into subject-independent event classification in BCI systems and demonstrate the potential of FD as a discriminative feature for EEG signals.

Mental fatigue assessment by an arbitrary channel EEG based on morphological features and LSTM-CNN

In order to achieve more sensitive mental fatigue assessment (MFA) based on an arbitrary channel EEG, this study proposed a series of feature extraction methods that combine mathematical morphology (MM), as well as an LSTM-CNN architecture. Firstly, 37 subjects had their resting-state EEGs collected at rested wakefulness (RW) and after 24 h of sleep deprivation (SD) using a 30-channel EEG acquisition device, the RW and SD groups were regarded as the negative and positive groups of mental fatigue, respectively, and the EEG collection were further categorized into two conditions: eye-opened state (EO) and eye-closed state (EC). Then, since MM can reflect the morphological characteristics of EEG rhythms and their potentials relatively independently of the time-frequency analysis and phase calculation, the MM methods were found to better reflect the mental fatigue after SD statistically, whether for single features (ANOVA: p<0.000001), multiple features (clustering by K-means, t-test: p<0.01), or time series feature spaces (calculating CD, t-test: p<0.01) of a single channel. Finally, the LSTM-CNN enhanced the generalization ability when dealing with different single-channel EEG by combining GRUs with convolutional layers: comparing the AUCs of different architectures for MFA based on an arbitrary channel, LSTM-CNN (0.992) > LSTM network (0.94) > CNN (0.831) > MLP (0.754). Moreover, the use of MM also improved the accuracy of analyzed architectures, and the true/false positive rate (TPR/FPR) of the LSTM-CNN architecture for MFA based on an arbitrary channel reached 97.024 %/3.497 %, which provided a feasible solution for the arbitrary channel EEG-based MFA.

Mental Stress Classification Based on Selected Electroencephalography Channels Using Correlation Coefficient of Hjorth Parameters

Electroencephalography (EEG) signals offer invaluable insights into diverse activities of the human brain, including the intricate physiological and psychological responses associated with mental stress. A major challenge, however, is accurately identifying mental stress while mitigating the limitations associated with a large number of EEG channels. Such limitations encompass computational complexity, potential overfitting, and the prolonged setup time for electrode placement, all of which can hinder practical applications. To address these challenges, this study presents the novel CCHP method, aimed at identifying and ranking commonly optimal EEG channels based on their sensitivity to the mental stress state. This method's uniqueness lies in its ability not only to find common channels, but also to prioritize them according to their responsiveness to stress, ensuring consistency across subjects and making it potentially transformative for real-world applications. From our rigorous examinations, eight channels emerged as universally optimal in detecting stress variances across participants. Leveraging features from the time, frequency, and time-frequency domains of these channels, and employing machine learning algorithms, notably RLDA, SVM, and KNN, our approach achieved a remarkable accuracy of 81.56% with the SVM algorithm outperforming existing methodologies. The implications of this research are profound, offering a stepping stone toward the development of real-time stress detection devices, and consequently, enabling clinicians to make more informed therapeutic decisions based on comprehensive brain activity monitoring.

Domain knowledge-assisted multi-objective evolutionary algorithm for channel selection in brain-computer interface systems

Background

For non-invasive brain-computer interface systems (BCIs) with multiple electroencephalogram (EEG) channels, the key factor limiting their convenient application in the real world is how to perform reasonable channel selection while ensuring task accuracy, which can be modeled as a multi-objective optimization problem. Therefore, this paper proposed a two-objective problem model for the channel selection problem and introduced a domain knowledge-assisted multi-objective optimization algorithm (DK-MOEA) to solve the aforementioned problem.

Methods

The multi-objective optimization problem model was designed based on the channel connectivity matrix and comprises two objectives: one is the task accuracy and the other one can sensitively indicate the removal status of channels in BCIs. The proposed DK-MOEA adopted a two-space framework, consisting of the population space and the knowledge space. Furthermore, a knowledge-assisted update operator was introduced to enhance the search efficiency of the population space by leveraging the domain knowledge stored in the knowledge space.

Results

The proposed two-objective problem model and DK-MOEA were tested on a fatigue detection task and four state-of-the-art multi-objective evolutionary algorithms were used for comparison. The experimental results indicated that the proposed algorithm achieved the best results among all the comparative algorithms for most cases by the Wilcoxon rank sum test at a significance level of 0.05. DK-MOEA was also compared with a version without the utilization of domain knowledge and the experimental results validated the effectiveness of the knowledge-assisted mutation operator. Moreover, the comparison between DK-MOEA and a traditional classification algorithm using all channels demonstrated that DK-MOEA can strike the balance between task accuracy and the number of selected channels.

Conclusion

The formulated two-objective optimization model enabled the selection of a minimal number of channels without compromising classification accuracy. The utilization of domain knowledge improved the performance of DK-MOEA. By adopting the proposed two-objective problem model and DK-MOEA, a balance can be achieved between the number of the selected channels and the accuracy of the fatigue detection task. The methods proposed in this paper can reduce the complexity of subsequent data processing and enhance the convenience of practical applications.

An improved multi-source domain adaptation network for inter-subject mental fatigue detection based on DANN

OBJECTIVES

Electroencephalogram (EEG) is often used to detect mental fatigue because of its real-time characteristic and objective nature. However, because of the individual variability of EEG among different individuals, tedious and time-consuming calibration sessions are needed.

METHODS

Therefore, we propose a multi-source domain adaptation network for inter-subject mental fatigue detection named FLDANN, which is short for focal loss based domain-adversarial training of neural network. As for mental state feature extraction, power spectrum density is extracted based on the Welch method from four sub-bands of EEG signals. The features of the source domain and target domain are fed into the FLDANN network. The contributions of FLDANN include: (1) It uses the idea of adversarial to reduce feature differences between the source and target domain. (2) A loss function named focal loss is used to assign weights to source and target domain samples.

RESULTS

The experiment result shows that when the number of the source domains increases, the classification accuracy of domain-adversarial training of neural network (DANN) gradually decreases and finally tends to be stable. The proposed method achieves an accuracy of 84.10% ± 8.75% on the SEED-VIG dataset and 65.42% ± 7.47% on the self-designed dataset. In addition, the proposed method is compared with other domain adaptation methods and the results show that the proposed method outperforms those state-of-the-art methods.

CONCLUSIONS

The result proves that the proposed method is able to solve the problem of individual differences across subjects and to solve the problem of low classification performance of multi-source domain transfer learning.

Mental Fatigue Degree Recognition Based on Relative Band Power and Fuzzy Entropy of EEG

Mental fatigue is a common phenomenon in our daily lives. Long-term fatigue can lead to a decline in a person's operational functions and seriously affect work efficiency. In this paper, a method that recognizes the degree of mental fatigue based on relative band power and fuzzy entropy of Electroencephalogram (EEG) is proposed. The N-back experiment was used to induce mental fatigue in subjects, and the corresponding EEG signals were recorded during the experiment. A preprocessing method based on complementary ensemble empirical modal decomposition (CEEMD) and independent component analysis (ICA) was designed to remove noise from the raw EEG signal. The relative band power feature, which has been used extensively in fatigue recognition studies, was extracted from the EEG signals. Meanwhile, fuzzy entropy, a feature commonly used in attention recognition, was also extracted for fatigue recognition, based on previous findings that an increase in fatigue is accompanied by a decrease in attention. The two features were fed into an extreme gradient boosting (XGBoost) classifier to distinguish three different degrees of fatigue, which resulted in an average accuracy of 92.39% based on data from eight subjects. The promising results indicate the effectiveness of the proposed method in mental fatigue degree identification.

EEG-based mental fatigue detection using linear prediction cepstral coefficients and Riemann spatial covariance matrix

Objective. Establishing a mental fatigue monitoring system is of great importance as for severe fatigue may cause unimaginable consequences. Electroencephalogram (EEG) is often utilized for mental fatigue detection because of its high temporal resolution and ease of use. However, many EEG-based approaches for detecting mental fatigue only take into account the feature extraction of a single domain and do not fully exploit the information that EEG may offer.Approach. In our work, we propose a new algorithm for mental fatigue detection based on multi-domain feature extraction and fusion. EEG components representing fatigue are closely related in the past and present because fatigue is a dynamic and gradual process. Accordingly, the idea of linear prediction is used to fit the current value with a set of sample values in the past to calculate the linear prediction cepstral coefficients (LPCCs) as the time domain feature. Moreover, in order to better capture fatigue-related spatial domain information, the spatial covariance matrix of the original EEG signal is projected into the Riemannian tangent space using the Riemannian geometric method. Then multi-domain features are fused to obtain comprehensive spatio-temporal information.Main results. Experimental results prove the suggested algorithm outperforms existing state-of-the-art methods, achieving an average accuracy of 87.10% classification on the public dataset SEED-VIG (three categories) and 97.40% classification accuracy (two categories) on the dataset made by self-designed experiments.Significance. These findings show that our proposed strategy perform more effectively for mental fatigue detection based on EEG.

Multivariate Empirical Mode Decomposition of EEG for Mental State Detection at Localized Brain Lobes

In this study, the Multivariate Empirical Mode Decomposition (MEMD) approach is applied to extract features from multi-channel EEG signals for mental state classification. MEMD is a data-adaptive analysis approach which is suitable particularly for multi-dimensional non-linear signals like EEG. Applying MEMD results in a set of oscillatory modes called intrinsic mode functions (IMFs). As the decomposition process is data-dependent, the IMFs vary in accordance with signal variation caused by functional brain activity. Among the extracted IMFs, it is found that those corresponding to high-oscillation modes are most useful for detecting different mental states. Non-linear features are computed from the IMFs that contribute most to mental state detection. These MEMD features show a significant performance gain over the conventional tempo-spectral features obtained by Fourier transform and Wavelet transform. The dominance of specific brain region is observed by analysing the MEMD features extracted from associated EEG channels. The frontal region is found to be most significant with a classification accuracy of 98.06%. This multi-dimensional decomposition approach upholds joint channel properties and produces most discriminative features for EEG based mental state detection.

Developing a Deep Neural Network for Driver Fatigue Detection Using EEG Signals Based on Compressed Sensing

In recent years, driver fatigue has become one of the main causes of road accidents. As a result, fatigue detection systems have been developed to warn drivers, and, among the available methods, EEG signal analysis is recognized as the most reliable method for detecting driver fatigue. This study presents an automated system for a two-stage classification of driver fatigue, using a combination of compressed sensing (CS) theory and deep neural networks (DNNs), that is based on EEG signals. First, CS theory is used to compress the recorded EEG data in order to reduce the computational load. Then, the compressed EEG data is fed into the proposed deep convolutional neural network for automatic feature extraction/selection and classification purposes. The proposed network architecture includes seven convolutional layers together with three long short-term memory (LSTM) layers. For compression rates of 40, 50, 60, 70, 80, and 90, the simulation results for a single-channel recording show accuracies of 95, 94.8, 94.6, 94.4, 94.4, and 92%, respectively. Furthermore, by comparing the results to previous methods, the accuracy of the proposed method for the two-stage classification of driver fatigue has been improved and can be used to effectively detect driver fatigue.

Comparing between Different Sets of Preprocessing, Classifiers, and Channels Selection Techniques to Optimise Motor Imagery Pattern Classification System from EEG Pattern Recognition

The ability to control external devices through thought is increasingly becoming a reality. Human beings can use the electrical signals of their brain to interact or change the surrounding environment and more. The development of this technology called brain-computer interface (BCI) will increasingly allow people with motor disabilities to communicate or use assistive devices to walk, manipulate objects and communicate. Using data from the PhysioNet database, this study implemented a pattern classification system for use in a BCI on 109 healthy volunteers during real movement activities and motor imagery recorded by 64-channels electroencephalography (EEG) system. Different classifiers such as Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Decision Trees (TREE) were applied on different combinations of EEG channels. Starting from two channels (C3, C4 and CP3 and CP4) positioned on the contralateral and ipsilateral sensorimotor cortex, the Region of Interest (RoI) centred on C3/Cp3 and C4/Cp4 and, finally, a data-driven automatic channels selection was tested to explore the best channel combination able to increase the classification accuracy. The results showed that the proposed automatic channels selection was able to significantly improve the performance of each classifier achieving 98% of accuracy for classification of real and imagined hand movement (sensitivity = 97%, specificity = 99%, AUC = 0.99) by SVM. While the accuracy of the classification between the imagery of hand and foot movements was 91% (sensitivity = 87%, specificity = 86%, AUC = 0.93) also with SVM. In the proposed approach, the data-driven automatic channels selection outperforms classical a priori channel selection models such as C3/C4, Cp3/Cp4, or RoIs around those channels with the utmost accuracy to help remove the boundaries of human communication and improve the quality of life of people with disabilities.

Visual Object Recognition from Single-Trial EEG Signals using Machine Learning Wrapper Techniques

Responses of the human brain to different visual stimuli elicit specific patterns in electroencephalography (EEG) signals. It is confirmed that by analyzing these patterns, we can recognize the category of the visited objects. However, high levels of noise and artifacts in EEG signals and the discrepancies between the recorded data from different subjects in visual object recognition task make classification of cognitive states of subjects a serious challenge. In this research, we present a framework for evaluating machine learning and wrapper channel selection algorithms used for classifying single-trial EEG signals recorded in response to photographic stimuli. It is shown that by correctly mapping the entire EEG data space to informative feature spaces (IFS), the performance of the classification methods can improve significantly. Results outperform the state-of-the-art results and confirm efficiency of the proposed feature selection methods in capturing the most informative EEG channels. This can help to achieve high separability of object categories in single-trial visual object recognition task.