An adaptive optimal-Kernel time-frequency representation-based complex network method for characterizing fatigued behavior using the SSVEP-based BCI system

Enhancing SSVEP-BCI Performance Under Fatigue State Using Dynamic Stopping Strategy

Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have emerged as a prominent technology due to their high information transfer rate, rapid calibration time, and robust signal-to-noise ratio. However, a critical challenge for practical applications is performance degradation caused by user fatigue during prolonged use. This work proposes novel methods to address this challenge by dynamically adjusting data acquisition length and updating detection models based on a fatigue-aware stopping strategy. Two 16-target SSVEP-BCIs were employed, one using low-frequency and the other using high-frequency stimulation. A self-recorded fatigue dataset from 24 subjects was utilized for extensive evaluation. A simulated online experiment demonstrated that the proposed methods outperform the conventional fixed stopping strategy in terms of classification accuracy, information transfer rate, and selection time, irrespective of stimulation frequency. These findings suggest that the proposed approach can significantly improve SSVEP-BCI performance under fatigue conditions, leading to superior performance during extended use.

Time-Varying Effective Connectivity for Describing the Dynamic Brain Networks of Post-stroke Rehabilitation

Hemiplegia is a common motor dysfunction caused by a stroke. However, the dynamic network mechanism of brain processing information in post-stroke hemiplegic patients has not been revealed when performing motor imagery (MI) tasks. We acquire electroencephalography (EEG) data from healthy subjects and post-stroke hemiplegic patients and use the Fugl-Meyer assessment (FMA) to assess the degree of motor function damage in stroke patients. Time-varying MI networks are constructed using the adaptive directed transfer function (ADTF) method to explore the dynamic network mechanism of MI in post-stroke hemiplegic patients. Finally, correlation analysis has been conducted to study potential relationships between global efficiency and FMA scores. The performance of our proposed method has shown that the brain network pattern of stroke patients does not significantly change from laterality to bilateral symmetry when performing MI recognition. The main change is that the contralateral motor areas of the brain damage and the effective connection between the frontal lobe and the non-motor areas are enhanced, to compensate for motor dysfunction in stroke patients. We also find that there is a correlation between FMA scores and global efficiency. These findings help us better understand the dynamic brain network of patients with post-stroke when processing MI information. The network properties may provide a reliable biomarker for the objective evaluation of the functional rehabilitation diagnosis of stroke patients.

Attention-Based Parallel Multiscale Convolutional Neural Network for Visual Evoked Potentials EEG Classification

Electroencephalography (EEG) decoding is an important part of Visual Evoked Potentials-based Brain-Computer Interfaces (BCIs), which directly determines the performance of BCIs. However, long-time attention to repetitive visual stimuli could cause physical and psychological fatigue, resulting in weaker reliable response and stronger noise interference, which exacerbates the difficulty of Visual Evoked Potentials EEG decoding. In this state, subjects' attention could not be concentrated enough and the frequency response of their brains becomes less reliable. To solve these problems, we propose an attention-based parallel multiscale convolutional neural network (AMS-CNN). Specifically, the AMS-CNN first extract robust temporal representations via two parallel convolutional layers with small and large temporal filters respectively. Then, we employ two sequential convolution blocks for spatial fusion and temporal fusion to extract advanced feature representations. Further, we use attention mechanism to weight the features at different moments according to the output-related interest. Finally, we employ a full connected layer with softmax activation function for classification. Two fatigue datasets collected from our lab are implemented to validate the superior classification performance of the proposed method compared to the state-of-the-art methods. Analysis reveals the competitiveness of multiscale convolution and attention mechanism. These results suggest that the proposed framework is a promising solution to improving the decoding performance of Visual Evoked Potential BCIs.

Complex networks and deep learning for EEG signal analysis

Electroencephalogram (EEG) signals acquired from brain can provide an effective representation of the human's physiological and pathological states. Up to now, much work has been conducted to study and analyze the EEG signals, aiming at spying the current states or the evolution characteristics of the complex brain system. Considering the complex interactions between different structural and functional brain regions, brain network has received a lot of attention and has made great progress in brain mechanism research. In addition, characterized by autonomous, multi-layer and diversified feature extraction, deep learning has provided an effective and feasible solution for solving complex classification problems in many fields, including brain state research. Both of them show strong ability in EEG signal analysis, but the combination of these two theories to solve the difficult classification problems based on EEG signals is still in its infancy. We here review the application of these two theories in EEG signal research, mainly involving brain-computer interface, neurological disorders and cognitive analysis. Furthermore, we also develop a framework combining recurrence plots and convolutional neural network to achieve fatigue driving recognition. The results demonstrate that complex networks and deep learning can effectively implement functional complementarity for better feature extraction and classification, especially in EEG signal analysis.

Brain-Computer Interface Speller Based on Steady-State Visual Evoked Potential: A Review Focusing on the Stimulus Paradigm and Performance

The steady-state visual evoked potential (SSVEP), measured by the electroencephalograph (EEG), has high rates of information transfer and signal-to-noise ratio, and has been used to construct brain-computer interface (BCI) spellers. In BCI spellers, the targets of alphanumeric characters are assigned different visual stimuli and the fixation of each target generates a unique SSVEP. Matching the SSVEP to the stimulus allows users to select target letters and numbers. Many BCI spellers that harness the SSVEP have been proposed over the past two decades. Various paradigms of visual stimuli, including the procedure of target selection, layout of targets, stimulus encoding, and the combination with other triggering methods are used and considered to influence on the BCI speller performance significantly. This paper reviews these stimulus paradigms and analyzes factors influencing their performance. The fundamentals of BCI spellers are first briefly described. SSVEP-based BCI spellers, where only the SSVEP is used, are classified by stimulus paradigms and described in chronological order. Furthermore, hybrid spellers that involve the use of the SSVEP are presented in parallel. Factors influencing the performance and visual fatigue of BCI spellers are provided. Finally, prevailing challenges and prospective research directions are discussed to promote the development of BCI spellers.

Rhythm-Dependent Multilayer Brain Network for the Detection of Driving Fatigue

Fatigue driving has attracted a great deal of attention for its huge influence on automobile accidents. Recognizing driving fatigue provides a primary but significant way for addressing this problem. In this paper, we first conduct the simulated driving experiments to acquire the EEG signals in alert and fatigue states. Then, for multi-channel EEG signals without pre-processing, a novel rhythm-dependent multilayer brain network (RDMB network) is developed and analyzed for driving fatigue detection. We find that there exists a significant difference between alert and fatigue states from the view of network science. Further, key sub-RDMB network based on closeness centrality are extracted. We calculate six network measures from the key sub-RDMB network and construct feature vectors to classify the alert and fatigue states. The results show that our method can respectively achieve the average accuracy of 95.28% (with sample length of 5 s), 90.25% (2 s), and 87.69% (1 s), significantly higher than compared methods. All these validate the effectiveness of RDMB network for reliable driving fatigue detection via EEG.

A Multi-Scale Fusion Convolutional Neural Network Based on Attention Mechanism for the Visualization Analysis of EEG Signals Decoding

Brain-computer interface (BCI) based on motor imagery (MI) electroencephalogram (EEG) decoding helps motor-disabled patients to communicate with external devices directly, which can achieve the purpose of human-computer interaction and assisted living. MI EEG decoding has a core problem which is extracting as many multiple types of features as possible from the multi-channel time series of EEG to understand brain activity accurately. Recently, deep learning technology has been widely used in EEG decoding. However, the variability of the simple network framework is insufficient to satisfy the complex task of EEG decoding. A multi-scale fusion convolutional neural network based on the attention mechanism (MS-AMF) is proposed in this paper. The network extracts spatio temporal multi-scale features from multi-brain regions representation signals and is supplemented by a dense fusion strategy to retain the maximum information flow. The attention mechanism we added to the network has improved the sensitivity of the network. The experimental results show that the network has a better classification effect compared with the baseline method in the BCI Competition IV-2a dataset. We conducted visualization analysis in multiple parts of the network, and the results show that the attention mechanism is also convenient for analyzing the underlying information flow of EEG decoding, which verifies the effectiveness of the MS-AMF method.

Matrix of Lags: A tool for analysis of multiple dependent time series applied for CAP scoring

BACKGROUND

Multiple methods have been developed to assess what happens between and within time series. In a particular type of these series, the previous values of the currently observed series are contingent on the lagged values of another series. These cases can commonly be addressed by regression. However, a model selection criteria should be employed to evaluate the compromise between the amount of information provided and the model complexity. This is the basis for the development of the Matrix of Lags (MoL), a tool to study dependent time series.

METHODS

For each input, multiple regressions were applied to produce a model for each lag and a model selection criterion identifies the lags that will populate an auxiliary matrix. Afterwards, the energy of the lags (that are in the auxiliary matrix) was used to define a row of the MoL. Therefore, each input corresponds to a row of the MoL. To test the proposed tool, the heart rate variability and the electrocardiogram derived respiration were employed to perform the indirect estimation of the electroencephalography cyclic alternating pattern (CAP) cycles. Therefore, a support vector machine was fed with the MoL to perform the CAP cycle classification for each input signal. Multiple tests were carried out to further examine the proposed tool, including the effect of balancing the datasets, application of other regression methods and employment of two feature section models. The first was based on sequential backward selection while the second examined characteristics of a return map.

RESULTS

The best performance of the subject independent model was attained by feeding the lags, selected by sequential backward selection, to a support vector machine, achieving an average accuracy, sensitivity, specificity and area under the receiver operating characteristic curve of, respectively, 77%, 71%, 82% and 0.77.

CONCLUSIONS

The developed model allows to perform a measurement of a characteristic marker of sleep instability (the CAP cycle) and the results are in the upper bound of the specialist agreement range with visual analysis. Thus, the developed method could possibly be used for clinical diagnosis.

Disrupted Time-Dependent and Functional Connectivity Brain Network in Alzheimer's Disease: A Resting-State fMRI Study Based on Visibility Graph

Background:

Alzheimer's Disease (AD) is a progressive neurodegenerative disease with insidious onset, which is difficult to be reversed and cured. Therefore, discovering more precise biological information from neuroimaging biomarkers is crucial for accurate and automatic detection of AD.

Methods:

We innovatively used a Visibility Graph (VG) to construct the time-dependent brain networks as well as functional connectivity network to investigate the underlying dynamics of AD brain based on functional magnetic resonance imaging. There were 32 AD patients and 29 Normal Controls (NCs) from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. First, the VG method mapped the time series of single brain region into networks. By extracting topological properties of the networks, the most significant features were selected as discriminant features into a supporting vector machine for classification. Furthermore, in order to detect abnormalities of these brain regions in the whole AD brain, functional connectivity among different brain regions was calculated based on the correlation of regional degree sequences.

Results:

According to the topology abnormalities exploration of local complex networks, we found several abnormal brain regions, including left insular, right posterior cingulate gyrus and other cortical regions. The accuracy of characteristics of the brain regions extracted from local complex networks was 88.52%. Association analysis demonstrated that the left inferior opercular part of frontal gyrus, right middle occipital gyrus, right superior parietal gyrus and right precuneus played a tremendous role in AD.

Conclusion:

These results would be helpful in revealing the underlying pathological mechanism of the disease.

Study on the Effect of Man-Machine Response Mode to Relieve Driving Fatigue Based on EEG and EOG

Rapid and accurate detection of driver fatigue is of great significance to improve traffic safety. In the present work, we propose the man-machine response mode (MRM) to relieve driver fatigue caused by long-term driving. In this paper, the characteristics of the complex brain network, which can effectively reflect brain activity information, were used to detect the change of driving fatigue over time. Combined with the traditional eye movement characteristics and a subjective questionnaire (SQ), the changes in driving fatigue characteristics were comprehensively analyzed. The results show that driving fatigue can be effectively delayed using the MRM. Additionally, the response equipment is low in cost and practical, so it will be practical to use in actual driving situations in the future.

A recurrence network-based convolutional neural network for fatigue driving detection from EEG

Driver fatigue is an important cause of traffic accidents, which has triggered great concern for detecting drivers' fatigue. Numerous methods have been proposed to fulfill this challenging task, including feature methods and machine learning methods. Recently, with the development of deep learning techniques, many studies achieved better results than traditional feature methods, and the combination of traditional methods and deep learning techniques gradually received attention. In this paper, we propose a recurrence network-based convolutional neural network (RN-CNN) method to detect fatigue driving. To be specific, we first conduct a simulated driving experiment to collect electroencephalogram (EEG) signals of subjects under alert state and fatigue state. Then, we construct the multiplex recurrence network (RN) from EEG signals to fuse information from the original time series. Finally, CNN is employed to extract and learn the features of a multiplex RN for realizing a classification task. The results indicate that the proposed RN-CNN method can achieve an average accuracy of 92.95%. To verify the effectiveness of our method, some existing competitive methods are compared with ours. The results show that our method outperforms the existing methods, which demonstrate the effect of the RN-CNN method.

EEG-Based Spatio-Temporal Convolutional Neural Network for Driver Fatigue Evaluation

Driver fatigue evaluation is of great importance for traffic safety and many intricate factors would exacerbate the difficulty. In this paper, based on the spatial-temporal structure of multichannel electroencephalogram (EEG) signals, we develop a novel EEG-based spatial-temporal convolutional neural network (ESTCNN) to detect driver fatigue. First, we introduce the core block to extract temporal dependencies from EEG signals. Then, we employ dense layers to fuse spatial features and realize classification. The developed network could automatically learn valid features from EEG signals, which outperforms the classical two-step machine learning algorithms. Importantly, we carry out fatigue driving experiments to collect EEG signals from eight subjects being alert and fatigue states. Using 2800 samples under within-subject splitting, we compare the effectiveness of ESTCNN with eight competitive methods. The results indicate that ESTCNN fulfills a better classification accuracy of 97.37% than these compared methods. Furthermore, the spatial-temporal structure of this framework advantages in computational efficiency and reference time, which allows further implementations in the brain-computer interface online systems.

Characterization of SSMVEP-based EEG signals using multiplex limited penetrable horizontal visibility graph

The steady state motion visual evoked potential (SSMVEP)-based brain computer interface (BCI), which incorporates the motion perception capabilities of the human visual system to alleviate the negative effects caused by strong visual stimulation from steady-state VEP, has attracted a great deal of attention. In this paper, we design a SSMVEP-based experiment by Newton's ring paradigm. Then, we use the canonical correlation analysis and Support Vector Machines to classify SSMVEP signals for the SSMVEP-based electroencephalography (EEG) signal detection. We find that the classification accuracy of different subjects under fatigue state is much lower than that in the normal state. To probe into this, we develop a multiplex limited penetrable horizontal visibility graph method, which enables to infer a brain network from 62-channel EEG signals. Subsequently, we analyze the variation of the average weighted clustering coefficient and the weighted global efficiency corresponding to these two brain states and find that both network measures are lower under fatigue state. The results suggest that the associations and information transfer efficiency among different brain regions become weaker when the brain state changes from normal to fatigue, which provide new insights into the explanations for the reduced classification accuracy. The promising classification results and the findings render the proposed methods particularly useful for analyzing EEG recordings from SSMVEP-based BCI system.

Multiplex Limited Penetrable Horizontal Visibility Graph from EEG Signals for Driver Fatigue Detection

Driver fatigue is an important contributor to road accidents, and driver fatigue detection has attracted a great deal of attention on account of its significant importance. Numerous methods have been proposed to fulfill this challenging task, though, the characterization of the fatigue mechanism still, to a large extent, remains to be investigated. To address this problem, we, in this work, develop a novel Multiplex Limited Penetrable Horizontal Visibility Graph (Multiplex LPHVG) method, which allows in not only detecting fatigue driving but also probing into the brain fatigue behavior. Importantly, we use the method to construct brain networks from EEG signals recorded from different subjects performing simulated driving tasks under alert and fatigue driving states. We then employ clustering coefficient, global efficiency and characteristic path length to characterize the topological structure of the networks generated from different brain states. In addition, we combine average edge overlap with the network measures to distinguish alert and mental fatigue states. The high-accurate classification results clearly demonstrate and validate the efficacy of our multiplex LPHVG method for the fatigue detection from EEG signals. Furthermore, our findings show a significant increase of the clustering coefficient as the brain evolves from alert state to mental fatigue state, which yields novel insights into the brain behavior associated with fatigue driving.