Deep learning techniques for classification of electroencephalogram (EEG) motor imagery (MI) signals: a review

Sparse Bayesian correntropy learning for robust muscle activity reconstruction from noisy brain recordings

Sparse Bayesian learning has promoted many effective frameworks of brain activity decoding for the brain-computer interface, including the direct reconstruction of muscle activity using brain recordings. However, existing sparse Bayesian learning algorithms mainly use Gaussian distribution as error assumption in the reconstruction task, which is not necessarily the truth in the real-world application. On the other hand, brain recording is known to be highly noisy and contains many non-Gaussian noises, which could lead to large performance degradation for sparse Bayesian learning algorithms. The goal of this paper is to propose a novel robust implementation of sparse Bayesian learning so that robustness and sparseness can be realized simultaneously. Motivated by the exceptional robustness of maximum correntropy criterion (MCC), we proposed integrating MCC to the sparse Bayesian learning regime. To be specific, we derived the explicit error assumption inherent in the MCC, and then leveraged it for the likelihood function. Meanwhile, we utilized the automatic relevance determination technique as the sparse prior distribution. To fully evaluate the proposed method, a synthetic example and a real-world muscle activity reconstruction task with two different brain modalities were leveraged. Experimental results showed, our proposed sparse Bayesian correntropy learning framework significantly improves the robustness for the noisy regression tasks. Our proposed algorithm could realize higher correlation coefficients and lower root mean squared errors for the real-world muscle activity reconstruction scenario. Sparse Bayesian correntropy learning provides a powerful approach for brain activity decoding which will promote the development of brain-computer interface technology.

STIT-Net- A Wavelet based Convolutional Transformer Model for Motor Imagery EEG Signal Classification in the Sensorimotor Bands

Motor Imagery (MI) electroencephalographic (EEG) signal classification is a pioneer research branch essential for mobility rehabilitation. This paper proposes an end-to-end hybrid deep network "Spatio Temporal Inception Transformer Network (STIT-Net)" model for MI classification. Discrete Wavelet Transform (DWT) is used to derive the alpha (8-13) Hz and beta (13-30) Hz EEG sub bands which are dominant during motor tasks to enhance the performance of the proposed work. STIT-Net employs spatial and temporal convolutions to capture spatial dependencies and temporal information and an inception block with three parallel convolutions extracts multi-level features. Then the transformer encoder with self-attention mechanism highlights the similar task. The proposed model improves the classification of the Physionet EEG motor imagery dataset with an average accuracy of 93.52% and 95.70% for binary class in the alpha and beta bands respectively, and 85.26% and 87.34% for three class, for four class 81.95% and 82.66% were obtained in the alpha and beta band respective EEG based motor signals which is better compared to the results available in the literature. The proposed methodology is further evaluated on other motor imagery datasets, both for subject-independent and cross-subject conditions, to assess the performance of the model.

Enhancing Motor Imagery Classification with Residual Graph Convolutional Networks and Multi-Feature Fusion

Stroke, an abrupt cerebrovascular ailment resulting in brain tissue damage, has prompted the adoption of motor imagery (MI)-based brain-computer interface (BCI) systems in stroke rehabilitation. However, analyzing electroencephalogram (EEG) signals from stroke patients poses challenges. To address the issues of low accuracy and efficiency in EEG classification, particularly involving MI, the study proposes a residual graph convolutional network (M-ResGCN) framework based on the modified S-transform (MST), and introduces the self-attention mechanism into residual graph convolutional network (ResGCN). This study uses MST to extract EEG time-frequency domain features, derives spatial EEG features by calculating the absolute Pearson correlation coefficient (aPcc) between channels, and devises a method to construct the adjacency matrix of the brain network using aPcc to measure the strength of the connection between channels. Experimental results involving 16 stroke patients and 16 healthy subjects demonstrate significant improvements in classification quality and robustness across tests and subjects. The highest classification accuracy reached 94.91% and a Kappa coefficient of 0.8918. The average accuracy and F1 scores from 10 times 10-fold cross-validation are 94.38% and 94.36%, respectively. By validating the feasibility and applicability of brain networks constructed using the aPcc in EEG signal analysis and feature encoding, it was established that the aPcc effectively reflects overall brain activity. The proposed method presents a novel approach to exploring channel relationships in MI-EEG and improving classification performance. It holds promise for real-time applications in MI-based BCI systems.

A Bibliometric Review of Brain-Computer Interfaces in Motor Imagery and Steady-State Visually Evoked Potentials for Applications in Rehabilitation and Robotics

In this paper, a bibliometric review is conducted on brain-computer interfaces (BCI) in non-invasive paradigms like motor imagery (MI) and steady-state visually evoked potentials (SSVEP) for applications in rehabilitation and robotics. An exploratory and descriptive approach is used in the analysis. Computational tools such as the biblioshiny application for R-Bibliometrix and VOSViewer are employed to generate data on years, sources, authors, affiliation, country, documents, co-author, co-citation, and co-occurrence. This article allows for the identification of different bibliometric indicators such as the research process, evolution, visibility, volume, influence, impact, and production in the field of brain-computer interfaces for MI and SSVEP paradigms in rehabilitation and robotics applications from 2000 to August 2024.

ESI-GAL: EEG source imaging-based trajectory estimation for grasp and lift task

BACKGROUND

Electroencephalogram (EEG) signals-based motor kinematics prediction (MKP) has been an active area of research to develop Brain-computer interface (BCI) systems such as exosuits, prostheses, and rehabilitation devices. However, EEG source imaging (ESI) based kinematics prediction is sparsely explored in the literature.

METHOD

In this study, pre-movement EEG features are utilized to predict three-dimensional (3D) hand kinematics for the grasp-and-lift motor task. A public dataset, WAY-EEG-GAL, is utilized for MKP analysis. In particular, sensor-domain (EEG data) and source-domain (ESI data) based features from the frontoparietal region are explored for MKP. Deep learning-based models are explored to achieve efficient kinematics decoding. Various time-lagged and window sizes are analyzed for hand kinematics prediction. Subsequently, intra-subject and inter-subject MKP analysis is performed to investigate the subject-specific and subject-independent motor-learning capabilities of the neural decoders. The Pearson correlation coefficient (PCC) is used as the performance metric for kinematics trajectory decoding.

RESULTS

The rEEGNet neural decoder achieved the best performance with sensor-domain and source-domain features with the time lag and window size of 100ms and 450ms, respectively. The highest mean PCC values of 0.790, 0.795, and 0.637 are achieved using sensor-domain features, while 0.769, 0.777, and 0.647 are achieved using source-domain features in x, y, and z-directions, respectively.

CONCLUSION

This study explores the feasibility of trajectory prediction using EEG sensor-domain and source-domain features for the grasp-and-lift task. Furthermore, inter-subject trajectory estimation is performed using the proposed deep learning decoder with EEG source domain features.

A multi-feature fusion graph attention network for decoding motor imagery intention in spinal cord injury patients

Objective.Electroencephalogram (EEG) signals exhibit temporal-frequency-spatial multi-domain feature, and due to the nonplanar nature of the brain surface, the electrode distributions follow non-Euclidean topology. To fully resolve the EEG signals, this study proposes a temporal-frequency-spatial multi-domain feature fusion graph attention network (GAT) for motor imagery (MI) intention recognition in spinal cord injury (SCI) patients.Approach.The proposed model uses phase-locked value (PLV) to extract spatial phase connectivity information between EEG channels and continuous wavelet transform to extract valid EEG information in the time-frequency domain. It then models as a graph data structure containing multi-domain information. The gated recurrent unit and GAT learn EEG's dynamic temporal-spatial information. Finally, the fully connected layer outputs the MI intention recognition results.Main results.After 10 times 10-fold cross-validation, the proposed model can achieve an average accuracy of 95.82%. Furthermore, this study analyses the event-related desynchronization/event-related synchronization and PLV brain network to explore the brain activity of SCI patients during MI.Significance.This study confirms the potential of the proposed model in terms of EEG decoding performance and provides a reference for the mechanism of neural activity in SCI patients.

Motion Cognitive Decoding of Cross-Subject Motor Imagery Guided on Different Visual Stimulus Materials

BACKGROUND

Motor imagery (MI) plays an important role in brain-computer interfaces, especially in evoking event-related desynchronization and synchronization (ERD/S) rhythms in electroencephalogram (EEG) signals. However, the procedure for performing a MI task for a single subject is subjective, making it difficult to determine the actual situation of an individual's MI task and resulting in significant individual EEG response variations during motion cognitive decoding.

METHODS

To explore this issue, we designed three visual stimuli (arrow, human, and robot), each of which was used to present three MI tasks (left arm, right arm, and feet), and evaluated differences in brain response in terms of ERD/S rhythms. To compare subject-specific variations of different visual stimuli, a novel cross-subject MI-EEG classification method was proposed for the three visual stimuli. The proposed method employed a covariance matrix centroid alignment for preprocessing of EEG samples, followed by a model agnostic meta-learning method for cross-subject MI-EEG classification.

RESULTS AND CONCLUSION

The experimental results showed that robot stimulus materials were better than arrow or human stimulus materials, with an optimal cross-subject motion cognitive decoding accuracy of 79.04%. Moreover, the proposed method produced robust classification of cross-subject MI-EEG signal decoding, showing superior results to conventional methods on collected EEG signals.

Refining ADHD diagnosis with EEG: The impact of preprocessing and temporal segmentation on classification accuracy

BACKGROUND

EEG signals are commonly used in ADHD diagnosis, but they are often affected by noise and artifacts. Effective preprocessing and segmentation methods can significantly enhance the accuracy and reliability of ADHD classification.

METHODS

We applied filtering, ASR, and ICA preprocessing techniques to EEG data from children with ADHD and neurotypical controls. The EEG recordings were segmented, and features were extracted and selected based on statistical significance. Classification was performed using various EEG segments and channels with Machine Learning models (SVM, KNN, and XGBoost) to identify the most effective combinations for accurate ADHD diagnosis.

RESULTS

Our findings show that models trained on later EEG segments achieved significantly higher accuracy, indicating the potential role of cognitive fatigue in distinguishing ADHD. The highest classification accuracy (86.1%) was achieved using data from the P3, P4, and C3 channels, with key features such as Kurtosis, Katz fractal dimension, and power spectrums in the Delta, Theta, and Alpha bands contributing to the results.

CONCLUSION

This study highlights the importance of preprocessing and segmentation in improving the reliability of ADHD diagnosis through EEG. The results suggest that further research on cognitive fatigue and segmentation could enhance diagnostic accuracy in ADHD patients.

MACNet: A Multidimensional Attention-Based Convolutional Neural Network for Lower-Limb Motor Imagery Classification

Decoding lower-limb motor imagery (MI) is highly important in brain-computer interfaces (BCIs) and rehabilitation engineering. However, it is challenging to classify lower-limb MI from electroencephalogram (EEG) signals, because lower-limb motions (LLMs) including MI are excessively close to physiological representations in the human brain and generate low-quality EEG signals. To address this challenge, this paper proposes a multidimensional attention-based convolutional neural network (CNN), termed MACNet, which is specifically designed for lower-limb MI classification. MACNet integrates a temporal refining module and an attention-enhanced convolutional module by leveraging the local and global feature representation abilities of CNNs and attention mechanisms. The temporal refining module adaptively investigates critical information from each electrode channel to refine EEG signals along the temporal dimension. The attention-enhanced convolutional module extracts temporal and spatial features while refining the feature maps across the channel and spatial dimensions. Owing to the scarcity of public datasets available for lower-limb MI, a specified lower-limb MI dataset involving four routine LLMs is built, consisting of 10 subjects over 20 sessions. Comparison experiments and ablation studies are conducted on this dataset and a public BCI Competition IV 2a EEG dataset. The experimental results show that MACNet achieves state-of-the-art performance and outperforms alternative models for the subject-specific mode. Visualization analysis reveals the excellent feature learning capabilities of MACNet and the potential relationship between lower-limb MI and brain activity. The effectiveness and generalizability of MACNet are verified.

Decoding Imagined Speech from EEG Data: A Hybrid Deep Learning Approach to Capturing Spatial and Temporal Features

Neuroimaging is revolutionizing our ability to investigate the brain's structural and functional properties, enabling us to visualize brain activity during diverse mental processes and actions. One of the most widely used neuroimaging techniques is electroencephalography (EEG), which records electrical activity from the brain using electrodes positioned on the scalp. EEG signals capture both spatial (brain region) and temporal (time-based) data. While a high temporal resolution is achievable with EEG, spatial resolution is comparatively limited. Consequently, capturing both spatial and temporal information from EEG data to recognize mental activities remains challenging. In this paper, we represent spatial and temporal information obtained from EEG signals by transforming EEG data into sequential topographic brain maps. We then apply hybrid deep learning models to capture the spatiotemporal features of the EEG topographic images and classify imagined English words. The hybrid framework utilizes a sequential combination of three-dimensional convolutional neural networks (3DCNNs) and recurrent neural networks (RNNs). The experimental results reveal the effectiveness of the proposed approach, achieving an average accuracy of 77.8% in identifying imagined English speech.

Motor task-to-task transfer learning for motor imagery brain-computer interfaces

Motor imagery (MI) is one of the popular control paradigms in the non-invasive brain-computer interface (BCI) field. MI-BCI generally requires users to conduct the imagination of movement (e.g., left or right hand) to collect training data for generating a classification model during the calibration phase. However, this calibration phase is generally time-consuming and tedious, as users conduct the imagination of hand movement several times without being given feedback for an extended period. This obstacle makes MI-BCI non user-friendly and hinders its use. On the other hand, motor execution (ME) and motor observation (MO) are relatively easier tasks, yield lower fatigue than MI, and share similar neural mechanisms to MI. However, few studies have integrated these three tasks into BCIs. In this study, we propose a new task-to-task transfer learning approach of 3-motor tasks (ME, MO, and MI) for building a better user-friendly MI-BCI. For this study, 28 subjects participated in 3-motor tasks experiment, and electroencephalography (EEG) was acquired. User opinions regarding the 3-motor tasks were also collected through questionnaire survey. The 3-motor tasks showed a power decrease in the alpha rhythm, known as event-related desynchronization, but with slight differences in the temporal patterns. In the classification analysis, the cross-validated accuracy (within-task) was 67.05 % for ME, 65.93 % for MI, and 73.16 % for MO on average. Consistently with the results, the subjects scored MI (3.16) as the most difficult task compared with MO (1.42) and ME (1.41), with p < 0.05. In the analysis of task-to-task transfer learning, where training and testing are performed using different task datasets, the ME-trained model yielded an accuracy of 65.93 % (MI test), which is statistically similar to the within-task accuracy (p > 0.05). The MO-trained model achieved an accuracy of 60.82 % (MI test). On the other hand, combining two datasets yielded interesting results. ME and 50 % of the MI-trained model (50-shot) classified MI with a 69.21 % accuracy, which outperformed the within-task accuracy (p < 0.05), and MO and 50 % of the MI-trained model showed an accuracy of 66.75 %. Of the low performers with a within-task accuracy of 70 % or less, 90 % (n = 21) of the subjects improved in training with ME, and 76.2 % (n = 16) improved in training with MO on the MI test at 50-shot. These results demonstrate that task-to-task transfer learning is possible and could be a promising approach to building a user-friendly training protocol in MI-BCI.

IMH-Net: a convolutional neural network for end-to-end EEG motor imagery classification

As the main component of Brain-computer interface (BCI) technology, the classification algorithm based on EEG has developed rapidly. The previous algorithms were often based on subject-dependent settings, resulting in BCI needing to be calibrated for new users. In this work, we propose IMH-Net, an end-to-end subject-independent model. The model first uses Inception blocks extracts the frequency domain features of the data, then further compresses the feature vectors to extract the spatial domain features, and finally learns the global information and classification through Multi-Head Attention mechanism. On the OpenBMI dataset, IMH-Net obtained 73.90 ± 13.10% accuracy and 73.09 ± 14.99% F1-score in subject-independent manner, which improved the accuracy by 1.96% compared with the comparison model. On the BCI competition IV dataset 2a, this model also achieved the highest accuracy and F1-score in subject-dependent manner. The IMH-Net model we proposed can improve the accuracy of subject-independent Motor Imagery (MI), and the robustness of the algorithm is high, which has strong practical value in the field of BCI.

Leveraging temporal dependency for cross-subject-MI BCIs by contrastive learning and self-attention

Brain-computer interfaces (BCIs) built based on motor imagery paradigm have found extensive utilization in motor rehabilitation and the control of assistive applications. However, traditional MI-BCI systems often exhibit suboptimal classification performance and require significant time for new users to collect subject-specific training data. This limitation diminishes the user-friendliness of BCIs and presents significant challenges in developing effective subject-independent models. In response to these challenges, we propose a novel subject-independent framework for learning temporal dependency for motor imagery BCIs by Contrastive Learning and Self-attention (CLS). In CLS model, we incorporate self-attention mechanism and supervised contrastive learning into a deep neural network to extract important information from electroencephalography (EEG) signals as features. We evaluate the CLS model using two large public datasets encompassing numerous subjects in a subject-independent experiment condition. The results demonstrate that CLS outperforms six baseline algorithms, achieving a mean classification accuracy improvement of 1.3 % and 4.71 % than the best algorithm on the Giga dataset and OpenBMI dataset, respectively. Our findings demonstrate that CLS can effectively learn invariant discriminative features from training data obtained from non-target subjects, thus showcasing its potential for building models for new users without the need for calibration.

Attention-based cross-frequency graph convolutional network for driver fatigue estimation

Fatigue driving significantly contributes to global vehicle accidents and fatalities, making driver fatigue level estimation crucial. Electroencephalography (EEG) is a proven reliable predictor of brain states. With Deep Learning (DL) advancements, brain state estimation algorithms have improved significantly. Nonetheless, EEG's multi-domain nature and the intricate spatial-temporal-frequency correlations among EEG channels present challenges in developing precise DL models. In this work, we introduce an innovative Attention-based Cross-Frequency Graph Convolutional Network (ACF-GCN) for estimating drivers' reaction times using EEG signals from theta, alpha, and beta bands. This method utilizes a multi-head attention mechanism to detect long-range dependencies between EEG channels across frequencies. Concurrently, the transformer's encoder module learns node-level feature maps from the attention-score matrix. Subsequently, the Graph Convolutional Network (GCN) integrates this matrix with feature maps to estimate driver reaction time. Our validation on a publicly available dataset shows that ACF-GCN outperforms several state-of-the-art methods. We also explore the brain dynamics within the cross-frequency attention-score matrix, identifying theta and alpha bands as key influencers in fatigue estimating performance. The ACF-GCN method advances brain state estimation and provides insights into the brain dynamics underlying multi-channel EEG signals.

An effective classification approach for EEG-based motor imagery tasks combined with attention mechanisms

Currently, electroencephalogram (EEG)-based motor imagery (MI) signals have been received extensive attention, which can assist disabled subjects to control wheelchair, automatic driving and other activities. However, EEG signals are easily affected by some factors, such as muscle movements, wireless devices, power line, etc., resulting in the low signal-to-noise ratios and the worse recognition results on EEG decoding. Therefore, it is crucial to develop a stable model for decoding MI-EEG signals. To address this issue and further improve the decoding performance for MI tasks, a hybrid structure combining convolutional neural networks and bidirectional long short-term memory (BLSTM) model, namely CBLSTM, is developed in this study to handle the various EEG-based MI tasks. Besides, the attention mechanism (AM) model is further adopted to adaptively assign the weight of EEG vital features and enhance the expression which beneficial to classification for MI tasks. First of all, the spatial features and the time series features are extracted by CBLSTM from preprocessed MI-EEG data, respectively. Meanwhile, more effective features information can be mined by the AM model, and the softmax function is utilized to recognize intention categories. Ultimately, the numerical results illustrate that the model presented achieves an average accuracy of 98.40% on the public physioNet dataset and faster training process for decoding MI tasks, which is superior to some other advanced models. Ablation experiment performed also verifies the effectiveness and feasibility of the developed model. Moreover, the established network model provides a good basis for the application of brain-computer interface in rehabilitation medicine.

A Comprehensive Review of Hardware Acceleration Techniques and Convolutional Neural Networks for EEG Signals

This paper comprehensively reviews hardware acceleration techniques and the deployment of convolutional neural networks (CNNs) for analyzing electroencephalogram (EEG) signals across various application areas, including emotion classification, motor imagery, epilepsy detection, and sleep monitoring. Previous reviews on EEG have mainly focused on software solutions. However, these reviews often overlook key challenges associated with hardware implementation, such as scenarios that require a small size, low power, high security, and high accuracy. This paper discusses the challenges and opportunities of hardware acceleration for wearable EEG devices by focusing on these aspects. Specifically, this review classifies EEG signal features into five groups and discusses hardware implementation solutions for each category in detail, providing insights into the most suitable hardware acceleration strategies for various application scenarios. In addition, it explores the complexity of efficient CNN architectures for EEG signals, including techniques such as pruning, quantization, tensor decomposition, knowledge distillation, and neural architecture search. To the best of our knowledge, this is the first systematic review that combines CNN hardware solutions with EEG signal processing. By providing a comprehensive analysis of current challenges and a roadmap for future research, this paper provides a new perspective on the ongoing development of hardware-accelerated EEG systems.

EEG-based motor imagery channel selection and classification using hybrid optimization and two-tier deep learning

Brain-computer interface (BCI) technology holds promise for individuals with profound motor impairments, offering the potential for communication and control. Motor imagery (MI)-based BCI systems are particularly relevant in this context. Despite their potential, achieving accurate and robust classification of MI tasks using electroencephalography (EEG) data remains a significant challenge. In this paper, we employed the Minimum Redundancy Maximum Relevance (MRMR) algorithm to optimize channel selection. Furthermore, we introduced a hybrid optimization approach that combines the War Strategy Optimization (WSO) and Chimp Optimization Algorithm (ChOA). This hybridization significantly enhances the classification model's overall performance and adaptability. A two-tier deep learning architecture is proposed for classification, consisting of a Convolutional Neural Network (CNN) and a modified Deep Neural Network (M-DNN). The CNN focuses on capturing temporal correlations within EEG data, while the M-DNN is designed to extract high-level spatial characteristics from selected EEG channels. Integrating optimal channel selection, hybrid optimization, and the two-tier deep learning methodology in our BCI framework presents an enhanced approach for precise and effective BCI control. Our model got 95.06% accuracy with high precision. This advancement has the potential to significantly impact neurorehabilitation and assistive technology applications, facilitating improved communication and control for individuals with motor impairments.

CTNet: a convolutional transformer network for EEG-based motor imagery classification

Brain-computer interface (BCI) technology bridges the direct communication between the brain and machines, unlocking new possibilities for human interaction and rehabilitation. EEG-based motor imagery (MI) plays a pivotal role in BCI, enabling the translation of thought into actionable commands for interactive and assistive technologies. However, the constrained decoding performance of brain signals poses a limitation to the broader application and development of BCI systems. In this study, we introduce a convolutional Transformer network (CTNet) designed for EEG-based MI classification. Firstly, CTNet employs a convolutional module analogous to EEGNet, dedicated to extracting local and spatial features from EEG time series. Subsequently, it incorporates a Transformer encoder module, leveraging a multi-head attention mechanism to discern the global dependencies of EEG's high-level features. Finally, a straightforward classifier module comprising fully connected layers is followed to categorize EEG signals. In subject-specific evaluations, CTNet achieved remarkable decoding accuracies of 82.52% and 88.49% on the BCI IV-2a and IV-2b datasets, respectively. Furthermore, in the challenging cross-subject assessments, CTNet achieved recognition accuracies of 58.64% on the BCI IV-2a dataset and 76.27% on the BCI IV-2b dataset. In both subject-specific and cross-subject evaluations, CTNet holds a leading position when compared to some of the state-of-the-art methods. This underscores the exceptional efficacy of our approach and its potential to set a new benchmark in EEG decoding.

Evaluation of perceived urgency from single-trial EEG data elicited by upper-body vibration feedback using deep learning

Notification systems that convey urgency without adding cognitive burden are crucial in human-computer interaction. Haptic feedback systems, particularly those utilizing vibration feedback, have emerged as a compelling solution, capable of providing desirable levels of urgency depending on the application. High-risk applications require an evaluation of the urgency level elicited during critical notifications. Traditional evaluations of perceived urgency rely on subjective self-reporting and performance metrics, which, while useful, are not real-time and can be distracting from the task at hand. In contrast, EEG technology offers a direct, non-intrusive method of assessing the user's cognitive state. Leveraging deep learning, this study introduces a novel approach to evaluate perceived urgency from single-trial EEG data, induced by vibration stimuli on the upper body, utilizing our newly collected urgency-via-vibration dataset. The proposed model combines a 2D convolutional neural network with a temporal convolutional network to capture spatial and temporal EEG features, outperforming several established EEG models. The proposed model achieves an average classification accuracy of 83% through leave-one-subject-out cross-validation across three urgency classes (not urgent, urgent, and very urgent) from a single trial of EEG data. Furthermore, explainability analysis showed that the prefrontal brain region, followed by the central brain region, are the most influential in predicting the urgency level. A follow-up neural statistical analysis revealed an increase in event-related synchronization (ERS) in the theta frequency band (4-7 Hz) with the increased level of urgency, which is associated with high arousal and attention in the neuroscience literature. A limitation of this study is that the proposed model's performance was tested only the urgency-via-vibration dataset, which may affect the generalizability of the findings.

Riemannian Locality Preserving Method for Transfer Learning With Applications on Brain-Computer Interface

Brain-computer interfaces (BCIs) have been widely focused and extensively studied in recent years for their huge prospect of medical rehabilitation and commercial applications. Transfer learning exploits the information in the source domain and applies in another different but related domain (target domain), and is therefore introduced into the BCIs to figure out the inter-subject variances of electroencephalography (EEG) signals. In this article, a novel transfer learning method is proposed to preserve the Riemannian locality of data structure in both the source and target domains and simultaneously realize the joint distribution adaptation of both domains to enhance the effectiveness of transfer learning. Specifically, a Riemannian graph is first defined and constructed based on the Riemannian distance to represent the Riemannian geometry information. To simultaneously align the marginal and conditional distribution of source and target domains and preserve the Riemannian locality of data structure in both domains, the Riemannian graph is embedded in the joint distribution adaptation (JDA) framework and forms the proposed Riemannian locality preserving-based transfer learning (RLPTL). To validate the effect of the proposed method, it is compared with several existing methods on two open motor imagery datasets, and both multi-source domains (MSD) and single-source domains (SSD) experiments are considered. Experimental results show that the proposed method achieves the highest accuracies in MSD and SSD experiments on three datasets and outperforms eight baseline methods, which demonstrates that the proposed method creates a feasible and efficient way to realize transfer learning.

Suppression of negative transfer in motor imagery brain-computer interface based on mutual information and Pearson correlation coefficient

The focus of this paper is on the main challenges in brain-computer interface transfer learning: how to address data characteristic length and the source domain sample selection problems caused by individual differences. To overcome the negative migration that results from feature length, we propose a migration algorithm based on mutual information transfer (MIT), which selects effective features by calculating the entropy value of the probability distribution and conditional distribution, thereby reducing negative migration and improving learning efficiency. Source domain participants who differ too much from the target domain distribution can affect the overall classification performance. On the basis of MIT, we propose the Pearson correlation coefficient source domain automatic selection algorithm (PDAS algorithm). The PDAS algorithm can automatically select the appropriate source domain participants according to the target domain distribution, which reduces the negative migration of participant data among the source domain participants, improves experimental accuracy, and greatly reduces training time. The two proposed algorithms were tested offline and online on two public datasets, and the results were compared with those from existing advanced algorithms. The experimental results showed that the MIT algorithm and the MIT + PDAS algorithm had obvious advantages.

A review on evaluating mental stress by deep learning using EEG signals

Mental stress is a common problem that affects individuals all over the world. Stress reduces human functionality during routine work and may lead to severe health defects. Early detection of stress is important for preventing diseases and other negative health-related consequences of stress. Several neuroimaging techniques have been utilized to assess mental stress, however, due to its ease of use, robustness, and non-invasiveness, electroencephalography (EEG) is commonly used. This paper aims to fill a knowledge gap by reviewing the different EEG-related deep learning algorithms with a focus on Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs) for the evaluation of mental stress. The review focuses on data representation, individual deep neural network model architectures, hybrid models, and results amongst others. The contributions of the paper address important issues such as data representation and model architectures. Out of all reviewed papers, 67% used CNN, 9% LSTM, and 24% hybrid models. Based on the reviewed literature, we found that dataset size and different representations contributed to the performance of the proposed networks. Raw EEG data produced classification accuracy around 62% while using spectral and topographical representation produced up to 88%. Nevertheless, the roles of generalizability across different deep learning models and individual differences remain key areas of inquiry. The review encourages the exploration of innovative avenues, such as EEG data image representations concurrently with graph convolutional neural networks (GCN), to mitigate the impact of inter-subject variability. This novel approach not only allows us to harmonize structural nuances within the data but also facilitates the integration of temporal dynamics, thereby enabling a more comprehensive assessment of mental stress levels.

MSE-VGG: A Novel Deep Learning Approach Based on EEG for Rapid Ischemic Stroke Detection

Ischemic stroke is a type of brain dysfunction caused by pathological changes in the blood vessels of the brain which leads to brain tissue ischemia and hypoxia and ultimately results in cell necrosis. Without timely and effective treatment in the early time window, ischemic stroke can lead to long-term disability and even death. Therefore, rapid detection is crucial in patients with ischemic stroke. In this study, we developed a deep learning model based on fusion features extracted from electroencephalography (EEG) signals for the fast detection of ischemic stroke. Specifically, we recruited 20 ischemic stroke patients who underwent EEG examination during the acute phase of stroke and collected EEG signals from 19 adults with no history of stroke as a control group. Afterwards, we constructed correlation-weighted Phase Lag Index (cwPLI), a novel feature, to explore the synchronization information and functional connectivity between EEG channels. Moreover, the spatio-temporal information from functional connectivity and the nonlinear information from complexity were fused by combining the cwPLI matrix and Sample Entropy (SaEn) together to further improve the discriminative ability of the model. Finally, the novel MSE-VGG network was employed as a classifier to distinguish ischemic stroke from non-ischemic stroke data. Five-fold cross-validation experiments demonstrated that the proposed model possesses excellent performance, with accuracy, sensitivity, and specificity reaching 90.17%, 89.86%, and 90.44%, respectively. Experiments on time consumption verified that the proposed method is superior to other state-of-the-art examinations. This study contributes to the advancement of the rapid detection of ischemic stroke, shedding light on the untapped potential of EEG and demonstrating the efficacy of deep learning in ischemic stroke identification.

A deep neural network and transfer learning combined method for cross-task classification of error-related potentials

Background

Error-related potentials (ErrPs) are electrophysiological responses that naturally occur when humans perceive wrongdoing or encounter unexpected events. It offers a distinctive means of comprehending the error-processing mechanisms within the brain. A method for detecting ErrPs with high accuracy holds significant importance for various ErrPs-based applications, such as human-in-the-loop Brain-Computer Interface (BCI) systems. Nevertheless, current methods fail to fulfill the generalization requirements for detecting such ErrPs due to the high non-stationarity of EEG signals across different tasks and the limited availability of ErrPs datasets.

Methods

This study introduces a deep learning-based model that integrates convolutional layers and transformer encoders for the classification of ErrPs. Subsequently, a model training strategy, grounded in transfer learning, is proposed for the effective training of the model. The datasets utilized in this study are available for download from the publicly accessible databases.

Results

In cross-task classification, an average accuracy of about 78% was achieved, exceeding the baseline. Furthermore, in the leave-one-subject-out, within-session, and cross-session classification scenarios, the proposed model outperformed the existing techniques with an average accuracy of 71.81, 78.74, and 77.01%, respectively.

Conclusions

Our approach contributes to mitigating the challenge posed by limited datasets in the ErrPs field, achieving this by reducing the requirement for extensive training data for specific target tasks. This may serve as inspiration for future studies that concentrate on ErrPs and their applications.

Comprehensive evaluation methods for translating BCI into practical applications: usability, user satisfaction and usage of online BCI systems

Although brain-computer interface (BCI) is considered a revolutionary advancement in human-computer interaction and has achieved significant progress, a considerable gap remains between the current technological capabilities and their practical applications. To promote the translation of BCI into practical applications, the gold standard for online evaluation for classification algorithms of BCI has been proposed in some studies. However, few studies have proposed a more comprehensive evaluation method for the entire online BCI system, and it has not yet received sufficient attention from the BCI research and development community. Therefore, the qualitative leap from analyzing and modeling for offline BCI data to the construction of online BCI systems and optimizing their performance is elaborated, and then user-centred is emphasized, and then the comprehensive evaluation methods for translating BCI into practical applications are detailed and reviewed in the article, including the evaluation of the usability (including effectiveness and efficiency of systems), the evaluation of the user satisfaction (including BCI-related aspects, etc.), and the evaluation of the usage (including the match between the system and user, etc.) of online BCI systems. Finally, the challenges faced in the evaluation of the usability and user satisfaction of online BCI systems, the efficacy of online BCI systems, and the integration of BCI and artificial intelligence (AI) and/or virtual reality (VR) and other technologies to enhance the intelligence and user experience of the system are discussed. It is expected that the evaluation methods for online BCI systems elaborated in this review will promote the translation of BCI into practical applications.

EEG-based emotion recognition systems; comprehensive study

Emotion recognition technology through EEG signal analysis is currently a fundamental concept in artificial intelligence. This recognition has major practical implications in emotional health care, human-computer interaction, and so on. This paper provides a comprehensive study of different methods for extracting electroencephalography (EEG) features for emotion recognition from four different perspectives, including time domain features, frequency domain features, time-frequency features, and nonlinear features. We summarize the current pattern recognition methods adopted in most related works, and with the rapid development of deep learning (DL) attracting the attention of researchers in this field, we pay more attention to deep learning-based studies and analyse the characteristics, advantages, disadvantages, and applicable scenarios. Finally, the current challenges and future development directions in this field were summarized. This paper can help novice researchers in this field gain a systematic understanding of the current status of emotion recognition research based on EEG signals and provide ideas for subsequent related research.

Optimal Channel Selection of Multiclass Motor Imagery Classification Based on Fusion Convolutional Neural Network with Attention Blocks

The widely adopted paradigm in brain-computer interfaces (BCIs) involves motor imagery (MI), enabling improved communication between humans and machines. EEG signals derived from MI present several challenges due to their inherent characteristics, which lead to a complex process of classifying and finding the potential tasks of a specific participant. Another issue is that BCI systems can result in noisy data and redundant channels, which in turn can lead to increased equipment and computational costs. To address these problems, the optimal channel selection of a multiclass MI classification based on a Fusion convolutional neural network with Attention blocks (FCNNA) is proposed. In this study, we developed a CNN model consisting of layers of convolutional blocks with multiple spatial and temporal filters. These filters are designed specifically to capture the distribution and relationships of signal features across different electrode locations, as well as to analyze the evolution of these features over time. Following these layers, a Convolutional Block Attention Module (CBAM) is used to, further, enhance EEG signal feature extraction. In the process of channel selection, the genetic algorithm is used to select the optimal set of channels using a new technique to deliver fixed as well as variable channels for all participants. The proposed methodology is validated showing 6.41% improvement in multiclass classification compared to most baseline models. Notably, we achieved the highest results of 93.09% for binary classes involving left-hand and right-hand movements. In addition, the cross-subject strategy for multiclass classification yielded an impressive accuracy of 68.87%. Following channel selection, multiclass classification accuracy was enhanced, reaching 84.53%. Overall, our experiments illustrated the efficiency of the proposed EEG MI model in both channel selection and classification, showing superior results with either a full channel set or a reduced number of channels.

A Data Augmentation Method for Motor Imagery EEG Signals Based on DCGAN-GP Network

Motor imagery electroencephalography (EEG) signals have garnered attention in brain-computer interface (BCI) research due to their potential in promoting motor rehabilitation and control. However, the limited availability of labeled data poses challenges for training robust classifiers. In this study, we propose a novel data augmentation method utilizing an improved Deep Convolutional Generative Adversarial Network with Gradient Penalty (DCGAN-GP) to address this issue. We transformed raw EEG signals into two-dimensional time-frequency maps and employed a DCGAN-GP network to generate synthetic time-frequency representations resembling real data. Validation experiments were conducted on the BCI IV 2b dataset, comparing the performance of classifiers trained with augmented and unaugmented data. Results demonstrated that classifiers trained with synthetic data exhibit enhanced robustness across multiple subjects and achieve higher classification accuracy. Our findings highlight the effectiveness of utilizing a DCGAN-GP-generated synthetic EEG data to improve classifier performance in distinguishing different motor imagery tasks. Thus, the proposed data augmentation method based on a DCGAN-GP offers a promising avenue for enhancing BCI system performance, overcoming data scarcity challenges, and bolstering classifier robustness, thereby providing substantial support for the broader adoption of BCI technology in real-world applications.

EISATC-Fusion: Inception Self-Attention Temporal Convolutional Network Fusion for Motor Imagery EEG Decoding

The motor imagery brain-computer interface (MI-BCI) based on electroencephalography (EEG) is a widely used human-machine interface paradigm. However, due to the non-stationarity and individual differences among subjects in EEG signals, the decoding accuracy is limited, affecting the application of the MI-BCI. In this paper, we propose the EISATC-Fusion model for MI EEG decoding, consisting of inception block, multi-head self-attention (MSA), temporal convolutional network (TCN), and layer fusion. Specifically, we design a DS Inception block to extract multi-scale frequency band information. And design a new cnnCosMSA module based on CNN and cos attention to solve the attention collapse and improve the interpretability of the model. The TCN module is improved by the depthwise separable convolution to reduces the parameters of the model. The layer fusion consists of feature fusion and decision fusion, fully utilizing the features output by the model and enhances the robustness of the model. We improve the two-stage training strategy for model training. Early stopping is used to prevent model overfitting, and the accuracy and loss of the validation set are used as indicators for early stopping. The proposed model achieves within-subject classification accuracies of 84.57% and 87.58% on BCI Competition IV Datasets 2a and 2b, respectively. And the model achieves cross-subject classification accuracies of 67.42% and 71.23% (by transfer learning) when training the model with two sessions and one session of Dataset 2a, respectively. The interpretability of the model is demonstrated through weight visualization method.

Prefrontal Cortex Activity During Gait in People With Persistent Symptoms After Concussion

BACKGROUND

Concussions result in transient symptoms stemming from a cortical metabolic energy crisis. Though this metabolic energy crisis typically resolves in a month, symptoms can persist for years. The symptomatic period is associated with gait dysfunction, the cortical underpinnings of which are poorly understood. Quantifying prefrontal cortex (PFC) activity during gait may provide insight into post-concussion gait dysfunction. The purpose of this study was to explore the effects of persisting concussion symptoms on PFC activity during gait. We hypothesized that adults with persisting concussion symptoms would have greater PFC activity during gait than controls. Within the concussed group, we hypothesized that worse symptoms would relate to increased PFC activity during gait, and that increased PFC activity would relate to worse gait characteristics.

METHODS

The Neurobehavior Symptom Inventory (NSI) characterized concussion symptoms. Functional near-infrared spectroscopy quantified PFC activity (relative concentration changes of oxygenated hemoglobin [HbO2]) in 14 people with a concussion and 25 controls. Gait was assessed using six inertial sensors in the concussion group.

RESULTS

Average NSI total score was 26.4 (13.2). HbO2 was significantly higher (P = .007) for the concussed group (0.058 [0.108]) compared to the control group (-0.016 [0.057]). Within the concussion group, HbO2 correlated with NSI total symptom score (ρ = .62; P = .02), sagittal range of motion (r = .79; P = .001), and stride time variability (r = -.54; P = .046).

CONCLUSION

These data suggest PFC activity relates to symptom severity and some gait characteristics in people with persistent concussion symptoms. Identifying the neurophysiological underpinnings to gait deficits post-concussion expands our knowledge of motor behavior deficits in people with persistent concussion symptoms.

Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

Bidirectional feature pyramid attention-based temporal convolutional network model for motor imagery electroencephalogram classification

Introduction

As an interactive method gaining popularity, brain-computer interfaces (BCIs) aim to facilitate communication between the brain and external devices. Among the various research topics in BCIs, the classification of motor imagery using electroencephalography (EEG) signals has the potential to greatly improve the quality of life for people with disabilities.

Methods

This technology assists them in controlling computers or other devices like prosthetic limbs, wheelchairs, and drones. However, the current performance of EEG signal decoding is not sufficient for real-world applications based on Motor Imagery EEG (MI-EEG). To address this issue, this study proposes an attention-based bidirectional feature pyramid temporal convolutional network model for the classification task of MI-EEG. The model incorporates a multi-head self-attention mechanism to weigh significant features in the MI-EEG signals. It also utilizes a temporal convolution network (TCN) to separate high-level temporal features. The signals are enhanced using the sliding-window technique, and channel and time-domain information of the MI-EEG signals is extracted through convolution.

Results

Additionally, a bidirectional feature pyramid structure is employed to implement attention mechanisms across different scales and multiple frequency bands of the MI-EEG signals. The performance of our model is evaluated on the BCI Competition IV-2a dataset and the BCI Competition IV-2b dataset, and the results showed that our model outperformed the state-of-the-art baseline model, with an accuracy of 87.5 and 86.3% for the subject-dependent, respectively.

Discussion

In conclusion, the BFATCNet model offers a novel approach for EEG-based motor imagery classification in BCIs, effectively capturing relevant features through attention mechanisms and temporal convolutional networks. Its superior performance on the BCI Competition IV-2a and IV-2b datasets highlights its potential for real-world applications. However, its performance on other datasets may vary, necessitating further research on data augmentation techniques and integration with multiple modalities to enhance interpretability and generalization. Additionally, reducing computational complexity for real-time applications is an important area for future work.

Toward Domain-Free Transformer for Generalized EEG Pre-Training

Electroencephalography (EEG) signals are the brain signals acquired using the non-invasive approach. Owing to the high portability and practicality, EEG signals have found extensive application in monitoring human physiological states across various domains. In recent years, deep learning methodologies have been explored to decode the intricate information embedded in EEG signals. However, since EEG signals are acquired from humans, it has issues with acquiring enormous amounts of data for training the deep learning models. Therefore, previous research has attempted to develop pre-trained models that could show significant performance improvement through fine-tuning when data are scarce. Nonetheless, existing pre-trained models often struggle with constraints, such as the necessity to operate within datasets of identical configurations or the need to distort the original data to apply the pre-trained model. In this paper, we proposed the domain-free transformer, called DFformer, for generalizing the EEG pre-trained model. In addition, we presented the pre-trained model based on DFformer, which is capable of seamless integration across diverse datasets without necessitating architectural modification or data distortion. The proposed model achieved competitive performance across motor imagery and sleep stage classification datasets. Notably, even when fine-tuned on datasets distinct from the pre-training phase, DFformer demonstrated marked performance enhancements. Hence, we demonstrate the potential of DFformer to overcome the conventional limitations in pre-trained model development, offering robust applicability across a spectrum of domains.

ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface

Convolutional neural networks (CNNs) have been successfully applied to motor imagery (MI)-based brain-computer interface (BCI). Nevertheless, single-scale CNN fail to extract abundant information over a wide spectrum from EEG signals, while typical multi-scale CNNs cannot effectively fuse information from different scales with concatenation-based methods. To overcome these challenges, we propose a new scheme equipped with attention-based dual-scale fusion convolutional neural network (ADFCNN), which jointly extracts and fuses EEG spectral and spatial information at different scales. This scheme also provides novel insight through self-attention for effective information fusion from different scales. Specifically, temporal convolutions with two different kernel sizes identify EEG μ and β rhythms, while spatial convolutions at two different scales generate global and detailed spatial information, respectively, and the self-attention mechanism performs feature fusion based on the internal similarity of the concatenated features extracted by the dual-scale CNN. The proposed scheme achieves the superior performance compared with state-of-the-art methods in subject-specific motor imagery recognition on BCI Competition IV dataset 2a, 2b and OpenBMI dataset, with the cross-session average classification accuracies of 79.39% and significant improvements of 9.14% on BCI-IV2a, 87.81% and 7.66% on BCI-IV2b, 65.26% and 7.2% on OpenBMI dataset, and the within-session average classification accuracies of 86.87% and significant improvements of 10.89% on BCI-IV2a, 87.26% and 8.07% on BCI-IV2b, 84.29% and 5.17% on OpenBMI dataset, respectively. What is more, ablation experiments are conducted to investigate the mechanism and demonstrate the effectiveness of the dual-scale joint temporal-spatial CNN and self-attention modules. Visualization is also used to reveal the learning process and feature distribution of the model.

A time segment adaptive optimization method based on separability criterion and correlation analysis for motor imagery BCIs

Motor imagery (MI) plays a crucial role in brain-computer interface (BCI), and the classification of MI tasks using electroencephalogram (EEG) is currently under extensive investigation. During MI classification, individual differences among subjects in terms of response and time latency need to be considered. Optimizing the time segment for different subjects can enhance subsequent classification performance. In view of the individual differences of subjects in motor imagery tasks, this article proposes a Time Segment Adaptive Optimization method based on Separability criterion and Correlation analysis (TSAOSC). The fundamental principle of this method involves applying the separability criterion to various sizes of time windows within the training data, identifying the optimal raw reference signal, and adaptively adjusting the time segment position for each trial's data by analyzing its relationship with the optimal reference signal. We evaluated our method on three BCI competition datasets, respectively. The utilization of the TSAOSC method in the experiments resulted in an enhancement of 4.90% in average classification accuracy compared to its absence. Additionally, building upon the TSAOSC approach, this study proposes a Nonlinear-TSAOSC method (N-TSAOSC) for analyzing EEG signals with nonlinearity, which shows improvements in the classification accuracy of certain subjects. The results of the experiments demonstrate that the proposed method is an effective time segment optimization method, and it can be integrated into other algorithms to further improve their accuracy.

Machine learning and deep learning-based approach in smart healthcare: Recent advances, applications, challenges and opportunities

In recent years, machine learning (ML) and deep learning (DL) have been the leading approaches to solving various challenges, such as disease predictions, drug discovery, medical image analysis, etc., in intelligent healthcare applications. Further, given the current progress in the fields of ML and DL, there exists the promising potential for both to provide support in the realm of healthcare. This study offered an exhaustive survey on ML and DL for the healthcare system, concentrating on vital state of the art features, integration benefits, applications, prospects and future guidelines. To conduct the research, we found the most prominent journal and conference databases using distinct keywords to discover scholarly consequences. First, we furnished the most current along with cutting-edge progress in ML-DL-based analysis in smart healthcare in a compendious manner. Next, we integrated the advancement of various services for ML and DL, including ML-healthcare, DL-healthcare, and ML-DL-healthcare. We then offered ML and DL-based applications in the healthcare industry. Eventually, we emphasized the research disputes and recommendations for further studies based on our observations.

EEG-FMCNN: A fusion multi-branch 1D convolutional neural network for EEG-based motor imagery classification

Motor imagery (MI) electroencephalogram (EEG) signal is recognized as a promising paradigm for brain-computer interface (BCI) systems and has been extensively employed in various BCI applications, including assisting disabled individuals, controlling devices and environments, and enhancing human capabilities. The high-performance decoding capability of MI-EEG signals is a key issue that impacts the development of the industry. However, decoding MI-EEG signals is challenging due to the low signal-to-noise ratio and inter-subject variability. In response to the aforementioned core problems, this paper proposes a novel end-to-end network, a fusion multi-branch 1D convolutional neural network (EEG-FMCNN), to decode MI-EEG signals without pre-processing. The utilization of multi-branch 1D convolution not only exhibits a certain level of noise tolerance but also addresses the issue of inter-subject variability to some extent. This is attributed to the ability of multi-branch architectures to capture information from different frequency bands, enabling the establishment of optimal convolutional scales and depths. Furthermore, we incorporate 1D squeeze-and-excitation (SE) blocks and shortcut connections at appropriate locations to further enhance the generalization and robustness of the network. In the BCI Competition IV-2a dataset, our proposed model has obtained good experimental results, achieving accuracies of 78.82% and 68.41% for subject-dependent and subject-independent modes, respectively. In addition, extensive ablative experiments and fine-tuning experiments were conducted, resulting in a notable 7% improvement in the average performance of the network, which holds significant implications for the generalization and application of the network.

Fog Computing for Control of Cyber-Physical Systems in Industry Using BCI

Brain-computer interfaces use signals from the brain, such as EEG, to determine brain states, which in turn can be used to issue commands, for example, to control industrial machinery. While Cloud computing can aid in the creation and operation of industrial multi-user BCI systems, the vast amount of data generated from EEG signals can lead to slow response time and bandwidth problems. Fog computing reduces latency in high-demand computation networks. Hence, this paper introduces a fog computing solution for BCI processing. The solution consists in using fog nodes that incorporate machine learning algorithms to convert EEG signals into commands to control a cyber-physical system. The machine learning module uses a deep learning encoder to generate feature images from EEG signals that are subsequently classified into commands by a random forest. The classification scheme is compared using various classifiers, being the random forest the one that obtained the best performance. Additionally, a comparison was made between the fog computing approach and using only cloud computing through the use of a fog computing simulator. The results indicate that the fog computing method resulted in less latency compared to the solely cloud computing approach.

A Combined Virtual Electrode-Based ESA and CNN Method for MI-EEG Signal Feature Extraction and Classification

A Brain-Computer Interface (BCI) is a medium for communication between the human brain and computers, which does not rely on other human neural tissues, but only decodes Electroencephalography (EEG) signals and converts them into commands to control external devices. Motor Imagery (MI) is an important BCI paradigm that generates a spontaneous EEG signal without external stimulation by imagining limb movements to strengthen the brain's compensatory function, and it has a promising future in the field of computer-aided diagnosis and rehabilitation technology for brain diseases. However, there are a series of technical difficulties in the research of motor imagery-based brain-computer interface (MI-BCI) systems, such as: large individual differences in subjects and poor performance of the cross-subject classification model; a low signal-to-noise ratio of EEG signals and poor classification accuracy; and the poor online performance of the MI-BCI system. To address the above problems, this paper proposed a combined virtual electrode-based EEG Source Analysis (ESA) and Convolutional Neural Network (CNN) method for MI-EEG signal feature extraction and classification. The outcomes reveal that the online MI-BCI system developed based on this method can improve the decoding ability of multi-task MI-EEG after training, it can learn generalized features from multiple subjects in cross-subject experiments and has some adaptability to the individual differences of new subjects, and it can decode the EEG intent online and realize the brain control function of the intelligent cart, which provides a new idea for the research of an online MI-BCI system.

A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding

Motor imagery (MI) decoding plays a crucial role in the advancement of electroencephalography (EEG)-based brain-computer interface (BCI) technology. Currently, most researches focus on complex deep learning structures for MI decoding. The growing complexity of networks may result in overfitting and lead to inaccurate decoding outcomes due to the redundant information. To address this limitation and make full use of the multi-domain EEG features, a multi-domain temporal-spatial-frequency convolutional neural network (TSFCNet) is proposed for MI decoding. The proposed network provides a novel mechanism that utilize the spatial and temporal EEG features combined with frequency and time-frequency characteristics. This network enables powerful feature extraction without complicated network structure. Specifically, the TSFCNet first employs the MixConv-Residual block to extract multiscale temporal features from multi-band filtered EEG data. Next, the temporal-spatial-frequency convolution block implements three shallow, parallel and independent convolutional operations in spatial, frequency and time-frequency domain, and captures high discriminative representations from these domains respectively. Finally, these features are effectively aggregated by average pooling layers and variance layers, and the network is trained with the joint supervision of the cross-entropy and the center loss. Our experimental results show that the TSFCNet outperforms the state-of-the-art models with superior classification accuracy and kappa values (82.72% and 0.7695 for dataset BCI competition IV 2a, 86.39% and 0.7324 for dataset BCI competition IV 2b). These competitive results demonstrate that the proposed network is promising for enhancing the decoding performance of MI BCIs.

SincMSNet: a Sinc filter convolutional neural network for EEG motor imagery classification

Objective.Motor imagery (MI) is widely used in brain-computer interfaces (BCIs). However, the decode of MI-EEG using convolutional neural networks (CNNs) remains a challenge due to individual variability.Approach.We propose a fully end-to-end CNN called SincMSNet to address this issue. SincMSNet employs the Sinc filter to extract subject-specific frequency band information and utilizes mixed-depth convolution to extract multi-scale temporal information for each band. It then applies a spatial convolutional block to extract spatial features and uses a temporal log-variance block to obtain classification features. The model of SincMSNet is trained under the joint supervision of cross-entropy and center loss to achieve inter-class separable and intra-class compact representations of EEG signals.Main results.We evaluated the performance of SincMSNet on the BCIC-IV-2a (four-class) and OpenBMI (two-class) datasets. SincMSNet achieves impressive results, surpassing benchmark methods. In four-class and two-class inter-session analysis, it achieves average accuracies of 80.70% and 71.50% respectively. In four-class and two-class single-session analysis, it achieves average accuracies of 84.69% and 76.99% respectively. Additionally, visualizations of the learned band-pass filter bands by Sinc filters demonstrate the network's ability to extract subject-specific frequency band information from EEG.Significance.This study highlights the potential of SincMSNet in improving the performance of MI-EEG decoding and designing more robust MI-BCIs. The source code for SincMSNet can be found at:https://github.com/Want2Vanish/SincMSNet.

Enhancing Cross-Subject Motor Imagery Classification in EEG-Based Brain-Computer Interfaces by Using Multi-Branch CNN

A brain-computer interface (BCI) is a computer-based system that allows for communication between the brain and the outer world, enabling users to interact with computers using neural activity. This brain signal is obtained from electroencephalogram (EEG) signals. A significant obstacle to the development of BCIs based on EEG is the classification of subject-independent motor imagery data since EEG data are very individualized. Deep learning techniques such as the convolutional neural network (CNN) have illustrated their influence on feature extraction to increase classification accuracy. In this paper, we present a multi-branch (five branches) 2D convolutional neural network that employs several hyperparameters for every branch. The proposed model achieved promising results for cross-subject classification and outperformed EEGNet, ShallowConvNet, DeepConvNet, MMCNN, and EEGNet_Fusion on three public datasets. Our proposed model, EEGNet Fusion V2, achieves 89.6% and 87.8% accuracy for the actual and imagined motor activity of the eegmmidb dataset and scores of 74.3% and 84.1% for the BCI IV-2a and IV-2b datasets, respectively. However, the proposed model has a bit higher computational cost, i.e., it takes around 3.5 times more computational time per sample than EEGNet_Fusion.

An improved model using convolutional sliding window-attention network for motor imagery EEG classification

Introduction

The classification model of motor imagery-based electroencephalogram (MI-EEG) is a new human-computer interface pattern and a new neural rehabilitation assessment method for diseases such as Parkinson's and stroke. However, existing MI-EEG models often suffer from insufficient richness of spatiotemporal feature extraction, learning ability, and dynamic selection ability.

Methods

To solve these problems, this work proposed a convolutional sliding window-attention network (CSANet) model composed of novel spatiotemporal convolution, sliding window, and two-stage attention blocks.

Results

The model outperformed existing state-of-the-art (SOTA) models in within- and between-individual classification tasks on commonly used MI-EEG datasets BCI-2a and Physionet MI-EEG, with classification accuracies improved by 4.22 and 2.02%, respectively.

Discussion

The experimental results also demonstrated that the proposed type token, sliding window, and local and global multi-head self-attention mechanisms can significantly improve the model's ability to construct, learn, and adaptively select multi-scale spatiotemporal features in MI-EEG signals, and accurately identify electroencephalogram signals in the unilateral motor area. This work provided a novel and accurate classification model for MI-EEG brain-computer interface tasks and proposed a feasible neural rehabilitation assessment scheme based on the model, which could promote the further development and application of MI-EEG methods in neural rehabilitation.

Multiclass classification of imagined speech EEG using noise-assisted multivariate empirical mode decomposition and multireceptive field convolutional neural network

Introduction

In this study, we classified electroencephalography (EEG) data of imagined speech using signal decomposition and multireceptive convolutional neural network. The imagined speech EEG with five vowels /a/, /e/, /i/, /o/, and /u/, and mute (rest) sounds were obtained from ten study participants.

Materials and methods

First, two different signal decomposition methods were applied for comparison: noise-assisted multivariate empirical mode decomposition and wavelet packet decomposition. Six statistical features were calculated from the decomposed eight sub-frequency bands EEG. Next, all features obtained from each channel of the trial were vectorized and used as the input vector of classifiers. Lastly, EEG was classified using multireceptive field convolutional neural network and several other classifiers for comparison.

Results

We achieved an average classification rate of 73.09 and up to 80.41% in a multiclass (six classes) setup (Chance: 16.67%). In comparison with various other classifiers, significant improvements for other classifiers were achieved (p-value < 0.05). From the frequency sub-band analysis, high-frequency band regions and the lowest-frequency band region contain more information about imagined vowel EEG data. The misclassification and classification rate of each vowel imaginary EEG was analyzed through a confusion matrix.

Discussion

Imagined speech EEG can be classified successfully using the proposed signal decomposition method and a convolutional neural network. The proposed classification method for imagined speech EEG can contribute to developing a practical imagined speech-based brain-computer interfaces system.

Encrypt with Your Mind: Reliable and Revocable Brain Biometrics via Multidimensional Gaussian Fitted Bit Allocation

Biometric features, e.g., fingerprints, the iris, and the face, have been widely used to authenticate individuals. However, most biometrics are not cancellable, i.e., once these biometric features are cloned or stolen, they cannot be replaced easily. Unlike traditional biometrics, brain biometrics are extremely difficult to clone or forge due to the natural randomness across different individuals, which makes them an ideal option for identity authentication. Most existing brain biometrics are based on electroencephalogram (EEG), which is usually demonstrated unstable performance due to the low signal-to-noise ratio (SNR). For the first time, we propose the use of intracortical brain signals, which have higher resolution and SNR, to realize the construction of the high-performance brain biometrics. Specifically, we put forward a novel brain-based key generation approach called multidimensional Gaussian fitted bit allocation (MGFBA). The proposed MGFBA method extracts keys from the local field potential of ten rats with high reliability and high entropy. We found that with the proposed MGFBA, the average effective key length of the brain biometrics was 938 bits, while achieving high authentication accuracy of 88.1% at a false acceptance rate of 1.9%, which is significantly improved compared to conventional EEG-based approaches. In addition, the proposed MGFBA-based keys can be conveniently revoked using different motor behaviors with high entropy. Experimental results demonstrate the potential of using intracortical brain signals for reliable authentication and other security applications.

Statistically significant features improve binary and multiple Motor Imagery task predictions from EEGs

In recent studies, in the field of Brain-Computer Interface (BCI), researchers have focused on Motor Imagery tasks. Motor Imagery-based electroencephalogram (EEG) signals provide the interaction and communication between the paralyzed patients and the outside world for moving and controlling external devices such as wheelchair and moving cursors. However, current approaches in the Motor Imagery-BCI system design require effective feature extraction methods and classification algorithms to acquire discriminative features from EEG signals due to the non-linear and non-stationary structure of EEG signals. This study investigates the effect of statistical significance-based feature selection on binary and multi-class Motor Imagery EEG signal classifications. In the feature extraction process performed 24 different time-domain features, 15 different frequency-domain features which are energy, variance, and entropy of Fourier transform within five EEG frequency subbands, 15 different time-frequency domain features which are energy, variance, and entropy of Wavelet transform based on five EEG frequency subbands, and 4 different Poincare plot-based non-linear parameters are extracted from each EEG channel. A total of 1,364 Motor Imagery EEG features are supplied from 22 channel EEG signals for each input EEG data. In the statistical significance-based feature selection process, the best one among all possible combinations of these features is tried to be determined using the independent t-test and one-way analysis of variance (ANOVA) test on binary and multi-class Motor Imagery EEG signal classifications, respectively. The whole extracted feature set and the feature set that contain statistically significant features only are classified in this study. We implemented 6 and 7 different classifiers in multi-class and binary (two-class) classification tasks, respectively. The classification process is evaluated using the five-fold cross-validation method, and each classification algorithm is tested 10 times. These repeated tests provide to check the repeatability of the results. The maximum of 61.86 and 47.36% for the two-class and four-class scenarios, respectively, are obtained with Ensemble Subspace Discriminant among all these classifiers using selected features including only statistically significant features. The results reveal that the introduced statistical significance-based feature selection approach improves the classifier performances by achieving higher classifier performances with fewer relevant components in Motor Imagery task classification. In conclusion, the main contribution of the presented study is two-fold evaluation of non-linear parameters as an alternative to the commonly used features and the prediction of multiple Motor Imagery tasks using statistically significant features.

Optimal Feature Analysis for Identification Based on Intracranial Brain Signals with Machine Learning Algorithms

Biometrics, e.g., fingerprints, the iris, and the face, have been widely used to authenticate individuals. However, most biometrics are not cancellable, i.e., once these traditional biometrics are cloned or stolen, they cannot be replaced easily. Unlike traditional biometrics, brain biometrics are extremely difficult to clone or forge due to the natural randomness across different individuals, which makes them an ideal option for identity authentication. Most existing brain biometrics are based on an electroencephalogram (EEG), which typically demonstrates unstable performance due to the low signal-to-noise ratio (SNR). Thus, in this paper, we propose the use of intracortical brain signals, which have higher resolution and SNR, to realize the construction of a high-performance brain biometric. Significantly, this is the first study to investigate the features of intracortical brain signals for identification. Specifically, several features based on local field potential are computed for identification, and their performance is compared with different machine learning algorithms. The results show that frequency domain features and time-frequency domain features are excellent for intra-day and inter-day identification. Furthermore, the energy features perform best among all features with 98% intra-day and 93% inter-day identification accuracy, which demonstrates the great potential of intracraial brain signals to be biometrics. This paper may serve as a guidance for future intracranial brain researches and the development of more reliable and high-performance brain biometrics.

Transformer-Based Network with Optimization for Cross-Subject Motor Imagery Identification

Exploring the effective signal features of electroencephalogram (EEG) signals is an important issue in the research of brain-computer interface (BCI), and the results can reveal the motor intentions that trigger electrical changes in the brain, which has broad research prospects for feature extraction from EEG data. In contrast to previous EEG decoding methods that are based solely on a convolutional neural network, the traditional convolutional classification algorithm is optimized by combining a transformer mechanism with a constructed end-to-end EEG signal decoding algorithm based on swarm intelligence theory and virtual adversarial training. The use of a self-attention mechanism is studied to expand the receptive field of EEG signals to global dependence and train the neural network by optimizing the global parameters in the model. The proposed model is evaluated on a real-world public dataset and achieves the highest average accuracy of 63.56% in cross-subject experiments, which is significantly higher than that found for recently published algorithms. Additionally, good performance is achieved in decoding motor intentions. The experimental results show that the proposed classification framework promotes the global connection and optimization of EEG signals, which can be further applied to other BCI tasks.

Review of public motor imagery and execution datasets in brain-computer interfaces

The demand for public datasets has increased as data-driven methodologies have been introduced in the field of brain-computer interfaces (BCIs). Indeed, many BCI datasets are available in various platforms or repositories on the web, and the studies that have employed these datasets appear to be increasing. Motor imagery is one of the significant control paradigms in the BCI field, and many datasets related to motor tasks are open to the public already. However, to the best of our knowledge, these studies have yet to investigate and evaluate the datasets, although data quality is essential for reliable results and the design of subject- or system-independent BCIs. In this study, we conducted a thorough investigation of motor imagery/execution EEG datasets recorded from healthy participants published over the past 13 years. The 25 datasets were collected from six repositories and subjected to a meta-analysis. In particular, we reviewed the specifications of the recording settings and experimental design, and evaluated the data quality measured by classification accuracy from standard algorithms such as Common Spatial Pattern (CSP) and Linear Discriminant Analysis (LDA) for comparison and compatibility across the datasets. As a result, we found that various stimulation types, such as text, figure, or arrow, were used to instruct subjects what to imagine and the length of each trial also differed, ranging from 2.5 to 29 s with a mean of 9.8 s. Typically, each trial consisted of multiple sections: pre-rest (2.38 s), imagination ready (1.64 s), imagination (4.26 s, ranging from 1 to 10 s), the post-rest (3.38 s). In a meta-analysis of the total of 861 sessions from all datasets, the mean classification accuracy of the two-class (left-hand vs. right-hand motor imagery) problem was 66.53%, and the population of the BCI poor performers, those who are unable to reach proficiency in using a BCI system, was 36.27% according to the estimated accuracy distribution. Further, we analyzed the CSP features and found that each dataset forms a cluster, and some datasets overlap in the feature space, indicating a greater similarity among them. Finally, we checked the minimal essential information (continuous signals, event type/latency, and channel information) that should be included in the datasets for convenient use, and found that only 71% of the datasets met those criteria. Our attempts to evaluate and compare the public datasets are timely, and these results will contribute to understanding the dataset's quality and recording settings as well as the use of using public datasets for future work on BCIs.

Robust Classification and Detection of Big Medical Data Using Advanced Parallel K-Means Clustering, YOLOv4, and Logistic Regression

Big-medical-data classification and image detection are crucial tasks in the field of healthcare, as they can assist with diagnosis, treatment planning, and disease monitoring. Logistic regression and YOLOv4 are popular algorithms that can be used for these tasks. However, these techniques have limitations and performance issue with big medical data. In this study, we presented a robust approach for big-medical-data classification and image detection using logistic regression and YOLOv4, respectively. To improve the performance of these algorithms, we proposed the use of advanced parallel k-means pre-processing, a clustering technique that identified patterns and structures in the data. Additionally, we leveraged the acceleration capabilities of a neural engine processor to further enhance the speed and efficiency of our approach. We evaluated our approach on several large medical datasets and showed that it could accurately classify large amounts of medical data and detect medical images. Our results demonstrated that the combination of advanced parallel k-means pre-processing, and the neural engine processor resulted in a significant improvement in the performance of logistic regression and YOLOv4, making them more reliable for use in medical applications. This new approach offers a promising solution for medical data classification and image detection and may have significant implications for the field of healthcare.