Optimization of preprocessing stage in EEG based BCI systems in terms of accuracy and timing cost

Application of cluster repeated mini-batch training method to classify electroencephalography for grab and lift tasks

Modern deep neural network training is based on mini-batch stochastic gradient optimization. While using extensive mini-batches improves the computational parallelism, the small batch training proved that it delivers improved generalization performance and allows a significantly smaller memory, which might also improve machine throughput. However, mini-batch size and characteristics, a key factor for training deep neural networks, has not been sufficiently investigated in training correlated group features and looping with highly complex ones. In addition, the unsupervised learning method clusters the data into different groups with similar properties to make the training process more stable and faster. Then, the supervised learning algorithm was applied with the cluster repeated mini-batch training (CRMT) methods. The CRMT algorithm changed the random minibatch characteristics in the training step into training in order of clusters. Specifically, the self-organizing maps (SOM) were used to cluster the information into n groups based on the dataset's labels Then, neural network models (ANN) were trained with each cluster using the cluster repeated mini-batch training method. Experiments conducted on EEG datasets demonstrate the survey of the proposed method and optimize it. In addition, the results in our research outperform other state-of-the-art methods.

Survey on the research direction of EEG-based signal processing

Electroencephalography (EEG) is increasingly important in Brain-Computer Interface (BCI) systems due to its portability and simplicity. In this paper, we provide a comprehensive review of research on EEG signal processing techniques since 2021, with a focus on preprocessing, feature extraction, and classification methods. We analyzed 61 research articles retrieved from academic search engines, including CNKI, PubMed, Nature, IEEE Xplore, and Science Direct. For preprocessing, we focus on innovatively proposed preprocessing methods, channel selection, and data augmentation. Data augmentation is classified into conventional methods (sliding windows, segmentation and recombination, and noise injection) and deep learning methods [Generative Adversarial Networks (GAN) and Variation AutoEncoder (VAE)]. We also pay attention to the application of deep learning, and multi-method fusion approaches, including both conventional algorithm fusion and fusion between conventional algorithms and deep learning. Our analysis identifies 35 (57.4%), 18 (29.5%), and 37 (60.7%) studies in the directions of preprocessing, feature extraction, and classification, respectively. We find that preprocessing methods have become widely used in EEG classification (96.7% of reviewed papers) and comparative experiments have been conducted in some studies to validate preprocessing. We also discussed the adoption of channel selection and data augmentation and concluded several mentionable matters about data augmentation. Furthermore, deep learning methods have shown great promise in EEG classification, with Convolutional Neural Networks (CNNs) being the main structure of deep neural networks (92.3% of deep learning papers). We summarize and analyze several innovative neural networks, including CNNs and multi-structure fusion. However, we also identified several problems and limitations of current deep learning techniques in EEG classification, including inappropriate input, low cross-subject accuracy, unbalanced between parameters and time costs, and a lack of interpretability. Finally, we highlight the emerging trend of multi-method fusion approaches (49.2% of reviewed papers) and analyze the data and some examples. We also provide insights into some challenges of multi-method fusion. Our review lays a foundation for future studies to improve EEG classification performance.

OptEF-BCI: An Optimization-Based Hybrid EEG and fNIRS-Brain Computer Interface

Multimodal data fusion (electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS)) has been developed as an important neuroimaging research field in order to circumvent the inherent limitations of individual modalities by combining complementary information from other modalities. This study employed an optimization-based feature selection algorithm to systematically investigate the complementary nature of multimodal fused features. After preprocessing the acquired data of both modalities (i.e., EEG and fNIRS), the temporal statistical features were computed separately with a 10 s interval for each modality. The computed features were fused to create a training vector. A wrapper-based binary enhanced whale optimization algorithm (E-WOA) was used to select the optimal/efficient fused feature subset using the support-vector-machine-based cost function. An online dataset of 29 healthy individuals was used to evaluate the performance of the proposed methodology. The findings suggest that the proposed approach enhances the classification performance by evaluating the degree of complementarity between characteristics and selecting the most efficient fused subset. The binary E-WOA feature selection approach showed a high classification rate (94.22 ± 5.39%). The classification performance exhibited a 3.85% increase compared with the conventional whale optimization algorithm. The proposed hybrid classification framework outperformed both the individual modalities and traditional feature selection classification (p < 0.01). These findings indicate the potential efficacy of the proposed framework for several neuroclinical applications.

Early-stage fusion of EEG and fNIRS improves classification of motor imagery

Introduction

Many research papers have reported successful implementation of hybrid brain-computer interfaces by complementarily combining EEG and fNIRS, to improve classification performance. However, modality or feature fusion of EEG and fNIRS was usually designed for specific user cases, which were generally customized and hard to be generalized. How to effectively utilize information from the two modalities was still unclear.

Methods

In this paper, we conducted a study to investigate the stage of bi-modal fusion based on EEG and fNIRS. A Y-shaped neural network was proposed and evaluated on an open dataset, which fuses the bimodal information in different stages.

Results

The results suggests that the early-stage fusion of EEG and fNIRS have significantly higher performance compared to middle-stage and late-stage fusion network configuration (N = 57, P < 0.05). With the proposed framework, the average accuracy of 29 participants reaches 76.21% in the left-or-right hand motor imagery task in leave-one-out cross-validation, using bi-modal data as network inputs respectively, which is in the same level as the state-of-the-art hybrid BCI methods based on EEG and fNIRS data.

Cortico-muscular coupling to control a hybrid brain-computer interface for upper limb motor rehabilitation: A pseudo-online study on stroke patients

Brain-Computer Interface (BCI) systems for motor rehabilitation after stroke have proven their efficacy to enhance upper limb motor recovery by reinforcing motor related brain activity. Hybrid BCIs (h-BCIs) exploit both central and peripheral activation and are frequently used in assistive BCIs to improve classification performances. However, in a rehabilitative context, brain and muscular features should be extracted to promote a favorable motor outcome, reinforcing not only the volitional control in the central motor system, but also the effective projection of motor commands to target muscles, i.e., central-to-peripheral communication. For this reason, we considered cortico-muscular coupling (CMC) as a feature for a h-BCI devoted to post-stroke upper limb motor rehabilitation. In this study, we performed a pseudo-online analysis on 13 healthy participants (CTRL) and 12 stroke patients (EXP) during executed (CTRL, EXP unaffected arm) and attempted (EXP affected arm) hand grasping and extension to optimize the translation of CMC computation and CMC-based movement detection from offline to online. Results showed that updating the CMC computation every 125 ms (shift of the sliding window) and accumulating two predictions before a final classification decision were the best trade-off between accuracy and speed in movement classification, independently from the movement type. The pseudo-online analysis on stroke participants revealed that both attempted and executed grasping/extension can be classified through a CMC-based movement detection with high performances in terms of classification speed (mean delay between movement detection and EMG onset around 580 ms) and accuracy (hit rate around 85%). The results obtained by means of this analysis will ground the design of a novel non-invasive h-BCI in which the control feature is derived from a combined EEG and EMG connectivity pattern estimated during upper limb movement attempts.

Subject-Dependent Artifact Removal for Enhancing Motor Imagery Classifier Performance under Poor Skills

The Electroencephalography (EEG)-based motor imagery (MI) paradigm is one of the most studied technologies for Brain-Computer Interface (BCI) development. Still, the low Signal-to-Noise Ratio (SNR) poses a challenge when constructing EEG-based BCI systems. Moreover, the non-stationary and nonlinear signal issues, the low-spatial data resolution, and the inter- and intra-subject variability hamper the extraction of discriminant features. Indeed, subjects with poor motor skills have difficulties in practicing MI tasks against low SNR scenarios. Here, we propose a subject-dependent preprocessing approach that includes the well-known Surface Laplacian Filtering and Independent Component Analysis algorithms to remove signal artifacts based on the MI performance. In addition, power- and phase-based functional connectivity measures are studied to extract relevant and interpretable patterns and identify subjects of inefficency. As a result, our proposal, Subject-dependent Artifact Removal (SD-AR), improves the MI classification performance in subjects with poor motor skills. Consequently, electrooculography and volume-conduction EEG artifacts are mitigated within a functional connectivity feature-extraction strategy, which favors the classification performance of a straightforward linear classifier.

Stockwell transform and semi-supervised feature selection from deep features for classification of BCI signals

Over the past few years, the processing of motor imagery (MI) electroencephalography (EEG) signals has been attracted for developing brain-computer interface (BCI) applications, since feature extraction and classification of these signals are extremely difficult due to the inherent complexity and tendency to artifact properties of them. The BCI systems can provide a direct interaction pathway/channel between the brain and a peripheral device, hence the MI EEG-based BCI systems seem crucial to control external devices for patients suffering from motor disabilities. The current study presents a semi-supervised model based on three-stage feature extraction and machine learning algorithms for MI EEG signal classification in order to improve the classification accuracy with smaller number of deep features for distinguishing right- and left-hand MI tasks. Stockwell transform is employed at the first phase of the proposed feature extraction method to generate two-dimensional time–frequency maps (TFMs) from one-dimensional EEG signals. Next, the convolutional neural network (CNN) is applied to find deep feature sets from TFMs. Then, the semi-supervised discriminant analysis (SDA) is utilized to minimize the number of descriptors. Finally, the performance of five classifiers, including support vector machine, discriminant analysis, k-nearest neighbor, decision tree, random forest, and the fusion of them are compared. The hyperparameters of SDA and mentioned classifiers are optimized by Bayesian optimization to maximize the accuracy. The presented model is validated using BCI competition II dataset III and BCI competition IV dataset 2b. The performance metrics of the proposed method indicate its efficiency for classifying MI EEG signals.

IENet: a robust convolutional neural network for EEG based brain-computer interfaces

OBJECTIVE

Brain-computer interfaces (BCIs) based on electroencephalogram (EEG) develop into novel application areas with more complex scenarios, which put forward higher requirements for the robustness of EEG signal processing algorithms. Deep learning can automatically extract discriminative features and potential dependencies via deep structures, demonstrating strong analytical capabilities in numerous domains such as computer vision (CV) and natural language processing (NLP). Making full use of deep learning technology to design a robust algorithm that is capable of analyzing EEG across BCI paradigms is our main work in this paper.

APPROACH

Inspired by InceptionV4 and InceptionTime architecture, we introduce a neural network ensemble named InceptionEEG-Net (IENet), where multi-scale convolutional layer and convolution of length 1 enable model to extract rich high-dimensional features with limited parameters. In addition, we propose the average receptive field gain for convolutional neural networks (CNNs), which optimizes IENet to detect long patterns at a smaller cost. We compare with the current state-of-the-art method across five EEG-BCI paradigms: steady-state visual evoked potentials, epilepsy EEG, overt attention P300 visual-evoked potentials, covert attention P300 visual-evoked potentials and movement-related cortical potentials.

MAIN RESULTS

The classification results show that the generalizability of IENet is on par with the state-of-the-art paradigm-agnostic models on test datasets. Furthermore, the feature explainability analysis of IENet illustrates its capability to extract neurophysiologically interpretable features for different BCI paradigms, ensuring the reliability of algorithm. Significance. It can be seen from our results that IENet can generalize to different BCI paradigms. And it is essential for deep CNNs to increase the receptive field size using average receptive field gain.

Suppressing of Power Line Artifact From Electroencephalogram Measurements Using Sparsity in Frequency Domain

Electroencephalogram (EEG) plays an important role in brain disease diagnosis and research of brain-computer interface (BCI). However, the measurements of EEG are often exposed to strong interference of power line artifact (PLA). Digital notch filters (DNFs) can be applied to remove the PLA effectively, but it also results in severe signal distortions in the time domain. To address this problem, spectrum correction (SC) based methods can be utilized. These methods estimate harmonic parameters of the PLA such that compensation signals are produced to remove the noise. In order to ensure high accuracy during harmonic parameter estimations, a novel approach is proposed in this paper. This novel approach is based on the combination of sparse representation (SR) and SC. It can deeply mine the information of PLA in the frequency domain. Firstly, a ratio-based spectrum correction (RBSC) using rectangular window is employed to make rough estimation of the harmonic parameters of PLA. Secondly, the two spectral line closest to the estimated frequency are calculated. Thirdly, the two spectral lines with high amplitudes can be utilized as input of RBSC to make finer estimations of the harmonic parameters. Finally, a compensation signal, based on the extracted harmonic parameters, is generated to suppress PLA. Numerical simulations and actual EEG signals with PLA were used to evaluate the effectiveness of the improved approach. It is verified that this approach can effectively suppress the PLA without distorting the time-domain waveform of the EEG signal.

Brain-Computer Interface: Advancement and Challenges

Brain-Computer Interface (BCI) is an advanced and multidisciplinary active research domain based on neuroscience, signal processing, biomedical sensors, hardware, etc. Since the last decades, several groundbreaking research has been conducted in this domain. Still, no comprehensive review that covers the BCI domain completely has been conducted yet. Hence, a comprehensive overview of the BCI domain is presented in this study. This study covers several applications of BCI and upholds the significance of this domain. Then, each element of BCI systems, including techniques, datasets, feature extraction methods, evaluation measurement matrices, existing BCI algorithms, and classifiers, are explained concisely. In addition, a brief overview of the technologies or hardware, mostly sensors used in BCI, is appended. Finally, the paper investigates several unsolved challenges of the BCI and explains them with possible solutions.