Decoding Premovement Patterns with Task-Related Component Analysis

Preparatory movement state enhances premovement EEG representations for brain-computer interfaces

Objective. Motor-related brain-computer interface (BCI) have a broad range of applications, with the detection of premovement intentions being a prominent use case. However, the electroencephalography (EEG) features during the premovement phase are not distinctly evident and are susceptible to attentional influences. These limitations impede the enhancement of performance in motor-based BCI. The objective of this study is to establish a premovement BCI encoding paradigm that integrates the preparatory movement state and validates its feasibility in improving the detection of movement intentions.Methods. Two button tasks were designed to induce subjects into a preparation state for two movement intentions (left and right) based on visual guidance, in contrast to spontaneous premovement. The low frequency movement-related cortical potentials (MRCPs) and high frequency event-related desynchronization (ERD) EEG data of 14 subjects were recorded. Extracted features were fused and classified using task related common spatial patterns (CSP) and CSP algorithms. Differences between prepared premovement and spontaneous premovement were compared in terms of time domain, frequency domain, and classification accuracy.Results. In the time domain, MRCPs features reveal that prepared premovement induce lower amplitude and earlier latency on both contralateral and ipsilateral motor cortex compared to spontaneous premovement, with susceptibility to the dominant hand's influence. Frequency domain ERD features indicate that prepared premovement induce lower ERD values bilaterally, and the ERD recovery speed after button press is the fastest. By using the fusion approach, the classification accuracy increased from 78.92% for spontaneous premovement to 83.59% for prepared premovement (p< 0.05). Along with the 4.67% improvement in classification accuracy, the standard deviation decreased by 0.95.Significance. The research findings confirm that incorporating a preparatory state into premovement enhances neural representations related to movement. This encoding enhancement paradigm effectively improves the performance of motor-based BCI. Additionally, this concept has the potential to broaden the range of decodable movement intentions and related information in motor-related BCI.

A robust multi-branch multi-attention-mechanism EEGNet for motor imagery BCI decoding

BACKGROUND

Motor-Imagery-based Brain-Computer Interface (MI-BCI) is a promising technology to assist communication, movement, and neurological rehabilitation for motor-impaired individuals. Electroencephalography (EEG) decoding techniques using deep learning (DL) possess noteworthy advantages due to automatic feature extraction and end-to-end learning. However, the DL-based EEG decoding models tend to show large variations due to intersubject variability of EEG, which results from inconsistencies of different subjects' optimal hyperparameters.

NEW METHODS

This study proposes a multi-branch multi-attention mechanism EEGNet model (MBMANet) for robust decoding. It applies the multi-branch EEGNet structure to achieve various feature extractions. Further, the different attention mechanisms introduced in each branch attain diverse adaptive weight adjustments. This combination of multi-branch and multi-attention mechanisms allows for multi-level feature fusion to provide robust decoding for different subjects.

RESULTS

The MBMANet model has a four-classification accuracy of 83.18% and kappa of 0.776 on the BCI Competition IV-2a dataset, which outperforms other eight CNN-based decoding models. This consistently satisfactory performance across all nine subjects indicates that the proposed model is robust.

CONCLUSIONS

The combine of multi-branch and multi-attention mechanisms empowers the DL-based models to adaptively learn different EEG features, which provides a feasible solution for dealing with data variability. It also gives the MBMANet model more accurate decoding of motion intentions and lower training costs, thus improving the MI-BCI's utility and robustness.

Personalized motor imagery prediction model based on individual difference of ERP

Objective. Motor imagery-based brain-computer interaction (MI-BCI) is a novel method of achieving human and external environment interaction that can assist individuals with motor disorders to rehabilitate. However, individual differences limit the utility of the MI-BCI. In this study, a personalized MI prediction model based on the individual difference of event-related potential (ERP) is proposed to solve the MI individual difference.Approach.A novel paradigm named action observation-based multi-delayed matching posture task evokes ERP during a delayed matching posture task phase by retrieving picture stimuli and videos, and generates MI electroencephalogram through action observation and autonomous imagery in an action observation-based motor imagery phase. Based on the correlation between the ERP and MI, a logistic regression-based personalized MI prediction model is built to predict each individual's suitable MI action. 32 subjects conducted the MI task with or without the help of the prediction model to select the MI action. Then classification accuracy of the MI task is used to evaluate the proposed model and three traditional MI methods.Main results.The personalized MI prediction model successfully predicts suitable action among 3 sets of daily actions. Under suitable MI action, the individual's ERP amplitude and event-related desynchronization (ERD) intensity are the largest, which helps to improve the accuracy by 14.25%.Significance.The personalized MI prediction model that uses the temporal ERP features to predict the classification accuracy of MI is feasible for improving the individual's MI-BCI performance, providing a new personalized solution for the individual difference and practical BCI application.

TSPNet: a time-spatial parallel network for classification of EEG-based multiclass upper limb motor imagery BCI

The classification of electroencephalogram (EEG) motor imagery signals has emerged as a prominent research focus within the realm of brain-computer interfaces. Nevertheless, the conventional, limited categories (typically just two or four) offered by brain-computer interfaces fail to provide an extensive array of control modes. To address this challenge, we propose the Time-Spatial Parallel Network (TSPNet) for recognizing six distinct categories of upper limb motor imagery. Within TSPNet, temporal and spatial features are extracted separately, with the time dimension feature extractor and spatial dimension feature extractor performing their respective functions. Following this, the Time-Spatial Parallel Feature Extractor is employed to decouple the connection between temporal and spatial features, thus diminishing feature redundancy. The Time-Spatial Parallel Feature Extractor deploys a gating mechanism to optimize weight distribution and parallelize time-spatial features. Additionally, we introduce a feature visualization algorithm based on signal occlusion frequency to facilitate a qualitative analysis of TSPNet. In a six-category scenario, TSPNet achieved an accuracy of 49.1% ± 0.043 on our dataset and 49.7% ± 0.029 on a public dataset. Experimental results conclusively establish that TSPNet outperforms other deep learning methods in classifying data from these two datasets. Moreover, visualization results vividly illustrate that our proposed framework can generate distinctive classifier patterns for multiple categories of upper limb motor imagery, discerned through signals of varying frequencies. These findings underscore that, in comparison to other deep learning methods, TSPNet excels in intention recognition, which bears immense significance for non-invasive brain-computer interfaces.

Multi-Class Classification of Upper Limb Movements With Filter Bank Task-Related Component Analysis

The classification of limb movements can provide with control commands in non-invasive brain-computer interface. Previous studies on the classification of limb movements have focused on the classification of left/right limbs; however, the classification of different types of upper limb movements has often been ignored despite that it provides more active-evoked control commands in the brain-computer interface. Nevertheless, few machine learning method can be used as the state-of-the-art method in the multi-class classification of limb movements. This work focuses on the multi-class classification of upper limb movements and proposes the multi-class filter bank task-related component analysis (mFBTRCA) method, which consists of three steps: spatial filtering, similarity measuring and filter bank selection. The spatial filter, namely the task-related component analysis, is first used to remove noise from EEG signals. The canonical correlation measures the similarity of the spatial-filtered signals and is used for feature extraction. The correlation features are extracted from multiple low-frequency filter banks. The minimum-redundancy maximum-relevance selects the essential features from all the correlation features, and finally, the support vector machine is used to classify the selected features. The proposed method compared against previously used models is evaluated using two datasets. mFBTRCA achieved a classification accuracy of 0.4193 ± 0.0780 (7 classes) and 0.4032 ± 0.0714 (5 classes), respectively, which improves on the best accuracies achieved using the compared methods (0.3590 ± 0.0645 and 0.3159 ± 0.0736, respectively). The proposed method is expected to provide more control commands in the applications of non-invasive brain-computer interfaces.

Improving pre-movement pattern detection with filter bank selection

Objective. Pre-movement decoding plays an important role in detecting the onsets of actions using low-frequency electroencephalography (EEG) signals before the movement of an upper limb. In this work, a binary classification method is proposed between two different states.Approach. The proposed method, referred to as filter bank standard task-related component analysis (FBTRCA), is to incorporate filter bank selection into the standard task-related component analysis (STRCA) method. In FBTRCA, the EEG signals are first divided into multiple sub-bands which start at specific fixed frequencies and end frequencies that follow in an arithmetic sequence. The STRCA method is then applied to the EEG signals in these bands to extract CCPs. The minimum redundancy maximum relevance feature selection method is used to select essential features from these correlation patterns in all sub-bands. Finally, the selected features are classified using the binary support vector machine classifier. A convolutional neural network (CNN) is an alternative approach to select canonical correlation patterns.Main Results. Three methods were evaluated using EEG signals in the time window from 2 s before the movement onset to 1 s after the movement onset. In the binary classification between a movement state and the resting state, the FBTRCA achieved an average accuracy of 0.8968 ± 0.0847 while the accuracies of STRCA and CNN were 0.8228 ± 0.1149 and 0.8828 ± 0.0917, respectively. In the binary classification between two actions, the accuracies of STRCA, CNN, and FBTRCA were 0.6611 ± 0.1432, 0.6993 ± 0.1271, 0.7178 ± 0.1274, respectively. Feature selection using filter banks, as in FBTRCA, produces comparable results to STRCA.Significance. The proposed method provides a way to select filter banks in pre-movement decoding, and thus it improves the classification performance. The improved pre-movement decoding of single upper limb movements is expected to provide people with severe motor disabilities with a more natural, non-invasive control of their external devices.

Deep Residual Convolutional Neural Networks for Brain-Computer Interface to Visualize Neural Processing of Hand Movements in the Human Brain

Concomitant with the development of deep learning, brain-computer interface (BCI) decoding technology has been rapidly evolving. Convolutional neural networks (CNNs), which are generally used as electroencephalography (EEG) classification models, are often deployed in BCI prototypes to improve the estimation accuracy of a participant's brain activity. However, because most BCI models are trained, validated, and tested via within-subject cross-validation and there is no corresponding generalization model, their applicability to unknown participants is not guaranteed. In this study, to facilitate the generalization of BCI model performance to unknown participants, we trained a model comprising multiple layers of residual CNNs and visualized the reasons for BCI classification to reveal the location and timing of neural activities that contribute to classification. Specifically, to develop a BCI that can distinguish between rest, left-hand movement, and right-hand movement tasks with high accuracy, we created multilayers of CNNs, inserted residual networks into the multilayers, and used a larger dataset than in previous studies. The constructed model was analyzed with gradient-class activation mapping (Grad-CAM). We evaluated the developed model via subject cross-validation and found that it achieved significantly improved accuracy (85.69 ± 1.10%) compared with conventional models or without residual networks. Grad-CAM analysis of the classification of cases in which our model produced correct answers showed localized activity near the premotor cortex. These results confirm the effectiveness of inserting residual networks into CNNs for tuning BCI. Further, they suggest that recording EEG signals over the premotor cortex and some other areas contributes to high classification accuracy.

Characterization of Bimanual Cyclical Tasks from Single-trial EEG-fNIRS Measurements

Robot-assisted bimanual training is promising to improve motor function and cortical reorganization for hemiparetic stroke patients. Closing the rehabilitation training loop with neurofeedback can help refine training protocols in time for better engagements and outcomes. However, due to the low signal-to- noise ratio (SNR) and non-stationary properties of neural signals, reliable characterization of bimanual training-induced neural activities from single-trial measurement is challenging. In this study, ten human participants were recruited conducting robot-assisted bimanual cyclical tasks (in-phase, 90° out-of-phase, and anti-phase) when concurrent electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were recorded. A unified EEG-fNIRS bimodal signal processing framework was proposed to characterize neural activities induced by three types of bimanual cyclical tasks. In this framework, novel artifact removal methods were used to improve the SNR and the task-related component analysis (TRCA) was introduced to increase the reproducibility of EEG-fNIRS bimodal features. The optimized features were transformed into low-dimensional indicators to reliably characterize bimanual training-induced neural activation. The SVM classification results of three bimanual cyclical tasks revealed a good discrimination ability of EEG-fNIRS bimodal indicators (90.1%), which was higher than that using EEG (74.8%) or fNIRS (82.2%) alone, supporting the proposed method as a feasible technique to characterize neural activities during robot-assisted bimanual training.