EEG-Based Brain-Computer Interfaces Using Motor-Imagery: Techniques and Challenges

Preparatory movement state enhances premovement EEG representations for brain-computer interfaces

Objective. Motor-related brain-computer interface (BCI) have a broad range of applications, with the detection of premovement intentions being a prominent use case. However, the electroencephalography (EEG) features during the premovement phase are not distinctly evident and are susceptible to attentional influences. These limitations impede the enhancement of performance in motor-based BCI. The objective of this study is to establish a premovement BCI encoding paradigm that integrates the preparatory movement state and validates its feasibility in improving the detection of movement intentions.Methods. Two button tasks were designed to induce subjects into a preparation state for two movement intentions (left and right) based on visual guidance, in contrast to spontaneous premovement. The low frequency movement-related cortical potentials (MRCPs) and high frequency event-related desynchronization (ERD) EEG data of 14 subjects were recorded. Extracted features were fused and classified using task related common spatial patterns (CSP) and CSP algorithms. Differences between prepared premovement and spontaneous premovement were compared in terms of time domain, frequency domain, and classification accuracy.Results. In the time domain, MRCPs features reveal that prepared premovement induce lower amplitude and earlier latency on both contralateral and ipsilateral motor cortex compared to spontaneous premovement, with susceptibility to the dominant hand's influence. Frequency domain ERD features indicate that prepared premovement induce lower ERD values bilaterally, and the ERD recovery speed after button press is the fastest. By using the fusion approach, the classification accuracy increased from 78.92% for spontaneous premovement to 83.59% for prepared premovement (p< 0.05). Along with the 4.67% improvement in classification accuracy, the standard deviation decreased by 0.95.Significance. The research findings confirm that incorporating a preparatory state into premovement enhances neural representations related to movement. This encoding enhancement paradigm effectively improves the performance of motor-based BCI. Additionally, this concept has the potential to broaden the range of decodable movement intentions and related information in motor-related BCI.

An EEG channel selection method for motor imagery based on Fisher score and local optimization

Objective. Multi-channel electroencephalogram (EEG) technology in brain-computer interface (BCI) research offers the advantage of enhanced spatial resolution and system performance. However, this also implies that more time is needed in the data processing stage, which is not conducive to the rapid response of BCI. Hence, it is a necessary and challenging task to reduce the number of EEG channels while maintaining decoding effectiveness.Approach. In this paper, we propose a local optimization method based on the Fisher score for within-subject EEG channel selection. Initially, we extract the common spatial pattern characteristics of EEG signals in different bands, calculate Fisher scores for each channel based on these characteristics, and rank them accordingly. Subsequently, we employ a local optimization method to finalize the channel selection.Main results. On the BCI Competition IV Dataset IIa, our method selects an average of 11 channels across four bands, achieving an average accuracy of 79.37%. This represents a 6.52% improvement compared to using the full set of 22 channels. On our self-collected dataset, our method similarly achieves a significant improvement of 24.20% with less than half of the channels, resulting in an average accuracy of 76.95%.Significance. This research explores the importance of channel combinations in channel selection tasks and reveals that appropriately combining channels can further enhance the quality of channel selection. The results indicate that the model selected a small number of channels with higher accuracy in two-class motor imagery EEG classification tasks. Additionally, it improves the portability of BCI systems through channel selection and combinations, offering the potential for the development of portable BCI systems.

Optimizing motor imagery BCI models with hard trials removal and model refinement

Deep learning models have demonstrated remarkable performance in the classification of motor imagery BCI systems. However, these models exhibit sensitivity to challenging trials, often called hard trials, leading to performance degradation. In this paper, we address this issue by proposing two novel methods for identifying and mitigating the impact of hard trials on model performance. The first method leverages model prediction scores to discern hard trials. The second approach employs a quantitative explainable artificial intelligence (XAI) approach, enabling a more transparent and interpretable means of hard trial identification. The identified hard trials are removed from the entire motor imagery training and validation dataset, and the deep learning model is further re-trained using the dataset without hard trials. To evaluate the efficacy of these proposed methods, experiments were conducted on the Open BMI dataset. The results for hold-out analysis show that the proposed quantitative XAI- based hard trial removal method has statistically improved the average classification accuracy of the baseline deep CNN model from 63.77% to 68.70%, withp-value =7.66e-11for the subject-specific MI classification. Additionally, analyzing the scalp map representing the average relevance scores of correctly classified trials compared to a misclassified trial provides a deeper insight into identifying hard trials. The results indicate that the proposed quantitaive-based XAI approach outperformes the prediction-score-based approach in hard trial identification.

Authentication with a one-dimensional CNN model using EEG-based brain-computer interface

Brain-computer interface (BCI) technology uses electroencephalogram (EEG) signals to create a direct interaction between the human body and its surroundings. Motor imagery (MI) classification using EEG signals is an important application that can help a rehabilitated or motor-impaired stroke patient perform certain tasks. Robust classification of these signals is an important step toward making the use of EEG more practical in many applications and less dependent on trained professionals. Deep learning methods have produced impressive results in BCI in recent years, especially with the availability of large electroencephalography (EEG) data sets. Dealing with EEG-MI signals is difficult because noise and other signal sources can interfere with the electrical amplitude of the brain, and its generalization ability is limited, so it is difficult to improve EEG classifiers. To address these issues, this paper presents a methodology based on one-dimensional convolutional neural networks (1-D CNN) for motor imagery (MI) recognition for the right hand, left hand, feet, and sedentary task. The proposed model is a lightweight model with fewer parameters and has an accuracy of 91.75%. Then, in an innovative exploitation of the four output classes, there is an idea that allows people with disabilities who are deprived of security measures, such as entering a secret code, to use the output classification, such as password codes. It is also an idea for a unique authentication system that is more secure and less vulnerable to theft or the like for a healthy person at the same time.

Classification of motor imagery using chaotic entropy based on sub-band EEG source localization

Objective.Electroencephalography (EEG) has been widely used in motor imagery (MI) research by virtue of its high temporal resolution and low cost, but its low spatial resolution is still a major criticism. The EEG source localization (ESL) algorithm effectively improves the spatial resolution of the signal by inverting the scalp EEG to extrapolate the cortical source signal, thus enhancing the classification accuracy.Approach.To address the problem of poor spatial resolution of EEG signals, this paper proposed a sub-band source chaotic entropy feature extraction method based on sub-band ESL. Firstly, the preprocessed EEG signals were filtered into 8 sub-bands. Each sub-band signal was source localized respectively to reveal the activation patterns of specific frequency bands of the EEG signals and the activities of specific brain regions in the MI task. Then, approximate entropy, fuzzy entropy and permutation entropy were extracted from the source signal as features to quantify the complexity and randomness of the signal. Finally, the classification of different MI tasks was achieved using support vector machine.Main result.The proposed method was validated on two MI public datasets (brain-computer interface (BCI) competition III IVa, BCI competition IV 2a) and the results showed that the classification accuracies were higher than the existing methods.Significance.The spatial resolution of the signal was improved by sub-band EEG localization in the paper, which provided a new idea for EEG MI research.

Immersive VR for upper-extremity rehabilitation in patients with neurological disorders: a scoping review

BACKGROUND

Neurological disorders, such as stroke and chronic pain syndromes, profoundly impact independence and quality of life, especially when affecting upper extremity (UE) function. While conventional physical therapy has shown effectiveness in providing some neural recovery in affected individuals, there remains a need for improved interventions. Virtual reality (VR) has emerged as a promising technology-based approach for neurorehabilitation to make the patient's experience more enjoyable. Among VR-based rehabilitation paradigms, those based on fully immersive systems with headsets have gained significant attention due to their potential to enhance patient's engagement.

METHODS

This scoping review aims to investigate the current state of research on the use of immersive VR for UE rehabilitation in individuals with neurological diseases, highlighting benefits and limitations. We identified thirteen relevant studies through comprehensive searches in Scopus, PubMed, and IEEE Xplore databases. Eligible studies incorporated immersive VR for UE rehabilitation in patients with neurological disorders and evaluated participants' neurological and motor functions before and after the intervention using clinical assessments.

RESULTS

Most of the included studies reported improvements in the participants rehabilitation outcomes, suggesting that immersive VR represents a valuable tool for UE rehabilitation in individuals with neurological disorders. In addition, immersive VR-based interventions hold the potential for personalized and intensive training within a telerehabilitation framework. However, further studies with better design are needed for true comparison with traditional therapy. Also, the potential side effects associated with VR head-mounted displays, such as dizziness and nausea, warrant careful consideration in the development and implementation of VR-based rehabilitation programs.

CONCLUSION

This review provides valuable insights into the application of immersive VR in UE rehabilitation, offering the foundation for future research and clinical practice. By leveraging immersive VR's potential, researchers and rehabilitation specialists can design more tailored and patient-centric rehabilitation strategies, ultimately improving the functional outcome and enhancing the quality of life of individuals with neurological diseases.

BCI-controlled wheelchairs: end-users' perceptions, needs, and expectations, an interview-based study

PURPOSE

Brain-computer interface (BCI)-controlled wheelchairs have the potential to improve the independence of people with mobility impairments. The low uptake of BCI devices has been linked to a lack of knowledge among researchers of the needs of end-users that should influence BCI development.

MATERIALS AND METHODS

This study used semi-structured interviews to learn about the perceptions, needs, and expectations of spinal cord injury (SCI) patients with regards to a BCI-controlled wheelchair. Topics discussed in the interview include: paradigms, shared control, safety, robustness, channel selection, hardware, and experimental design. The interviews were recorded and then transcribed. Analysis was carried out using coding based on grounded theory principles.

RESULTS

The majority of participants had a positive view of BCI-controlled wheelchair technology and were willing to use the technology. Core issues were raised regarding safety, cost and aesthetics. Interview discussions were linked to state-of-the-art BCI technology. The results challenge the current reliance of researchers on the motor-imagery paradigm by suggesting end-users expect highly intuitive paradigms. There also needs to be a stronger focus on obstacle avoidance and safety features in BCI wheelchairs. Finally, the development of control approaches that can be personalized for individual users may be instrumental for widespread adoption of these devices.

CONCLUSIONS

This study, based on interviews with SCI patients, indicates that BCI-controlled wheelchairs are a promising assistive technology that would be well received by end-users. Recommendations for a more person-centered design of BCI controlled wheelchairs are made and clear avenues for future research are identified.

Brain-machine interface based on deep learning to control asynchronously a lower-limb robotic exoskeleton: a case-of-study

BACKGROUND

This research focused on the development of a motor imagery (MI) based brain-machine interface (BMI) using deep learning algorithms to control a lower-limb robotic exoskeleton. The study aimed to overcome the limitations of traditional BMI approaches by leveraging the advantages of deep learning, such as automated feature extraction and transfer learning. The experimental protocol to evaluate the BMI was designed as asynchronous, allowing subjects to perform mental tasks at their own will.

METHODS

A total of five healthy able-bodied subjects were enrolled in this study to participate in a series of experimental sessions. The brain signals from two of these sessions were used to develop a generic deep learning model through transfer learning. Subsequently, this model was fine-tuned during the remaining sessions and subjected to evaluation. Three distinct deep learning approaches were compared: one that did not undergo fine-tuning, another that fine-tuned all layers of the model, and a third one that fine-tuned only the last three layers. The evaluation phase involved the exclusive closed-loop control of the exoskeleton device by the participants' neural activity using the second deep learning approach for the decoding.

RESULTS

The three deep learning approaches were assessed in comparison to an approach based on spatial features that was trained for each subject and experimental session, demonstrating their superior performance. Interestingly, the deep learning approach without fine-tuning achieved comparable performance to the features-based approach, indicating that a generic model trained on data from different individuals and previous sessions can yield similar efficacy. Among the three deep learning approaches compared, fine-tuning all layer weights demonstrated the highest performance.

CONCLUSION

This research represents an initial stride toward future calibration-free methods. Despite the efforts to diminish calibration time by leveraging data from other subjects, complete elimination proved unattainable. The study's discoveries hold notable significance for advancing calibration-free approaches, offering the promise of minimizing the need for training trials. Furthermore, the experimental evaluation protocol employed in this study aimed to replicate real-life scenarios, granting participants a higher degree of autonomy in decision-making regarding actions such as walking or stopping gait.

Evolving Therapeutic Landscape of Intracerebral Hemorrhage: Emerging Cutting-Edge Advancements in Surgical Robots, Regenerative Medicine, and Neurorehabilitation Techniques

Intracerebral hemorrhage (ICH) is the most serious form of stroke and has limited available therapeutic options. As knowledge on ICH rapidly develops, cutting-edge techniques in the fields of surgical robots, regenerative medicine, and neurorehabilitation may revolutionize ICH treatment. However, these new advances still must be translated into clinical practice. In this review, we examined several emerging therapeutic strategies and their major challenges in managing ICH, with a particular focus on innovative therapies involving robot-assisted minimally invasive surgery, stem cell transplantation, in situ neuronal reprogramming, and brain-computer interfaces. Despite the limited expansion of the drug armamentarium for ICH over the past few decades, the judicious selection of more efficacious therapeutic modalities and the exploration of multimodal combination therapies represent opportunities to improve patient prognoses after ICH.

Evaluation of temporal, spatial and spectral filtering in CSP-based methods for decoding pedaling-based motor tasks using EEG signals

Stroke is a neurological syndrome that usually causes a loss of voluntary control of lower/upper body movements, making it difficult for affected individuals to perform Activities of Daily Living (ADLs). Brain-Computer Interfaces (BCIs) combined with robotic systems, such as Motorized Mini Exercise Bikes (MMEB), have enabled the rehabilitation of people with disabilities by decoding their actions and executing a motor task. However, Electroencephalography (EEG)-based BCIs are affected by the presence of physiological and non-physiological artifacts. Thus, movement discrimination using EEG become challenging, even in pedaling tasks, which have not been well explored in the literature. In this study, Common Spatial Patterns (CSP)-based methods were proposed to classify pedaling motor tasks. To address this, Filter Bank Common Spatial Patterns (FBCSP) and Filter Bank Common Spatial-Spectral Patterns (FBCSSP) were implemented with different spatial filtering configurations by varying the time segment with different filter bank combinations for the three methods to decode pedaling tasks. An in-house EEG dataset during pedaling tasks was registered for 8 participants. As results, the best configuration corresponds to a filter bank with two filters (8-19 Hz and 19-30 Hz) using a time window between 1.5 and 2.5 s after the cue and implementing two spatial filters, which provide accuracy of approximately 0.81, False Positive Rates lower than 0.19, andKappaindex of 0.61. This work implies that EEG oscillatory patterns during pedaling can be accurately classified using machine learning. Therefore, our method can be applied in the rehabilitation context, such as MMEB-based BCIs, in the future.

Liquid Metal Composites-Enabled Real-Time Hand Gesture Recognizer with Superior Recognition Speed and Accuracy

Prosthetic hands play a vital role in restoring forearm functionality for patients who have suffered hand loss or deformity. The hand gesture intention recognition system serves as a critical component within the prosthetic hand system. However, accurately and swiftly identifying hand gesture intentions remains a challenge in existing approaches. Here, a real-time motion intention recognition system utilizing liquid metal composite sensor bracelets is proposed. The sensor bracelet detects pressure signals generated by forearm muscle movements to recognize hand gesture intent. Leveraging the remarkable pressure sensitivity of liquid metal composites and the efficient classifier based on the optimized recognition algorithm, this system achieves an average offline and real-time recognition accuracy of 98.2% and 92.04%, respectively, with an average recognition speed of 0.364 s. Thus, this wearable system shows advantages in superior recognition speed and accuracy. Furthermore, this system finds applications in master-slave control of prosthetic hands in unmanned scenarios, such as electrically powered operations, space exploration, and telemedicine. The proposed system promises significant advances in next-generation intent-controlled prosthetic hands and robots.

An EEG motor imagery dataset for brain computer interface in acute stroke patients

The brain-computer interface (BCI) is a technology that involves direct communication with parts of the brain and has evolved rapidly in recent years; it has begun to be used in clinical practice, such as for patient rehabilitation. Patient electroencephalography (EEG) datasets are critical for algorithm optimization and clinical applications of BCIs but are rare at present. We collected data from 50 acute stroke patients with wireless portable saline EEG devices during the performance of two tasks: 1) imagining right-handed movements and 2) imagining left-handed movements. The dataset consists of four types of data: 1) the motor imagery instructions, 2) raw recording data, 3) pre-processed data after removing artefacts and other manipulations, and 4) patient characteristics. This is the first open dataset to address left- and right-handed motor imagery in acute stroke patients. We believe that the dataset will be very helpful for analysing brain activation and designing decoding methods that are more applicable for acute stroke patients, which will greatly facilitate research in the field of motor imagery-BCI.

EEG-BCI-based motor imagery classification using double attention convolutional network

This article aims to improve and diversify signal processing techniques to execute a brain-computer interface (BCI) based on neurological phenomena observed when performing motor tasks using motor imagery (MI). The noise present in the original data, such as intermodulation noise, crosstalk, and other unwanted noise, is removed by Modify Least Mean Square (M-LMS) in the pre-processing stage. Traditional LMSs were unable to extract all the noise from the images. After pre-processing, the required features, such as statistical features, entropy features, etc., were extracted using Common Spatial Pattern (CSP) and Pearson's Correlation Coefficient (PCC) instead of the traditional single feature extraction model. The arithmetic optimization algorithm cannot select the features accurately and fails to reduce the feature dimensionality of the data. Thus, an Extended Arithmetic operation optimization (ExAo) algorithm is used to select the most significant attributes from the extracted features. The proposed model uses Double Attention Convolutional Neural Networks (DAttnConvNet) to classify the types of EEG signals based on optimal feature selection. Here, the attention mechanism is used to select and optimize the features to improve the classification accuracy and efficiency of the model. In EEG motor imagery datasets, the proposed model has been analyzed under class, which obtained an accuracy of 99.98% in class Baseline (B), 99.82% in class Imagined movement of a right fist (R) and 99.61% in class Imagined movement of both fists (RL). In the EEG dataset, the proposed model can obtain a high accuracy of 97.94% compared to EEG datasets of other models.

Multi-domain feature joint optimization based on multi-view learning for improving the EEG decoding

Background

Brain-computer interface (BCI) systems based on motor imagery (MI) have been widely used in neurorehabilitation. Feature extraction applied by the common spatial pattern (CSP) is very popular in MI classification. The effectiveness of CSP is highly affected by the frequency band and time window of electroencephalogram (EEG) segments and channels selected.

Objective

In this study, the multi-domain feature joint optimization (MDFJO) based on the multi-view learning method is proposed, which aims to select the discriminative features enhancing the classification performance.

Method

The channel patterns are divided using the Fisher discriminant criterion (FDC). Furthermore, the raw EEG is intercepted for multiple sub-bands and time interval signals. The high-dimensional features are constructed by extracting features from CSP on each EEG segment. Specifically, the multi-view learning method is used to select the optimal features, and the proposed feature sparsification strategy on the time level is proposed to further refine the optimal features.

Results

Two public EEG datasets are employed to validate the proposed MDFJO method. The average classification accuracy of the MDFJO in Data 1 and Data 2 is 88.29 and 87.21%, respectively. The classification result of MDFJO was significantly better than MSO (p < 0.05), FBCSP32 (p < 0.01), and other competing methods (p < 0.001).

Conclusion

Compared with the CSP, sparse filter band common spatial pattern (SFBCSP), and filter bank common spatial pattern (FBCSP) methods with channel numbers 16, 32 and all channels as well as MSO, the MDFJO significantly improves the test accuracy. The feature sparsification strategy proposed in this article can effectively enhance classification accuracy. The proposed method could improve the practicability and effectiveness of the BCI system.

Riemannian geometric and ensemble learning for decoding cross-session motor imagery electroencephalography signals

Objective.Brain-computer interfaces (BCIs) enable a direct communication pathway between the human brain and external devices, without relying on the traditional peripheral nervous and musculoskeletal systems. Motor imagery (MI)-based BCIs have attracted significant interest for their potential in motor rehabilitation. However, current algorithms fail to account for the cross-session variability of electroencephalography signals, limiting their practical application.Approach.We proposed a Riemannian geometry-based adaptive boosting and voting ensemble (RAVE) algorithm to address this issue. Our approach segmented the MI period into multiple sub-datasets using a sliding window approach and extracted features from each sub-dataset using Riemannian geometry. We then trained adaptive boosting (AdaBoost) ensemble learning classifiers for each sub-dataset, with the final BCI output determined by majority voting of all classifiers. We tested our proposed RAVE algorithm and eight other competing algorithms on four datasets (Pan2023, BNCI001-2014, BNCI001-2015, BNCI004-2015).Main results.Our results showed that, in the cross-session scenario, the RAVE algorithm outperformed the eight other competing algorithms significantly under different within-session training sample sizes. Compared to traditional algorithms that involved a large number of training samples, the RAVE algorithm achieved similar or even better classification performance on the datasets (Pan2023, BNCI001-2014, BNCI001-2015), even when it did not use or only used a small number of within-session training samples.Significance.These findings indicate that our cross-session decoding strategy could enable MI-BCI applications that require no or minimal training process.

Spatio-Temporal Explanation of 3D-EEGNet for Motor Imagery EEG Classification Using Permutation and Saliency

Recently, convolutional neural network (CNN)-based classification models have shown good performance for motor imagery (MI) brain-computer interfaces (BCI) using electroencephalogram (EEG) in end-to-end learning. Although a few explainable artificial intelligence (XAI) techniques have been developed, it is still challenging to interpret the CNN models for EEG-based BCI classification effectively. In this research, we propose 3D-EEGNet as a 3D CNN model to improve both the explainability and performance of MI EEG classification. The proposed approach exhibited better performances on two MI EEG datasets than the existing EEGNet, which uses a 2D input shape. The MI classification accuracies are improved around 1.8% and 6.1% point in average on the datasets, respectively. The permutation-based XAI method is first applied for the reliable explanation of the 3D-EEGNet. Next, to find a faster XAI method for spatio-temporal explanation, we design a novel technique based on the normalized discounted cumulative gain (NDCG) for selecting the best among a few saliency-based methods due to their higher time complexity than the permutation-based method. Among the saliency-based methods, DeepLIFT was selected because the NDCG scores indicated its results are the most similar to the permutation-based results. Finally, the fast spatio-temporal explanation using DeepLIFT provides deeper understanding for the classification results of the 3D-EEGNet and the important properties in the MI EEG experiments.

Patients with epilepsy without cognitive impairment show altered brain networks in multiple frequency bands in an audiovisual integration task

OBJECTIVES

Comorbid cognitive and behavioral deficits are often observed in patients with epilepsy. It is not clear whether the brain networks of patients with epilepsy without cognitive decline differs from that of healthy controls in different frequency bands in the task-state. The purpose of our study was to explore whether epilepsy affects the structure of brain networks associated with cognitive processing, even when patients with epilepsy do not have cognitive impairment.

METHODS

We designed an audiovisual discrimination task and recorded electroencephalogram (EEG) data from healthy controls and patients with epilepsy. We established constructed time-varying brain networks across the delta, theta, alpha, and beta bands on the task-state EEG data during audiovisual integration processing.

RESULTS

The results showed changes in the structure of the brain networks in the theta, alpha, and beta bands in patients with epilepsy who had no cognitive deficit. No significant difference in the connectivity strength, clustering coefficient, characteristic path length, or global efficiency was noted between patients and healthy controls. Moreover, the structure of brain networks in patients showed no correlation with the behavioral performance.

CONCLUSION

The repeated abnormal firing of neurons in the brain of patients with epilepsy may inhibit it from optimizing networks into more efficient structures. Epilepsy might affect decision-making ability by damaging the neural activity in the beta band and preventing its correlation with decision-making behaviors.

Self-attention-based convolutional neural network and time-frequency common spatial pattern for enhanced motor imagery classification

BACKGROUND

Motor imagery (MI) based brain-computer interfaces (BCIs) have promising potentials in the field of neuro-rehabilitation. However, due to individual variations in active brain regions during MI tasks, the challenge of decoding MI EEG signals necessitates improved classification performance for practical application.

NEW METHOD

This study proposes a self-attention-based Convolutional Neural Network (CNN) in conjunction with a time-frequency common spatial pattern (TFCSP) for enhanced MI classification. Due to the limited availability of training data, a data augmentation strategy is employed to expand the scale of MI EEG datasets. The self-attention-based CNN is trained to automatically extract the temporal and spatial information from EEG signals, allowing the self-attention module to select active channels by calculating EEG channel weights. TFCSP is further implemented to extract multiscale time-frequency-space features from EEG data. Finally, the EEG features derived from TFCSP are concatenated with those from the self-attention-based CNN for MI classification.

RESULTS

The proposed method is evaluated on two publicly accessible datasets, BCI Competition IV IIa and BCI Competition III IIIa, yielding mean accuracies of 79.28 % and 86.39 %, respectively.

CONCLUSIONS

Compared with state-of-the-art methods, our approach achieves superior classification results in accuracy. Self-attention-based CNN combining with TFCSP can make full use of the time-frequency-space information of EEG, and enhance the classification performance.

Enhancing Cross-Subject Motor Imagery Classification in EEG-Based Brain-Computer Interfaces by Using Multi-Branch CNN

A brain-computer interface (BCI) is a computer-based system that allows for communication between the brain and the outer world, enabling users to interact with computers using neural activity. This brain signal is obtained from electroencephalogram (EEG) signals. A significant obstacle to the development of BCIs based on EEG is the classification of subject-independent motor imagery data since EEG data are very individualized. Deep learning techniques such as the convolutional neural network (CNN) have illustrated their influence on feature extraction to increase classification accuracy. In this paper, we present a multi-branch (five branches) 2D convolutional neural network that employs several hyperparameters for every branch. The proposed model achieved promising results for cross-subject classification and outperformed EEGNet, ShallowConvNet, DeepConvNet, MMCNN, and EEGNet_Fusion on three public datasets. Our proposed model, EEGNet Fusion V2, achieves 89.6% and 87.8% accuracy for the actual and imagined motor activity of the eegmmidb dataset and scores of 74.3% and 84.1% for the BCI IV-2a and IV-2b datasets, respectively. However, the proposed model has a bit higher computational cost, i.e., it takes around 3.5 times more computational time per sample than EEGNet_Fusion.

Transformers for autonomous recognition of psychiatric dysfunction via raw and imbalanced EEG signals

Early identification of mental disorders, based on subjective interviews, is extremely challenging in the clinical setting. There is a growing interest in developing automated screening tools for potential mental health problems based on biological markers. Here, we demonstrate the feasibility of an AI-powered diagnosis of different mental disorders using EEG data. Specifically, this work aims to classify different mental disorders in the following ecological context accurately: (1) using raw EEG data, (2) collected during rest, (3) during both eye open, and eye closed conditions, (4) at short 2-min duration, (5) on participants with different psychiatric conditions, (6) with some overlapping symptoms, and (7) with strongly imbalanced classes. To tackle this challenge, we designed and optimized a transformer-based architecture, where class imbalance is addressed through focal loss and class weight balancing. Using the recently released TDBRAIN dataset (n= 1274 participants), our method classifies each participant as either a neurotypical or suffering from major depressive disorder (MDD), attention deficit hyperactivity disorder (ADHD), subjective memory complaints (SMC), or obsessive-compulsive disorder (OCD). We evaluate the performance of the proposed architecture on both the window-level and the patient-level. The classification of the 2-min raw EEG data into five classes achieved a window-level accuracy of 63.2% and 65.8% for open and closed eye conditions, respectively. When the classification is limited to three main classes (MDD, ADHD, SMC), window level accuracy improved to 75.1% and 69.9% for eye open and eye closed conditions, respectively. Our work paves the way for developing novel AI-based methods for accurately diagnosing mental disorders using raw resting-state EEG data.

Predicting Motor Imagery BCI Performance Based on EEG Microstate Analysis

Motor imagery (MI) electroencephalography (EEG) is natural and comfortable for controllers, and has become a research hotspot in the field of the brain-computer interface (BCI). Exploring the inter-subject MI-BCI performance variation is one of the fundamental problems in MI-BCI application. EEG microstates with high spatiotemporal resolution and multichannel information can represent brain cognitive function. In this paper, four EEG microstates (MS1, MS2, MS3, MS4) were used in the analysis of the differences in the subjects' MI-BCI performance, and the four microstate feature parameters (the mean duration, the occurrences per second, the time coverage ratio, and the transition probability) were calculated. The correlation between the resting-state EEG microstate feature parameters and the subjects' MI-BCI performance was measured. Based on the negative correlation of the occurrence of MS1 and the positive correlation of the mean duration of MS3, a resting-state microstate predictor was proposed. Twenty-eight subjects were recruited to participate in our MI experiments to assess the performance of our resting-state microstate predictor. The experimental results show that the average area under curve (AUC) value of our resting-state microstate predictor was 0.83, and increased by 17.9% compared with the spectral entropy predictor, representing that the microstate feature parameters can better fit the subjects' MI-BCI performance than spectral entropy predictor. Moreover, the AUC of microstate predictor is higher than that of spectral entropy predictor at both the single-session level and average level. Overall, our resting-state microstate predictor can help MI-BCI researchers better select subjects, save time, and promote MI-BCI development.

Implementation of artificial intelligence and machine learning-based methods in brain-computer interaction

Brain-computer interfaces are used for direct two-way communication between the human brain and the computer. Brain signals contain valuable information about the mental state and brain activity of the examined subject. However, due to their non-stationarity and susceptibility to various types of interference, their processing, analysis and interpretation are challenging. For these reasons, the research in the field of brain-computer interfaces is focused on the implementation of artificial intelligence, especially in five main areas: calibration, noise suppression, communication, mental condition estimation, and motor imagery. The use of algorithms based on artificial intelligence and machine learning has proven to be very promising in these application domains, especially due to their ability to predict and learn from previous experience. Therefore, their implementation within medical technologies can contribute to more accurate information about the mental state of subjects, alleviate the consequences of serious diseases or improve the quality of life of disabled patients.

Functional Connectivity and Feature Fusion Enhance Multiclass Motor-Imagery Brain-Computer Interface Performance

(1) Background: in the field of motor-imagery brain-computer interfaces (MI-BCIs), obtaining discriminative features among multiple MI tasks poses a significant challenge. Typically, features are extracted from single electroencephalography (EEG) channels, neglecting their interconnections, which leads to limited results. To address this limitation, there has been growing interest in leveraging functional brain connectivity (FC) as a feature in MI-BCIs. However, the high inter- and intra-subject variability has so far limited its effectiveness in this domain. (2) Methods: we propose a novel signal processing framework that addresses this challenge. We extracted translation-invariant features (TIFs) obtained from a scattering convolution network (SCN) and brain connectivity features (BCFs). Through a feature fusion approach, we combined features extracted from selected channels and functional connectivity features, capitalizing on the strength of each component. Moreover, we employed a multiclass support vector machine (SVM) model to classify the extracted features. (3) Results: using a public dataset (IIa of the BCI Competition IV), we demonstrated that the feature fusion approach outperformed existing state-of-the-art methods. Notably, we found that the best results were achieved by merging TIFs with BCFs, rather than considering TIFs alone. (4) Conclusions: our proposed framework could be the key for improving the performance of a multiclass MI-BCI system.

An evaluation of transfer learning models in EEG-based authentication

Electroencephalogram(EEG)-based authentication has received increasing attention from researchers as they believe it could serve as an alternative to more conventional personal authentication methods. Unfortunately, EEG signals are non-stationary and could be easily contaminated by noise and artifacts. Therefore, further processing of data analysis is needed to retrieve useful information. Various machine learning approaches have been proposed and implemented in the EEG-based domain, with deep learning being the most current trend. However, retaining the performance of a deep learning model requires substantial computational effort and a vast amount of data, especially when the models go deeper to generate consistent results. Deep learning models trained with small data sets from scratch may experience an overfitting issue. Transfer learning becomes an alternative solution. It is a technique to recognize and apply the knowledge and skills learned from the previous tasks to a new domain with limited training data. This study attempts to explore the applicability of transferring various pre-trained models' knowledge to the EEG-based authentication domain. A self-collected database that consists of 30 subjects was utilized in the analysis. The database enrolment is divided into two sessions, with each session producing two sets of EEG recording data. The frequency spectrums of the preprocessed EEG signals are extracted and fed into the pre-trained models as the input data. Three experimental tests are carried out and the best performance is reported with accuracy in the range of 99.1-99.9%. The acquired results demonstrate the efficiency of transfer learning in authenticating an individual in this domain.

Subject-Independent EEG Classification of Motor Imagery Based on Dual-Branch Feature Fusion

A brain computer interface (BCI) system helps people with motor dysfunction interact with the external environment. With the advancement of technology, BCI systems have been applied in practice, but their practicability and usability are still greatly challenged. A large amount of calibration time is often required before BCI systems are used, which can consume the patient's energy and easily lead to anxiety. This paper proposes a novel motion-assisted method based on a novel dual-branch multiscale auto encoder network (MSAENet) to decode human brain motion imagery intentions, while introducing a central loss function to compensate for the shortcomings of traditional classifiers that only consider inter-class differences and ignore intra-class coupling. The effectiveness of the method is validated on three datasets, namely BCIIV2a, SMR-BCI and OpenBMI, to achieve zero calibration of the MI-BCI system. The results show that our proposed network displays good results on all three datasets. In the case of subject-independence, the MSAENet outperformed the other four comparison methods on the BCIIV2a and SMR-BCI datasets, while achieving F1_score values as high as 69.34% on the OpenBMI dataset. Our method maintains better classification accuracy with a small number of parameters and short prediction times, and the method achieves zero calibration of the MI-BCI system.

Transfer Learning with CNN Models for Brain-Machine Interfaces to command lower-limb exoskeletons: A Solution for Limited Data. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083615 DOI: 10.1109/embc40787.2023.10340008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

This study evaluates the performance of two convolutional neural networks (CNNs) in a brain-machine interface (BMI) based on motor imagery (MI) by using a small dataset collected from five participants wearing a lower-limb exoskeleton. To address the issue of limited data availability, transfer learning was employed by training models on EEG signals from other subjects and subsequently fine-tuning them to specific users. A combination of common spatial patterns (CSP) and linear discriminant analysis (LDA) was used as a benchmark for comparison. The study's primary aim is to examine the potential of CNNs and transfer learning in the development of an automatic neural classification system for a BMI based on MI to command a lower-limb exoskeleton that can be used by individuals without specialized training.Clinical Relevance- BMI can be used in rehabilitation for patients with motor impairment by using mental simulation of movement to activate robotic exoskeletons. This can promote neural plasticity and aid in recovery.

Autoregressive models for biomedical signal processing

Autoregressive models are ubiquitous tools for the analysis of time series in many domains such as computational neuroscience and biomedical engineering. In these domains, data is, for example, collected from measurements of brain activity. Crucially, this data is subject to measurement errors as well as uncertainties in the underlying system model. As a result, standard signal processing using autoregressive model estimators may be biased. We present a framework for autoregressive modelling that incorporates these uncertainties explicitly via an overparameterised loss function. To optimise this loss, we derive an algorithm that alternates between state and parameter estimation. Our work shows that the procedure is able to successfully denoise time series and successfully reconstruct system parameters.Clinical relevance- This new paradigm can be used in a multitude of applications in neuroscience such as brain-computer interface data analysis and better understanding of brain dynamics in diseases such as epilepsy.

EEG motor imagery classification using deep learning approaches in naïve BCI users

Motor Imagery (MI)-Brain Computer-Interfaces (BCI) illiteracy defines that not all subjects can achieve a good performance in MI-BCI systems due to different factors related to the fatigue, substance consumption, concentration, and experience in the use. To reduce the effects of lack of experience in the use of BCI systems (naïve users), this paper presents the implementation of three Deep Learning (DL) methods with the hypothesis that the performance of BCI systems could be improved compared with baseline methods in the evaluation of naïve BCI users. The methods proposed here are based on Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM)/Bidirectional Long Short-Term Memory (BiLSTM), and a combination of CNN and LSTM used for upper limb MI signal discrimination on a dataset of 25 naïve BCI users. The results were compared with three widely used baseline methods based on the Common Spatial Pattern (CSP), Filter Bank Common Spatial Pattern (FBCSP), and Filter Bank Common Spatial-Spectral Pattern (FBCSSP), in different temporal window configurations. As results, the LSTM-BiLSTM-based approach presented the best performance, according to the evaluation metrics of Accuracy, F-score, Recall, Specificity, Precision, and ITR, with a mean performance of 80% (maximum 95%) and ITR of 10 bits/min using a temporal window of 1.5 s. The DL Methods represent a significant increase of 32% compared with the baseline methods (p< 0.05). Thus, with the outcomes of this study, it is expected to increase the controllability, usability, and reliability of the use of robotic devices in naïve BCI users.

Decoding Multi-Class Motor Imagery and Motor Execution Tasks Using Riemannian Geometry Algorithms on Large EEG Datasets

The use of Riemannian geometry decoding algorithms in classifying electroencephalography-based motor-imagery brain-computer interfaces (BCIs) trials is relatively new and promises to outperform the current state-of-the-art methods by overcoming the noise and nonstationarity of electroencephalography signals. However, the related literature shows high classification accuracy on only relatively small BCI datasets. The aim of this paper is to provide a study of the performance of a novel implementation of the Riemannian geometry decoding algorithm using large BCI datasets. In this study, we apply several Riemannian geometry decoding algorithms on a large offline dataset using four adaptation strategies: baseline, rebias, supervised, and unsupervised. Each of these adaptation strategies is applied in motor execution and motor imagery for both scenarios 64 electrodes and 29 electrodes. The dataset is composed of four-class bilateral and unilateral motor imagery and motor execution of 109 subjects. We run several classification experiments and the results show that the best classification accuracy is obtained for the scenario where the baseline minimum distance to Riemannian mean has been used. The mean accuracy values up to 81.5% for motor execution, and up to 76.4% for motor imagery. The accurate classification of EEG trials helps to realize successful BCI applications that allow effective control of devices.

Motor Imagery Classification Based on EEG Sensing with Visual and Vibrotactile Guidance

Motor imagery (MI) is a technique of imagining the performance of a motor task without actually using the muscles. When employed in a brain-computer interface (BCI) supported by electroencephalographic (EEG) sensors, it can be used as a successful method of human-computer interaction. In this paper, the performance of six different classifiers, namely linear discriminant analysis (LDA), support vector machine (SVM), random forest (RF), and three classifiers from the family of convolutional neural networks (CNN), is evaluated using EEG MI datasets. The study investigates the effectiveness of these classifiers on MI, guided by a static visual cue, dynamic visual guidance, and a combination of dynamic visual and vibrotactile (somatosensory) guidance. The effect of filtering passband during data preprocessing was also investigated. The results show that the ResNet-based CNN significantly outperforms the competing classifiers on both vibrotactile and visually guided data when detecting different directions of MI. Preprocessing the data using low-frequency signal features proves to be a better solution to achieve higher classification accuracy. It has also been shown that vibrotactile guidance has a significant impact on classification accuracy, with the associated improvement particularly evident for architecturally simpler classifiers. These findings have important implications for the development of EEG-based BCIs, as they provide valuable insight into the suitability of different classifiers for different contexts of use.

Brain-computer interface enhanced by virtual reality training for controlling a lower limb exoskeleton

This study explores the use of a brain-computer interface (BCI) based on motor imagery (MI) for the control of a lower limb exoskeleton to aid in motor recovery after a neural injury. The BCI was evaluated in ten able-bodied subjects and two patients with spinal cord injuries. Five able-bodied subjects underwent a virtual reality (VR) training session to accelerate training with the BCI. Results from this group were compared with a control group of five able-bodied subjects, and it was found that the employment of shorter training by VR did not reduce the effectiveness of the BCI and even improved it in some cases. Patients gave positive feedback about the system and were able to handle experimental sessions without reaching high levels of physical and mental exertion. These results are promising for the inclusion of BCI in rehabilitation programs, and future research should investigate the potential of the MI-based BCI system.

Lower-limb kinematic reconstruction during pedaling tasks from EEG signals using Unscented Kalman filter

Kinematic reconstruction of lower-limb movements using electroencephalography (EEG) has been used in several rehabilitation systems. However, the nonlinear relationship between neural activity and limb movement may challenge decoders in real-time Brain-Computer Interface (BCI) applications. This paper proposes a nonlinear neural decoder using an Unscented Kalman Filter (UKF) to infer lower-limb kinematics from EEG signals during pedaling. The results demonstrated maximum decoding accuracy using slow cortical potentials in the delta band (0.1-4 Hz) of 0.33 for Pearson's r-value and 8 for the signal-to-noise ratio (SNR). This leaves an open door to the development of closed-loop EEG-based BCI systems for kinematic monitoring during pedaling rehabilitation tasks.

Multimodal motor imagery decoding method based on temporal spatial feature alignment and fusion

Objective. A motor imagery-based brain-computer interface (MI-BCI) translates spontaneous movement intention from the brain to outside devices. Multimodal MI-BCI that uses multiple neural signals contains rich common and complementary information and is promising for enhancing the decoding accuracy of MI-BCI. However, the heterogeneity of different modalities makes the multimodal decoding task difficult. How to effectively utilize multimodal information remains to be further studied.Approach. In this study, a multimodal MI decoding neural network was proposed. Spatial feature alignment losses were designed to enhance the feature representations extracted from the heterogeneous data and guide the fusion of features from different modalities. An attention-based modality fusion module was built to align and fuse the features in the temporal dimension. To evaluate the proposed decoding method, a five-class MI electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) dataset were constructed.Main results and significance. The comparison experimental results showed that the proposed decoding method achieved higher decoding accuracy than the compared methods on both the self-collected dataset and a public dataset. The ablation results verified the effectiveness of each part of the proposed method. Feature distribution visualization results showed that the proposed losses enhance the feature representation of EEG and fNIRS modalities. The proposed method based on EEG and fNIRS modalities has significant potential for improving decoding performance of MI tasks.

Frontal midline theta rhythm and gamma activity measured by sheet-type wearable EEG device

Introduction

The current study measured the frontal midline theta rhythm (Fmθ), which appears in the frontal midline region during the attentional focus state, using the sheet-type wearable electroencephalograph (EEG) device HARU-1, and examined the modulation of frontal gamma band activity by cognitive tasks.

Methods

We measured the frontal EEG of 20 healthy subjects using HARU-1 for 2 min during the rest eyes-closed condition and simple mental calculation task condition, respectively. Statistical analyses were conducted using permutation testing based on t-test and cluster analysis to compare the results between the resting state and the task condition.

Results

Twelve of 20 subjects showed Fmθ during the task condition. The 12 subjects with Fmθ showed significantly higher activity of the theta and gamma bands, and significantly low activity of the alpha band during the task condition compared to the resting condition. In the eight subjects without Fmθ were significantly low activity of the alpha and beta bands and no significant activity in the theta and gamma band activity during the task condition compared to the resting condition.

Discussion

These results indicate that it is possible to measure Fmθ using HARU-1. A novel finding was the gamma band activity appearing with Fmθ in the left and right frontal forehead regions, suggesting that it reflects the function of the prefrontal cortex in working memory tasks.

Long-Range α-Synchronization as Control Signal for BCI: A Feasibility Study

Shifts in spatial attention are associated with variations in α band (α, 8-14 Hz) activity, specifically in interhemispheric imbalance. The underlying mechanism is attributed to local α-synchronization, which regulates local inhibition of neural excitability, and frontoparietal synchronization reflecting long-range communication. The direction-specific nature of this neural correlate brings forward its potential as a control signal in brain-computer interfaces (BCIs). In the present study, we explored whether long-range α-synchronization presents lateralized patterns dependent on voluntary attention orienting and whether these neural patterns can be picked up at a single-trial level to provide a control signal for active BCI. We collected electroencephalography (EEG) data from a cohort of healthy adults (n = 10) while performing a covert visuospatial attention (CVSA) task. The data show a lateralized pattern of α-band phase coupling between frontal and parieto-occipital regions after target presentation, replicating previous findings. This pattern, however, was not evident during the cue-to-target orienting interval, the ideal time window for BCI. Furthermore, decoding the direction of attention trial-by-trial from cue-locked synchronization with support vector machines (SVMs) was at chance level. The present findings suggest EEG may not be capable of detecting long-range α-synchronization in attentional orienting on a single-trial basis and, thus, highlight the limitations of this metric as a reliable signal for BCI control.

Continual Learning of a Transformer-Based Deep Learning Classifier Using an Initial Model from Action Observation EEG Data to Online Motor Imagery Classification

The motor imagery (MI)-based brain computer interface (BCI) is an intuitive interface that enables users to communicate with external environments through their minds. However, current MI-BCI systems ask naïve subjects to perform unfamiliar MI tasks with simple textual instruction or a visual/auditory cue. The unclear instruction for MI execution not only results in large inter-subject variability in the measured EEG patterns but also causes the difficulty of grouping cross-subject data for big-data training. In this study, we designed an BCI training method in a virtual reality (VR) environment. Subjects wore a head-mounted device (HMD) and executed action observation (AO) concurrently with MI (i.e., AO + MI) in VR environments. EEG signals recorded in AO + MI task were used to train an initial model, and the initial model was continually improved by the provision of EEG data in the following BCI training sessions. We recruited five healthy subjects, and each subject was requested to participate in three kinds of tasks, including an AO + MI task, an MI task, and the task of MI with visual feedback (MI-FB) three times. This study adopted a transformer- based spatial-temporal network (TSTN) to decode the user's MI intentions. In contrast to other convolutional neural network (CNN) or recurrent neural network (RNN) approaches, the TSTN extracts spatial and temporal features, and applies attention mechanisms along spatial and temporal dimensions to perceive the global dependencies. The mean detection accuracies of TSTN were 0.63, 0.68, 0.75, and 0.77 in the MI, first MI-FB, second MI-FB, and third MI-FB sessions, respectively. This study demonstrated the AO + MI gave an easier way for subjects to conform their imagery actions, and the BCI performance was improved with the continual learning of the MI-FB training process.

A study of action difference on motor imagery based on delayed matching posture task

Objective. Motor imagery (MI)-based brain-computer interfaces (BCIs) provide an additional control pathway for people by decoding the intention of action imagination. The way people imagine greatly affects MI-BCI performance. Action itself is one of the factors that influence the way people imagine. Whether the different actions cause a difference in the MI performance is unknown. What is more important is how to manifest this action difference in the process of imagery, which has the potential to guide people to use their individualized actions to imagine more effectively.Approach.To explore action differences, this study proposes a novel paradigm named as action observation based delayed matching posture task. Ten subjects are required to observe, memorize, match, and imagine three types of actions (cutting, grasping and writing) given by visual images or videos, to accomplish the phases of encoding, retrieval and reinforcement of MI. Event-related potential (ERP), MI features, and classification accuracy of the left or the right hand are used to evaluate the effect of the action difference on the MI difference.Main results.Action differences cause different feature distributions, resulting in that the accuracy with high event-related (de)synchronization (ERD/ERS) is 27.75% higher than the ones with low ERD/ERS (p< 0.05), which indicates that the action difference has impact on the MI difference and the BCI performance. In addition, significant differences in the ERP amplitudes exists among the three actions: the amplitude of P300-N200 potential reaches 9.28μV of grasping, 5.64μV and 5.25μV higher than the cutting and the writing, respectively (p< 0.05).Significance.The ERP amplitudes derived from the supplementary motor area shows positive correlation to the MI classification accuracy, implying that the ERP might be an index of the MI performance when the people is faced with action selection. This study demonstrates that the MI difference is related to the action difference, and can be manifested by the ERP, which is important for improving MI training by selecting suitable action; the relationship between the ERP and the MI provides a novel index to find the suitable action to set up an individualized BCI and improve the performance further.

Enhancement of motor imagery training efficiency by an online adaptive training paradigm integrated with error related potential

Objective. Motor imagery (MI) is a process of autonomously modulating the motor area to rehearse action mentally without actual execution. Based on the neuroplasticity of the cerebral cortex, MI can promote the functional rehabilitation of the injured cerebral cortex motor area. However, it usually takes several days to a few months to train individuals to acquire the necessary MI ability to control rehabilitation equipment in current studies, which greatly limits the clinical application of rehabilitation training systems based on the MI brain-computer interface (BCI).Approach. A novel MI training paradigm combined with the error related potential (ErrP) is proposed, and online adaptive training of the MI classifier was performed using ErrP. ErrP is used to correct the output of the MI classification to obtain a higher accuracy of kinesthetic feedback based on the imagination intention of subjects while generating simulated labels for MI online adaptive training. In this way, we improved the MI training efficiency. Thirteen subjects were randomly divided into an experimental group using the proposed paradigm and a control group using the traditional MI training paradigm to participate in six MI training experiments.Main results. The proposed paradigm enabled the experimental group to obtain a higher event-related desynchronization modulation level in the contralateral brain region compared with the control group and 69.76% online classification accuracy of MI after three MI training experiments. The online classification accuracy reached 72.76% and the whole system recognized the MI intention of the subjects with an online accuracy of 82.61% after six experiments.Significance. Compared with the conventional unimodal MI training strategy, the proposed approach enables subjects to use the MI-BCI based system directly and achieve a better performance after only three training experiments with training left and right hands simultaneously. This greatly improves the usability of the MI-BCI-based rehabilitation system and makes it more convenient for clinical use.

A Review of Online Classification Performance in Motor Imagery-Based Brain–Computer Interfaces for Stroke Neurorehabilitation

Motor imagery (MI)-based brain–computer interfaces (BCI) have shown increased potential for the rehabilitation of stroke patients; nonetheless, their implementation in clinical practice has been restricted due to their low accuracy performance. To date, although a lot of research has been carried out in benchmarking and highlighting the most valuable classification algorithms in BCI configurations, most of them use offline data and are not from real BCI performance during the closed-loop (or online) sessions. Since rehabilitation training relies on the availability of an accurate feedback system, we surveyed articles of current and past EEG-based BCI frameworks who report the online classification of the movement of two upper limbs in both healthy volunteers and stroke patients. We found that the recently developed deep-learning methods do not outperform the traditional machine-learning algorithms. In addition, patients and healthy subjects exhibit similar classification accuracy in current BCI configurations. Lastly, in terms of neurofeedback modality, functional electrical stimulation (FES) yielded the best performance compared to non-FES systems.

Investigating User Proficiency of Motor Imagery for EEG-Based BCI System to Control Simulated Wheelchair

The research on the electroencephalography (EEG)-based brain-computer interface (BCI) is widely utilized for wheelchair control. The ability of the user is one factor of BCI efficiency. Therefore, we focused on BCI tasks and protocols to yield high efficiency from the robust EEG features of individual users. This study proposes a task-based brain activity to gain the power of the alpha band, which included eyes closed for alpha response at the occipital area, attention to an upward arrow for alpha response at the frontal area, and an imagined left/right motor for alpha event-related desynchronization at the left/right motor cortex. An EPOC X neuroheadset was used to acquire the EEG signals. We also proposed user proficiency in motor imagery sessions with limb movement paradigms by recommending motor imagination tasks. Using the proposed system, we verified the feature extraction algorithms and command translation. Twelve volunteers participated in the experiment, and the conventional paradigm of motor imagery was used to compare the efficiencies. With utilized user proficiency in motor imagery, an average accuracy of 83.7% across the left and right commands was achieved. The recommended MI paradigm via user proficiency achieved an approximately 4% higher accuracy than the conventional MI paradigm. Moreover, the real-time control results of a simulated wheelchair revealed a high efficiency based on the time condition. The time results for the same task as the joystick-based control were still approximately three times longer. We suggest that user proficiency be used to recommend an individual MI paradigm for beginners. Furthermore, the proposed BCI system can be used for electric wheelchair control by people with severe disabilities.

Multibranch convolutional neural network with contrastive representation learning for decoding same limb motor imagery tasks

Introduction

Emerging deep learning approaches to decode motor imagery (MI) tasks have significantly boosted the performance of brain-computer interfaces. Although recent studies have produced satisfactory results in decoding MI tasks of different body parts, the classification of such tasks within the same limb remains challenging due to the activation of overlapping brain regions. A single deep learning model may be insufficient to effectively learn discriminative features among tasks.

Methods

The present study proposes a framework to enhance the decoding of multiple hand-MI tasks from the same limb using a multi-branch convolutional neural network. The CNN framework utilizes feature extractors from established deep learning models, as well as contrastive representation learning, to derive meaningful feature representations for classification.

Results

The experimental results suggest that the proposed method outperforms several state-of-the-art methods by obtaining a classification accuracy of 62.98% with six MI classes and 76.15 % with four MI classes on the Tohoku University MI-BCI and BCI Competition IV datasets IIa, respectively.

Discussion

Despite requiring heavy data augmentation and multiple optimization steps, resulting in a relatively long training time, this scheme is still suitable for online use. However, the trade-of between the number of base learners, training time, prediction time, and system performance should be carefully considered.

Improving pre-movement pattern detection with filter bank selection

Objective. Pre-movement decoding plays an important role in detecting the onsets of actions using low-frequency electroencephalography (EEG) signals before the movement of an upper limb. In this work, a binary classification method is proposed between two different states.Approach. The proposed method, referred to as filter bank standard task-related component analysis (FBTRCA), is to incorporate filter bank selection into the standard task-related component analysis (STRCA) method. In FBTRCA, the EEG signals are first divided into multiple sub-bands which start at specific fixed frequencies and end frequencies that follow in an arithmetic sequence. The STRCA method is then applied to the EEG signals in these bands to extract CCPs. The minimum redundancy maximum relevance feature selection method is used to select essential features from these correlation patterns in all sub-bands. Finally, the selected features are classified using the binary support vector machine classifier. A convolutional neural network (CNN) is an alternative approach to select canonical correlation patterns.Main Results. Three methods were evaluated using EEG signals in the time window from 2 s before the movement onset to 1 s after the movement onset. In the binary classification between a movement state and the resting state, the FBTRCA achieved an average accuracy of 0.8968 ± 0.0847 while the accuracies of STRCA and CNN were 0.8228 ± 0.1149 and 0.8828 ± 0.0917, respectively. In the binary classification between two actions, the accuracies of STRCA, CNN, and FBTRCA were 0.6611 ± 0.1432, 0.6993 ± 0.1271, 0.7178 ± 0.1274, respectively. Feature selection using filter banks, as in FBTRCA, produces comparable results to STRCA.Significance. The proposed method provides a way to select filter banks in pre-movement decoding, and thus it improves the classification performance. The improved pre-movement decoding of single upper limb movements is expected to provide people with severe motor disabilities with a more natural, non-invasive control of their external devices.

Graph Theoretical Analysis of EEG Functional Connectivity Patterns and Fusion with Physiological Signals for Emotion Recognition

Emotion recognition is a key attribute for realizing advances in human-computer interaction, especially when using non-intrusive physiological sensors, such as electroencephalograph (EEG) and electrocardiograph. Although functional connectivity of EEG has been utilized for emotion recognition, the graph theory analysis of EEG connectivity patterns has not been adequately explored. The exploitation of brain network characteristics could provide valuable information regarding emotions, while the combination of EEG and peripheral physiological signals can reveal correlation patterns of human internal state. In this work, a graph theoretical analysis of EEG functional connectivity patterns along with fusion between EEG and peripheral physiological signals for emotion recognition has been proposed. After extracting functional connectivity from EEG signals, both global and local graph theory features are extracted. Those features are concatenated with statistical features from peripheral physiological signals and fed to different classifiers and a Convolutional Neural Network (CNN) for emotion recognition. The average accuracy on the DEAP dataset using CNN was 55.62% and 57.38% for subject-independent valence and arousal classification, respectively, and 83.94% and 83.87% for subject-dependent classification. Those scores went up to 75.44% and 78.77% for subject-independent classification and 88.27% and 90.84% for subject-dependent classification using a feature selection algorithm, exceeding the current state-of-the-art results.

EEG channel selection based on sequential backward floating search for motor imagery classification

Brain-computer interfaces (BCIs) based on motor imagery (MI) utilizing multi-channel electroencephalogram (EEG) data are commonly used to improve motor function of people with motor disabilities. EEG channel selection can enhance MI classification accuracy by selecting informative channels, accordingly reducing redundant information. The sequential backward floating search (SBFS) approach has been considered as one of the best feature selection methods. In this paper, SBFS is first implemented to select the optimal EEG channels in MI-BCI. Further, to reduce the time complexity of SBFS, the modified SBFS is proposed and applied to left and right hand MI tasks. In the modified SBFS, based on the map of EEG channels at the scalp, the symmetrical channels are selected as channel pairs and acceleration is thus realized by removing or adding multiple channels in each iteration. Extensive experiments were conducted on four public BCI datasets. Experimental results show that the SBFS achieves significantly higher classification accuracy (p < 0.001) than using all channels and conventional MI channels (i.e., C3, C4, and Cz). Moreover, the proposed method outperforms the state-of-the-art selection methods.

Riemannian distance based channel selection and feature extraction combining discriminative time-frequency bands and Riemannian tangent space for MI-BCIs

OBJECTIVE

Motor imagery-based brain computer interfaces (MI-BCIs) have been widely researched because they do not demand external stimuli and have a high degree of maneuverability. In most scenarios, superabundant selected channels, fixed time windows, and frequency bands would certainly affect the performance of MI-BCIs due to the neurophysiological diversities between different individuals. In this study, we tended to effectively use the Riemannian geometry of spatial covariance matrix to extract more robust features and thus enhance the decoding efficiency.

APPROACH

First, we propose a Riemannian distance-based EEG channel selection method (RDCS), which preliminarily reduces the information redundancy in the first stage. Second, we extract discriminative Riemannian Tangent Space features of EEG signals of selected channels from the most discriminant time-frequency bands (DTFRTS) to further enhance decoding accuracy for MI-BCIs. Finally, we trained a support vector machine (SVM) model with a linear kernel to classify our extracted discriminative Riemannian features and evaluated our proposed method using publicly available BCI Competition Ⅳ dataset Ⅰ (DS1) and Competition Ⅲ dataset Ⅲa (DS2).

MAIN RESULTS

The experimental results showed that the average classification accuracy with the selected 10-channel EEG signals of our method is 88.1% and 91.6% in DS1 and DS2 respectively. The average improvements are 24.3% & 27.1% on DS1 and 4.4% & 14.2% on DS2 for 10 & 20 selected channels, respectively.

SIGNIFICANCE

These results showed that our proposed method is a promising candidate for performance improvement of MI-BCIs.

Automatic detection of abnormal EEG signals using multiscale features with ensemble learning

Electroencephalogram (EEG) is an economical and convenient auxiliary test to aid in the diagnosis and analysis of brain-related neurological diseases. In recent years, machine learning has shown great potential in clinical EEG abnormality detection. However, existing methods usually fail to consider the issue of feature redundancy when extracting the relevant EEG features. In addition, the importance of utilizing the patient age information in EEG detection is ignored. In this paper, a new framework is proposed for distinguishing an unknown EEG recording as either normal or abnormal by identifying different types of EEG-derived significant features. In the proposed framework, different hierarchical salient features are extracted using a time-wise multi-scale aggregation strategy, based on a selected group of statistical characteristics calculated from the optimum discrete wavelet transform coefficients. We also fuse the age information with multi-scale features for further improving discrimination. The integrated features are classified using three ensemble learning classifiers, CatBoost, LightGBM, and random forest. Experimental results show that our method with CatBoost classifier can yield superior performance vis-a-vis competing techniques, which indicates the great promise of our methodology in EEG pathology detection.

Decoding EEG rhythms offline and online during motor imagery for standing and sitting based on a brain-computer interface

Motor imagery (MI)-based brain-computer interface (BCI) systems have shown promising advances for lower limb motor rehabilitation. The purpose of this study was to develop an MI-based BCI for the actions of standing and sitting. Thirty-two healthy subjects participated in the study using 17 active EEG electrodes. We used a combination of the filter bank common spatial pattern (FBCSP) method and the regularized linear discriminant analysis (RLDA) technique for decoding EEG rhythms offline and online during motor imagery for standing and sitting. The offline analysis indicated the classification of motor imagery and idle state provided a mean accuracy of 88.51 ± 1.43% and 85.29 ± 1.83% for the sit-to-stand and stand-to-sit transitions, respectively. The mean accuracies of the sit-to-stand and stand-to-sit online experiments were 94.69 ± 1.29% and 96.56 ± 0.83%, respectively. From these results, we believe that the MI-based BCI may be useful to future brain-controlled standing systems.