Improvement Motor Imagery EEG Classification Based on Regularized Linear Discriminant Analysis

An end-to-end multi-task motor imagery EEG classification neural network based on dynamic fusion of spectral-temporal features

Electroencephalograph (EEG) brain-computer interfaces (BCI) have potential to provide new paradigms for controlling computers and devices. The accuracy of brain pattern classification in EEG BCI is directly affected by the quality of features extracted from EEG signals. Currently, feature extraction heavily relies on prior knowledge to engineer features (for example from specific frequency bands); therefore, better extraction of EEG features is an important research direction. In this work, we propose an end-to-end deep neural network that automatically finds and combines features for motor imagery (MI) based EEG BCI with 4 or more imagery classes (multi-task). First, spectral domain features of EEG signals are learned by compact convolutional neural network (CCNN) layers. Then, gated recurrent unit (GRU) neural network layers automatically learn temporal patterns. Lastly, an attention mechanism dynamically combines (across EEG channels) the extracted spectral-temporal features, reducing redundancy. We test our method using BCI Competition IV-2a and a data set we collected. The average classification accuracy on 4-class BCI Competition IV-2a was 85.1 % ± 6.19 %, comparable to recent work in the field and showing low variability among participants; average classification accuracy on our 6-class data was 64.4 % ± 8.35 %. Our dynamic fusion of spectral-temporal features is end-to-end and has relatively few network parameters, and the experimental results show its effectiveness and potential.

Local Embedding Learning via Landmark-Based Dynamic Connections

Linear discriminant analysis (LDA) is one of the most effective and popular methods to reduce the dimensionality of data with Gaussian assumption. However, LDA cannot handle non-Gaussian data because the center point is incompetent to represent the distribution of data. Some existing methods based on graph embedding focus on exploring local structures via pairwise relationships of data for addressing the non-Gaussian issue. Due to massive pairwise relationships, the computational complexity is high as well as the locally optimal solution is hard to find. To address these issues, we propose a novel and efficient local embedding learning via landmark-based dynamic connections (LDC) in which we leverage several landmarks to represent different subclusters in the same class and establish the connections between each point and landmark. Furthermore, in order to explore the relationship of landmarks pairwise more precisely, the relationship between each point and their corresponding neighbor landmarks are found in the optimal subspace, rather than the original space, which can avoid the negative influence of the noises. We also propose an efficient iterative algorithm to deal with the proposed ratio minimization problem. Extensive experiments conducted on several real-world datasets have demonstrated the advantages of the proposed method.

MEDUSA©: A novel Python-based software ecosystem to accelerate brain-computer interface and cognitive neuroscience research

BACKGROUND AND OBJECTIVE

Neurotechnologies have great potential to transform our society in ways that are yet to be uncovered. The rate of development in this field has increased significantly in recent years, but there are still barriers that need to be overcome before bringing neurotechnologies to the general public. One of these barriers is the difficulty of performing experiments that require complex software, such as brain-computer interfaces (BCI) or cognitive neuroscience experiments. Current platforms have limitations in terms of functionality and flexibility to meet the needs of researchers, who often need to implement new experimentation settings. This work was aimed to propose a novel software ecosystem, called MEDUSA©, to overcome these limitations.

METHODS

We followed strict development practices to optimize MEDUSA© for research in BCI and cognitive neuroscience, making special emphasis in the modularity, flexibility and scalability of our solution. Moreover, it was implemented in Python, an open-source programming language that reduces the development cost by taking advantage from its high-level syntax and large number of community packages.

RESULTS

MEDUSA© provides a complete suite of signal processing functions, including several deep learning architectures or connectivity analysis, and ready-to-use BCI and neuroscience experiments, making it one of the most complete solutions nowadays. We also put special effort in providing tools to facilitate the development of custom experiments, which can be easily shared with the community through an app market available in our website to promote reproducibility.

CONCLUSIONS

MEDUSA© is a novel software ecosystem for modern BCI and neurotechnology experimentation that provides state-of-the-art tools and encourages the participation of the community to make a difference for the progress of these fields. Visit the official website at https://www.medusabci.com/ to know more about this project.

Decoding EEG rhythms offline and online during motor imagery for standing and sitting based on a brain-computer interface

Motor imagery (MI)-based brain-computer interface (BCI) systems have shown promising advances for lower limb motor rehabilitation. The purpose of this study was to develop an MI-based BCI for the actions of standing and sitting. Thirty-two healthy subjects participated in the study using 17 active EEG electrodes. We used a combination of the filter bank common spatial pattern (FBCSP) method and the regularized linear discriminant analysis (RLDA) technique for decoding EEG rhythms offline and online during motor imagery for standing and sitting. The offline analysis indicated the classification of motor imagery and idle state provided a mean accuracy of 88.51 ± 1.43% and 85.29 ± 1.83% for the sit-to-stand and stand-to-sit transitions, respectively. The mean accuracies of the sit-to-stand and stand-to-sit online experiments were 94.69 ± 1.29% and 96.56 ± 0.83%, respectively. From these results, we believe that the MI-based BCI may be useful to future brain-controlled standing systems.

Real-Time Process Monitoring Based on Multivariate Control Chart for Anomalies Driven by Frequency Signal via Sound and Electrocardiography Cases

Recent developments in network technologies have led to the application of cloud computing and big data analysis to industrial automation. However, the automation of process monitoring still has numerous issues that need to be addressed. Traditionally, offline statistical processes are generally used for process monitoring; thus, problems are often detected too late. This study focused on the construction of an automated process monitoring system based on sound and vibration frequency signals. First, empirical mode decomposition was combined with intrinsic mode functions to construct different sound frequency combinations and differentiate sound frequencies according to anomalies. Then, linear discriminant analysis (LDA) was adopted to classify abnormal and normal sound frequency signals, and a control line was constructed to monitor the sound frequency. In a case study, the proposed method was applied to detect abnormal sounds at high and low frequencies, and a detection accuracy of over 90% was realized. In another case study, the proposed method was applied to analyze electrocardiography signals and was similarly able to identify abnormal situations. Thus, the proposed method can be applied to real-time process monitoring and the detection of abnormalities with high accuracy in various situations.

Brain Signals Classification Based on Fuzzy Lattice Reasoning

Cyber-Physical System (CPS) applications including human-robot interaction call for automated reasoning for rational decision-making. In the latter context, typically, audio-visual signals are employed. Τhis work considers brain signals for emotion recognition towards an effective human-robot interaction. An ElectroEncephaloGraphy (EEG) signal here is represented by an Intervals’ Number (IN). An IN-based, optimizable parametric k Nearest Neighbor (kNN) classifier scheme for decision-making by fuzzy lattice reasoning (FLR) is proposed, where the conventional distance between two points is replaced by a fuzzy order function (σ) for reasoning-by-analogy. A main advantage of the employment of INs is that no ad hoc feature extraction is required since an IN may represent all-order data statistics, the latter are the features considered implicitly. Four different fuzzy order functions are employed in this work. Experimental results demonstrate comparably the good performance of the proposed techniques.

A Fuzzy Shell for Developing an Interpretable BCI Based on the Spatiotemporal Dynamics of the Evoked Oscillations

Researchers in neuroscience computing experience difficulties when they try to carry out neuroanalysis in practice or when they need to design an explainable brain-computer interface (BCI) with quick setup and minimal training phase. There is a need of interpretable computational intelligence techniques and new brain states decoding for more understandable interpretation of the sensory, cognitive, and motor brain processing. We propose a general-purpose fuzzy software system shell for developing a custom EEG BCI system. It relies on the bursts of the ongoing EEG frequency power synchronization/desynchronization at scalp level and supports quick BCI setup by linguistic features, ad hoc fuzzy membership construction, explainable IF-THEN rules, and the concept of the Internet of Things (IoT), which makes the BCI system device and service independent. It has a potential for designing both passive and event-related BCIs with options for visual representation at scalp-source level in response to time. The feasibility of the proposed system has been proven by real experiments and bursts for β and γ frequency power have been detected in real time in response to evoked visuospatial selective attention. The presence of the proposed new brain state decoding can be used as a feasible metric for interpretation of the spatiotemporal dynamics of the passive or evoked neural oscillations.

Motor Imagery Classification Based on a Recurrent-Convolutional Architecture to Control a Hexapod Robot

Advances in the field of Brain-Computer Interfaces (BCIs) aim, among other applications, to improve the movement capacities of people suffering from the loss of motor skills. The main challenge in this area is to achieve real-time and accurate bio-signal processing for pattern recognition, especially in Motor Imagery (MI). The significant interaction between brain signals and controllable machines requires instantaneous brain data decoding. In this study, an embedded BCI system based on fist MI signals is developed. It uses an Emotiv EPOC+ Brainwear®, an Altera SoCKit® development board, and a hexapod robot for testing locomotion imagery commands. The system is tested to detect the imagined movements of closing and opening the left and right hand to control the robot locomotion. Electroencephalogram (EEG) signals associated with the motion tasks are sensed on the human sensorimotor cortex. Next, the SoCKit processes the data to identify the commands allowing the controlled robot locomotion. The classification of MI-EEG signals from the F3, F4, FC5, and FC6 sensors is performed using a hybrid architecture of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. This method takes advantage of the deep learning recognition model to develop a real-time embedded BCI system, where signal processing must be seamless and precise. The proposed method is evaluated using k-fold cross-validation on both created and public Scientific-Data datasets. Our dataset is comprised of 2400 trials obtained from four test subjects, lasting three seconds of closing and opening fist movement imagination. The recognition tasks reach 84.69% and 79.2% accuracy using our data and a state-of-the-art dataset, respectively. Numerical results support that the motor imagery EEG signals can be successfully applied in BCI systems to control mobile robots and related applications such as intelligent vehicles.

A multi-modal modified feedback self-paced BCI to control the gait of an avatar

Brain-computer interfaces (BCI) have been used to control the gait of a virtual self-avatar with a proposed application in the field of gait rehabilitation.

OBJECTIVE

to develop a high performance multi-modal BCI to control single steps and forward walking of an immersive virtual reality avatar. This system will overcome the limitation of existing systems.

APPROACH

This system used MI of these actions, in cue-paced and self-paced modes. Twenty healthy participants participated in this 4 sessions study across 4 different days. They were cued to imagine a single step forward with their right or left foot, or to imagine walking forward. They were instructed to reach a target by using the MI of multiple steps (self-paced switch-control mode) or by maintaining MI of forward walking (continuous-control mode). The movement of the avatar was controlled by two calibrated RLDA classifiers that used the µ power spectral density (PSD) over the foot area of the motor cortex as a feature. The classifiers were retrained after every session. For a subset of the trials, positive modified feedback was presented to half of the participants.

MAIN RESULTS

All participants were able to operate the BCI. Their average offline performance, after retraining the classifiers was 86.0 ± 6.1%, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9 ± 8.4% showing that modified feedback enhanced BCI performance (p =0.001). The average performance was 83% at self-paced switch control and 92% at continuous control mode.

SIGNIFICANCE

This study reports on the first novel integration of different design approaches, different control modes and different performance enhancement techniques, all in parallel in one single high performance and multi-modal BCI system, to control single steps and forward walking of an immersive virtual reality avatar.

Motor Imagery EEG Decoding Method Based on a Discriminative Feature Learning Strategy

With the rapid development of deep learning, more and more deep learning-based motor imagery electroencephalograph (EEG) decoding methods have emerged in recent years. However, the existing deep learning-based methods usually only adopt the constraint of classification loss, which hardly obtains the features with high discrimination and limits the improvement of EEG decoding accuracy. In this paper, a discriminative feature learning strategy is proposed to improve the discrimination of features, which includes the central distance loss (CD-loss), the central vector shift strategy, and the central vector update process. First, the CD-loss is proposed to make the same class of samples converge to the corresponding central vector. Then, the central vector shift strategy extends the distance between different classes of samples in the feature space. Finally, the central vector update process is adopted to avoid the non-convergence of CD-loss and weaken the influence of the initial value of central vectors on the final results. In addition, overfitting is another severe challenge for deep learning-based EEG decoding methods. To deal with this problem, a data augmentation method based on circular translation strategy is proposed to expand the experimental datasets without introducing any extra noise or losing any information of the original data. To validate the effectiveness of the proposed method, we conduct some experiments on two public motor imagery EEG datasets (BCI competition IV 2a and 2b dataset), respectively. The comparison with current state-of-the-art methods indicates that our method achieves the highest average accuracy and good stability on the two experimental datasets.

Discrimination Improvement Through Undesirable Feedback in Coupling Object Manipulation Tasks

Subjective effort can significantly affect the ability of humans to act optimally in dynamic manipulation tasks. In a previous study, we designed a complex object coupling manipulation task that required tight performance and induced high cognitive workload. We hypothesize that strong-effort-related physiological reactivity during the dynamic manipulation task improves the user performance in an undesired task feedback situation. To test this hypothesis, using the motor intentions' discrimination from electroencephalogram (EEG) measurements, we evaluate the effort expended by 20 participants in a controlling task with constraints involving complex coupling objects. Specifically, the finer motor decisions are obtained from the controlling information in EEG by using two fingers from the same hand rather than two hands. The motor intention is decoded from a task-dependent EEG through a regularized discriminant analysis, and the area under the curve is [Formula: see text]. Furthermore, we compare the undesired and desired task feedback conditions along with the individual's effort dynamic adjustment, and investigate whether the undesired task feedback improved the discrimination of the motor activities. A stronger effort to attain the desired feedback state corresponds to improved motor activity discrimination from the EEG in the undesired task feedback scenario. The differences in the brain activities under the undesired and desired task feedback conditions are analyzed using brain-network-based topographical scalp maps. Our experiment provides preliminary evidence that inducing strong effort can improve discrimination performance during highly demanding tasks. This finding can advance our understanding of human attention, potentially improve the accuracy of intention recognition, and may inspire better EEG acquisition contexts.

Cluster decomposing and multi-objective optimization based-ensemble learning framework for motor imagery-based brain-computer interfaces

OBJECTIVE

Motor imagery (MI) is a mental representation of motor behavior and a widely used pattern in electroencephalogram (EEG) based brain-computer interface (BCI) systems. EEG is known for its non-stationary, non-linear features and sensitivity to artifacts from various sources. This study aimed to design a powerful classifier with a strong generalization capability for MI based BCIs.

APPROACH

In this study, we proposed a cluster decomposing based ensemble learning framework (CDECL) for EEG classification of MI based BCIs. The EEG data was decomposed into sub-data sets with different distributions by clustering decomposition. Then a set of heterogeneous classifiers was trained on each sub-data set for generating a diversified classifier search space. To obtain the optimal classifier combination, the ensemble learning was formulated as a multi-objective optimization problem and a stochastic fractal based binary multi-objective fruit fly optimization algorithm was proposed for solving the ensemble learning problem.

MAIN RESULTS

The proposed method was validated on two public EEG datasets (BCI Competition IV datasets IIb and BCI Competition IV dataset IIa) and compared with several other competing classification methods. Experimental results showed that the proposed CDECL based methods can effectively construct a diversity ensemble classifier and exhibits superior classification performance in comparison with several competing methods.

SIGNIFICANCE

The proposed method is promising for improving the performance of MI-based BCIs.

Early classification of motor tasks using dynamic functional connectivity graphs from EEG

OBJECTIVE

Classification of electroencephalography (EEG) signals with high accuracy using short recording intervals has been a challenging problem in developing brain computer interfaces (BCIs). This paper presents a novel feature extraction method for EEG recordings to tackle this problem.

APPROACH

The proposed approach is based on the concept that the brain functions in a dynamic manner, and utilizes dynamic functional connectivity graphs. The EEG data is first segmented into intervals during which functional networks sustain their connectivity. Functional connectivity networks for each identified segment are then localized, and graphs are constructed, which will be used as features. To take advantage of the dynamic nature of the generated graphs, a Long Short Term Memory (LSTM) classifier is employed for classification.

MAIN RESULTS

Features extracted from various durations of post-stimulus EEG data associated with motor execution and imagery tasks are used to test the performance of the classifier. Results show an average accuracy of 85.32% using features extracted from only 500 ms of the post-stimulus data.

SIGNIFICANCE

Our results demonstrate, for the first time, that using the proposed feature extraction method, it is possible to classify motor tasks from EEG recordings using a short interval of the data in the order of hundreds of milliseconds (e.g. 500 ms). This duration is considerably shorter than what has been reported before. These results will have significant implications for improving the effectiveness and the speed of BCIs, particularly for those used in assistive technologies.

Motor-Imagery Classification Using Riemannian Geometry with Median Absolute Deviation

Motor imagery (MI) from human brain signals can diagnose or aid specific physical activities for rehabilitation, recreation, device control, and technology assistance. It is a dynamic state in learning and practicing movement tracking when a person mentally imitates physical activity. Recently, it has been determined that a brain–computer interface (BCI) can support this kind of neurological rehabilitation or mental practice of action. In this context, MI data have been captured via non-invasive electroencephalogram (EEGs), and EEG-based BCIs are expected to become clinically and recreationally ground-breaking technology. However, determining a set of efficient and relevant features for the classification step was a challenge. In this paper, we specifically focus on feature extraction, feature selection, and classification strategies based on MI-EEG data. In an MI-based BCI domain, covariance metrics can play important roles in extracting discriminatory features from EEG datasets. To explore efficient and discriminatory features for the enhancement of MI classification, we introduced a median absolute deviation (MAD) strategy that calculates the average sample covariance matrices (SCMs) to select optimal accurate reference metrics in a tangent space mapping (TSM)-based MI-EEG. Furthermore, all data from SCM were projected using TSM according to the reference matrix that represents the featured vector. To increase performance, we reduced the dimensions and selected an optimum number of features using principal component analysis (PCA) along with an analysis of variance (ANOVA) that could classify MI tasks. Then, the selected features were used to develop linear discriminant analysis (LDA) training for classification. The benchmark datasets were considered for the evaluation and the results show that it provides better accuracy than more sophisticated methods.

Scaling Analysis of Phase Fluctuations of Brain Networks in Dynamic Constrained Object Manipulation

In this study, we investigated the dynamic properties of oscillatory activities in the scalp electro-encephalographs (EEGs) of 20 participants involved in a novel dynamic manipulating task using a physical interface and a virtual feedback. The complexity of such a task a rises from the unexpected relationship between the magnitude of the motion and the feedback. The characterization of complex patterns arising from EEG is an important problem in identifying different mental intentions. We proposed a scaling analysis of phase fluctuation in the scalp EEG to discriminate the network states related to different EEG patterns, which correspond to manipulating the task with right or left movement intention. These intentions are generated while the participant is engaged in such a complex task. The phase characterization method was used to calculate the instantaneous phase from the operational EEG. Then, functional brain networks (FBNs) of 20 subjects based on the task-related EEG were constructed by phase synchronization. The degree features representing the structures and scaling components of brain networks are sensitive to the EEG patterns with left or right motor intention. The correlation between features and mental intentions was investigated by discriminant analysis. For 20 subjects, the average accuracy of state detection is [Formula: see text], and the average mean-squared error (MSE) is [Formula: see text]. The brain state depicted by the results is related to high awareness, the phase characterization is of the effectiveness in EEG processing and FBN construction and the difference of control intentions can be explored by the phase characterization method. This finding may be relevant to understanding some neuronal mechanisms underlying the attention and some applications of closed-loop control for the safety operation of tools.

A performance based feature selection technique for subject independent MI based BCI

Purpose

Significant research has been conducted in the field of brain computer interface (BCI) algorithm development, however, many of the resulting algorithms are both complex, and specific to a particular user as the most successful methodology can vary between individuals and sessions. The objective of this study was to develop a simple yet effective method of feature selection to improve the accuracy of a subject independent BCI algorithm and streamline the process of BCI algorithm development. Over the past several years, several high precision features have been suggested by researchers to classify different motor imagery tasks. This research applies fourteen of these features as a feature pool that can be used as a reference for future researchers. Additionally, we look for the most efficient feature or feature set with four different classifiers that best differentiates several motor imagery tasks. In this work we have successfully employed a feature fusion method to obtain the best sub-set of features. We have proposed a novel computer aided feature selection method to determine the best set of features for distinguishing between motor imagery tasks in lieu of the manual feature selection that has been performed in past studies. The features selected by this method were then fed into a Linear Discriminant Analysis, K-nearest neighbor, decision tree, or support vector machine classifier for classification to determine the overall performance.

Methods

The methods used were a novel performance based additive feature fusion algorithm working in conjunction with machine learning in order to classify the motor imagery signals into particular states. The data used for this study was collected from BCI competition III dataset IVa.

Result

The result of this algorithm was a classification accuracy of 99% for a subject independent algorithm with less computation cost compared to traditional methods, in addition to multiple feature/classifier combinations that outperform current subject independent methods.

Conclusion

The conclusion of this study and its significance is that it developed a viable methodology for simple, efficient feature selection and BCI algorithm development, which leads to an overall increase in algorithm classification accuracy.