An enhanced time-frequency-spatial approach for motor imagery classification

Optimal channel-based sparse time-frequency blocks common spatial pattern feature extraction method for motor imagery classification

Common spatial pattern (CSP) as a spatial filtering method has been most widely applied to electroencephalogram (EEG) feature extraction to classify motor imagery (MI) in brain-computer interface (BCI) applications. The effectiveness of CSP is determined by the quality of interception in a specific time window and frequency band. Although numerous algorithms have been designed to optimize CSP by splitting the EEG data with a sliding time window and dividing the frequency bands with a set of band-pass filters, simultaneously. However, they did not consider the drawbacks of the rapid increase in data volume and feature dimensions brought about by this method, which would reduce the classification accuracy and calculation efficiency of the model. Therefore, we propose an optimal channel-based sparse time-frequency blocks common spatial pattern (OCSB-CSP) feature extraction method to improve the model classification accuracy and computational efficiency. Comparative experiments on two public EEG datasets show that the proposed method can quickly select significant time-frequency blocks and improve classification performance. The average classification accuracies are higher than those of other winners' methods, providing a new idea for the improvement of BCI applications.

Enhanced Multiple Instance Representation Using Time-Frequency Atoms in Motor Imagery Classification

Selection of the time-window mainly affects the effectiveness of piecewise feature extraction procedures. We present an enhanced bag-of-patterns representation that allows capturing the higher-level structures of brain dynamics within a wide window range. So, we introduce augmented instance representations with extended window lengths for the short-time Common Spatial Pattern algorithm. Based on multiple-instance learning, the relevant bag-of-patterns are selected by a sparse regression to feed a bag classifier. The proposed higher-level structure representation promotes two contributions: (i) accuracy improvement of bi-conditional tasks, (ii) A better understanding of dynamic brain behavior through the learned sparse regression fits. Using a support vector machine classifier, the achieved performance on a public motor imagery dataset (left-hand and right-hand tasks) shows that the proposed framework performs very competitive results, providing robustness to the time variation of electroencephalography recordings and favoring the class separability.

A Lasso quantile periodogram based feature extraction for EEG-based motor imagery

BACKGROUND

The extraction of relevant and distinct features from the electroencephalogram (EEG) signals is one of the most challenging task when implementing Brain Computer Interface (BCI) based systems. Frequency analysis techniques are recognised as one of the most suitable methods to have distinct information from EEG signals. However, existing studies use mostly classical approaches assuming that the signal is Gaussian, stationary and linear. These properties are not verified in the EEG case considering the complexity of the brain electrical activity.

NEW METHOD

This paper proposes two new spectral estimators that are robust against non-Gaussian, non-linear and non-stationary signals. These two approaches use quantile regression and L₁-norm regularisation to estimate the spectrum of the motor imagery (MI) related EEG.

RESULTS

A dataset collected during a study of BCI motor imagery project conducted at Tshwane University of Technology (TUT), Pretoria, South Africa, is used to validate the proposed estimators. Experimental results demonstrate that the newly proposed approaches help improve the classification performance of MI.

COMPARISON WITH EXISTING METHODS

In order to show the effectiveness of the proposed estimators, a comparative study is conducted, considering classical commonly used techniques such as FFT and Welch periodogram through 5 classification algorithms.

CONCLUSIONS

The proposed Quantile-based spectral estimators are potential methods to improve the classification performance of the EEG-Based motor imagery systems.

Human motor decoding from neural signals: a review

Many people suffer from movement disability due to amputation or neurological diseases. Fortunately, with modern neurotechnology now it is possible to intercept motor control signals at various points along the neural transduction pathway and use that to drive external devices for communication or control. Here we will review the latest developments in human motor decoding. We reviewed the various strategies to decode motor intention from human and their respective advantages and challenges. Neural control signals can be intercepted at various points in the neural signal transduction pathway, including the brain (electroencephalography, electrocorticography, intracortical recordings), the nerves (peripheral nerve recordings) and the muscles (electromyography). We systematically discussed the sites of signal acquisition, available neural features, signal processing techniques and decoding algorithms in each of these potential interception points. Examples of applications and the current state-of-the-art performance were also reviewed. Although great strides have been made in human motor decoding, we are still far away from achieving naturalistic and dexterous control like our native limbs. Concerted efforts from material scientists, electrical engineers, and healthcare professionals are needed to further advance the field and make the technology widely available in clinical use.

Temporally Constrained Sparse Group Spatial Patterns for Motor Imagery BCI

Common spatial pattern (CSP)-based spatial filtering has been most popularly applied to electroencephalogram (EEG) feature extraction for motor imagery (MI) classification in brain-computer interface (BCI) application. The effectiveness of CSP is highly affected by the frequency band and time window of EEG segments. Although numerous algorithms have been designed to optimize the spectral bands of CSP, most of them selected the time window in a heuristic way. This is likely to result in a suboptimal feature extraction since the time period when the brain responses to the mental tasks occurs may not be accurately detected. In this paper, we propose a novel algorithm, namely temporally constrained sparse group spatial pattern (TSGSP), for the simultaneous optimization of filter bands and time window within CSP to further boost classification accuracy of MI EEG. Specifically, spectrum-specific signals are first derived by bandpass filtering from raw EEG data at a set of overlapping filter bands. Each of the spectrum-specific signals is further segmented into multiple subseries using sliding window approach. We then devise a joint sparse optimization of filter bands and time windows with temporal smoothness constraint to extract robust CSP features under a multitask learning framework. A linear support vector machine classifier is trained on the optimized EEG features to accurately identify the MI tasks. An experimental study is implemented on three public EEG datasets (BCI Competition III dataset IIIa, BCI Competition IV datasets IIa, and BCI Competition IV dataset IIb) to validate the effectiveness of TSGSP in comparison to several other competing methods. Superior classification performance (averaged accuracies are 88.5%, 83.3%, and 84.3% for the three datasets, respectively) based on the experimental results confirms that the proposed algorithm is a promising candidate for performance improvement of MI-based BCIs.

Continuous Wavelet Transform for Decoding Finger Movements From Single-Channel EEG

Human body movements can be reflected in brain signals and collected noninvasively with electroencephalography (EEG). Motor-related signals include sensory motor rhythms (also known as the Mu wave) in the upper-alpha band of 8-13 Hz and slow cortical potentials (SCPs) in the low frequency range of 0.1-5 Hz. This study compares the two signals for decoding finger movements. Human subjects were asked to individually lift each of the five digits of their right hand, at the rate of one every 10 s. EEG was recorded using a bipolar montage between ipsilateral and contralateral motor cortices. Electromyograms were obtained for identifying movement onsets. Linear discriminant analysis (LDA) generated significant performance with SCPs but not with Mu. Meanwhile, continuous wavelet transform (CWT) was applied to SCPs or Mu to create a spectrogram for each finger, showing distinctive amplitude and phase patterns. A dprime-based weighting algorithm was used to extract time-frequency features. With a template-matching paradigm, both SCP and Mu spectrograms yielded significant classification accuracies for multiple subjects, with the highest being >50% (chance = 20%). Interestingly, the index finger was better distinguished with Mu for most of the subjects, whereas the ring finger was better distinguished with SCPs. The CWT algorithm outperformed LDA by better decoding the thumb. This study suggests that the time-frequency characteristics of a single-channel EEG, when phase is preserved, contain critical information on finger movements. SCPs and Mu seem to work in an independent but complementary manner.

A Ternary Hybrid EEG-NIRS Brain-Computer Interface for the Classification of Brain Activation Patterns during Mental Arithmetic, Motor Imagery, and Idle State

The performance of a brain-computer interface (BCI) can be enhanced by simultaneously using two or more modalities to record brain activity, which is generally referred to as a hybrid BCI. To date, many BCI researchers have tried to implement a hybrid BCI system by combining electroencephalography (EEG) and functional near-infrared spectroscopy (NIRS) to improve the overall accuracy of binary classification. However, since hybrid EEG-NIRS BCI, which will be denoted by hBCI in this paper, has not been applied to ternary classification problems, paradigms and classification strategies appropriate for ternary classification using hBCI are not well investigated. Here we propose the use of an hBCI for the classification of three brain activation patterns elicited by mental arithmetic, motor imagery, and idle state, with the aim to elevate the information transfer rate (ITR) of hBCI by increasing the number of classes while minimizing the loss of accuracy. EEG electrodes were placed over the prefrontal cortex and the central cortex, and NIRS optodes were placed only on the forehead. The ternary classification problem was decomposed into three binary classification problems using the "one-versus-one" (OVO) classification strategy to apply the filter-bank common spatial patterns filter to EEG data. A 10 × 10-fold cross validation was performed using shrinkage linear discriminant analysis (sLDA) to evaluate the average classification accuracies for EEG-BCI, NIRS-BCI, and hBCI when the meta-classification method was adopted to enhance classification accuracy. The ternary classification accuracies for EEG-BCI, NIRS-BCI, and hBCI were 76.1 ± 12.8, 64.1 ± 9.7, and 82.2 ± 10.2%, respectively. The classification accuracy of the proposed hBCI was thus significantly higher than those of the other BCIs (p < 0.005). The average ITR for the proposed hBCI was calculated to be 4.70 ± 1.92 bits/minute, which was 34.3% higher than that reported for a previous binary hBCI study.

Improving the Accuracy and Training Speed of Motor Imagery Brain-Computer Interfaces Using Wavelet-Based Combined Feature Vectors and Gaussian Mixture Model-Supervectors

In this paper, we propose a set of wavelet-based combined feature vectors and a Gaussian mixture model (GMM)-supervector to enhance training speed and classification accuracy in motor imagery brain–computer interfaces. The proposed method is configured as follows: first, wavelet transforms are applied to extract the feature vectors for identification of motor imagery electroencephalography (EEG) and principal component analyses are used to reduce the dimensionality of the feature vectors and linearly combine them. Subsequently, the GMM universal background model is trained by the expectation–maximization (EM) algorithm to purify the training data and reduce its size. Finally, a purified and reduced GMM-supervector is used to train the support vector machine classifier. The performance of the proposed method was evaluated for three different motor imagery datasets in terms of accuracy, kappa, mutual information, and computation time, and compared with the state-of-the-art algorithms. The results from the study indicate that the proposed method achieves high accuracy with a small amount of training data compared with the state-of-the-art algorithms in motor imagery EEG classification.

Classifying three imaginary states of the same upper extremity using time-domain features

Brain-computer interface (BCI) allows collaboration between humans and machines. It translates the electrical activity of the brain to understandable commands to operate a machine or a device. In this study, we propose a method to improve the accuracy of a 3-class BCI using electroencephalographic (EEG) signals. This BCI discriminates rest against imaginary grasps and elbow movements of the same limb. This classification task is challenging because imaginary movements within the same limb have close spatial representations on the motor cortex area. The proposed method extracts time-domain features and classifies them using a support vector machine (SVM) with a radial basis kernel function (RBF). An average accuracy of 74.2% was obtained when using the proposed method on a dataset collected, prior to this study, from 12 healthy individuals. This accuracy was higher than that obtained when other widely used methods, such as common spatial patterns (CSP), filter bank CSP (FBCSP), and band power methods, were used on the same dataset. These results are encouraging and the proposed method could potentially be used in future applications including BCI-driven robotic devices, such as a portable exoskeleton for the arm, to assist individuals with impaired upper extremity functions in performing daily tasks.

Classification Scheme for Arm Motor Imagery

Facilitating independent living of individuals with upper extremity impairment is a compelling goal for our society. The degree of disability of these individuals could potentially be reduced by using robotic devices that assist their movements in activities of daily living. One approach to control such robotic systems is the use of a brain–computer interface, which detects the user’s intention. This study proposes a method for estimating the user’s intention using electroencephalographic (EEG) signals. The proposed method is capable of discriminating rest from various imagined arm movements, including grasping and elbow flexion. The features extracted from EEG signals are autoregressive model coefficients, root-mean-square amplitude, and waveform length. Support vector machine was used as a classifier, distinguishing class labels corresponding to rest and imagined arm movements. The performance of the proposed method was evaluated using cross-validation. Average accuracies of 91.8 ± 5.8 and 90 ± 4.1 % were obtained for distinguishing rest versus grasping and rest versus elbow flexion. The results show that the proposed scheme provides 18.9, 17.1, and 16.5 % higher classification accuracies for distinguishing rest versus grasping and 21.9, 17.6, and 18.1 % higher classification accuracies for distinguishing rest versus elbow flexion compared with those obtained using filter bank common spatial pattern, band power, and common spatial pattern methods, respectively, which are widely used in the literature.

Development of a Wearable Motor-Imagery-Based Brain-Computer Interface

A motor-imagery-based brain-computer interface (BCI) is a translator that converts the motor intention of the brain into a control command to control external machines without muscles. Numerous motor-imagery-based BCIs have been successfully proposed in previous studies. However, several electroencephalogram (EEG) channels are typically required for providing sufficient information to maintain a specific accuracy and bit rate, and the bulk volume of these EEG machines is also inconvenient. A wearable motor imagery-based BCI system was proposed and implemented in this study. A wearable mechanical design with novel active comb-shaped dry electrodes was developed to measure EEG signals without conductive gels at hair sites, which is easy and convenient for users wearing the EEG machine. In addition, a wireless EEG acquisition module was also designed to measure EEG signals, which provides a user with more freedom of motion. The proposed wearable motor-imagery-based BCI system was validated using an electrical specifications test and a hand motor imagery experiment. Experimental results showed that the proposed wearable motor-imagery-based BCI system provides favorable signal quality for measuring EEG signals and detecting motor imagery.

Hybrid EEG-fNIRS Asynchronous Brain-Computer Interface for Multiple Motor Tasks

Non-invasive Brain-Computer Interfaces (BCI) have demonstrated great promise for neuroprosthetics and assistive devices. Here we aim to investigate methods to combine Electroencephalography (EEG) and functional Near-Infrared Spectroscopy (fNIRS) in an asynchronous Sensory Motor rhythm (SMR)-based BCI. We attempted to classify 4 different executed movements, namely, Right-Arm—Left-Arm—Right-Hand—Left-Hand tasks. Previous studies demonstrated the benefit of EEG-fNIRS combination. However, since normally fNIRS hemodynamic response shows a long delay, we investigated new features, involving slope indicators, in order to immediately detect changes in the signals. Moreover, Common Spatial Patterns (CSPs) have been applied to both EEG and fNIRS signals. 15 healthy subjects took part in the experiments and since 25 trials per class were available, CSPs have been regularized with information from the entire population of participants and optimized using genetic algorithms. The different features have been compared in terms of performance and the dynamic accuracy over trials shows that the introduced methods diminish the fNIRS delay in the detection of changes.

Brain-computer interfaces using sensorimotor rhythms: current state and future perspectives

Many studies over the past two decades have shown that people can use brain signals to convey their intent to a computer using brain-computer interfaces (BCIs). BCI systems extract specific features of brain activity and translate them into control signals that drive an output. Recently, a category of BCIs that are built on the rhythmic activity recorded over the sensorimotor cortex, i.e., the sensorimotor rhythm (SMR), has attracted considerable attention among the BCIs that use noninvasive neural recordings, e.g., electroencephalography (EEG), and have demonstrated the capability of multidimensional prosthesis control. This paper reviews the current state and future perspectives of SMR-based BCI and its clinical applications, in particular focusing on the EEG SMR. The characteristic features of SMR from the human brain are described and their underlying neural sources are discussed. The functional components of SMR-based BCI, together with its current clinical applications, are reviewed. Finally, limitations of SMR-BCIs and future outlooks are also discussed.

A frequency-temporal-spatial method for motor-related electroencephalography pattern recognition by comprehensive feature optimization

Either imagined or actual movements lead to a combination of electroencephalography signals with distinctive frequency, temporal and spatial characteristics, which correspond to various motor-related neural activities. This frequency-temporal-spatial pattern is the key of motor intention decoding which is the basis of brain-computer interfaces by motor imagery. We present a new method for motor-related electroencephalography recognition which comprehensively optimizes the frequency-time-space features in a user-specific way. The recognition work focuses on three points: proper time and frequency domain segmentation, spatial optimization based on common spatial pattern filters and feature importance evaluation. We show that by combining the advantages of these optimizational methods, the proposed algorithm effectively improves motor task classification, and the recognized signal chanracteristics can be used to visualize the motor related electroencephalography patterns under different conditions.

Time-frequency selection in two bipolar channels for improving the classification of motor imagery EEG

Time and frequency information is essential to feature extraction in a motor imagery BCI, in particular for systems based on a few channels. In this paper, we propose a novel time-frequency selection method based on a criterion called Time-frequency Discrimination Factor (TFDF) to extract discriminative event-related desynchronization (ERD) features for BCI data classification. Compared to existing methods, the proposed approach generates better classification performances (mean kappa coefficient= 0.62) on experimental data from the BCI competition IV dataset IIb, with only two bipolar channels.

Optimum spatio-spectral filtering network for brain-computer interface

This paper proposes a feature extraction method for motor imagery brain-computer interface (BCI) using electroencephalogram. We consider the primary neurophysiologic phenomenon of motor imagery, termed event-related desynchronization, and formulate the learning task for feature extraction as maximizing the mutual information between the spatio-spectral filtering parameters and the class labels. After introducing a nonparametric estimate of mutual information, a gradient-based learning algorithm is devised to efficiently optimize the spatial filters in conjunction with a band-pass filter. The proposed method is compared with two existing methods on real data: a BCI Competition IV dataset as well as our data collected from seven human subjects. The results indicate the superior performance of the method for motor imagery classification, as it produced higher classification accuracy with statistical significance ( ≥ 95% confidence level) in most cases.

Identification of task parameters from movement-related cortical potentials

The study investigates the accuracy in discriminating rate of torque development (RTD) and target torque (TT) (task parameters) from electroencephalography (EEG) signals generated during imaginary motor tasks. Signals were acquired from nine healthy subjects during four imaginary isometric plantar-flexions of the right foot involving two RTDs (ballistic and moderate) and two TTs (30 and 60% of the maximal voluntary contraction torque), each repeated 60 times in random order. The single-trial EEG traces were classified with a pattern recognition approach based on wavelet coefficients as features and support vector machine (SVM) as classifier. Average misclassification rates were (mean +/- SD) 16 +/- 9% and 26 +/- 13% for discrimination of the two TTs under ballistic and moderate RTDs, respectively. RTDs could be discriminated with misclassification rates of 16 +/- 11% and 19 +/- 10% under high and low TT, respectively. These results indicate that differences in both TT and RTD can be detected from single-trial EEG traces during imaginary tasks.

Identification of arm movements using correlation of electrocorticographic spectral components and kinematic recordings

The purpose of this study was to explore the possibility of using electrocorticographic (ECoG) recordings from subdural electrodes placed over the motor cortex to identify the upper limb motion performed by a human subject. More specifically, we were trying to identify features in the ECoG signals that could help us determine the type of movement performed by an individual. Two subjects who had subdural electrodes implanted over the motor cortex were asked to perform various motor tasks with the upper limb contralateral to the site of electrode implantation. ECoG signals and upper limb kinematics were recorded while the participants were performing the movements. ECoG frequency components were identified that correlated well with the performed movements measured along 6D coordinates (X, Y, Z, roll, yaw and pitch). These frequencies were grouped using histograms. The resulting histograms had consistent and unique shapes that were representative of individual upper limb movements performed by the participants. Thus, it was possible to identify which movement was performed by the participant without prior knowledge of the arm and hand kinematics. To confirm these findings, a nearest neighbour classifier was applied to identify the specific movement that each participant had performed. The achieved classification accuracy was 89%.

The wadsworth BCI research and development program: at home with BCI

The ultimate goal of brain-computer interface (BCI) technology is to provide communication and control capacities to people with severe motor disabilities. BCI research at the Wadsworth Center focuses primarily on noninvasive, electroencephalography (EEG)-based BCI methods. We have shown that people, including those with severe motor disabilities, can learn to use sensorimotor rhythms (SMRs) to move a cursor rapidly and accurately in one or two dimensions. We have also improved P300-based BCI operation. We are now translating this laboratory-proven BCI technology into a system that can be used by severely disabled people in their homes with minimal ongoing technical oversight. To accomplish this, we have: improved our general-purpose BCI software (BCI2000); improved online adaptation and feature translation for SMR-based BCI operation; improved the accuracy and bandwidth of P300-based BCI operation; reduced the complexity of system hardware and software and begun to evaluate home system use in appropriate users. These developments have resulted in prototype systems for every day use in people's homes.