Classification of motor imagery tasks for brain-computer interface applications by means of two equivalent dipoles analysis

Progress in Brain Computer Interface: Challenges and Opportunities

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

Chaotic behaviour of EEG responses with an identical grasp posture

Individuals with severe neuromuscular ailments can benefit from restoring their grasp activities with a brain-controlled upper-limb neuroprosthesis. EEG signals can be utilized as the driving source, and to implement natural human-like grasping abilities. Although good accuracy has already been achieved in classifying the various grasp patterns for specific sets of objects, unseen objects are still a hurdle in real-life implementation. Generalization of grasp patterns should be explored without any prior knowledge of the objects. In this regard, the similarity of motor imagery for different objects requiring similar grasp pattern can be utilized. It is also necessary to identify the brain regions that exhibit prominent distinguishability during different grasp patterns. In this study, we propose a chaos-based method to decode the motor imagery of two quite similar Power grasp patterns-cylindrical and spherical-for holding various objects. Three distinct suitable objects were chosen for each of the two patterns, and a 29-channel EEG was taken of 18 healthy participants to explore motor imagery for grasping the objects. Nonlinear correlation dimension was employed on the EEG data, at sub-band levels α, upper β, and γ, to analyse the distinguishability, as well as the similarity of grasp patterns for the objects. ANOVA was subsequently performed on the obtained CD parameters to identify the contribution of each electrode channel. Furthermore, using an SVM classifier, more than 80% accuracy was obtained in classifying the grasping patterns at the upper β sub-band. The outcome may lead to identification of optimum feature sets of motor imagery from specific brain regions for random objects grasps.

A space-frequency localized approach of spatial filtering for motor imagery classification

Classification of Motor Imagery (MI) signals is the heart of Brain-Computer Interface (BCI) based applications. Spatial filtering is an important step in this process that produce new set of signals for better discrimination of two classes of EEG signals. In this work, a new approach of spatial filtering called Space-Frequency Localized Spatial Filtering (SFLSF) is proposed to enhance the performances of MI classification. The SFLSF method initially divides the scalp-EEG channels into local overlapping spatial windows. Then a filter bank is used to divide the signals into local frequency bands. The group of channels, localized in space and frequency, are then processed with spatial filter, and features are subsequently extracted for classification task. Experimental results corroborate that the proposed space localization helps to increase the classification accuracy when compared to the existing methods using spatial filters. The classification performance is further improved when frequency localization is incorporated. Thus, the proposed space-frequency localized approach of spatial filtering helps to deliver better classification result which is consistently 3-5% higher than traditional methods.

A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN

OBJECTIVE

To develop and implement a novel approach which combines the technique of scout EEG source imaging (ESI) with convolutional neural network (CNN) for the classification of motor imagery (MI) tasks.

APPROACH

The technique of ESI uses a boundary element method (BEM) and weighted minimum norm estimation (WMNE) to solve the EEG forward and inverse problems, respectively. Ten scouts are then created within the motor cortex to select the region of interest (ROI). We extract features from the time series of scouts using a Morlet wavelet approach. Lastly, CNN is employed for classifying MI tasks.

MAIN RESULTS

The overall mean accuracy on the Physionet database reaches 94.5% and the individual accuracy of each task reaches 95.3%, 93.3%, 93.6%, 96% for the left fist, right fist, both fists and both feet, correspondingly, validated using ten-fold cross validation. We report an increase of up to 14.4% for overall classification compared with the competitive results from the state-of-the-art MI classification methods. Then, we add four new subjects to verify the validity of the method and the overall mean accuracy is 92.5%. Furthermore, the global classifier was adapted to single subjects improving the overall mean accuracy to 94.54%.

SIGNIFICANCE

The combination of scout ESI and CNN enhances BCI performance of decoding EEG four-class MI tasks.

A performance based feature selection technique for subject independent MI based BCI

Purpose

Significant research has been conducted in the field of brain computer interface (BCI) algorithm development, however, many of the resulting algorithms are both complex, and specific to a particular user as the most successful methodology can vary between individuals and sessions. The objective of this study was to develop a simple yet effective method of feature selection to improve the accuracy of a subject independent BCI algorithm and streamline the process of BCI algorithm development. Over the past several years, several high precision features have been suggested by researchers to classify different motor imagery tasks. This research applies fourteen of these features as a feature pool that can be used as a reference for future researchers. Additionally, we look for the most efficient feature or feature set with four different classifiers that best differentiates several motor imagery tasks. In this work we have successfully employed a feature fusion method to obtain the best sub-set of features. We have proposed a novel computer aided feature selection method to determine the best set of features for distinguishing between motor imagery tasks in lieu of the manual feature selection that has been performed in past studies. The features selected by this method were then fed into a Linear Discriminant Analysis, K-nearest neighbor, decision tree, or support vector machine classifier for classification to determine the overall performance.

Methods

The methods used were a novel performance based additive feature fusion algorithm working in conjunction with machine learning in order to classify the motor imagery signals into particular states. The data used for this study was collected from BCI competition III dataset IVa.

Result

The result of this algorithm was a classification accuracy of 99% for a subject independent algorithm with less computation cost compared to traditional methods, in addition to multiple feature/classifier combinations that outperform current subject independent methods.

Conclusion

The conclusion of this study and its significance is that it developed a viable methodology for simple, efficient feature selection and BCI algorithm development, which leads to an overall increase in algorithm classification accuracy.

Reducing the Computational Complexity of EEG Source Localization With Cortical Patch Decomposition and Optimal Electrode Selection

OBJECTIVE

Real-time implementation of EEG source localization can be employed in a broad area of applications such as clinical diagnosis of neurologic diseases and brain-computer interface. However, a power-efficient, low-complexity, and real-time implementation of EEG source localization is still challenging due to extensive iterations in the solutions. In this study, two techniques are introduced to reduce the computational burden of the subspace-based MUltiple SIgnal Classification (MUSIC) algorithm.

METHODS

To shrink the exhaustive search inherent in MUSIC, the cortex is parsed into cortical regions. A novel nomination procedure involving a dictionary learning step will pick a number of regions to be searched for the active sources. In addition, a new electrode selection algorithm based on the Cramer-Rao bound of the errors is introduced to pick the best set of an arbitrary number of electrodes out of the total.

RESULTS

The performance of the proposed techniques were evaluated using simulated EEG signal under variation of different parameters such as the number of nominated regions, the signal to noise ratio, and the number of electrodes. The proposed techniques can reduce the computational complexity by up to $90\%$. Furthermore, the proposed techniques were tested on EEG data from an auditory oddball experiment.

CONCLUSION

A good concordance was observed in the comparison of the topographies and the localization errors derived from the proposed technique and regular MUSIC.

SIGNIFICANCE

Such reduction can be exploited in the real-time, long-run, and mobile monitoring of cortical activity for clinical diagnosis and research purposes.

Functional Brain Connectivity during Multiple Motor Imagery Tasks in Spinal Cord Injury

Reciprocal communication of the central and peripheral nervous systems is compromised during spinal cord injury due to neurotrauma of ascending and descending pathways. Changes in brain organization after spinal cord injury have been associated with differences in prognosis. Changes in functional connectivity may also serve as injury biomarkers. Most studies on functional connectivity have focused on chronic complete injury or resting-state condition. In our study, ten right-handed patients with incomplete spinal cord injury and ten age- and gender-matched healthy controls performed multiple visual motor imagery tasks of upper extremities and walking under high-resolution electroencephalography recording. Directed transfer function was used to study connectivity at the cortical source space between sensorimotor nodes. Chronic disruption of reciprocal communication in incomplete injury could result in permanent significant decrease of connectivity in a subset of the sensorimotor network, regardless of positive or negative neurological outcome. Cingulate motor areas consistently contributed the larger outflow (right) and received the higher inflow (left) among all nodes, across all motor imagery categories, in both groups. Injured subjects had higher outflow from left cingulate than healthy subjects and higher inflow in right cingulate than healthy subjects. Alpha networks were less dense, showing less integration and more segregation than beta networks. Spinal cord injury patients showed signs of increased local processing as adaptive mechanism. This trial is registered with NCT02443558.

Anodal Transcranial Direct Current Stimulation Increases Bilateral Directed Brain Connectivity during Motor-Imagery Based Brain-Computer Interface Control

Transcranial direct current stimulation (tDCS) has been shown to affect motor and cognitive task performance and learning when applied to brain areas involved in the task. Targeted stimulation has also been found to alter connectivity within the stimulated hemisphere during rest. However, the connectivity effect of the interaction of endogenous task specific activity and targeted stimulation is unclear. This study examined the aftereffects of concurrent anodal high-definition tDCS over the left sensorimotor cortex with motor network connectivity during a one-dimensional EEG based sensorimotor rhythm brain-computer interface (SMR-BCI) task. Directed connectivity following anodal tDCS illustrates altered connections bilaterally between frontal and parietal regions, and these alterations occur in a task specific manner; connections between similar cortical regions are altered differentially during left and right imagination trials. During right-hand imagination following anodal tDCS, there was an increase in outflow from the left premotor cortex (PMC) to multiple regions bilaterally in the motor network and increased inflow to the stimulated sensorimotor cortex from the ipsilateral PMC and contralateral sensorimotor cortex. During left-hand imagination following anodal tDCS, there was increased outflow from the stimulated sensorimotor cortex to regions across the motor network. Significant correlations between connectivity and the behavioral measures of total correct trials and time-to-hit (TTH) correct trials were also found, specifically that the input to the left PMC correlated with decreased right hand imagination performance and that flow from the ipsilateral posterior parietal cortex (PPC) to midline sensorimotor cortex correlated with improved performance for both right and left hand imagination. These results indicate that tDCS interacts with task-specific endogenous activity to alter directed connectivity during SMR-BCI. In order to predict and maximize the targeted effect of tDCS, the interaction of stimulation with the dynamics of endogenous activity needs to be examined comprehensively and understood.

Feasibility of Equivalent Dipole Models for Electroencephalogram-Based Brain Computer Interfaces

This article examines the localization errors of equivalent dipolar sources inverted from the surface electroencephalogram in order to determine the feasibility of using their location as classification parameters for non-invasive brain computer interfaces. Inverse localization errors are examined for two head models: a model represented by four concentric spheres and a realistic model based on medical imagery. It is shown that the spherical model results in localization ambiguity such that a number of dipolar sources, with different azimuths and varying orientations, provide a near match to the electroencephalogram of the best equivalent source. No such ambiguity exists for the elevation of inverted sources, indicating that for spherical head models, only the elevation of inverted sources (and not the azimuth) can be expected to provide meaningful classification parameters for brain-computer interfaces. In a realistic head model, all three parameters of the inverted source location are found to be reliable, providing a more robust set of parameters. In both cases, the residual error hypersurfaces demonstrate local minima, indicating that a search for the best-matching sources should be global. Source localization error vs. signal-to-noise ratio is also demonstrated for both head models.

As above, so below? Towards understanding inverse models in BCI

OBJECTIVE

In brain-computer interfaces (BCI), measurements of the user's brain activity are classified into commands for the computer. With EEG-based BCIs, the origins of the classified phenomena are often considered to be spatially localized in the cortical volume and mixed in the EEG. We investigate if more accurate BCIs can be obtained by reconstructing the source activities in the volume.

APPROACH

We contrast the physiology-driven source reconstruction with data-driven representations obtained by statistical machine learning. We explain these approaches in a common linear dictionary framework and review the different ways to obtain the dictionary parameters. We consider the effect of source reconstruction on some major difficulties in BCI classification, namely information loss, feature selection and nonstationarity of the EEG.

MAIN RESULTS

Our analysis suggests that the approaches differ mainly in their parameter estimation. Physiological source reconstruction may thus be expected to improve BCI accuracy if machine learning is not used or where it produces less optimal parameters. We argue that the considered difficulties of surface EEG classification can remain in the reconstructed volume and that data-driven techniques are still necessary. Finally, we provide some suggestions for comparing approaches.

SIGNIFICANCE

The present work illustrates the relationships between source reconstruction and machine learning-based approaches for EEG data representation. The provided analysis and discussion should help in understanding, applying, comparing and improving such techniques in the future.

Sensorimotor Rhythm BCI with Simultaneous High Definition-Transcranial Direct Current Stimulation Alters Task Performance

BACKGROUND

Transcranial direct current stimulation (tDCS) has been used to alter the excitability of neurons within the cerebral cortex. Improvements in motor learning have been found in multiple studies when tDCS was applied to the motor cortex before or during task learning. The motor cortex is also active during the performance of motor imagination, a cognitive task during which a person imagines, but does not execute, a movement. Motor imagery can be used with noninvasive brain computer interfaces (BCIs) to control virtual objects in up to three dimensions, but to master control of such devices requires long training times.

OBJECTIVE

To evaluate the effect of high-definition tDCS on the performance and underlying electrophysiology of motor imagery based BCI.

METHODS

We utilize high-definition tDCS to investigate the effect of stimulation on motor imagery-based BCI performance across and within sessions over multiple training days.

RESULTS

We report a decreased time-to-hit with anodal stimulation both within and across sessions. We also found differing electrophysiological changes of the stimulated sensorimotor cortex during online BCI task performance for left vs. right trials. Cathodal stimulation led to a decrease in alpha and beta band power during task performance compared to sham stimulation for right hand imagination trials.

CONCLUSION

These results suggest that unilateral tDCS over the sensorimotor motor cortex differentially affects cortical areas based on task specific neural activation.

EEG classification for motor imagery and resting state in BCI applications using multi-class Adaboost extreme learning machine

Brain-computer interface (BCI) systems provide an alternative communication and control approach for people with limited motor function. Therefore, the feature extraction and classification approach should differentiate the relative unusual state of motion intention from a common resting state. In this paper, we sought a novel approach for multi-class classification in BCI applications. We collected electroencephalographic (EEG) signals registered by electrodes placed over the scalp during left hand motor imagery, right hand motor imagery, and resting state for ten healthy human subjects. We proposed using the Kolmogorov complexity (Kc) for feature extraction and a multi-class Adaboost classifier with extreme learning machine as base classifier for classification, in order to classify the three-class EEG samples. An average classification accuracy of 79.5% was obtained for ten subjects, which greatly outperformed commonly used approaches. Thus, it is concluded that the proposed method could improve the performance for classification of motor imagery tasks for multi-class samples. It could be applied in further studies to generate the control commands to initiate the movement of a robotic exoskeleton or orthosis, which finally facilitates the rehabilitation of disabled people.

Discriminating hand gesture motor imagery tasks using cortical current density estimation

Current EEG based brain computer interface (BCI) systems have achieved successful control in up to 3 dimensions; however the current paradigm may be unnatural for many rehabilitative and recreational applications. Therefore there is a great need to find motor imagination (MI) tasks that are realistic for output device control. In this paper we present our results on classifying hand gesture MI tasks, including right hand flexion, extension, supination and pronation using a novel EEG inverse imaging approach. By using both temporal and spatial specificity in the source domain we were able to separate MI tasks with up to 95% accuracy for binary classification of any two tasks compared to a maximum of only 79% in the sensor domain.

EEG Source Imaging Enhances the Decoding of Complex Right-Hand Motor Imagery Tasks

GOAL

Sensorimotor-based brain-computer interfaces (BCIs) have achieved successful control of real and virtual devices in up to three dimensions; however, the traditional sensor-based paradigm limits the intuitive use of these systems. Many control signals for state-of-the-art BCIs involve imagining the movement of body parts that have little to do with the output command, revealing a cognitive disconnection between the user's intent and the action of the end effector. Therefore, there is a need to develop techniques that can identify with high spatial resolution the self-modulated neural activity reflective of the actions of a helpful output device.

METHODS

We extend previous EEG source imaging (ESI) work to decoding natural hand/wrist manipulations by applying a novel technique to classifying four complex motor imaginations of the right hand: flexion, extension, supination, and pronation.

RESULTS

We report an increase of up to 18.6% for individual task classification and 12.7% for overall classification using the proposed ESI approach over the traditional sensor-based method.

CONCLUSION

ESI is able to enhance BCI performance of decoding complex right-hand motor imagery tasks.

SIGNIFICANCE

This study may lead to the development of BCI systems with naturalistic and intuitive motor imaginations, thus facilitating broad use of noninvasive BCIs.

Noninvasive Brain-Computer Interfaces Based on Sensorimotor Rhythms

Brain-computer interfaces (BCIs) have been explored in the field of neuroengineering to investigate how the brain can use these systems to control external devices. We review the principles and approaches we have taken to develop a sensorimotor rhythm EEG based brain-computer interface (BCI). The methods include developing BCI systems incorporating the control of physical devices to increase user engagement, improving BCI systems by inversely mapping scalp-recorded EEG signals to the cortical source domain, integrating BCI with noninvasive neuromodulation strategies to improve learning, and incorporating mind-body awareness training to enhance BCI learning and performance. The challenges and merits of these strategies are discussed, together with recent findings. Our work indicates that the sensorimotor-rhythm-based noninvasive BCI has the potential to provide communication and control capabilities as an alternative to physiological motor pathways.

Electroencephalography-based real-time cortical monitoring system that uses hierarchical Bayesian estimations for the brain-machine interface

In this study, a real-time cortical activity monitoring system was constructed, which could estimate cortical activities every 125 milliseconds over 2,240 vertexes from 64 channel electroencephalography signals through the Hierarchical Bayesian estimation that uses functional magnetic resonance imaging data as its prior information. Recently, functional magnetic resonance imaging has mostly been used in the neurofeedback field because it allows for high spatial resolution. However, in functional magnetic resonance imaging, the time for the neurofeedback information to reach the patient is delayed several seconds because of its poor temporal resolution. Therefore, a number of problems need to be solved to effectively implement feedback training paradigms in patients. To address this issue, this study used a new cortical activity monitoring system that improved both spatial and temporal resolution by using both functional magnetic resonance imaging data and electroencephalography signals in conjunction with one another. This system is advantageous as it can improve applications in the fields of real-time diagnosis, neurofeedback, and the brain-machine interface.

Brain-computer interfaces using sensorimotor rhythms: current state and future perspectives

Many studies over the past two decades have shown that people can use brain signals to convey their intent to a computer using brain-computer interfaces (BCIs). BCI systems extract specific features of brain activity and translate them into control signals that drive an output. Recently, a category of BCIs that are built on the rhythmic activity recorded over the sensorimotor cortex, i.e., the sensorimotor rhythm (SMR), has attracted considerable attention among the BCIs that use noninvasive neural recordings, e.g., electroencephalography (EEG), and have demonstrated the capability of multidimensional prosthesis control. This paper reviews the current state and future perspectives of SMR-based BCI and its clinical applications, in particular focusing on the EEG SMR. The characteristic features of SMR from the human brain are described and their underlying neural sources are discussed. The functional components of SMR-based BCI, together with its current clinical applications, are reviewed. Finally, limitations of SMR-BCIs and future outlooks are also discussed.

EEG-based motor imagery classification accuracy improves with gradually increased channel number

The question of how many channels should be sed for classification remains a key issue in the study of Brain-Computer Interface. Several studies have shown that a reduced number of channels can achieve the optimal classification accuracy in the offline analysis of motor imagery paradigm, which does not have real-time feedback as in the online control. However, for the cursor movement control paradigm, it remains unclear as to how many channels should be selected in order to achieve the optimal classification. In the present study, we gradually increased the number of channels, and adopted the time-frequency-spatial synthesized method for left and right motor imagery classification. We compared the effect of increasing channel number in two datasets, an imagery-based cursor movement control dataset and a motor imagery tasks dataset. Our results indicated that for the former dataset, the more channels we used, the higher the accuracy rate was achieved, which is in contrast to the finding in the latter dataset that optimal performance was obtained at a subset number of channels. When gradually increasing the number of channels from 2 to all in the analysis of cursor movement control dataset, the average training and testing accuracies from three subjects improved from 68.7% to 90.4% and 63.7% to 87.7%, respectively.

Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain-computer interface

OBJECTIVE

At the balanced intersection of human and machine adaptation is found the optimally functioning brain-computer interface (BCI). In this study, we report a novel experiment of BCI controlling a robotic quadcopter in three-dimensional (3D) physical space using noninvasive scalp electroencephalogram (EEG) in human subjects. We then quantify the performance of this system using metrics suitable for asynchronous BCI. Lastly, we examine the impact that the operation of a real world device has on subjects' control in comparison to a 2D virtual cursor task.

APPROACH

Five human subjects were trained to modulate their sensorimotor rhythms to control an AR Drone navigating a 3D physical space. Visual feedback was provided via a forward facing camera on the hull of the drone.

MAIN RESULTS

Individual subjects were able to accurately acquire up to 90.5% of all valid targets presented while travelling at an average straight-line speed of 0.69 m s(-1).

SIGNIFICANCE

Freely exploring and interacting with the world around us is a crucial element of autonomy that is lost in the context of neurodegenerative disease. Brain-computer interfaces are systems that aim to restore or enhance a user's ability to interact with the environment via a computer and through the use of only thought. We demonstrate for the first time the ability to control a flying robot in 3D physical space using noninvasive scalp recorded EEG in humans. Our work indicates the potential of noninvasive EEG-based BCI systems for accomplish complex control in 3D physical space. The present study may serve as a framework for the investigation of multidimensional noninvasive BCI control in a physical environment using telepresence robotics.

Performance assessment in brain-computer interface-based augmentative and alternative communication

A large number of incommensurable metrics are currently used to report the performance of brain-computer interfaces (BCI) used for augmentative and alterative communication (AAC). The lack of standard metrics precludes the comparison of different BCI-based AAC systems, hindering rapid growth and development of this technology. This paper presents a review of the metrics that have been used to report performance of BCIs used for AAC from January 2005 to January 2012. We distinguish between Level 1 metrics used to report performance at the output of the BCI Control Module, which translates brain signals into logical control output, and Level 2 metrics at the Selection Enhancement Module, which translates logical control to semantic control. We recommend that: (1) the commensurate metrics Mutual Information or Information Transfer Rate (ITR) be used to report Level 1 BCI performance, as these metrics represent information throughput, which is of interest in BCIs for AAC; 2) the BCI-Utility metric be used to report Level 2 BCI performance, as it is capable of handling all current methods of improving BCI performance; (3) these metrics should be supplemented by information specific to each unique BCI configuration; and (4) studies involving Selection Enhancement Modules should report performance at both Level 1 and Level 2 in the BCI system. Following these recommendations will enable efficient comparison between both BCI Control and Selection Enhancement Modules, accelerating research and development of BCI-based AAC systems.

Event-related desynchronization and synchronization quantification in motor-related EEG by Kolmogorov entropy

OBJECTIVE

Various approaches have been applied for the quantification of event-related desynchronization/synchronization (ERD/ERS) in EEG/MEG data analysis, but most of them are based on band power analysis. In this paper, we sought a novel method using a nonlinear measurement to quantify the ERD/ERS time course of motor-related EEG.

APPROACH

We applied Kolmogorov entropy to quantify the ERD/ERS time course of motor-related EEG in relation to hand movement imagination and execution for the first time. To further test the validity of the Kolmogorov entropy measure, we tested it on five human subjects for feature extraction to classify the left and right hand motor tasks.

MAIN RESULTS

The results show that the relative increase and decrease of Kolmogorov entropy indicates the ERD and ERS respectively. An average classification accuracy of 87.3% was obtained for five subjects.

SIGNIFICANCE

The results prove that Kolmogorov entropy can effectively quantify the dynamic process of event-related EEG, and it also provides a novel method of classifying motor imagery tasks from scalp EEG by Kolmogorov entropy measurement with promising classification accuracy.

Corticomuscular coherence analysis on hand movement distinction for active rehabilitation

Active rehabilitation involves patient's voluntary thoughts as the control signals of restore device to assist stroke rehabilitation. Although restoration of hand opening stands importantly in patient's daily life, it is difficult to distinguish the voluntary finger extension from thumb adduction and finger flexion using stroke patients' electroencephalography (EMG) on single muscle activity. We propose to implement corticomuscular coherence analysis on electroencephalography (EEG) and EMG signals on Extensor Digitorum to extract their intention involved in hand opening. EEG and EMG signals of 8 subjects are simultaneously collected when executing 4 hand movement tasks (finger extension, thumb adduction, finger flexion, and rest). We explore the spatial and temporal distribution of the coherence and observe statistically significant corticomuscular coherence appearing at left motor cortical area and different patterns within beta frequency range for 4 movement tasks. Linear discriminate analysis is applied on the coherence pattern to distinguish finger extension from thumb adduction, finger flexion, and rest. The classification results are greater than those by EEG only. The results indicate the possibility to detect voluntary hand opening based on coherence analysis between single muscle EMG signal and single EEG channel located in motor cortical area, which potentially helps active hand rehabilitation for stroke patients.

Evaluation of feature extraction methods for EEG-based brain-computer interfaces in terms of robustness to slight changes in electrode locations

To date, most EEG-based brain-computer interface (BCI) studies have focused only on enhancing BCI performance in such areas as classification accuracy and information transfer rate. In practice, however, test-retest reliability of the developed BCI systems must also be considered for use in long-term, daily life applications. One factor that can affect the reliability of BCI systems is the slight displacement of EEG electrode locations that often occurs due to the removal and reattachment of recording electrodes. The aim of this study was to evaluate and compare various feature extraction methods for motor-imagery-based BCI in terms of robustness to slight changes in electrode locations. To this end, EEG signals were recorded from three reference electrodes (Fz, C3, and C4) and from six additional electrodes located close to the reference electrodes with a 1-cm inter-electrode distance. Eight healthy participants underwent 180 trials of left- and right-hand motor imagery tasks. The performance of four different feature extraction methods [power spectral density (PSD), phase locking value (PLV), a combination of PSD and PLV, and cross-correlation (CC)] were evaluated using five-fold cross-validation and linear discriminant analysis, in terms of robustness to electrode location changes as well as regarding absolute classification accuracy. The quantitative evaluation results demonstrated that the use of either PSD- or CC-based features led to higher classification accuracy than the use of PLV-based features, while PSD-based features showed much higher sensitivity to changes in EEG electrode location than CC- or PLV-based features. Our results suggest that CC can be used as a promising feature extraction method in motor-imagery-based BCI studies, since it provides high classification accuracy along with being little affected by slight changes in the EEG electrode locations.

Continuous three-dimensional control of a virtual helicopter using a motor imagery based brain-computer interface

Brain-computer interfaces (BCIs) allow a user to interact with a computer system using thought. However, only recently have devices capable of providing sophisticated multi-dimensional control been achieved non-invasively. A major goal for non-invasive BCI systems has been to provide continuous, intuitive, and accurate control, while retaining a high level of user autonomy. By employing electroencephalography (EEG) to record and decode sensorimotor rhythms (SMRs) induced from motor imaginations, a consistent, user-specific control signal may be characterized. Utilizing a novel method of interactive and continuous control, we trained three normal subjects to modulate their SMRs to achieve three-dimensional movement of a virtual helicopter that is fast, accurate, and continuous. In this system, the virtual helicopter's forward-backward translation and elevation controls were actuated through the modulation of sensorimotor rhythms that were converted to forces applied to the virtual helicopter at every simulation time step, and the helicopter's angle of left or right rotation was linearly mapped, with higher resolution, from sensorimotor rhythms associated with other motor imaginations. These different resolutions of control allow for interplay between general intent actuation and fine control as is seen in the gross and fine movements of the arm and hand. Subjects controlled the helicopter with the goal of flying through rings (targets) randomly positioned and oriented in a three-dimensional space. The subjects flew through rings continuously, acquiring as many as 11 consecutive rings within a five-minute period. In total, the study group successfully acquired over 85% of presented targets. These results affirm the effective, three-dimensional control of our motor imagery based BCI system, and suggest its potential applications in biological navigation, neuroprosthetics, and other applications.

Neural representations involved in observed, imagined, and imitated actions are dissociable and hierarchically organized

The fact that action observation, motor imagery and execution are associated with partially overlapping increases in parieto-frontal areas has been interpreted as evidence for reliance of these behaviors on a common system of motor representations. However, studies that include all three conditions within a single paradigm are rare, and consequently, there is a dearth of knowledge concerning the distinct mechanisms involved in these functions. Here we report key differences in neural representations subserving observation, imagery, and synchronous imitation of a repetitive bimanual finger-tapping task using fMRI under conditions in which visual stimulation is carefully controlled. Relative to rest, observation, imagery, and synchronous imitation are all associated with widespread increases in cortical activity. Importantly, when effects of visual stimulation are properly controlled, each of these conditions is found to have its own unique neural signature. Relative to observation or imagery, synchronous imitation shows increased bilateral activity along the central sulcus (extending into precentral and postcentral gyri), in the cerebellum, supplementary motor area (SMA), parietal operculum, and several motor-related subcortical areas. No areas show greater increases for imagery vs. synchronous imitation; however, relative to synchronous imitation, observation is associated with greater increases in caudal SMA activity than synchronous imitation. Compared to observation, imagery increases activation in pre-SMA and left inferior frontal cortex, while no areas show the inverse effect. Region-of-interest (ROI) analyses reveal that areas involved in bimanual open-loop movements respond most to synchronous imitation (primary sensorimotor, classic SMA, and cerebellum), and less vigorously to imagery and observation. The differential activity between conditions suggests an alternative hierarchical model in which these behaviors all rely on partially independent mechanisms.

Electrophysiological imaging of brain activity and connectivity-challenges and opportunities

Unlocking the dynamic inner workings of the brain continues to remain a grand challenge of the 21st century. To this end, functional neuroimaging modalities represent an outstanding approach to better understand the mechanisms of both normal and abnormal brain functions. The ability to image brain function with ever increasing spatial and temporal resolution has made a significant leap over the past several decades. Further delineation of functional networks could lead to improved understanding of brain function in both normal and diseased states. This paper reviews recent advancements and current challenges in dynamic functional neuroimaging techniques, including electrophysiological source imaging, multimodal neuroimaging integrating fMRI with EEG/MEG, and functional connectivity imaging.

Relationship between speed and EEG activity during imagined and executed hand movements

The relationship between primary motor cortex and movement kinematics has been shown in nonhuman primate studies of hand reaching or drawing tasks. Studies have demonstrated that the neural activities accompanying or immediately preceding the movement encode the direction, speed and other information. Here we investigated the relationship between the kinematics of imagined and actual hand movement, i.e. the clenching speed, and the EEG activity in ten human subjects. Study participants were asked to perform and imagine clenching of the left hand and right hand at various speeds. The EEG activity in the alpha (8-12 Hz) and beta (18-28 Hz) frequency bands were found to be linearly correlated with the speed of imagery clenching. Similar parametric modulation was also found during the execution of hand movements. A single equation relating the EEG activity to the speed and the hand (left versus right) was developed. This equation, which contained a linear independent combination of the two parameters, described the time-varying neural activity during the tasks. Based on the model, a regression approach was developed to decode the two parameters from the multiple-channel EEG signals. We demonstrated the continuous decoding of dynamic hand and speed information of the imagined clenching. In particular, the time-varying clenching speed was reconstructed in a bell-shaped profile. Our findings suggest an application to providing continuous and complex control of noninvasive brain-computer interface for movement-impaired paralytics.

Negative covariation between task-related responses in alpha/beta-band activity and BOLD in human sensorimotor cortex: an EEG and fMRI study of motor imagery and movements

Similar to the occipital alpha rhythm, electroencephalographic (EEG) signals in the alpha- and beta-frequency bands can be suppressed by movement or motor imagery and have thus been thought to represent the "idling state" of the sensorimotor cortex. A negative correlation between spontaneous alpha EEG and blood-oxygen-level-dependent (BOLD) signals has been reported in combined EEG and fMRI (functional Magnetic Resonance Imaging) experiments when subjects stayed at the resting state or alternated between the resting state and a task. However, the precise nature of the task-induced alpha modulation remains elusive. It was not clear whether alpha/beta rhythm suppressions may co-vary with BOLD when conducting tasks involving varying activations of the cortex. Here, we quantified the task-evoked responses of BOLD and alpha/beta-band power of EEG directly in the cortical source domain, by using source imaging technology, and examined their covariation across task conditions in a mixed block and event-related design. In this study, 13 subjects performed tasks of right-hand, right-foot or left-hand movement and motor imagery when EEG and fMRI data were separately collected. Task-induced increase of BOLD signal and decrease of EEG amplitudes in alpha and beta bands were shown to be co-localized at the somatotopic sensorimotor cortex. At the corresponding regions, the reciprocal changes of the two signals co-varied in the magnitudes across imagination and movement conditions. The spatial correspondence and negative covariation between the two measurements were further shown to exist at somatotopic brain regions associated with different body parts. These results suggest an inverse functional coupling relationship between task-induced changes of BOLD and low-frequency EEG signals.

Cortical imaging of event-related (de)synchronization during online control of brain-computer interface using minimum-norm estimates in frequency domain

It is of wide interest to study the brain activity that correlates to the control of brain-computer interface (BCI). In the present study, we have developed an approach to image the cortical rhythmic modulation associated with motor imagery using minimum-norm estimates in the frequency domain (MNEFD). The distribution of cortical sources of mu activity during online control of BCI was obtained with the MNEFD. Contralateral decrease (event-related desynchronization) and ipsilateral increase (event-related synchronization) are localized in the sensorimotor cortex during online control of BCI in a group of human subjects. Statistical source analysis revealed that maximum correlation with movement imagination is localized in sensorimotor cortex.

Cortical imaging of sensorimotor rhythms for BCI applications

Rhythmic electroencephalographic (EEG) activities associated with movement imaginations are widely used in developing noninvasive Brain-Computer Interface (BCI) towards replacing or restoring the lost motor function in the paralytics. And it is of great importance to develop imaging techniques to enhance the spatial resolution and specificity of the EEG modality. In our work, we developed an innovative approach of imaging the distributed rhythmic brain activity in the spectral domain. In the present study, we evaluated the proposed technique in experimental data of offline and online imaginations in naive and well-trained BCI subjects. Our results identified the cortical origins of sensorimotor rhythms. We also applied the source imaging approach to classifying mental states for BCI applications and demonstrated its feasibility and superior performance.

Exploration of computational methods for classification of movement intention during human voluntary movement from single trial EEG

OBJECTIVE

To explore effective combinations of computational methods for the prediction of movement intention preceding the production of self-paced right and left hand movements from single trial scalp electroencephalogram (EEG).

METHODS

Twelve naïve subjects performed self-paced movements consisting of three key strokes with either hand. EEG was recorded from 128 channels. The exploration was performed offline on single trial EEG data. We proposed that a successful computational procedure for classification would consist of spatial filtering, temporal filtering, feature selection, and pattern classification. A systematic investigation was performed with combinations of spatial filtering using principal component analysis (PCA), independent component analysis (ICA), common spatial patterns analysis (CSP), and surface Laplacian derivation (SLD); temporal filtering using power spectral density estimation (PSD) and discrete wavelet transform (DWT); pattern classification using linear Mahalanobis distance classifier (LMD), quadratic Mahalanobis distance classifier (QMD), Bayesian classifier (BSC), multi-layer perceptron neural network (MLP), probabilistic neural network (PNN), and support vector machine (SVM). A robust multivariate feature selection strategy using a genetic algorithm was employed.

RESULTS

The combinations of spatial filtering using ICA and SLD, temporal filtering using PSD and DWT, and classification methods using LMD, QMD, BSC and SVM provided higher performance than those of other combinations. Utilizing one of the better combinations of ICA, PSD and SVM, the discrimination accuracy was as high as 75%. Further feature analysis showed that beta band EEG activity of the channels over right sensorimotor cortex was most appropriate for discrimination of right and left hand movement intention.

CONCLUSIONS

Effective combinations of computational methods provide possible classification of human movement intention from single trial EEG. Such a method could be the basis for a potential brain-computer interface based on human natural movement, which might reduce the requirement of long-term training.

SIGNIFICANCE

Effective combinations of computational methods can classify human movement intention from single trial EEG with reasonable accuracy.

Neuroelectrical source imaging of mu rhythm control for BCI applications

In the last decade, the possibility to noninvasively estimate cortical activity has been highlighted by the application of the techniques known as high resolution EEG. These techniques include a subject's multi-compartment head model (scalp, skull, dura mater, cortex) constructed from individual magnetic resonance images, multi-dipole source model, and regularized linear inverse source estimates of cortical current density. The aim of this paper is to demonstrate that the use of cortical activity estimated from noninvasive EEG recordings of motor imagery is useful in the context of a brain computer interface as compared with others scalp spatial filters usually used on-line.

An EEG-based real-time cortical rhythmic activity monitoring system

In the present study, we introduce an electroencephalography (EEG)-based, real-time, cortical rhythmic activity monitoring system which can monitor spatiotemporal changes of cortical rhythmic activity on a subject's cortical surface, not on the subject's scalp surface, with a high temporal resolution. In the monitoring system, a frequency domain inverse operator is preliminarily constructed, considering the subject's anatomical information and sensor configurations, and then the spectral current power at each cortical vertex is calculated for the Fourier transforms of successive sections of continuous data, when a particular frequency band is given. A preliminary offline simulation study using four sets of artifact-free, eye-closed, resting EEG data acquired from two dementia patients and two normal subjects demonstrates that spatiotemporal changes of cortical rhythmic activity can be monitored at the cortical level with a maximal delay time of about 200 ms, when 18 channel EEG data are analyzed under a Pentium4 3.4 GHz environment. The first pilot system is applied to two human experiments-(1) cortical alpha rhythm changes induced by opening and closing eyes and (2) cortical mu rhythm changes originated from the arm movements-and demonstrated the feasibility of the developed system.

A review of classification algorithms for EEG-based brain–computer interfaces

In this paper we review classification algorithms used to design brain-computer interface (BCI) systems based on electroencephalography (EEG). We briefly present the commonly employed algorithms and describe their critical properties. Based on the literature, we compare them in terms of performance and provide guidelines to choose the suitable classification algorithm(s) for a specific BCI.

Classification of motor imagery by means of cortical current density estimation and Von Neumann entropy

The goal of the present study is to employ the source imaging methods such as cortical current density estimation for the classification of left- and right-hand motor imagery tasks, which may be used for brain-computer interface (BCI) applications. The scalp recorded EEG was first preprocessed by surface Laplacian filtering, time-frequency filtering, noise normalization and independent component analysis. Then the cortical imaging technique was used to solve the EEG inverse problem. Cortical current density distributions of left and right trials were classified from each other by exploiting the concept of Von Neumann entropy. The proposed method was tested on three human subjects (180 trials each) and a maximum accuracy of 91.5% and an average accuracy of 88% were obtained. The present results confirm the hypothesis that source analysis methods may improve accuracy for classification of motor imagery tasks. The present promising results using source analysis for classification of motor imagery enhances our ability of performing source analysis from single trial EEG data recorded on the scalp, and may have applications to improved BCI systems.

An enhanced time-frequency-spatial approach for motor imagery classification

Human motor imagery (MI) tasks evoke electroencephalogram (EEG) signal changes. The features of these changes appear as subject-specific temporal traces of EEG rhythmic components at specific channels located over the scalp. Accurate classification of MI tasks based upon EEG may lead to a noninvasive brain-computer interface (BCI) to decode and convey intention of human subjects. We have previously proposed two novel methods on time-frequency feature extraction, expression and classification for high-density EEG recordings (Wang and He 2004; Wang, Deng, and He, 2004). In the present study, we refined the above time-frequency-spatial approach and applied it to a one-dimensional "cursor control" BCI experiment with online feedback. Through offline analysis of the collected data, we evaluated the capability of the present refined method in comparison with the original time-frequency-spatial methods. The enhanced performance in terms of classification accuracy was found for the proposed approach, with a mean accuracy rate of 91.1% for two subjects studied.

Limb (hand vs. foot) and response conflict have similar effects on event-related potentials (ERPs) recorded during motor imagery and overt execution

Although there is substantial evidence that motor execution (M-Ex) and motor imagery (M-Im) share a common neural substrate, the role of the primary motor cortex (M1) during imagery is still a matter of debate. The present ERP study tries to clarify the functional similarity between the two processes in respect of (i) the engagement of the corresponding somatotopic M1 areas during execution and imagery of hand vs. foot movements; and (ii) the effect of conflicting information on response preparation. To this end, we recorded ERPs from 28 electrode sites in 19 participants while they performed a conflict task with congruent (target and flanker arrowheads pointing in the same direction) and incongruent (target pointing in the opposite direction to the flanker arrowheads) trials. We obtained the lateralized readiness potential (LRP), a component generated in M1, while subjects physically executed or mentally simulated the task. As expected by the somatotopic organization of M1, the LRP was of opposite polarity when foot, rather than hand, movements were prepared. The inversion of polarity also occurred during M-Im, a result that strongly argues in favour of the participation of M1 in motor imagery. In incongruent trials, longer LRP latencies, a premature preparation of the incorrect response (positive deflection in LRP waveform) and a fronto-central N2 component associated with response conflict appeared during both M-Ex and M-Im. Altogether, the results support the functional equivalence of the two processes and give support to the clinical use of M-Im for the improvement and recovery of motor functions.

BCI meeting 2005-workshop on BCI signal processing: feature extraction and translation

This paper describes the outcome of discussions held during the Third International BCI Meeting at a workshop charged with reviewing and evaluating the current state of and issues relevant to brain-computer interface (BCI) feature extraction and translation. The issues discussed include a taxonomy of methods and applications, time-frequency spatial analysis, optimization schemes, the role of insight in analysis, adaptation, and methods for quantifying BCI feedback.