Phase synchronization for the recognition of mental tasks in a brain-computer interface

Detecting cognitive traits and occupational proficiency using EEG and statistical inference

Machine learning (ML) is widely used in classification tasks aimed at detecting various cognitive states or neurological diseases using noninvasive electroencephalogram (EEG) time series. However, successfully detecting specific cognitive skills in a healthy population, independent of subject, remains challenging. This study compared the subject-independent classification performance of three different pipelines: supervised and Riemann projections with logistic regression and handcrafted power spectral features with light gradient boosting machine (LightGBM). 128-channel EEGs were recorded from 26 healthy volunteers while they solved arithmetic, logical, and verbal tasks. The participants were divided into two groups based on their higher education and occupation: specialists in mathematics and humanities. The balanced accuracy of the education type was significantly above chance for all pipelines: 0.84-0.89, 0.85-0.88, and 0.86-0.88 for each type of task, respectively. All three pipelines allowed us to distinguish mathematical proficiency based on learning experience with different trade-offs between performance and explainability. Our results suggest that ML approaches could also be effective for recognizing individual cognitive traits using EEG.

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study

While the term task load (TL) refers to external task demands, the amount of work, or the number of tasks to be performed, mental workload (MWL) refers to the individual's effort, mental capacity, or cognitive resources utilized while performing a task. MWL in multitasking scenarios is often closely linked with the quantity of tasks a person is handling within a given timeframe. In this study, we challenge this hypothesis from the perspective of electroencephalography (EEG) using a deep learning approach. We conducted an EEG experiment with 50 participants performing NASA Multi-Attribute Task Battery II (MATB-II) under 4 different task load levels. We designed a convolutional neural network (CNN) to help with two distinct classification tasks. In one setting, the CNN was used to classify EEG segments based on their task load level. In another setting, the same CNN architecture was trained again to detect the presence of individual MATB-II subtasks. Results show that, while the model successfully learns to detect whether a particular subtask is active in a given segment (i.e., to differentiate between different subtasks-related EEG patterns), it struggles to differentiate between the two highest levels of task load (i.e., to distinguish MWL-related EEG patterns). We speculate that the challenge comes from two factors: first, the experiment was designed in a way that these two highest levels differed only in the quantity of work within a given timeframe; and second, the participants' effective adaptation to increased task demands, as evidenced by low error rates. Consequently, this indicates that under such conditions in multitasking, EEG may not reflect distinct enough patterns to differentiate higher levels of task load.

A study on the combination of functional connection features and Riemannian manifold in EEG emotion recognition

Introduction

Affective computing is the core for Human-computer interface (HCI) to be more intelligent, where electroencephalogram (EEG) based emotion recognition is one of the primary research orientations. Besides, in the field of brain-computer interface, Riemannian manifold is a highly robust and effective method. However, the symmetric positive definiteness (SPD) of the features limits its application.

Methods

In the present work, we introduced the Laplace matrix to transform the functional connection features, i.e., phase locking value (PLV), Pearson correlation coefficient (PCC), spectral coherent (COH), and mutual information (MI), to into semi-positive, and the max operator to ensure the transformed feature be positive. Then the SPD network is employed to extract the deep spatial information and a fully connected layer is employed to validate the effectiveness of the extracted features. Particularly, the decision layer fusion strategy is utilized to achieve more accurate and stable recognition results, and the differences of classification performance of different feature combinations are studied. What's more, the optimal threshold value applied to the functional connection feature is also studied.

Results

The public emotional dataset, SEED, is adopted to test the proposed method with subject dependent cross-validation strategy. The result of average accuracies for the four features indicate that PCC outperform others three features. The proposed model achieve best accuracy of 91.05% for the fusion of PLV, PCC, and COH, followed by the fusion of all four features with the accuracy of 90.16%.

Discussion

The experimental results demonstrate that the optimal thresholds for the four functional connection features always kept relatively stable within a fixed interval. In conclusion, the experimental results demonstrated the effectiveness of the proposed method.

A hybrid unsupervised and supervised learning approach for postictal generalized EEG suppression detection

Sudden unexpected death of epilepsy (SUDEP) is a catastrophic and fatal complication of epilepsy and is the primary cause of mortality in those who have uncontrolled seizures. While several multifactorial processes have been implicated including cardiac, respiratory, autonomic dysfunction leading to arrhythmia, hypoxia, and cessation of cerebral and brainstem function, the mechanisms underlying SUDEP are not completely understood. Postictal generalized electroencephalogram (EEG) suppression (PGES) is a potential risk marker for SUDEP, as studies have shown that prolonged PGES was significantly associated with a higher risk of SUDEP. Automated PGES detection techniques have been developed to efficiently obtain PGES durations for SUDEP risk assessment. However, real-world data recorded in epilepsy monitoring units (EMUs) may contain high-amplitude signals due to physiological artifacts, such as breathing, muscle, and movement artifacts, making it difficult to determine the end of PGES. In this paper, we present a hybrid approach that combines the benefits of unsupervised and supervised learning for PGES detection using multi-channel EEG recordings. A K-means clustering model is leveraged to group EEG recordings with similar artifact features. We introduce a new learning strategy for training a set of random forest (RF) models based on clustering results to improve PGES detection performance. Our approach achieved a 5-second tolerance-based detection accuracy of 64.92%, a 10-second tolerance-based detection accuracy of 79.85%, and an average predicted time distance of 8.26 seconds with 286 EEG recordings using leave-one-out (LOO) cross-validation. The results demonstrated that our hybrid approach provided better performance compared to other existing approaches.

An Intelligent Motor Imagery Detection System Using Electroencephalography with Adaptive Wavelets

Classification of motor imagery (MI) tasks provides a robust solution for specially-abled people to connect with the milieu for brain-computer interface. Precise selection of uniform tuning parameters of tunable Q wavelet transform (TQWT) for electroencephalography (EEG) signals is arduous. Therefore, this paper proposes robust TQWT for automatically selecting optimum tuning parameters to decompose non-stationary EEG signals accurately. Three evolutionary optimization algorithms are explored for automating the tuning parameters of robust TQWT. The fitness function of the mean square error of decomposition is used. This paper also exploits channel selection using a Laplacian score for dominant channel selection. Important features elicited from sub-bands of robust TQWT are classified using different kernels of the least square support vector machine classifier. The radial basis function kernel has provided the highest accuracy of 99.78%, proving that the proposed method is superior to other state-of-the-art using the same database.

Using linear parameter varying autoregressive models to measure cross frequency couplings in EEG signals

For years now, phase-amplitude cross frequency coupling (CFC) has been observed across multiple brain regions under different physiological and pathological conditions. It has been suggested that CFC serves as a mechanism that facilitates communication and information transfer between local and spatially separated neuronal populations. In non-invasive brain computer interfaces (BCI), CFC has not been thoroughly explored. In this work, we propose a CFC estimation method based on Linear Parameter Varying Autoregressive (LPV-AR) models and we assess its performance using both synthetic data and electroencephalographic (EEG) data recorded during attempted arm/hand movements of spinal cord injured (SCI) participants. Our results corroborate the potentiality of CFC as a feature for movement attempt decoding and provide evidence of the superiority of our proposed CFC estimation approach compared to other commonly used techniques.

Functional connectivity ensemble method to enhance BCI performance (FUCONE)

OBJECTIVE

Relying on the idea that functional connectivity provides important insights on the underlying dynamic of neuronal interactions, we propose a novel framework that combines functional connectivity estimators and covariance-based pipelines to improve the classification of mental states, such as motor imagery.

METHODS

A Riemannian classifier is trained for each estimator and an ensemble classifier combines the decisions in each feature space. A thorough assessment of the functional connectivity estimators is provided and the best performing pipeline among those tested, called FUCONE, is evaluated on different conditions and datasets.

RESULTS

Using a meta-analysis to aggregate results across datasets, FUCONE performed significantly better than all state-of-the-art methods.

CONCLUSION

The performance gain is mostly imputable to the improved diversity of the feature spaces, increasing the robustness of the ensemble classifier with respect to the inter- and intra-subject variability.

SIGNIFICANCE

Our results offer new insights into the need to consider functional connectivity-based methods to improve the BCI performance.

Phase/Amplitude Synchronization of Brain Signals During Motor Imagery BCI Tasks

In the last decade, functional connectivity (FC) has been increasingly adopted based on its ability to capture statistical dependencies between multivariate brain signals. However, the role of FC in the context of brain-computer interface applications is still poorly understood. To address this gap in knowledge, we considered a group of 20 healthy subjects during an EEG-based hand motor imagery (MI) task. We studied two well-established FC estimators, i.e. spectral- and imaginary-coherence, and we investigated how they were modulated by the MI task. We characterized the resulting FC networks by extracting the strength of connectivity of each EEG sensor and we compared the discriminant power with respect to standard power spectrum features. At the group level, results showed that while spectral-coherence based network features were increasing in the sensorimotor areas, those based on imaginary-coherence were significantly decreasing. We demonstrated that this opposite, but complementary, behavior was respectively determined by the increase in amplitude and phase synchronization between the brain signals. At the individual level, we eventually assessed the potential of these network connectivity features in a simple off-line classification scenario. Taken together, our results provide fresh insights into the oscillatory mechanisms subserving brain network changes during MI and offer new perspectives to improve BCI performance.

Interacting brains coming in sync through their minds: an interbrain neurofeedback study

Neurophysiological evidence shows that interpersonal action coordination is accompanied by interbrain synchronization (IBS). However, the functional significance of this association remains unclear. Using two experimental designs, we explored whether IBS is amenable to neurofeedback (NFB). Feedback was provided either as two balls approaching each other (so-called ball design), or as two pendula, each reflecting the oscillatory activity of one of the two participants (so-called pendulum design). The NFB was provided at delta (i.e., 2.5 Hz) and theta (i.e., 5 Hz) electroencephalography frequencies, and manipulated by enhanced and inverse feedback. We showed that the participants were able to increase IBS by using NFB, especially when it was fed back at the theta frequency. Apart from intra- and interbrain coupling, other oscillatory activities (e.g., power spectral density, peak amplitude, and peak frequency) also changed during the task compared with the rest. Moreover, all the measures showed specific correlations with the subjective postsurvey item scores, reflecting subjective feeling and appraisal. We conclude that the use of IBS for NFB might help in specifying the contribution of IBS to interpersonal action coordination and in providing important information about the neural mechanisms of social interaction and the causal dimension of IBS.

Toward New Modalities in VEP-Based BCI Applications Using Dynamical Stimuli: Introducing Quasi-Periodic and Chaotic VEP-Based BCI

Visual evoked potentials (VEPs) to periodic stimuli are commonly used in brain computer interfaces for their favorable properties such as high target identification accuracy, less training time, and low surrounding target interference. Conventional periodic stimuli can lead to subjective visual fatigue due to continuous and high contrast stimulation. In this study, we compared quasi-periodic and chaotic complex stimuli to common periodic stimuli for use with VEP-based brain computer interfaces (BCIs). Canonical correlation analysis (CCA) and coherence methods were used to evaluate the performance of the three stimulus groups. Subjective fatigue caused by the presented stimuli was evaluated by the Visual Analogue Scale (VAS). Using CCA with the M2 template approach, target identification accuracy was highest for the chaotic stimuli (M = 86.8, SE = 1.8) compared to the quasi-periodic (M = 78.1, SE = 2.6, p = 0.008) and periodic (M = 64.3, SE = 1.9, p = 0.0001) stimulus groups. The evaluation of fatigue rates revealed that the chaotic stimuli caused less fatigue compared to the quasi-periodic (p = 0.001) and periodic (p = 0.0001) stimulus groups. In addition, the quasi-periodic stimuli led to lower fatigue rates compared to the periodic stimuli (p = 0.011). We conclude that the target identification results were better for the chaotic group compared to the other two stimulus groups with CCA. In addition, the chaotic stimuli led to a less subjective visual fatigue compared to the periodic and quasi-periodic stimuli and can be suitable for designing new comfortable VEP-based BCIs.

Dynamic Changes of Brain Networks during Working Memory Tasks in Schizophrenia

Electroencephalograph (EEG) signals and graph theory measures have been widely used to characterize the brain functional networks of healthy individuals and patients by calculating the correlations between different electrodes over an entire time series. Although EEG signals have a high temporal resolution and can provide relatively stable results, the process of constructing and analyzing brain functional networks is inevitably complicated by high time complexity. Our goal in this research was to distinguish the brain function networks of schizophrenia patients from those of healthy participants during working memory tasks. Consequently, we utilized a method involving microstates, which are each characterized by a unique topography of electric potentials over an entire channel array, to reduce the dimension of the EEG signals during working memory tasks and then compared and analyzed the brain functional networks using the microstates time series (MTS) and original time series (OTS) of the schizophrenia patients and healthy individuals. We found that the right frontal and parietal-occipital regions neurons of the schizophrenia patients were less active than those of the healthy participants during working memory tasks. Notably, compared with OTS, the time needed to construct the brain functional networks was significantly reduced by using MTS. In conclusion, our results show that, like OTS, MTS can well distinguish the brain functional network of schizophrenia patients from those of healthy individuals during working memory tasks while greatly decreasing time complexity. MTS can thus provide a method for characterizing the original time series for the construction and analysis of EEG brain functional networks.

Detection of Postictal Generalized Electroencephalogram Suppression: Random Forest Approach

BACKGROUND

Sudden unexpected death in epilepsy (SUDEP) is second only to stroke in neurological events resulting in years of potential life lost. Postictal generalized electroencephalogram (EEG) suppression (PGES) is a period of suppressed brain activity often occurring after generalized tonic-clonic seizure, a most significant risk factor for SUDEP. Therefore, PGES has been considered as a potential biomarker for SUDEP risk. Automatic PGES detection tools can address the limitations of labor-intensive, and sometimes inconsistent, visual analysis. A successful approach to automatic PGES detection must overcome computational challenges involved in the detection of subtle amplitude changes in EEG recordings, which may contain physiological and acquisition artifacts.

OBJECTIVE

This study aimed to present a random forest approach for automatic PGES detection using multichannel human EEG recordings acquired in epilepsy monitoring units.

METHODS

We used a combination of temporal, frequency, wavelet, and interchannel correlation features derived from EEG signals to train a random forest classifier. We also constructed and applied confidence-based correction rules based on PGES state changes. Motivated by practical utility, we introduced a new, time distance-based evaluation method for assessing the performance of PGES detection algorithms.

RESULTS

The time distance-based evaluation showed that our approach achieved a 5-second tolerance-based positive prediction rate of 0.95 for artifact-free signals. For signals with different artifact levels, our prediction rates varied from 0.68 to 0.81.

CONCLUSIONS

We introduced a feature-based, random forest approach for automatic PGES detection using multichannel EEG recordings. Our approach achieved increasingly better time distance-based performance with reduced signal artifact levels. Further study is needed for PGES detection algorithms to perform well irrespective of the levels of signal artifacts.

Phase-synchrony evaluation of EEG signals for Multiple Sclerosis diagnosis based on bivariate empirical mode decomposition during a visual task

BACKGROUND AND OBJECTIVE

Despite the widespread prevalence of Multiple Sclerosis (MS), the study of brain interactions is still poorly understood. Moreover, there has always been a great need to automate the MS diagnosis procedure to eliminate the evaluation errors thereby improving its consistency and reliability. To address these issues, in this work, we proposed a robust pattern recognition algorithm as a computer-aided diagnosis system. This method is based on calculating the pairwise phase-synchrony of EEG recordings during a visual task. Initially, the bivariate empirical mode decomposition (BEMD) was applied to extract the intrinsic mode functions (IMFs). The phases of these IMFs were then obtained using the Hilbert transform to be utilized in the mean phase coherence (MPC), a measure for phase-synchrony calculation. After the construction of the feature space using MPC values, the ReliefF algorithm was applied for dimension reduction. Finally, the best distinguishing features were input to a k-nearest neighbor (KNN) classifier. The results revealed a higher level of network synchronization in the posterior regions of the brain and desynchronization in the anterior regions among the MS group as compared with the normal subjects. In the validation phase, the leave-one-subject-out cross-validation (LOOCV) method was used to assess the validity of the proposed algorithm. We achieved an accuracy, sensitivity, and specificity of 93.09%, 91.07%, and 95.24% for red-green, 90.44%, 88.39%, and 92.62% for luminance, and 87.44%, 87.05%, and 87.86% for blue-yellow tasks, respectively. The experimental results demonstrated the reliability of the presented method to be generalized in the field of automated MS diagnosis systems.

Adjustment of Synchronization Stability of Dynamic Brain-Networks Based on Feature Fusion

When the brain is active, the neural activities of different regions are integrated on various spatial and temporal scales; this is termed the synchronization phenomenon in neurobiological theory. This synchronicity is also the main underlying mechanism for information integration and processing in the brain. Clinical medicine has found that some of the neurological diseases that are difficult to cure have deficiencies or abnormalities in the whole or local integration processes of the brain. By studying the synchronization capabilities of the brain-network, we can intensively describe and characterize both the state of the interactions between brain regions and their differences between people with a mental illness and a set of controls by measuring the rapid changes in brain activity in patients with psychiatric disorders and the strength and integrity of their entire brain network. This is significant for the study of mental illness. Because static brain network connection methods are unable to assess the dynamic interactions within the brain, we introduced the concepts of dynamics and variability in a constructed EEG brain functional network based on dynamic connections, and used it to analyze the variability in the time characteristics of the EEG functional network. We used the spectral features of the brain network to extract its synchronization features and used the synchronization features to describe the process of change and the differences in the brain network's synchronization ability between a group of patients and healthy controls during a working memory task. We propose a method based on the fusion of traditional features and spectral features to achieve an adjustment of the patient's brain network synchronization ability, so that its synchronization ability becomes consistent with that of healthy controls, theoretically achieving the purpose of the treatment of the diseases. Studying the stability of brain network synchronization can provide new insights into the pathogenic mechanism and cure of mental diseases and has a wide range of potential applications.

Introducing chaotic codes for the modulation of code modulated visual evoked potentials (c-VEP) in normal adults for visual fatigue reduction

Code modulated Visual Evoked Potentials (c-VEP) based BCI studies usually employ m-sequences as a modulating codes for their broadband spectrum and correlation property. However, subjective fatigue of the presented codes has been a problem. In this study, we introduce chaotic codes containing broadband spectrum and similar correlation property. We examined whether the introduced chaotic codes could be decoded from EEG signals and also compared the subjective fatigue level with m-sequence codes in normal subjects. We generated chaotic code from one-dimensional logistic map and used it with conventional 31-bit m-sequence code. In a c-VEP based study in normal subjects (n = 44, 21 females) we presented these codes visually and recorded EEG signals from the corresponding codes for their four lagged versions. Canonical correlation analysis (CCA) and spatiotemporal beamforming (STB) methods were used for target identification and comparison of responses. Additionally, we compared the subjective self-declared fatigue using VAS caused by presented m-sequence and chaotic codes. The introduced chaotic code was decoded from EEG responses with CCA and STB methods. The maximum total accuracy values of 93.6 ± 11.9% and 94 ± 14.4% were achieved with STB method for chaotic and m-sequence codes for all subjects respectively. The achieved accuracies in all subjects were not significantly different in m-sequence and chaotic codes. There was significant reduction in subjective fatigue caused by chaotic codes compared to the m-sequence codes. Both m-sequence and chaotic codes were similar in their accuracies as evaluated by CCA and STB methods. The chaotic codes significantly reduced subjective fatigue compared to the m-sequence codes.

Developing a Three- to Six-State EEG-Based Brain-Computer Interface for a Virtual Robotic Manipulator Control

OBJECTIVE

We develop an electroencephalography (EEG)-based noninvasive brain-computer interface (BCI) system having short training time (15 min) that can be applied for high-performance control of robotic prosthetic systems.

METHODS

A signal processing system for detecting user's mental intent from EEG data based on up to six-state BCI paradigm is developed and used.

RESULTS

We examine the performance of the developed system on experimental data collected from 12 healthy participants and analyzed offline. Out of 12 participants 3 achieve an accuracy of six-state communication in 80%-90% range, while 2 participants do not achieve a satisfactory accuracy. We further implement an online BCI system for control of a virtual 3 degree-of-freedom (dof) prosthetic manipulator and test it with our three best participants. Two participants are able to successfully complete 100% of the test tasks, demonstrating on average the accuracy rate of 80% and requiring 5-10 s to execute a manipulator move. One participant failed to demonstrate a satisfactory performance in online trials.

CONCLUSION

We show that our offline EEG BCI system can correctly identify different motor imageries in EEG data with high accuracy and our online BCI system can be used for control of a virtual 3 dof prosthetic manipulator.

SIGNIFICANCE

Our results prepare foundation for further development of higher performance EEG BCI-based robotic assistive systems and demonstrate that EEG-based BCI may be feasible for robotic control by paralyzed and immobilized individuals.

Data-Driven Transducer Design and Identification for Internally-Paced Motor Brain Computer Interfaces: A Review

Brain-Computer Interfaces (BCIs) are systems that establish a direct communication pathway between the users' brain activity and external effectors. They offer the potential to improve the quality of life of motor-impaired patients. Motor BCIs aim to permit severely motor-impaired users to regain limb mobility by controlling orthoses or prostheses. In particular, motor BCI systems benefit patients if the decoded actions reflect the users' intentions with an accuracy that enables them to efficiently interact with their environment. One of the main challenges of BCI systems is to adapt the BCI's signal translation blocks to the user to reach a high decoding accuracy. This paper will review the literature of data-driven and user-specific transducer design and identification approaches and it focuses on internally-paced motor BCIs. In particular, continuous kinematic biomimetic and mental-task decoders are reviewed. Furthermore, static and dynamic decoding approaches, linear and non-linear decoding, offline and real-time identification algorithms are considered. The current progress and challenges related to the design of clinical-compatible motor BCI transducers are additionally discussed.

A PCA aided cross-covariance scheme for discriminative feature extraction from EEG signals

BACKGROUND AND OBJECTIVES

Feature extraction of EEG signals plays a significant role in Brain-computer interface (BCI) as it can significantly affect the performance and the computational time of the system. The main aim of the current work is to introduce an innovative algorithm for acquiring reliable discriminating features from EEG signals to improve classification performances and to reduce the time complexity.

METHODS

This study develops a robust feature extraction method combining the principal component analysis (PCA) and the cross-covariance technique (CCOV) for the extraction of discriminatory information from the mental states based on EEG signals in BCI applications. We apply the correlation based variable selection method with the best first search on the extracted features to identify the best feature set for characterizing the distribution of mental state signals. To verify the robustness of the proposed feature extraction method, three machine learning techniques: multilayer perceptron neural networks (MLP), least square support vector machine (LS-SVM), and logistic regression (LR) are employed on the obtained features. The proposed methods are evaluated on two publicly available datasets. Furthermore, we evaluate the performance of the proposed methods by comparing it with some recently reported algorithms.

RESULTS

The experimental results show that all three classifiers achieve high performance (above 99% overall classification accuracy) for the proposed feature set. Among these classifiers, the MLP and LS-SVM methods yield the best performance for the obtained feature. The average sensitivity, specificity and classification accuracy for these two classifiers are same, which are 99.32%, 100%, and 99.66%, respectively for the BCI competition dataset IVa and 100%, 100%, and 100%, for the BCI competition dataset IVb. The results also indicate the proposed methods outperform the most recently reported methods by at least 0.25% average accuracy improvement in dataset IVa. The execution time results show that the proposed method has less time complexity after feature selection.

CONCLUSIONS

The proposed feature extraction method is very effective for getting representatives information from mental states EEG signals in BCI applications and reducing the computational complexity of classifiers by reducing the number of extracted features.

Electroencephalographic Motor Imagery Brain Connectivity Analysis for BCI: A Review

Recent research has reached a consensus on the feasibility of motor imagery brain-computer interface (MI-BCI) for different applications, especially in stroke rehabilitation. Most MI-BCI systems rely on temporal, spectral, and spatial features of single channels to distinguish different MI patterns. However, no successful communication has been established for a completely locked-in subject. To provide more useful and informative features, it has been recommended to take into account the relationships among electroencephalographic (EEG) sensor/source signals in the form of brain connectivity as an efficient tool of neuroscience. In this review, we briefly report the challenges and limitations of conventional MI-BCIs. Brain connectivity analysis, particularly functional and effective, has been described as one of the most promising approaches for improving MI-BCI performance. An extensive literature on EEG-based MI brain connectivity analysis of healthy subjects is reviewed. We subsequently discuss the brain connectomes during left and right hand, feet, and tongue MI movements. Moreover, key components involved in brain connectivity analysis that considerably affect the results are explained. Finally, possible technical shortcomings that may have influenced the results in previous research are addressed and suggestions are provided.

Functional Brain Connectivity as a New Feature for P300 Speller

The brain is a large-scale complex network often referred to as the "connectome". Cognitive functions and information processing are mainly based on the interactions between distant brain regions. However, most of the 'feature extraction' methods used in the context of Brain Computer Interface (BCI) ignored the possible functional relationships between different signals recorded from distinct brain areas. In this paper, the functional connectivity quantified by the phase locking value (PLV) was introduced to characterize the evoked responses (ERPs) obtained in the case of target and non-targets visual stimuli. We also tested the possibility of using the functional connectivity in the context of 'P300 speller'. The proposed approach was compared to the well-known methods proposed in the state of the art of "P300 Speller", mainly the peak picking, the area, time/frequency based features, the xDAWN spatial filtering and the stepwise linear discriminant analysis (SWLDA). The electroencephalographic (EEG) signals recorded from ten subjects were analyzed offline. The results indicated that phase synchrony offers relevant information for the classification in a P300 speller. High synchronization between the brain regions was clearly observed during target trials, although no significant synchronization was detected for a non-target trial. The results showed also that phase synchrony provides higher performance than some existing methods for letter classification in a P300 speller principally when large number of trials is available. Finally, we tested the possible combination of both approaches (classical features and phase synchrony). Our findings showed an overall improvement of the performance of the P300-speller when using Peak picking, the area and frequency based features. Similar performances were obtained compared to xDAWN and SWLDA when using large number of trials.

Multiscale AM-FM methods on EEG signals for motor task classification

In this manuscript, we present the use of customized, multiscale amplitude-modulation frequency-modulation (AMFM) methods on electroencephalography (EEG) brain signals during the subject development a motor task: right hand and left hand. This approach is compared to various non-linear patterns and methods that have been applied in order to characterize and understand the dynamic behavior of the EEG signals. The AM-FM methods have been optimized in terms of multiscale filters for the mu band (8-12 Hz). The instantaneous AM-FM values are processed using their probability density function and classified using multiple layer perceptron (MLP) and the partial least squares regression (PLS). The system is tested using the standard BCI dataset with results with a precision to 89% and an area under the ROC to 91%.

A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching

OBJECTIVE

In order to increase the number of states classified by a brain-computer interface (BCI), we utilized a motor imagery task where subjects imagined both force and speed of hand clenching.

APPROACH

The BCI utilized simultaneously recorded electroencephalographic (EEG) and functional near-infrared spectroscopy (fNIRS) signals. The time-phase-frequency feature was extracted from EEG, whereas the HbD [the difference of oxy-hemoglobin (HbO) and deoxy-hemoglobin (Hb)] feature was used to improve the classification accuracy of fNIRS. The EEG and fNIRS features were combined and optimized using the joint mutual information (JMI) feature selection criterion; then the extracted features were classified with the extreme learning machines (ELMs).

MAIN RESULTS

In this study, the averaged classification accuracy of EEG signals achieved by the time-phase-frequency feature improved by 7%, to 18%, more than the single-type feature, and improved by 15% more than common spatial pattern (CSP) feature. The HbD feature of fNIRS signals improved the accuracy by 1%, to 4%, more than Hb, HbO, or HbT (total hemoglobin). The EEG-fNIRS feature for decoding motor imagery of both force and speed of hand clenching achieved an accuracy of 89% ± 2%, and improved the accuracy by 1% to 5% more than the sole EEG or fNIRS feature.

SIGNIFICANCE

Our novel motor imagery paradigm improves BCI performance by increasing the number of extracted commands. Both the time-phase-frequency and the HbD feature improve the classification accuracy of EEG and fNIRS signals, respectively, and the hybrid EEG-fNIRS technique achieves a higher decoding accuracy for two-class motor imagery, which may provide the framework for future multi-modal online BCI systems.

Enhancing the performance of motor imagery EEG classification using phase features

An electroencephalogram recognition system considering phase features is proposed to enhance the performance of motor imagery classification in this study. It mainly consists of feature extraction, feature selection and classification. Surface Laplacian filter is used for background removal. Several potential features, including phase features, are then extracted to enhance the classification accuracy. Next, genetic algorithm is used to select sub-features from feature combination. Finally, selected features are classified by extreme learning machine. Compared with "without phase features" and linear discriminant analysis on motor imagery data from 2 data sets, the results denote that the proposed system achieves enhanced performance, which is suitable for the brain-computer interface applications.

Modulation effect of transcranial direct current stimulation on phase synchronization in motor imagery brain-computer interface

Transcranial direct current stimulation (tDCS) has been demonstrated that it can enhance the cortex excitability and modulate the event-related desynchronization (ERD) in motor imagery (MI). Phase synchronization is an important signature in the brain that reflects the neural interaction and integration, which has been adopted as an important EEG pattern for Brian-Computer Interface (BCI) control. In this study, we designed an experiment paradigm and investigated whether the tDCS can modulate the phase synchronization between the primary motor cortex (M1) and the supplementary motor area (SMA) in MI. Ten healthy subjects were selected and separated into two groups randomly. They performed the left and right hand MI task in two successive sessions. According to the different groups, anodal or sham stimulation were administrated to the right side of the M1. The phase locking value (PLV), which is a reliable measurement of phase synchronization in MI, was calculated. The pre and post-stimulation normalized PLV in the left hand MI task were compared. The result manifests that the normalized PLV of the entire subjects in anodal stimulation group increases after the stimulation, which shows a statistically significant difference (paired t-test p = 0.0371, n = 5). Our study reveals that the tDCS can impact the neural coupling between different brain regions and modulate phase synchronization in MI. Moreover, intervention of phase synchronization by tDCS might contribute to the rehabilitation of people with motor disorder and neurological disorders.

Enhanced performance by time-frequency-phase feature for EEG-based BCI systems

We introduce a new motor parameter imagery paradigm using clench speed and clench force motor imagery. The time-frequency-phase features are extracted from mu rhythm and beta rhythms, and the features are optimized using three process methods: no-scaled feature using “MIFS” feature selection criterion, scaled feature using “MIFS” feature selection criterion, and scaled feature using “mRMR” feature selection criterion. Support vector machines (SVMs) and extreme learning machines (ELMs) are compared for classification between clench speed and clench force motor imagery using the optimized feature. Our results show that no significant difference in the classification rate between SVMs and ELMs is found. The scaled feature combinations can get higher classification accuracy than the no-scaled feature combinations at significant level of 0.01, and the “mRMR” feature selection criterion can get higher classification rate than the “MIFS” feature selection criterion at significant level of 0.01. The time-frequency-phase feature can improve the classification rate by about 20% more than the time-frequency feature, and the best classification rate between clench speed motor imagery and clench force motor imagery is 92%. In conclusion, the motor parameter imagery paradigm has the potential to increase the direct control commands for BCI control and the time-frequency-phase feature has the ability to improve BCI classification accuracy.

Modified CC-LR algorithm with three diverse feature sets for motor imagery tasks classification in EEG based brain-computer interface

Motor imagery (MI) tasks classification provides an important basis for designing brain-computer interface (BCI) systems. If the MI tasks are reliably distinguished through identifying typical patterns in electroencephalography (EEG) data, a motor disabled people could communicate with a device by composing sequences of these mental states. In our earlier study, we developed a cross-correlation based logistic regression (CC-LR) algorithm for the classification of MI tasks for BCI applications, but its performance was not satisfactory. This study develops a modified version of the CC-LR algorithm exploring a suitable feature set that can improve the performance. The modified CC-LR algorithm uses the C3 electrode channel (in the international 10-20 system) as a reference channel for the cross-correlation (CC) technique and applies three diverse feature sets separately, as the input to the logistic regression (LR) classifier. The present algorithm investigates which feature set is the best to characterize the distribution of MI tasks based EEG data. This study also provides an insight into how to select a reference channel for the CC technique with EEG signals considering the anatomical structure of the human brain. The proposed algorithm is compared with eight of the most recently reported well-known methods including the BCI III Winner algorithm. The findings of this study indicate that the modified CC-LR algorithm has potential to improve the identification performance of MI tasks in BCI systems. The results demonstrate that the proposed technique provides a classification improvement over the existing methods tested.

The Unlock Project: a Python-based framework for practical brain-computer interface communication "app" development

In this paper we present a framework for reducing the development time needed for creating applications for use in non-invasive brain-computer interfaces (BCI). Our framework is primarily focused on facilitating rapid software "app" development akin to current efforts in consumer portable computing (e.g. smart phones and tablets). This is accomplished by handling intermodule communication without direct user or developer implementation, instead relying on a core subsystem for communication of standard, internal data formats. We also provide a library of hardware interfaces for common mobile EEG platforms for immediate use in BCI applications. A use-case example is described in which a user with amyotrophic lateral sclerosis participated in an electroencephalography-based BCI protocol developed using the proposed framework. We show that our software environment is capable of running in real-time with updates occurring 50-60 times per second with limited computational overhead (5 ms system lag) while providing accurate data acquisition and signal analysis.

Combination of amplitude and phase features under a uniform framework with EMD in EEG-based Brain-Computer Interface

In a Brain-Computer Interface (BCI) system, the variations of the amplitude and the phase in EEG signal convey subjects' movement intention and underpin the differentiation of the various mental tasks. Combining these two kinds of information under a uniform feature extraction framework can better reflect the brain states and potentially contribute to BCI classification. Here the Common Spatial Pattern (CSP) and the Phase Locking Value (PLV) were used to capture the amplitude and the phase information. To integrate these two feature extraction procedures, the Empirical Mode Decomposition (EMD) is introduced in preprocessing which behaved as filter bank to optimize bands selection automatically for CSP and exactly calculate the instantaneous phase for PLV. The most discriminative features were selected from the feature pool by the sequential floating forward feature selection method (SFFS). The proposed method was applied to both public and recorded datasets (each n=4). Compared with the traditional CSP, the average increment of classification accuracy is 5.4% (2.0% for public and 8.7% for recorded datasets), which both manifests statistically significances (p<0.05). Moreover, we preliminarily investigate the possibility of the online realization of this method and it shows a comparable result with the offline result.

Evaluation of feature extraction methods for EEG-based brain-computer interfaces in terms of robustness to slight changes in electrode locations

To date, most EEG-based brain-computer interface (BCI) studies have focused only on enhancing BCI performance in such areas as classification accuracy and information transfer rate. In practice, however, test-retest reliability of the developed BCI systems must also be considered for use in long-term, daily life applications. One factor that can affect the reliability of BCI systems is the slight displacement of EEG electrode locations that often occurs due to the removal and reattachment of recording electrodes. The aim of this study was to evaluate and compare various feature extraction methods for motor-imagery-based BCI in terms of robustness to slight changes in electrode locations. To this end, EEG signals were recorded from three reference electrodes (Fz, C3, and C4) and from six additional electrodes located close to the reference electrodes with a 1-cm inter-electrode distance. Eight healthy participants underwent 180 trials of left- and right-hand motor imagery tasks. The performance of four different feature extraction methods [power spectral density (PSD), phase locking value (PLV), a combination of PSD and PLV, and cross-correlation (CC)] were evaluated using five-fold cross-validation and linear discriminant analysis, in terms of robustness to electrode location changes as well as regarding absolute classification accuracy. The quantitative evaluation results demonstrated that the use of either PSD- or CC-based features led to higher classification accuracy than the use of PLV-based features, while PSD-based features showed much higher sensitivity to changes in EEG electrode location than CC- or PLV-based features. Our results suggest that CC can be used as a promising feature extraction method in motor-imagery-based BCI studies, since it provides high classification accuracy along with being little affected by slight changes in the EEG electrode locations.

Correlation of fronto-central phase coupling with sensorimotor rhythm modulation

We investigated neurophysiologic correlates of individual differences in the modulation of sensorimotor rhythms (SMRs) in the human electroencephalography (EEG) during motor imagery. The ability of modulating SMRs to different motor imageries was correlated with the strength of alpha phase synchronization across frontal and central sensorimotor areas. The results suggest that fronto-central coupling may elucidate individual variations in SMR modulation that is essential for using SMR-based brain-computer interfaces.

The analytic common spatial patterns method for EEG-based BCI data

One of the most important stages in a brain-computer interface (BCI) system is that of extracting features that can reliably discriminate data recorded during different user states. A popular technique used for feature extraction in BCIs is the common spatial patterns (CSP) method, which provides a set of spatial filters that optimally discriminate between two classes of data in the least-squares sense. The method also yields a set of spatial patterns that are associated with the most relevant activity for distinguishing between the two classes. The high recognition rates that have been achieved with the method have led to its widespread adoption in the field. Here, a variant of the CSP method that considers EEG data in its complex form is described. By explicitly considering the amplitude and phase information in the data, the analytic CSP (ACSP) technique can provide a more comprehensive picture of the underlying activity, resulting in improved classification accuracies and more informative spatial patterns than the conventional CSP method. In this paper, we elaborate on the theoretical aspects of the ACSP algorithm and demonstrate the advantages of the method through a number of simulations and through tests on EEG data.

Connectivity measures applied to human brain electrophysiological data

Connectivity measures are (typically bivariate) statistical measures that may be used to estimate interactions between brain regions from electrophysiological data. We review both formal and informal descriptions of a range of such measures, suitable for the analysis of human brain electrophysiological data, principally electro- and magnetoencephalography. Methods are described in the space-time, space-frequency, and space-time-frequency domains. Signal processing and information theoretic measures are considered, and linear and nonlinear methods are distinguished. A novel set of cross-time-frequency measures is introduced, including a cross-time-frequency phase synchronization measure.

Decoding oscillatory representations and mechanisms in memory

A fundamental goal in memory research is to understand how information is represented in distributed brain networks and what mechanisms enable its reactivation. It is evident that progress towards this goal will greatly benefit from multivariate pattern classification (MVPC) techniques that can decode representations in brain activity with high temporal resolution. Recently, progress along these lines has been achieved by applying MVPC to neural oscillations recorded with electroencephalography (EEG) and magnetoencephalography (MEG). We highlight two examples of methodological approaches for MVPC of EEG and MEG data that can be used to study memory function. The first example aims at understanding the dynamic neural mechanisms that enable reactivation of memory representations, i.e., memory replay; we discuss how MVPC can help uncover the physiological mechanisms underlying memory replay during working memory maintenance and episodic memory. The second example aims at understanding representational differences between various types of memory, such as perceptual priming and conscious recognition memory. We also highlight the conceptual and methodological differences between these two examples. Finally, we discuss potential future applications for MVPC of EEG/MEG data in studies of memory. We conclude that despite its infancy and existing methodological challenges, MVPC of EEG and MEG data is a powerful tool with which to assess mechanistic models of memory.

Inferring functional neural connectivity with phase synchronization analysis: a review of methodology

Functional neural connectivity is drawing increasing attention in neuroscience research. To infer functional connectivity from observed neural signals, various methods have been proposed. Among them, phase synchronization analysis is an important and effective one which examines the relationship of instantaneous phase between neural signals but neglecting the influence of their amplitudes. In this paper, we review the advances in methodologies of phase synchronization analysis. In particular, we discuss the definitions of instantaneous phase, the indexes of phase synchronization and their significance test, the issues that may affect the detection of phase synchronization and the extensions of phase synchronization analysis. In practice, phase synchronization analysis may be affected by observational noise, insufficient samples of the signals, volume conduction, and reference in recording neural signals. We make comments and suggestions on these issues so as to better apply phase synchronization analysis to inferring functional connectivity from neural signals.

Ethanol reduces the phase locking of neural activity in human and rodent brain

How the neuromolecular actions of ethanol translate to its observed intoxicating effects remains poorly understood. Synchrony of phase (phase locking) of event-related oscillations (EROs) within and between different brain areas has been suggested to reflect communication exchange between neural networks and as such may be a sensitive and translational measure of ethanol's effects. Using a similar auditory event-related potential paradigm in both rats and humans we investigated the phase variability of EROs collected from 38 young men who had participated in an ethanol/placebo challenge protocol, and 46 adult male rats given intraperitoneal injections of ethanol/saline. Phase locking was significantly higher in the delta frequencies in humans than in rats. Phase locking was also higher for the rare (target) tone than the frequent (non-target) tone in both species. Significant reductions in phase locking to the rare (target) tone in the delta, theta, alpha, beta and gamma frequencies, within and between brain sites, was found at 1h following ethanol as compared to placebo/saline administration in both rats and humans. Reductions in phase locking in the alpha frequencies in the parietal cortex were found to be correlated with blood ethanol concentrations. These findings are consistent with the hypothesis that ethanol's intoxicating actions in the brain include reducing synchrony within and between neuronal networks, perhaps by increasing the level of noise in key neuromolecular interactions.

Value of amplitude, phase, and coherence features for a sensorimotor rhythm-based brain-computer interface

Measures that quantify the relationship between two or more brain signals are drawing attention as neuroscientists explore the mechanisms of large-scale integration that enable coherent behavior and cognition. Traditional Fourier-based measures of coherence have been used to quantify frequency-dependent relationships between two signals. More recently, several off-line studies examined phase-locking value (PLV) as a possible feature for use in brain-computer interface (BCI) systems. However, only a few individuals have been studied and full statistical comparisons among the different classes of features and their combinations have not been conducted. The present study examines the relative BCI performance of spectral power, coherence, and PLV, alone and in combination. The results indicate that spectral power produced classification at least as good as PLV, coherence, or any possible combination of these measures. This may be due to the fact that all three measures reflect mainly the activity of a single signal source (i.e., an area of sensorimotor cortex). This possibility is supported by the finding that EEG signals from different channels generally had near-zero phase differences. Coherence, PLV, and other measures of inter-channel relationships may be more valuable for BCIs that use signals from more than one distinct cortical source.

Comparison of classification methods for P300 brain-computer interface on disabled subjects

We report on tests with a mind typing paradigm based on a P300 brain-computer interface (BCI) on a group of amyotrophic lateral sclerosis (ALS), middle cerebral artery (MCA) stroke, and subarachnoid hemorrhage (SAH) patients, suffering from motor and speech disabilities. We investigate the achieved typing accuracy given the individual patient's disorder, and how it correlates with the type of classifier used. We considered 7 types of classifiers, linear as well as nonlinear ones, and found that, overall, one type of linear classifier yielded a higher classification accuracy. In addition to the selection of the classifier, we also suggest and discuss a number of recommendations to be considered when building a P300-based typing system for disabled subjects.

Critical issues in state-of-the-art brain-computer interface signal processing

This paper reviews several critical issues facing signal processing for brain-computer interfaces (BCIs) and suggests several recent approaches that should be further examined. The topics were selected based on discussions held during the 4th International BCI Meeting at a workshop organized to review and evaluate the current state of, and issues relevant to, feature extraction and translation of field potentials for BCIs. The topics presented in this paper include the relationship between electroencephalography and electrocorticography, novel features for performance prediction, time-embedded signal representations, phase information, signal non-stationarity, and unsupervised adaptation.