Discriminative codewaves: a symbolic dynamics approach to SSVEP recognition for asynchronous BCI

An efficient CNN-LSTM Network with spectral normalization and label smoothing technologies for SSVEP frequency recognition

OBJECTIVE

Steady-state visual evoked potentials(SSVEPs) based braincomputer interface(BCI) has received great interests owing to the high information transfer rate(ITR) and available large number of targets. However, the performance of frequency recognition methods heavily depends on the amount of the calibration data for intra-subject classification. Some research adopted the deep learning(DL) algorithm to conduct the inter-subject classification, which could reduce the calculation procedure, but the performance still has large room to improve compared with the intra-subject classification.

APPROACH

To address these issues, we proposed an efficient SSVEP DL NETwork (termed SSVEPNET) based on 1D convolution and long short-term memory (LSTM) module. To enhance the performance of SSVEPNT, we adopted the spectral normalization and label smoothing technologies during implementing the network architecture. We evaluated the SSVEPNET and compared it with other methods for the intra- and inter-subject classification under different conditions, i.e., two datasets, two time-window lengths (1 s and 0.5 s), three sizes of training data.

MAIN RESULTS

Under all the experimental settings, the proposed SSVEPNET achieved the highest average accuracy for the intra- and inter-subject classification on the two SSVEP datasets, when compared with other traditional and DL baseline methods. Signif icance. The extensive experimental results demonstrate that the proposed DL model holds promise to enhance frequency recognition performance in SSVEP-based BCIs. Besides, the mixed network structures with CNN and LSTM, and the spectral normalization and label smoothing could be useful optimization strategies to design efficient models for EEG data.

Covariation Informed Graph Slepians for Motor Imagery Decoding

Graph signal processing (GSP) provides signal analytic tools for data defined in irregular domains, as is the case of non-invasive electroencephalography (EEG). In this work, the recently introduced technique of Graph Slepian functions is exploited for the robust decoding of motor imagery (MI) brain activity. The particular technique builds over the concept of graph Fourier transform (GFT) and provides additional flexibility in the subsequent data analysis by incorporating domain knowledge. Based on contrastive learning, we introduce an algorithmic pipeline that attains a data driven and subject specific design of Graph Slepian functions. These functions, by incorporating both the topology of the sensor array and the empirical evidence about the differential functional covariation, act as spatial filters that enhance the information conveyed by the multichannel signal and specifically relates to the participant's intention. The proposed technique for crafting Graph Slepians is incorporated in a MI-decoding scheme, in which the informed projections are fed to a support vector machine (SVM) that casts a prediction regarding the type of intended movement. The employed MI-decoder is evaluated based on two publicly available datasets and its superiority against popular alternatives in the field is established. Computational efficiency is listed among its main advantages, since it involves only simple matrix operations, allowing to consider its use in real-time implementations.

Inter- and Intra-subject Template-Based Multivariate Synchronization Index Using an Adaptive Threshold for SSVEP-Based BCIs

The steady-state visually evoked potential (SSVEP) has been widely used in brain-computer interfaces (BCIs). Many studies have proved that the Multivariate synchronization index (MSI) is an efficient method for recognizing the frequency components in SSVEP-based BCIs. Despite its success, the recognition accuracy has not been satisfactory because the simplified pre-constructed sine-cosine waves lack abundant features from the real electroencephalogram (EEG) data. Recent advances in addressing this issue have achieved a significant improvement in recognition accuracy by using individual calibration data. In this study, a new extension based on inter- and intra-subject template signals is introduced to improve the performance of the standard MSI method. Through template transfer, inter-subject similarity and variability are employed to enhance the robustness of SSVEP recognition. Additionally, most existed methods for SSVEP recognition utilize a fixed time window (TW) to perform frequency domain analysis, which limits the information transfer rate (ITR) of BCIs. For addressing this problem, a novel adaptive threshold strategy is integrated into the extension of MSI, which uses a dynamic window to extract the temporal features of SSVEPs and recognizes the stimulus frequency based on a pre-set threshold. The pre-set threshold contributes to obtaining an appropriate and shorter signal length for frequency recognition and filtering ignored-invalid trials. The proposed method is evaluated on a 12-class SSVEP dataset recorded from 10 subjects, and the result shows that this achieves higher recognition accuracy and information transfer rate when compared with the CCA, MSI, Multi-set CCA, and Individual Template-based CCA. This paper demonstrates that the proposed method is a promising approach for developing high-speed BCIs.

Connectivity steered graph Fourier transform for motor imagery BCI decoding

OBJECTIVE

Graph signal processing (GSP) concepts are exploited for brain activity decoding and particularly the detection and recognition of a motor imagery (MI) movement. A novel signal analytic technique that combines graph Fourier transform (GFT) with estimates of cross-frequency coupling (CFC) and discriminative learning is introduced as a means to recover the subject's intention from the multichannel signal.

APPROACH

Adopting a multi-view perspective, based on the popular concept of co-existing and interacting brain rhythms, a multilayer network model is first built from empirical data and its connectivity graph is used to derive the GFT-basis. A personalized decoding scheme supporting a binary decision, either 'left versus right' or 'rest versus MI', is crafted from a small set of training trials. Electroencephalographic (EEG) activity from 12 volunteers recorded during two randomly alternating, externally cued, MI tasks (clenching either left or right fist) and a rest condition is used to introduce and validate our methodology. In addition, the introduced methodology was further validated based on dataset IVa of BCI III competition.

MAIN RESULTS

Our GFT-domain decoding scheme achieves nearly optimal performance and proves superior to alternative techniques that are very popular in the field.

SIGNIFICANCE

At a conceptual level, our work suggests a fruitful way to introduce network neuroscience in BCI research. At a more practical level, it is characterized by efficiency. Training is realized using a small number of exemplar trials and decoding requires very simple operations that leaves room for real-time implementation.

Exploiting the heightened phase synchrony in patients with neuromuscular disease for the establishment of efficient motor imagery BCIs

Background

Phase synchrony has extensively been studied for understanding neural coordination in health and disease. There are a few studies concerning the implications in the context of BCIs, but its potential for establishing a communication channel in patients suffering from neuromuscular disorders remains totally unexplored. We investigate, here, this possibility by estimating the time-resolved phase connectivity patterns induced during a motor imagery (MI) task and adopting a supervised learning scheme to recover the subject’s intention from the streaming data.

Methods

Electroencephalographic activity from six patients suffering from neuromuscular disease (NMD) and six healthy individuals was recorded during two randomly alternating, externally cued, MI tasks (clenching either left or right fist) and a rest condition. The metric of Phase locking value (PLV) was used to describe the functional coupling between all recording sites. The functional connectivity patterns and the associate network organization was first compared between the two cohorts. Next, working at the level of individual patients, we trained support vector machines (SVMs) to discriminate between “left” and “right” based on different instantiations of connectivity patterns (depending on the encountered brain rhythm and the temporal interval). Finally, we designed and realized a novel brain decoding scheme that could interpret the intention from streaming connectivity patterns, based on an ensemble of SVMs.

Results

The group-level analysis revealed increased phase synchrony and richer network organization in patients. This trend was also seen in the performance of the employed classifiers. Time-resolved connectivity led to superior performance, with distinct SVMs acting as local experts, specialized in the patterning emerged within specific temporal windows (defined with respect to the external trigger). This empirical finding was further exploited in implementing a decoding scheme that can be activated without the need of the precise timing of a trigger.

Conclusion

The increased phase synchrony in NMD patients can turn to a valuable tool for MI decoding. Considering the fast implementation for the PLV pattern computation in multichannel signals, we can envision the development of efficient personalized BCI systems in assistance of these patients.

Electronic supplementary material

The online version of this article (10.1186/s12984-018-0431-6) contains supplementary material, which is available to authorized users.

Enhancing Performance and Bit Rates in a Brain-Computer Interface System With Phase-to-Amplitude Cross-Frequency Coupling: Evidences From Traditional c-VEP, Fast c-VEP, and SSVEP Designs

A brain–computer interface (BCI) is a channel of communication that transforms brain activity into specific commands for manipulating a personal computer or other home or electrical devices. In other words, a BCI is an alternative way of interacting with the environment by using brain activity instead of muscles and nerves. For that reason, BCI systems are of high clinical value for targeted populations suffering from neurological disorders. In this paper, we present a new processing approach in three publicly available BCI data sets: (a) a well-known multi-class (N = 6) coded-modulated Visual Evoked potential (c-VEP)-based BCI system for able-bodied and disabled subjects; (b) a multi-class (N = 32) c-VEP with slow and fast stimulus representation; and (c) a steady-state Visual Evoked potential (SSVEP) multi-class (N = 5) flickering BCI system. Estimating cross-frequency coupling (CFC) and namely δ-θ [δ: (0.5–4 Hz), θ: (4–8 Hz)] phase-to-amplitude coupling (PAC) within sensor and across experimental time, we succeeded in achieving high classification accuracy and Information Transfer Rates (ITR) in the three data sets. Our approach outperformed the originally presented ITR on the three data sets. The bit rates obtained for both the disabled and able-bodied subjects reached the fastest reported level of 324 bits/min with the PAC estimator. Additionally, our approach outperformed alternative signal features such as the relative power (29.73 bits/min) and raw time series analysis (24.93 bits/min) and also the original reported bit rates of 10–25 bits/min. In the second data set, we succeeded in achieving an average ITR of 124.40 ± 11.68 for the slow 60 Hz and an average ITR of 233.99 ± 15.75 for the fast 120 Hz. In the third data set, we succeeded in achieving an average ITR of 106.44 ± 8.94. Current methodology outperforms any previous methodologies applied to each of the three free available BCI datasets.