Discriminative Feature Extraction via Multivariate Linear Regression for SSVEP-Based BCI

Comparative Study of Frequency Recognition Techniques for Steady-State Visual Evoked Potentials According to the Frequency Harmonics and Stimulus Number

Background

A key challenge in steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems is to effectively recognize frequencies within a short time window. To address this challenge, the specific characteristics of the data are needed to select the frequency recognition method. These characteristics include factors, such as the number of stimulation targets and the presence of harmonic frequencies, resulting in optimizing the performance and accuracy of SSVEP-based BCI systems.

Objective

The current study aimed to examine the effect of data characteristics on frequency recognition accuracy.

Material and Methods

In this analytical study, five commonly used frequency recognition methods were examined, used to various datasets containing different numbers of frequencies, including sub-data with and without frequency harmonics.

Results

The increase in the number of frequencies in the Multivariate Linear Regression (MLR) method has led to a decrease in frequency recognition accuracy by 9%. Additionally, the presence of harmonic frequencies resulted in an 8% decrease in accuracy for the MLR method.

Conclusion

Frequency recognition using the MLR method reduces the effect of the number of different frequencies and harmonics of the stimulation frequencies on the frequency recognition accuracy.

Filter bank temporally local multivariate synchronization index for SSVEP-based BCI

BACKGROUND

Multivariate synchronization index (MSI) has been successfully applied for frequency detection in steady state visual evoked potential (SSVEP) based brain-computer interface (BCI) systems. However, the standard MSI algorithm and its variants cannot simultaneously take full advantage of the time-local structure and the harmonic components in SSVEP signals, which are both crucial for frequency detection performance. To overcome the limitation, we propose a novel filter bank temporally local MSI (FBTMSI) algorithm to further improve SSVEP frequency detection accuracy. The method explicitly utilizes the temporal information of signal for covariance matrix estimation and employs filter bank decomposition to exploits SSVEP-related harmonic components.

RESULTS

We employed the cross-validation strategy on the public Benchmark dataset to optimize the parameters and evaluate the performance of the FBTMSI algorithm. Experimental results show that FBTMSI outperforms the standard MSI, temporally local MSI (TMSI) and filter bank driven MSI (FBMSI) algorithms across multiple experimental settings. In the case of data length of one second, the average accuracy of FBTMSI is 9.85% and 3.15% higher than that of the FBMSI and the TMSI, respectively.

CONCLUSIONS

The promising results demonstrate the effectiveness of the FBTMSI algorithm for frequency recognition and show its potential in SSVEP-based BCI applications.

TDLNet: Transfer Data Learning Network for Cross-Subject Classification Based on Multiclass Upper Limb Motor Imagery EEG

The limited number of brain-computer interface based on motor imagery (MI-BCI) instruction sets for different movements of single limbs makes it difficult to meet practical application requirements. Therefore, designing a single-limb, multi-category motor imagery (MI) paradigm and effectively decoding it is one of the important research directions in the future development of MI-BCI. Furthermore, one of the major challenges in MI-BCI is the difficulty of classifying brain activity across different individuals. In this article, the transfer data learning network (TDLNet) is proposed to achieve the cross-subject intention recognition for multiclass upper limb motor imagery. In TDLNet, the Transfer Data Module (TDM) is used to process cross-subject electroencephalogram (EEG) signals in groups and then fuse cross-subject channel features through two one-dimensional convolutions. The Residual Attention Mechanism Module (RAMM) assigns weights to each EEG signal channel and dynamically focuses on the EEG signal channels most relevant to a specific task. Additionally, a feature visualization algorithm based on occlusion signal frequency is proposed to qualitatively analyze the proposed TDLNet. The experimental results show that TDLNet achieves the best classification results on two datasets compared to CNN-based reference methods and transfer learning method. In the 6-class scenario, TDLNet obtained an accuracy of 65%±0.05 on the UML6 dataset and 63%±0.06 on the GRAZ dataset. The visualization results demonstrate that the proposed framework can produce distinct classifier patterns for multiple categories of upper limb motor imagery through signals of different frequencies. The ULM6 dataset is available at https://dx.doi.org/10.21227/8qw6-f578.

Dataset Evaluation Method and Application for Performance Testing of SSVEP-BCI Decoding Algorithm

Steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems have been extensively researched over the past two decades, and multiple sets of standard datasets have been published and widely used. However, there are differences in sample distribution and collection equipment across different datasets, and there is a lack of a unified evaluation method. Most new SSVEP decoding algorithms are tested based on self-collected data or offline performance verification using one or two previous datasets, which can lead to performance differences when used in actual application scenarios. To address these issues, this paper proposed a SSVEP dataset evaluation method and analyzed six datasets with frequency and phase modulation paradigms to form an SSVEP algorithm evaluation dataset system. Finally, based on the above datasets, performance tests were carried out on the four existing SSVEP decoding algorithms. The findings reveal that the performance of the same algorithm varies significantly when tested on diverse datasets. Substantial performance variations were observed among subjects, ranging from the best-performing to the worst-performing. The above results demonstrate that the SSVEP dataset evaluation method can integrate six datasets to form a SSVEP algorithm performance testing dataset system. This system can test and verify the SSVEP decoding algorithm from different perspectives such as different subjects, different environments, and different equipment, which is helpful for the research of new SSVEP decoding algorithms and has significant reference value for other BCI application fields.

An Analysis of Deep Learning Models in SSVEP-Based BCI: A Survey

The brain-computer interface (BCI), which provides a new way for humans to directly communicate with robots without the involvement of the peripheral nervous system, has recently attracted much attention. Among all the BCI paradigms, BCIs based on steady-state visual evoked potentials (SSVEPs) have the highest information transfer rate (ITR) and the shortest training time. Meanwhile, deep learning has provided an effective and feasible solution for solving complex classification problems in many fields, and many researchers have started to apply deep learning to classify SSVEP signals. However, the designs of deep learning models vary drastically. There are many hyper-parameters that influence the performance of the model in an unpredictable way. This study surveyed 31 deep learning models (2011-2023) that were used to classify SSVEP signals and analyzed their design aspects including model input, model structure, performance measure, etc. Most of the studies that were surveyed in this paper were published in 2021 and 2022. This survey is an up-to-date design guide for researchers who are interested in using deep learning models to classify SSVEP signals.

An Adaptive Task-Related Component Analysis Method for SSVEP Recognition

Steady-State Visual Evoked Potential (SSVEP) recognition methods use a subject's calibration data to differentiate between brain responses, hence, providing the SSVEP-based brain-computer interfaces (BCIs) with high performance. However, they require sufficient calibration EEG trials to achieve that. This study develops a new method to learn from limited calibration EEG trials, and it proposes and evaluates a novel adaptive data-driven spatial filtering approach for enhancing SSVEP detection. The spatial filter learned from each stimulus utilizes temporal information from the corresponding EEG trials. To introduce the temporal information into the overall procedure, a multitask learning approach, based on the Bayesian framework, is adopted. The performance of the proposed method was evaluated into two publicly available benchmark datasets, and the results demonstrated that our method outperformed competing methods by a significant margin.

An efficient CNN-LSTM Network with spectral normalization and label smoothing technologies for SSVEP frequency recognition

OBJECTIVE

Steady-state visual evoked potentials(SSVEPs) based braincomputer interface(BCI) has received great interests owing to the high information transfer rate(ITR) and available large number of targets. However, the performance of frequency recognition methods heavily depends on the amount of the calibration data for intra-subject classification. Some research adopted the deep learning(DL) algorithm to conduct the inter-subject classification, which could reduce the calculation procedure, but the performance still has large room to improve compared with the intra-subject classification.

APPROACH

To address these issues, we proposed an efficient SSVEP DL NETwork (termed SSVEPNET) based on 1D convolution and long short-term memory (LSTM) module. To enhance the performance of SSVEPNT, we adopted the spectral normalization and label smoothing technologies during implementing the network architecture. We evaluated the SSVEPNET and compared it with other methods for the intra- and inter-subject classification under different conditions, i.e., two datasets, two time-window lengths (1 s and 0.5 s), three sizes of training data.

MAIN RESULTS

Under all the experimental settings, the proposed SSVEPNET achieved the highest average accuracy for the intra- and inter-subject classification on the two SSVEP datasets, when compared with other traditional and DL baseline methods. Signif icance. The extensive experimental results demonstrate that the proposed DL model holds promise to enhance frequency recognition performance in SSVEP-based BCIs. Besides, the mixed network structures with CNN and LSTM, and the spectral normalization and label smoothing could be useful optimization strategies to design efficient models for EEG data.

An improved cross-subject spatial filter transfer method for SSVEP-based BCI

Steady-state visual evoked potential (SSVEP) training feature recognition algorithms utilize user training data to reduce the interference of spontaneous electroencephalogram (EEG) activities on SSVEP response for improved recognition accuracy. The data collection process can be tedious, increasing the mental fatigue of users and also seriously affecting the practicality of SSVEP-based brain-computer interface (BCI) systems. As an alternative, a cross-subject spatial filter transfer (CSSFT) method to transfer an existing user data model with good SSVEP response to new user test data has been proposed. The CSSFT method uses superposition averages of data for multiple blocks of data as transfer data. However, the amplitude and pattern of brain signals are often significantly different across trials. The goal of this study was to improve superposition averaging for the CSSFT method and propose an Ensemble scheme based on ensemble learning, and an Expansion scheme based on matrix expansion. The feature recognition performance was compared for CSSFT and the proposed improved CSSFT method using two public datasets. The results demonstrated that the improved CSSFT method can significantly improve the recognition accuracy and information transmission rate of existing methods. This strategy avoids a tedious data collection process, and promotes the potential practical application of BCI systems.

Cross-subject spatial filter transfer method for SSVEP-EEG feature recognition

Objective.Steady-state visual evoked potential (SSVEP) is an important control method of the brain-computer interface (BCI) system. The development of an efficient SSVEP feature decoding algorithm is the core issue in SSVEP-BCI. It has been proposed to use user training data to reduce the spontaneous electroencephalogram activity interference on SSVEP response, thereby improving the feature recognition accuracy of the SSVEP signal. Nevertheless, the tedious data collection process increases the mental fatigue of the user and severely affects the applicability of the BCI system.Approach.A cross-subject spatial filter transfer (CSSFT) method that transfer the existing user model with good SSVEP response to the new user test data without collecting any training data from the new user is proposed.Main results.Experimental results demonstrate that the transfer model increases the distinction of the feature discriminant coefficient between the gaze following target and the non-gaze following target and accurately identifies the wrong target in the fundamental algorithm model. The public datasets show that the CSSFT method significantly increases the recognition performance of canonical correlation analysis (CCA) and filter bank CCA. Additionally, when the data used to calculate the transfer model contains one data block only, the CSSFT method retains its effective feature recognition capabilities.Significance.The proposed method requires no tedious data calibration process for new users, provides an effective technical solution for the transfer of the cross-subject model, and has potential application value for promoting the application of the BCI system.

Control of a Robotic Arm With an Optimized Common Template-Based CCA Method for SSVEP-Based BCI

Recently, the robotic arm control system based on a brain-computer interface (BCI) has been employed to help the disabilities to improve their interaction abilities without body movement. However, it's the main challenge to implement the desired task by a robotic arm in a three-dimensional (3D) space because of the instability of electroencephalogram (EEG) signals and the interference by the spontaneous EEG activities. Moreover, the free motion control of a manipulator in 3D space is a complicated operation that requires more output commands and higher accuracy for brain activity recognition. Based on the above, a steady-state visual evoked potential (SSVEP)-based synchronous BCI system with six stimulus targets was designed to realize the motion control function of the seven degrees of freedom (7-DOF) robotic arm. Meanwhile, a novel template-based method, which builds the optimized common templates (OCTs) from various subjects and learns spatial filters from the common templates and the multichannel EEG signal, was applied to enhance the SSVEP recognition accuracy, called OCT-based canonical correlation analysis (OCT-CCA). The comparison results of offline experimental based on a public benchmark dataset indicated that the proposed OCT-CCA method achieved significant improvement of detection accuracy in contrast to CCA and individual template-based CCA (IT-CCA), especially using a short data length. In the end, online experiments with five healthy subjects were implemented for achieving the manipulator real-time control system. The results showed that all five subjects can accomplish the tasks of controlling the manipulator to reach the designated position in the 3D space independently.

Recent Advances and Current Trends in Brain-Computer Interface (BCI) Research and Their Applications

Brain-Computer Interface (BCI) provides direct communication between the brain and an external device. BCI systems have become a trendy field of research in recent years. These systems can be used in a variety of applications to help both disabled and healthy people. Concerning significant BCI progress, we may assume that these systems are not very far from real-world applications. This review has taken into account current trends in BCI research. In this survey, one hundred most cited articles from the WOS database were selected over the last four years. This survey is divided into several sectors. These sectors are Medicine, Communication and Control, Entertainment, and Other BCI applications. The application area, recording method, signal acquisition types, and countries of origin have been identified in each article. This survey provides an overview of the BCI articles published from 2016 to 2020 and their current trends and advances in different application areas.

Signal processing algorithms for SSVEP-based brain computer interface: State-of-the-art and recent developments

Over past two decades, steady-state evoked potentials (SSVEP)-based brain computer interface (BCI) systems have been extensively developed. As we all know, signal processing algorithms play an important role in this BCI. However, there is no comprehensive review of the latest development of signal processing algorithms for SSVEP-based BCI. By analyzing the papers published in authoritative journals in nearly five years, signal processing algorithms of preprocessing, feature extraction and classification modules are discussed in detail. In addition, other aspects existed in this BCI are mentioned. The following key problems are solved. (1) In recent years, which signal processing algorithms are frequently used in each module? (2) Which signal processing algorithms attract more attention in recent years? (3) Which modules are the key to signal processing in BCI field? This information is very important for choosing the appropriate algorithms, and can also be considered as a reference for further research. Simultaneously, we hope that this work can provide relevant BCI researchers with valuable information about the latest trends of signal processing algorithms for SSVEP-based BCI systems.

Filter Bank-Driven Multivariate Synchronization Index for Training-Free SSVEP BCI

In recent years, multivariate synchronization index (MSI) algorithm, as a novel frequency detection method, has attracted increasing attentions in the study of brain-computer interfaces (BCIs) based on steady state visual evoked potential (SSVEP). However, MSI algorithm is hard to fully exploit SSVEP-related harmonic components in the electroencephalogram (EEG), which limits the application of MSI algorithm in BCI systems. In this paper, we propose a novel filter bank-driven MSI algorithm (FBMSI) to overcome the limitation and further improve the accuracy of SSVEP recognition. We evaluate the efficacy of the FBMSI method by developing a 6-command SSVEP-NAO robot system with extensive experimental analyses. An offline experimental study is first performed with EEG collected from nine subjects to investigate the effects of varying parameters on the model performance. Offline results show that the proposed method has achieved a stable improvement effect. We further conduct an online experiment with six subjects to assess the efficacy of the developed FBMSI algorithm in a real-time BCI application. The online experimental results show that the FBMSI algorithm yields a promising average accuracy of 83.56% using a data length of even only one second, which was 12.26% higher than the standard MSI algorithm. These extensive experimental results confirmed the effectiveness of the FBMSI algorithm in SSVEP recognition and demonstrated its potential application in the development of improved BCI systems.

Enhancing detection of steady-state visual evoked potentials using channel ensemble method

OBJECTIVE

This study proposed and evaluated a channel ensemble approach to enhance detection of steady-state visual evoked potentials (SSVEPs).

APPROACH

Collected multi-channel electroencephalogram (EEG) signals were classified into multiple groups of new analysis signals based on correlation analysis, and each group of analysis signals contained signals from a different number of electrode channels. These groups of analysis signals were used as the input of a training-free feature extraction model, and the obtained feature coefficients were converted into feature probability values using the softmax function. The ensemble value of multiple sets of feature probability values was determined and used as the final discrimination coefficient.

MAIN RESULTS

Compared with canonical correlation analysis (CCA), likelihood ratio test (LRT), and multivariate synchronization index (MSI) analysis methods using a standard approach, the recognition accuracies of the methods using a channel ensemble approach were improved by 5.05%, 3.87%, and 3.42%, and the information transfer rates (ITRs) were improved by 6.00%, 4.61%, and 3.71%, respectively. The channel ensemble method also obtained better recognition results than the standard algorithm on the public dataset. This study validated the efficiency of the proposed method to enhance the detection of SSVEPs, demonstrating its potential use in practical brain-computer interface (BCI) systems.

SIGNIFICANCE

A SSVEP-based BCI system using a channel ensemble method could achieve high ITR, indicating great potential of this design for various applications with improved control and interaction.

OPTICAL+: a frequency-based deep learning scheme for recognizing brain wave signals

A human-computer interaction (HCI) system can be used to detect different categories of the brain wave signals that can be beneficial for neurorehabilitation, seizure detection and sleep stage classification. Research on developing HCI systems using brain wave signals has progressed a lot over the years. However, real-time implementation, computational complexity and accuracy are still a concern. In this work, we address the problem of selecting the appropriate filtering frequency band while also achieving a good system performance by proposing a frequency-based approach using long short-term memory network (LSTM) for recognizing different brain wave signals. Adaptive filtering using genetic algorithm is incorporated for a hybrid system utilizing common spatial pattern and LSTM network. The proposed method (OPTICAL+) achieved an overall average classification error rate of 30.41% and a kappa coefficient value of 0.398, outperforming the state-of-the-art methods. The proposed OPTICAL+ predictor can be used to develop improved HCI systems that will aid in neurorehabilitation and may also be beneficial for sleep stage classification and seizure detection.

Common Information Components Analysis

Wyner’s common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis (CICA). The first step can be interpreted as an extraction of Wyner’s common information. The second step is a form of back-projection of the common information onto the original variables, leading to the extracted features. A free parameter γ controls the complexity of the extracted features. We establish that, in the case of Gaussian statistics, CICA precisely reduces to Canonical Correlation Analysis (CCA), where the parameter γ determines the number of CCA components that are extracted. In this sense, we establish a novel rigorous connection between information measures and CCA, and CICA is a strict generalization of the latter. It is shown that CICA has several desirable features, including a natural extension to beyond just two data sets.

A Maximum Likelihood Perspective of Spatial Filter Design in SSVEP-Based BCIs

In steady-state visual-evoked potential (SSVEP) based brain-computer interfaces (BCIs), existing detection algorithms utilizing spatial filters like task-related component analysis (TRCA) derive the spatial filters mainly through maximizing the inter-trial similarity between the combined signals over the training set. Although they achieve by far the best classification performance in SSVEP-based BCIs, some important problems are still unresolved. Especially, the mechanism of how spatial filters cancel the background noise in brain signals and optimize the signal-to-noise ratio (SNR) of SSVEPs is still not figured out. Therefore, to solve these problems, in this paper a new perspective of spatial filter design is proposed. Specifically, a linear generative signal model of SSVEP is adopted and the spatial filters are obtained automatically through maximum likelihood estimation of source signals and channel vectors. In the same time, the relation between maximum likelihood estimation and signal-to-noise ratio (SNR) maximization is discussed. Through a step-by-step formulation, this paper provides a theoretical justification for those conventional algorithms utilizing spatial filters. As for the classification performance, the proposed scheme is tested on a benchmark dataset of 35 subjects. Experiment results show that the classification performance of the proposed scheme is competitive against three benchmark algorithms, which include TRCA. Especially, the proposed scheme achieves a fair performance improvement over the benchmark methods in the cases where a shorter time window, or a larger number of electrodes, or a smaller number of training blocks are adopted.

A Functional Model for Unifying Brain Computer Interface Terminology

Brain Computer Interface (BCI) technology is a critical area both for researchers and clinical practitioners. The IEEE P2731 working group is developing a comprehensive BCI lexicography and a functional model of BCI. The glossary and the functional model are inextricably intertwined. The functional model guides the development of the glossary. Terminology is developed from the basis of a BCI functional model. This paper provides the current status of the P2731 working group's progress towards developing a BCI terminology standard and functional model for the IEEE.

Temporal Combination Pattern Optimization Based on Feature Selection Method for Motor Imagery BCIs

Common spatial pattern (CSP) method is widely used for spatial filtering and brain pattern extraction from electroencephalogram (EEG) signals in motor imagery (MI)-based brain-computer interfaces (BCIs). The participant-specific time window relative to the visual cue has a significant impact on the effectiveness of the CSP. However, the time window is usually selected experientially or manually. To solve this problem, we propose a novel feature selection approach for MI-based BCIs. Specifically, multiple time segments were obtained by decomposing each EEG sample of the MI task. Furthermore, the features were extracted by CSP from each time segment and were combined to form a new feature vector. Finally, the optimal temporal combination patterns for the new feature vector were selected based on four feature selection algorithms, i.e., mutual information, least absolute shrinkage and selection operator, principal component analysis and stepwise linear discriminant analysis (denoted as MUIN, LASSO, PCA, and SWLDA, respectively), and the classification algorithm was employed to evaluate the average classification accuracy. With three BCI competition datasets, the results of the four proposed algorithms were compared with traditional CSP algorithm in classification accuracy. Experimental results show that compared with traditional algorithm, the proposed methods significantly improve performance. Specifically, the LASSO achieved the highest accuracy (88.58%) among the proposed methods. Importantly, the average classification accuracies using the proposed approaches significantly improved 10.14% (MUIN), 11.40% (LASSO), 6.08% (PCA), and 10.25% (SWLDA) compared to that using CSP. These results indicate that the proposed approach is expected to be practical in MI-based BCIs.

EEG-EOG based Virtual Keyboard: Toward Hybrid Brain Computer Interface

The past twenty years have ignited a new spark in the research of Electroencephalogram (EEG), which was pursued to develop innovative Brain Computer Interfaces (BCIs) in order to help severely disabled people live a better life with a high degree of independence. Current BCIs are more theoretical than practical and are suffering from numerous challenges. New trends of research propose combining EEG to other simple and efficient bioelectric inputs such as Electro-oculography (EOG) resulting from eye movements, to produce more practical and robust Hybrid Brain Computer Interface systems (hBCI) or Brain/Neuronal Computer Interface (BNCI). Working towards this purpose, existing research in EOG based Human Computer Interaction (HCI) applications, must be organized and surveyed in order to develop a vision on the potential benefits of combining both input modalities and give rise to new designs that maximize these benefits. Our aim is to support and inspire the design of new hBCI systems based on both EEG and EOG signals, in doing so; first the current EOG based HCI systems were surveyed with a particular focus on EOG based systems for communication using virtual keyboard. Then, a survey of the current EEG-EOG virtual keyboard was performed highlighting the design protocols employed. We concluded with a discussion of the potential advantages of combining both systems with recommendations to give deep insight for future design issues for all EEG-EOG hBCI systems. Finally, a general architecture was proposed for a new EEG-EOG hBCI system. The proposed hybrid system completely alters the traditional view of the eye movement features present in EEG signal as artifacts that should be removed; instead EOG traces are extracted from EEG in our proposed hybrid architecture and are considered as an additional input modality sharing control according to the chosen design protocol.

Decision-Making Selector (DMS) for Integrating CCA-Based Methods to Improve Performance of SSVEP-Based BCIs

OBJECTIVE

Recent research has demonstrated improved performance of a brain-computer interface (BCI) using fusion based approaches. This paper proposes a novel decision-making selector (DMS) to integrate classification decisions of different frequency recognition methods based on canonical correlation analysis (CCA) which were used in decoding steady state visual evoked potentials (SSVEPs).

METHODS

The DMS method selects a decision more likely to be correct from two methods namely as M1 and M2 by separating the M1-false and M2-false trials. To measure the uncertainty of each decision, feature vectors were extracted using the largest and second largest correlation coefficients corresponding to all the stimulus frequencies. The proposed method was evaluated by integrating all pairs of 7 CCA-based algorithms, including CCA, individual template-based CCA (ITCCA), multi-set CCA (MsetCCA), L1-regularized multi-way CCA (L1-MCCA), filter bank CCA (FBCCA), extended CCA (ECCA), and task-related component analysis (TRCA).

MAIN RESULTS

The experimental results obtained from a 40-target dataset of thirty-five subjects showed that the proposed DMS method was validated to obtain an enhanced performance by integrating the algorithms with close accuracies.

CONCLUSION

The results suggest that the proposed DMS method is effective in integrating decisions of different methods to improve the performance of SSVEP-based BCIs.

Spatial Filtering in SSVEP-Based BCIs: Unified Framework and New Improvements

OBJECTIVE

In the steady-state visual evoked potential (SSVEP)-based brain computer interfaces (BCIs), spatial filtering, which combines the multi-channel electroencephalography (EEG) signals in order to reduce the non-SSVEP-related component and thus enhance the signal-to-noise ratio (SNR), plays an important role in target recognition. Recently, various spatial filtering algorithms have been developed employing different prior knowledge and characteristics of SSVEPs, however how these algorithms interconnect and differ is not yet fully explored, leading to difficulties in further understanding, utilizing and improving them.

METHODS

We propose a unified framework under which the spatial filtering algorithms can be formulated as generalized eigenvalue problems (GEPs) with four different elements: data, temporal filter, orthogonal projection and spatial filter. Based on the framework, we design new spatial filtering algorithms for improvements through the choice of different elements.

RESULTS

The similarities, differences and relationships among nineteen mainstream spatial filtering algorithms are revealed under the proposed framework. Particularly, it is found that they originate from the canonical correlation analysis (CCA), principal component analysis (PCA), and multi-set CCA, respectively. Furthermore, three new spatial filtering algorithms are developed with enhanced performance validated on two public SSVEP datasets with 45 subjects.

CONCLUSION

The proposed framework provides insights into the underlying relationships among different spatial filtering algorithms and helps the design of new spatial filtering algorithms.

SIGNIFICANCE

This is a systematic study to explore, compare and improve the existing spatial filtering algorithms, which would be significant for further understanding and future development of high performance SSVEP-based BCIs.

Discrimination of SSVEP responses using a kernel based approach

Brain Computer Interfaces based on Steady State Visual Evoked Potentials have gained increased attention due to their low training requirements and higher information transfer rates. In this work, a method based on sparse kernel machines is proposed for the discrimination of Steady State Visual Evoked Potentials responses. More specifically, a new kernel based on Partial Least Squares is introduced to describe the similarities between EEG trials, while the estimation of regression weights is performed using the Sparse Bayesian Learning framework. The experimental results obtained on two benchmarking datasets, have shown that the proposed method provides significantly better performance compared to state of the art approaches of the related literature.

Group task-related component analysis (gTRCA): a multivariate method for inter-trial reproducibility and inter-subject similarity maximization for EEG data analysis

EEG is known to contain considerable inter-trial and inter-subject variability, which poses a challenge in any group-level EEG analyses. A true experimental effect must be reproducible even with variabilities in trials, sessions, and subjects. Extracting components that are reproducible across trials and subjects benefits both understanding common mechanisms in neural processing of cognitive functions and building robust brain-computer interfaces. This study extends our previous method (task-related component analysis, TRCA) by maximizing not only trial-by-trial reproducibility within single subjects but also similarity across a group of subjects, hence referred to as group TRCA (gTRCA). The problem of maximizing reproducibility of time series across trials and subjects is formulated as a generalized eigenvalue problem. We applied gTRCA to EEG data recorded from 35 subjects during a steady-state visual-evoked potential (SSVEP) experiment. The results revealed: (1) The group-representative data computed by gTRCA showed higher and consistent spectral peaks than other conventional methods; (2) Scalp maps obtained by gTRCA showed estimated source locations consistently within the occipital lobe; And (3) the high-dimensional features extracted by gTRCA are consistently mapped to a low-dimensional space. We conclude that gTRCA offers a framework for group-level EEG data analysis and brain-computer interfaces alternative in complement to grand averaging.

Machine Learning Algorithms for Smart Data Analysis in Internet of Things Environment: Taxonomies and Research Trends

Machine learning techniques will contribution towards making Internet of Things (IoT) symmetric applications among the most significant sources of new data in the future. In this context, network systems are endowed with the capacity to access varieties of experimental symmetric data across a plethora of network devices, study the data information, obtain knowledge, and make informed decisions based on the dataset at its disposal. This study is limited to supervised and unsupervised machine learning (ML) techniques, regarded as the bedrock of the IoT smart data analysis. This study includes reviews and discussions of substantial issues related to supervised and unsupervised machine learning techniques, highlighting the advantages and limitations of each algorithm, and discusses the research trends and recommendations for further study.

Data mining based approach to study the effect of consumption of caffeinated coffee on the generation of the steady-state visual evoked potential signals

The steady-state visual evoked potentials (SSVEP), are elicited at the parieto-occipital region of the cortex when a light source (3.5-75 Hz), flickering at a constant frequency, stimulates the retinal cells. In the last few decades, researchers have reported that caffeine enhances the vigilance and the executive control of visual attention. However, no study has investigated the effect of caffeinated coffee on the SSVEP response, which is used for controlling the brain-computer interface (BCI) devices for rehabilitative applications. The current work proposes a data mining-based approach to gain insight into the alterations in the SSVEP signals after the consumption of caffeinated coffee. Recurrence quantification analysis (RQA) of the electroencephalogram (EEG) signals was employed for this purpose. The EEG signals were acquired at seven frequencies of photic stimuli. The stimuli frequencies were chosen such that they were distributed throughout the EEG frequency bands. The prominent SSVEP signals were identified using the Canonical Correlation Analysis (CCA) method. Several statistical features were extracted from the recurrence plot of the SSVEP signals. Statistical analyses using the t-test and decision tree-based methods helped to select the most relevant features, which were then classified using Automated Neural Network (ANN). The relevant features could be classified with a maximum accuracy of 97%. This supports our hypothesis that the consumption of caffeinated coffee can alter the SSVEP response. In conclusion, utmost care should be taken in selecting the features for designing BCI devices.

An ICA based approach for steady-state and transient analysis of task fMRI data: Application to study of thermal pain response

BACKGROUND

Data driven analysis methods such as independent component analysis (ICA) offer the advantage of estimating subject contributions when used in a second-level analysis. With the traditionally used regression-based methods this is achieved with a design matrix that has to be specified a priori.

NEW METHOD

We show that the ability of ICA to estimate subject contributions can be effectively used to perform steady-state as well as transient analysis of task functional magnetic resonance imaging (fMRI) data, which can help reveal important group differences.

RESULTS

We apply the method to steady-state and transient analysis of block designed thermal pain stimulated fMRI data, and identify distinct sex differences, in parts of the pain matrix: brain stem, thalamus, amygdala, frontal pole (FP), temporal pole (TP), operculum (second somatosensory cortex, SII), anterior insular (AI), dorsal anterior cingulate cortex (dACC), and default mode network (DMN). We also show that the identified regions have significant correlation with weekly exercise and anxiety. Using transient analysis, we identify regions (SII, AI, dACC, DMN) specific to female group showing difference mainly in the initial stages of the experiments.

COMPARISON WITH EXISTING METHOD

With exact same spatial components input in the second level, permutation analysis of linear models cannot identify any significant group difference. In addition, the proposed transient analysis cannot be realized if user is required to input a design matrix as is the case with regression-based analyses.

CONCLUSIONS

The proposed two-level ICA is an effective multi-variate analysis method for both steady-state and transient analysis of task data.

Temporally Constrained Sparse Group Spatial Patterns for Motor Imagery BCI

Common spatial pattern (CSP)-based spatial filtering has been most popularly applied to electroencephalogram (EEG) feature extraction for motor imagery (MI) classification in brain-computer interface (BCI) application. The effectiveness of CSP is highly affected by the frequency band and time window of EEG segments. Although numerous algorithms have been designed to optimize the spectral bands of CSP, most of them selected the time window in a heuristic way. This is likely to result in a suboptimal feature extraction since the time period when the brain responses to the mental tasks occurs may not be accurately detected. In this paper, we propose a novel algorithm, namely temporally constrained sparse group spatial pattern (TSGSP), for the simultaneous optimization of filter bands and time window within CSP to further boost classification accuracy of MI EEG. Specifically, spectrum-specific signals are first derived by bandpass filtering from raw EEG data at a set of overlapping filter bands. Each of the spectrum-specific signals is further segmented into multiple subseries using sliding window approach. We then devise a joint sparse optimization of filter bands and time windows with temporal smoothness constraint to extract robust CSP features under a multitask learning framework. A linear support vector machine classifier is trained on the optimized EEG features to accurately identify the MI tasks. An experimental study is implemented on three public EEG datasets (BCI Competition III dataset IIIa, BCI Competition IV datasets IIa, and BCI Competition IV dataset IIb) to validate the effectiveness of TSGSP in comparison to several other competing methods. Superior classification performance (averaged accuracies are 88.5%, 83.3%, and 84.3% for the three datasets, respectively) based on the experimental results confirms that the proposed algorithm is a promising candidate for performance improvement of MI-based BCIs.

Combined Lateral and Longitudinal Control of EEG Signals-Based Brain-Controlled Vehicles

Using brain signals rather than limbs to control a vehicle may not only help persons with disabilities to acquire driving ability, but also provide healthy persons with a new alternative way to drive. In this paper, we propose a combined lateral and longitudinal control system for electroencephalogram (EEG) signals-based brain-controlled vehicles (BCVs). The proposed system is designed by integrating a user interface, a brain-computer interface (BCI), a control interface model, a lateral controller, and a longitudinal controller. We conduct driver-and-hardware-in-the-loop experiments under two control conditions (i.e., the brain- and manual-control conditions) with different subjects and three driving tests (i.e., the lane-changing, path-selection, and car-following tests). Experimental results show the feasibility of using brain signals to continuously perform both the lateral and longitudinal control of a vehicle. This study not only promotes the development of BCVs, but also provides some insights on how to apply BCIs in conjunction with assistant controllers to control other dynamic systems.

A semi-blind online dictionary learning approach for fMRI data

BACKGROUND

Online dictionary learning (ODL) has been applied to extract brain networks from functional magnetic resonance imaging (fMRI) data in recent year. Moreover, the supervised dictionary learning (SDL) that fixed the task stimulus curves as predefined atoms was proposed to improve ODL for functional networks separation. However, SDL cannot estimate the real time courses underlying the brain networks and cannot be applied to the inter-network connectivity analysis. This study aimed at investigating how to add the temporal prior information to ODL to extract the accurate task-related brain networks and the corresponding time courses.

NEW METHOD

To improve the performance of ODL, we propose a semi-blind ODL (semi-ODL) method that incorporates temporal prior information of the task paradigm into the dictionary updating process and optimizes the direction of one or more specific atoms "close" to the task time courses.

RESULTS

Results of the simulated and real fMRI experiment revealed that semi-ODL extracted more accurate task-related component and time courses than ODL and SDL. For one-task fMRI data, semi-ODL and Infomax-ICA showed similar detection power in most cases.

COMPARISON WITH EXISTING METHODS

The semi-ODL outperformed ODL, SDL in robustness to noise, spatial detection power and time course estimation. Moreover, semi-ODL showed comparable performance to Infomax-ICA for one-task fMRI data and outperformed Infomax-ICA in extracting the components related to each task from multi-task fMRI data.

CONCLUSIONS

The semi-ODL method is potentially useful to reveal brain networks underlying various cognitive tasks and the interactions between task-related brain networks.

Two-Stage Frequency Recognition Method Based on Correlated Component Analysis for SSVEP-Based BCI

A canonical correlation analysis (CCA) is a state-of-the-art method for frequency recognition in steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems. Various extended methods have been developed, and among such methods, a combination method of CCA and individual-template-based CCA has achieved the best performance. However, the CCA requires the canonical vectors to be orthogonal, which may not be a reasonable assumption for the EEG analysis. In this paper, we propose using the correlated component analysis (CORRCA) rather than CCA to implement frequency recognition. CORRCA can relax the constraint of canonical vectors in CCA and generate the same projection vector for two multichannel EEG signals. Furthermore, we propose a two-stage method based on the basic CORRCA method (termed TSCORRCA). Evaluated on a benchmark data set of 35 subjects, the experimental results demonstrate that CORRCA significantly outperformed CCA, and TSCORRCA obtained the best performance among the compared methods. This paper demonstrates that CORRCA-based methods have a great potential for implementing high-performance SSVEP-based BCI systems.

Correlated Component Analysis for Enhancing the Performance of SSVEP-Based Brain-Computer Interface

A new method for steady-state visual evoked potentials (SSVEPs) frequency recognition is proposed to enhance the performance of SSVEP-based brain-computer interface (BCI). Correlated component analysis (CORCA) is introduced, which originally was designed to find linear combinations of electrodes that are consistent across subjects and maximally correlated between them. We propose a CORCA algorithm to learn spatial filters with multiple blocks of individual training data for SSVEP-based BCI scenario. The spatial filters are used to remove background noises by combining the multichannel electroencephalogram signals. We conduct a comparison between the proposed CORCA-based and the task-related component analysis (TRCA) based methods using a 40-class SSVEP benchmark data set recorded from 35 subjects. Our experimental study validates the efficiency of the CORCA-based method, and the extensive comparison results indicate that the CORCA-based method significantly outperforms the TRCA-based method. Superior performance demonstrates that the proposed method holds the promising potential to achieve satisfactory performance for SSVEP-based BCI with a large number of targets.

Comparing Steady-State Visually Evoked Potentials Frequency Estimation Methods in Brain-Computer Interface With the Minimum Number of EEG Channels

Introduction

Brain-Computer Interface (BCI) systems provide a communication pathway between users and systems. BCI systems based on Steady-State Visually Evoked Potentials (SSVEP) are widely used in recent decades. Different feature extraction methods have been introduced in the literature to estimate SSVEP responses to BCI applications.

Methods

In this study, the new algorithms, including Canonical Correlation Analysis (CCA), Least Absolute Shrinkage and Selection Operator (LASSO), L1-regularized Multi-way CCA (L1-MCCA), Multi-set CCA (MsetCCA), Common Feature Analysis (CFA), and Multiple Logistic Regression (MLR) are compared using proper statistical methods to determine which one has better performance with the least number of EEG electrodes.

Results

It was found that MLR, MsetCCA, and CFA algorithms provided the highest performances and significantly outperformed CCA, LASSO, and L1-MCCA algorithms when using 8 EEG channels. However, when using only 1 or 2 EEG channels d, CFA method provided the highest F-scores. This algorithm not only outperformed MLR and MsetCCA when applied on different electrode montages but also provided the fastest computation time on the test set.

Conclusion

Although MLR method has already demonstrated to have higher performance in comparison with other frequency recognition algorithms, this study showed that in a practical SSVEP-based BCI system with 1 or 2 EEG channels and short-time windows, CFA method outperforms other algorithms. Therefore, it is proposed that CFA algorithm is a promising choice for the expansion of practical SSVEP-based BCI systems.

Feature selection using regularized neighbourhood component analysis to enhance the classification performance of motor imagery signals

In motor imagery (MI) based brain-computer interface (BCI) signal analysis, mu and beta rhythms of electroencephalograms (EEGs) are widely investigated due to their high temporal resolution and capability to define the different movement-related mental tasks separately. However, due to the high dimensions and subject-specific behaviour of EEG features, there is a need for a suitable feature selection algorithm that can select the optimal features to give the best classification performance along with increased computational efficiency. The present study proposes a feature selection algorithm based on neighbourhood component analysis (NCA) with modification of the regularization parameter. In the experiment, time, frequency, and phase features of the EEG are extracted using a dual-tree complex wavelet transform (DTCWT). Afterwards, the proposed algorithm selects the most significant EEG features, and using these selected features, a support vector machine (SVM) classifier performs the classification of MI signals. The proposed algorithm has been validated experimentally on two public BCI datasets (BCI Competition II Dataset III and BCI Competition IV Dataset 2b). The classification performance of the algorithm is quantified by the average accuracy and kappa coefficient, whose values are 80.7% and 0.615 respectively. The performance of the proposed algorithm is compared with standard feature selection methods based on Genetic Algorithm (GA), Principal Component Analysis (PCA), and ReliefF and performs better than these methods. Further, the proposed algorithm selects the lowest number of features and results in increased computational efficiency, which makes it a promising feature selection tool for an MI-based BCI system.

RNN-based longitudinal analysis for diagnosis of Alzheimer's disease

Alzheimer's disease (AD) is an irreversible neurodegenerative disorder with progressive impairment of memory and other mental functions. Magnetic resonance images (MRI) have been widely used as an important imaging modality of brain for AD diagnosis and monitoring the disease progression. The longitudinal analysis of sequential MRIs is important to model and measure the progression of the disease along the time axis for more accurate diagnosis. Most existing methods extracted the features capturing the morphological abnormalities of brain and their longitudinal changes using MRIs and then designed a classifier to discriminate different groups. However, these methods have several limitations. First, since the feature extraction and classifier model are independent, the extracted features may not capture the full characteristics of brain abnormalities related to AD. Second, longitudinal MR images may be missing at some time points for some subjects, which results in difficulties for extraction of consistent features for longitudinal analysis. In this paper, we present a classification framework based on combination of convolutional and recurrent neural networks for longitudinal analysis of structural MR images in AD diagnosis. First, Convolutional Neural Networks (CNN) is constructed to learn the spatial features of MR images for the classification task. After that, recurrent Neural Networks (RNN) with cascaded three bidirectional gated recurrent units (BGRU) layers is constructed on the outputs of CNN at multiple time points for extracting the longitudinal features for AD classification. Instead of independently performing feature extraction and classifier training, the proposed method jointly learns the spatial and longitudinal features and disease classifier, which can achieve optimal performance. In addition, the proposed method can model the longitudinal analysis using RNN from the imaging data at various time points. Our method is evaluated with the longitudinal T1-weighted MR images of 830 participants including 198 AD, 403 mild cognitive impairment (MCI), and 229 normal controls (NC) subjects from Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Experimental results show that the proposed method achieves classification accuracy of 91.33% for AD vs. NC and 71.71% for pMCI vs. sMCI, demonstrating the promising performance for longitudinal MR image analysis.

A Hybrid Proposed Fundus Image Enhancement Framework for Diabetic Retinopathy

Diabetic retinopathy (DR) is a complication of diabetes and is known as visual impairment, and is diagnosed in various ethnicities of the working-age population worldwide. Fundus angiography is a widely applicable modality used by ophthalmologists and computerized applications to detect DR-based clinical features such as microaneurysms (MAs), hemorrhages (HEMs), and exudates (EXs) for early screening of DR. Fundus images are usually acquired using funduscopic cameras in varied light conditions and angles. Therefore, these images are prone to non-uniform illumination, poor contrast, transmission error, low brightness, and noise problems. This paper presents a novel and real-time mechanism of fundus image enhancement used for early grading of diabetic retinopathy, macular degeneration, retinal neoplasms, and choroid disruptions. The proposed system is based on two folds: (i) An RGB fundus image is initially taken and converted into a color appearance module (called lightness and denoted as J) of the CIECAM02 color space model to obtain image information in grayscale with bright light. Afterwards, in step (ii), the achieved J component is processed using a nonlinear contrast enhancement approach to improve the textural and color features of the fundus image without any further extraction steps. To test and evaluate the strength of the proposed technique, several performance and quality parameters—namely peak signal-to-noise ratio (PSNR), contrast-to-noise ratio (CNR), entropy (content information), histograms (intensity variation), and a structure similarity index measure (SSIM)—were applied to 1240 fundus images comprised of two publicly available datasets, DRIVE and MESSIDOR. It was determined from the experiments that the proposed enhancement procedure outperformed histogram-based approaches in terms of contrast, sharpness of fundus features, and brightness. This further revealed that it can be a suitable preprocessing tool for segmentation and classification of DR-related features algorithms.

EEG Signals-Based Longitudinal Control System for a Brain-Controlled Vehicle

Directly using brain signals to drive a vehicle may not only help persons with disabilities to regain driving ability but also provide a new alternative way for healthy people to control a vehicle. In this paper, we propose a new longitudinal control system based on electroencephalogram signals for brain-controlled vehicles (BCVs) by combining a user interface, a brain-computer interface (BCI) system, and a longitudinal control module. Driver-in-the-loop experiments were conducted by using two driving tests (i.e., the destination-approaching and car-following tests) with different subjects under two control conditions, i.e., the brain and manual control conditions. Experimental results show the feasibility of alone using brain signals to continuously perform the longitudinal control of a vehicle at a relatively high speed, at least for some users. This paper not only promotes the development of BCVs but also provides some insights into the research on how to apply BCIs to control other high-speed dynamic systems.

An adaptive CSP filter to investigate user independence in a 3-class MI-BCI paradigm

This paper describes the implementation of a Brain Computer Interface (BCI) scheme using a common spatial patterns (CSP) filter in combination with a Recursive Least Squares (RLS) approach to iteratively update the coefficients of the CSP filter. The proposed adaptive CSP (ACSP) algorithm is made more robust by introducing regularization using Diagonal Loading (DL), and thus will be able to significantly reduce the length of training sessions when introducing new patients to the BCI system. The system is tested on a 4-class multi-limb motor imagery (MI) data set from the BCI competition IV (2a), and a more complex single limb 3-class MI dataset recorded in-house. The latter dataset is produced to mimic an upper limb rehabilitation session, e.g., after stroke. The findings indicate that when extensive calibration data is available, the ACSP performs comparably to the CSP (kappa value of 0.523 and 0.502, respectively, for the 4-class problem); for reduced calibration sessions, the ACSP significantly improved the performance of the system (up to 4-fold). The proposed paradigm proved feasible and the ACSP algorithm seems to enable a user or semi user independent scenario, where the need for long system calibration sessions without feedback is eliminated.