A high-speed brain speller using steady-state visual evoked potentials

An Adaptive SSVEP-Based Brain-Computer Interface to Compensate Fatigue-Induced Decline of Performance in Practical Application

Brain-computer interfaces based on steady-state visual evoked potentials are promising communication systems for people with speech and motor disabilities. However, reliable SSVEP response requires user's attention, which degrades over time due to significant eye-fatigue when low-frequency visual stimuli (5-15 Hz) are used. Previous studies have shown that eye-fatigue can be reduced using high-frequency flickering stimuli (>25 Hz). Here, it is quantitatively demonstrated that the performance of a high-frequency SSVEP BCI decreases over time, but this amount of decrease can be compensated effectively by using two proposed adaptive algorithms. This leaded to a robust alternative communication system for practical applications. The asynchronous spelling system implemented in this study uses a threshold-based version of LASSO algorithm for frequency recognition. In long online experiments, when participants typed a sentence with the BCI system for 16 times, accuracy of the system was close to its maximum along the experiment. However, regression analysis on typing speed of each sentence demonstrated a significant decrease in all 7 subjects ( ) when thresholds obtained from a calibration test were kept fixed over the experiment. In comparison, no significant change in typing speed was observed when the proposed adaptive algorithms were used. The analysis of variances revealed that the average typing speed of the last four sentences when using adaptive relational algorithm (8.7 char/min) was significantly higher than the tolerance-based algorithm (8.1 char/min) and significantly above 6 char/min when the fixed thresholds were used. Therefore, the relational algorithm proposed in this paper could successfully compensate for the effect of fatigue on performance of the SSVEP BCI system.

A High-Speed SSVEP-Based BCI Using Dry EEG Electrodes

A high-speed steady-state visual evoked potentials (SSVEP)-based brain-computer interface (BCI) system using dry EEG electrodes was demonstrated in this study. The dry electrode was fabricated in our laboratory. It was designed as claw-like structure with a diameter of 14 mm, featuring 8 small fingers of 6 mm length and 2 mm diameter. The structure and elasticity can help the fingers pass through the hair and contact the scalp when the electrode is placed on head. The electrode was capable of recording spontaneous EEG and evoked brain activities such as SSVEP with high signal-to-noise ratio. This study implemented a twelve-class SSVEP-based BCI system with eight electrodes embedded in a headband. Subjects also completed a comfort level questionnaire with the dry electrodes. Using a preprocessing algorithm of filter bank analysis (FBA) and a classification algorithm based on task-related component analysis (TRCA), the average classification accuracy of eleven participants was 93.2% using 1-second-long SSVEPs, leading to an average information transfer rate (ITR) of 92.35 bits/min. All subjects did not report obvious discomfort with the dry electrodes. This result represented the highest communication speed in the dry-electrode based BCI systems. The proposed system could provide a comfortable user experience and a stable control method for developing practical BCIs.

SSVEP recognition by modeling brain activity using system identification based on Box-Jenkins model

The steady-state visual evoked potential (SSVEP) based brain-computer interface (BCI) has received increasing attention in recent years. The present study proposes a new method for recognition based on system identification. The method relies on modeling the electroencephalogram (EEG) signals using the Box-Jenkins model. In this approach, the recorded EEG signal is considered as a combination of an SSVEP signal evoked by periodic visual stimulation and a background EEG signal whose components are modeled by a moving average (MA) process and an auto-regressive moving average (ARMA) process, respectively. Then, the target frequency is determined by comparing the modeled SSVEP signals for all stimulation frequencies. The experimental results of the proposed method for recorded EEG signals from five subjects (each subject with four stimulation frequencies) demonstrated a significant improvement in the accuracy of the SSVEP recognition in contrast to canonical correlation analysis, least absolute shrinkage and selection operator, and multivariate linear regression methods. The proposed method exhibits enhanced accuracy especially for short data length and a small number of channels. This superiority suggests that the proposed method is an appropriate choice for the implementation of real-time SSVEP based BCI systems.

Decoding Steady-State Visual Evoked Potentials From Electrocorticography

We report on a unique electrocorticography (ECoG) experiment in which Steady-State Visual Evoked Potentials (SSVEPs) to frequency- and phase-tagged stimuli were recorded from a large subdural grid covering the entire right occipital cortex of a human subject. The paradigm is popular in EEG-based Brain Computer Interfacing where selectable targets are encoded by different frequency- and/or phase-tagged stimuli. We compare the performance of two state-of-the-art SSVEP decoders on both ECoG- and scalp-recorded EEG signals, and show that ECoG-based decoding is more accurate for very short stimulation lengths (i.e., less than 1 s). Furthermore, whereas the accuracy of scalp-EEG decoding benefits from a multi-electrode approach, to address interfering EEG responses and noise, ECoG decoding enjoys only a marginal improvement as even a single electrode, placed over the posterior part of the primary visual cortex, seems to suffice. This study shows, for the first time, that EEG-based SSVEP decoders can in principle be applied to ECoG, and can be expected to yield faster decoding speeds using less electrodes.

A binary motor imagery tasks based brain-computer interface for two-dimensional movement control

OBJECTIVE

Two-dimensional movement control is a popular issue in brain-computer interface (BCI) research and has many applications in the real world. In this paper, we introduce a combined control strategy to a binary class-based BCI system that allows the user to move a cursor in a two-dimensional (2D) plane. Users focus on a single moving vector to control 2D movement instead of controlling vertical and horizontal movement separately.

APPROACH

Five participants took part in a fixed-target experiment and random-target experiment to verify the effectiveness of the combination control strategy under the fixed and random routine conditions. Both experiments were performed in a virtual 2D dimensional environment and visual feedback was provided on the screen.

MAIN RESULTS

The five participants achieved an average hit rate of 98.9% and 99.4% for the fixed-target experiment and the random-target experiment, respectively.

SIGNIFICANCE

The results demonstrate that participants could move the cursor in the 2D plane effectively. The proposed control strategy is based only on a basic two-motor imagery BCI, which enables more people to use it in real-life applications.

Complex sparse spatial filter for decoding mixed frequency and phase coded steady-state visually evoked potentials

BACKGROUND

Mixed frequency and phase coding (FPC) can achieve the significant increase of the number of commands in steady-state visual evoked potential-based brain-computer interface (SSVEP-BCI). However, the inconsistent phases of the SSVEP over channels in a trial and the existence of non-contributing channels due to noise effects can decrease accurate detection of stimulus frequency.

NEW METHOD

We propose a novel command detection method based on a complex sparse spatial filter (CSSF) by solving ℓ₁- and ℓ_2,1-regularization problems for a mixed-coded SSVEP-BCI. In particular, ℓ_2,1-regularization (aka group sparsification) can lead to the rejection of electrodes that are not contributing to the SSVEP detection.

RESULTS

A calibration data based canonical correlation analysis (CCA) and CSSF with ℓ₁- and ℓ_2,1-regularization cases were demonstrated for a 16-target stimuli with eleven subjects. The results of statistical test suggest that the proposed method with ℓ₁- and ℓ_2,1-regularization significantly achieved the highest ITR.

COMPARISON WITH EXISTING METHODS

The proposed approaches do not need any reference signals, automatically select prominent channels, and reduce the computational cost compared to the other mixed frequency-phase coding (FPC)-based BCIs.

CONCLUSIONS

The experimental results suggested that the proposed method can be usable implementing BCI effectively with reduce visual fatigue.

To train or not to train? A survey on training of feature extraction methods for SSVEP-based BCIs

OBJECTIVE

Despite the vast research aimed at improving the performance of steady-state visually evoked potential (SSVEP)-based brain-computer interfaces (BCIs), several limitations exist that restrict the use of such applications for long-term users in the real-world. One of the main challenges has been to reduce training time while maintaining good BCI performance. In view of this challenge, this survey identifies and compares the different training requirements of feature extraction methods for SSVEP-based BCIs.

APPROACH

This paper reviews the various state-of-the-art SSVEP feature extraction methods that have been developed and are most widely used in the literature.

MAIN RESULTS

The main contributions compared to existing reviews are the following: (i) a detailed summary, including a brief mathematical description of each feature extraction algorithm, providing a guide to the basic concepts of the state-of-the-art techniques for SSVEP-based BCIs found in literature; (ii) a categorisation of the training requirements of SSVEP-based methods into three categories, defined as training-free methods, subject-specific and subject-independent training methods; (iii) a comparative review of the training requirements of SSVEP feature extraction methods, providing a reference for future work on SSVEP-based BCIs.

SIGNIFICANCE

This review highlights the strengths and weaknesses of the three categories of SSVEP training methods. Training-free systems are more practical but their performance is limited due to inter-subject variability resulting from the complex EEG activity. Feature extraction methods that incorporate some training data address this issue and in fact have outperformed training-free methods: subject-specific BCIs are tuned to the individual yielding the best performance at the cost of long, tiring training sessions making these methods unsuitable for everyday use; subject-independent BCIs that make use of training data from various subjects offer a good trade-off between training effort and performance, making these BCIs better suited for practical use.

Incorporation of dynamic stopping strategy into the high-speed SSVEP-based BCIs

OBJECTIVE

Electroencephalography (EEG) is a non-linear and non-stationary process, as a result, its features are unstable and often vary in quality across trials, which poses significant challenges to brain-computer interfaces (BCIs). One remedy to this problem is to adaptively collect sufficient EEG evidence using dynamic stopping (DS) strategies. The high-speed steady-state visual evoked potential (SSVEP)-based BCI has experienced tremendous progress in recent years. This study aims to further improve the high-speed SSVEP-based BCI by incorporating the DS strategy.

APPROACH

This study involves the development of two different DS strategies for a high-speed SSVEP-based BCI, which were based on the Bayes estimation and the discriminant analysis, respectively. To evaluate their performance, they were compared with the conventional fixed stopping (FS) strategy using simulated online tests on both our collected data and a public dataset. Two most effective SSVEP recognition methods were used for comparison, including the extended canonical correlation analysis (CCA) and the ensemble task-related component analysis (TRCA).

MAIN RESULTS

The DS strategies achieved significantly higher information transfer rates (ITRs) than the FS strategy for both datasets, improving 9.78% for the Bayes-based DS and 6.7% for the discriminant-based DS. Specifically, the discriminant-based DS strategy using ensemble TRCA performed the best for our collected data, reaching an average ITR of 353.3  ±  67.1 bits min^-1 with a peak of 460 bits min^-1. The Bayes-based DS strategy using ensemble TRCA had the highest ITR for the public dataset, reaching an average of 230.2  ±  65.8 bits min^-1 with a peak of 304.1 bits min^-1.

SIGNIFICANCE

This study demonstrates that the proposed dynamic stopping strategies can further improve the performance of a SSVEP-based BCI, and hold promise for practical applications.

Novel spatial filter for SSVEP-based BCI: A generated reference filter approach

Steady state visual evoked potential (SSVEP)-based brain computer interface (BCI) systems can be realised using only one electrode; however, due to the inter-user and inter-trial differences, the handling of multiple electrode is preferred. This raises the problem of evaluating information from multiple electrode signals. To solve this problem, we developed a novel spatial filtering method (Generated Reference Filter) for SSVEP-based BCIs. In our method an artificial reference signal is generated by a combination of reference electrode signals. Multiple regression analysis (MRA) was used to determine the optimal weight coefficients for signal combination. The filtered signal was obtained by subtraction. The method was tested on a SSVEP dataset and compared with minimum energy combination and common reference methods, namely the surface Laplacian technique and common average referencing. The newly developed method provided more effective filtering and therefore higher SSVEP detection accuracy was obtained. It was also more robust against subject-to-subject and trial-to-trial variability as the artificial reference signal was recalculated for each detection round. No special preparation is required, and the method is easy to implement. These experimental results indicate that the proposed method can be used confidently with SSVEP-based BCI systems.

Control of a 7-DOF Robotic Arm System With an SSVEP-Based BCI

Although robot technology has been successfully used to empower people who suffer from motor disabilities to increase their interaction with their physical environment, it remains a challenge for individuals with severe motor impairment, who do not have the motor control ability to move robots or prosthetic devices by manual control. In this study, to mitigate this issue, a noninvasive brain-computer interface (BCI)-based robotic arm control system using gaze based steady-state visual evoked potential (SSVEP) was designed and implemented using a portable wireless electroencephalogram (EEG) system. A 15-target SSVEP-based BCI using a filter bank canonical correlation analysis (FBCCA) method allowed users to directly control the robotic arm without system calibration. The online results from 12 healthy subjects indicated that a command for the proposed brain-controlled robot system could be selected from 15 possible choices in 4[Formula: see text]s (i.e. 2[Formula: see text]s for visual stimulation and 2[Formula: see text]s for gaze shifting) with an average accuracy of 92.78%, resulting in a 15 commands/min transfer rate. Furthermore, all subjects (even naive users) were able to successfully complete the entire move-grasp-lift task without user training. These results demonstrated an SSVEP-based BCI could provide accurate and efficient high-level control of a robotic arm, showing the feasibility of a BCI-based robotic arm control system for hand-assistance.

A dynamic stopping method for improving performance of steady-state visual evoked potential based brain-computer interfaces

The performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) has been drastically improved in the past few years. In conventional SSVEP-based BCIs, the speed of a selection is fixed towards high performance based on preliminary offline analysis. However, due to inter-trial variability, the optimal selection time to achieve sufficient accuracy is different for each trial. To optimize the performance of SSVEP-based BCIs, this study proposed a dynamic stopping method that can adaptively determine a selection time in each trial by applying a threshold to the probability of detecting a target. A 12-class SSVEP dataset recorded from 10 subjects was used to evaluate the performance of the proposed method. Compared to the conventional method with a fixed selection time towards the highest accuracy, the proposed method could significantly reduce the averaged selection time (0.84±0.39 s vs. 1.44±0.63 s, p<;0.05) with comparable accuracy (99.44±1.57 % vs. 99.55±1.22 %). As a result, the simulated online information transfer rate (ITR) with the dynamic stopping method achieved a significant improvement compared to the conventional method (125.30±21.55 bits/min vs. 92.75±23.77 bits/min). These results suggest that the proposed dynamic stopping method is effective for improving the performance of SSVEP-based BCI systems.

Accurate Decoding of Short, Phase-Encoded SSVEPs

Four novel EEG signal features for discriminating phase-coded steady-state visual evoked potentials (SSVEPs) are presented, and their performance in view of target selection in an SSVEP-based brain-computer interfacing (BCI) is assessed. The novel features are based on phase estimation and correlations between target responses. The targets are decoded from the feature scores using the least squares support vector machine (LS-SVM) classifier, and it is shown that some of the proposed features compete with state-of-the-art classifiers when using short (0.5 s) EEG recordings in a binary classification setting.

Evaluating the Influence of Chromatic and Luminance Stimuli on SSVEPs from Behind-the-Ears and Occipital Areas

This work presents a study of chromatic and luminance stimuli in low-, medium-, and high-frequency stimulation to evoke steady-state visual evoked potential (SSVEP) in the behind-the-ears area. Twelve healthy subjects participated in this study. The electroencephalogram (EEG) was measured on occipital (Oz) and left and right temporal (TP9 and TP10) areas. The SSVEP was evaluated in terms of amplitude, signal-to-noise ratio (SNR), and detection accuracy using power spectral density analysis (PSDA), canonical correlation analysis (CCA), and temporally local multivariate synchronization index (TMSI) methods. It was found that stimuli based on suitable color and luminance elicited stronger SSVEP in the behind-the-ears area, and that the response of the SSVEP was related to the flickering frequency and the color of the stimuli. Thus, green-red stimulus elicited the highest SSVEP in medium-frequency range, and green-blue stimulus elicited the highest SSVEP in high-frequency range, reaching detection accuracy rates higher than 80%. These findings will aid in the development of more comfortable, accurate and stable BCIs with electrodes positioned on the behind-the-ears (hairless) areas.

Effects of Distracting Task with Different Mental Workload on Steady-State Visual Evoked Potential Based Brain Computer Interfaces-an Offline Study

Brain-computer interfaces (BCIs), independent of the brain's normal output pathways, are attracting an increasing amount of attention as devices that extract neural information. As a typical type of BCI system, the steady-state visual evoked potential (SSVEP)-based BCIs possess a high signal-to-noise ratio and information transfer rate. However, the current high speed SSVEP-BCIs were implemented with subjects concentrating on stimuli, and intentionally avoided additional tasks as distractors. This paper aimed to investigate how a distracting simultaneous task, a verbal n-back task with different mental workload, would affect the performance of SSVEP-BCI. The results from fifteen subjects revealed that the recognition accuracy of SSVEP-BCI was significantly impaired by the distracting task, especially under a high mental workload. The average classification accuracy across all subjects dropped by 8.67% at most from 1- to 4-back, and there was a significant negative correlation (maximum r = −0.48, p < 0.001) between accuracy and subjective mental workload evaluation of the distracting task. This study suggests a potential hindrance for the SSVEP-BCI daily use, and then improvements should be investigated in the future studies.

Enhancing Detection of SSVEPs for a High-Speed Brain Speller Using Task-Related Component Analysis

OBJECTIVE

This study proposes and evaluates a novel data-driven spatial filtering approach for enhancing steady-state visual evoked potentials (SSVEPs) detection toward a high-speed brain-computer interface (BCI) speller.

METHODS

Task-related component analysis (TRCA), which can enhance reproducibility of SSVEPs across multiple trials, was employed to improve the signal-to-noise ratio (SNR) of SSVEP signals by removing background electroencephalographic (EEG) activities. An ensemble method was further developed to integrate TRCA filters corresponding to multiple stimulation frequencies. This study conducted a comparison of BCI performance between the proposed TRCA-based method and an extended canonical correlation analysis (CCA)-based method using a 40-class SSVEP dataset recorded from 12 subjects. An online BCI speller was further implemented using a cue-guided target selection task with 20 subjects and a free-spelling task with 10 of the subjects.

RESULTS

The offline comparison results indicate that the proposed TRCA-based approach can significantly improve the classification accuracy compared with the extended CCA-based method. Furthermore, the online BCI speller achieved averaged information transfer rates (ITRs) of 325.33 ± 38.17 bits/min with the cue-guided task and 198.67 ± 50.48 bits/min with the free-spelling task.

CONCLUSION

This study validated the efficiency of the proposed TRCA-based method in implementing a high-speed SSVEP-based BCI.

SIGNIFICANCE

The high-speed SSVEP-based BCIs using the TRCA method have great potential for various applications in communication and control.

An Efficient Asynchronous High-Frequency Steady-State Visual Evoked Potential-Based Brain-Computer Interface speller: The Problem of Individual Differences

Background

Brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEPs) provide high rates of accuracy and information transfer rate, but need user's attention to flickering visual stimuli. This quickly leads to eye-fatigue when the flickering frequency is in the low-frequency range. High-frequency flickering stimuli (>30 Hz) have been proposed with significantly lower eye-fatigue. However, SSVEP responses in this frequency range are remarkably weaker, leading to doubts about usability of high-frequency stimuli to develop efficient BCI systems. The purpose of this study was to evaluate if a practical SSVEP Speller can be developed with Repetitive Visual Stimuli in the high-frequency range.

Methods

An asynchronous high-frequency (35-40 Hz) speller for typing in Persian language was developed using five flickering visual stimuli. Least absolute shrinkage and selection operator algorithm with two user-calibrated thresholds was used to detect the user's selections. A total of 14 volunteers evaluated the system in an ordinary office environment to type 9 sentences consist of 81 characters with a multistage virtual keyboard.

Results

Despite very high performance of 6.9 chars/min overall typing speed, average accuracy of 98.3%, and information transfer rate of 64.9 bpm for eight of the participants, the other six participants had serious difficulty in spelling with the system and could not complete the typing experiment.

Conclusions

The results of this study in accordance with some previous studies suggest that high rate of illiteracy in high-frequency SSVEP-based BCI systems may be a major burden for their practical application.

Spatiotemporal Beamforming: A Transparent and Unified Decoding Approach to Synchronous Visual Brain-Computer Interfacing

Brain-Computer Interfaces (BCIs) decode brain activity with the aim to establish a direct communication channel with an external device. Albeit they have been hailed to (re-)establish communication in persons suffering from severe motor- and/or communication disabilities, only recently BCI applications have been challenging other assistive technologies. Owing to their considerably increased performance and the advent of affordable technological solutions, BCI technology is expected to trigger a paradigm shift not only in assistive technology but also in the way we will interface with technology. However, the flipside of the quest for accuracy and speed is most evident in EEG-based visual BCI where it has led to a gamut of increasingly complex classifiers, tailored to the needs of specific stimulation paradigms and use contexts. In this contribution, we argue that spatiotemporal beamforming can serve several synchronous visual BCI paradigms. We demonstrate this for three popular visual paradigms even without attempting to optimizing their electrode sets. For each selectable target, a spatiotemporal beamformer is applied to assess whether the corresponding signal-of-interest is present in the preprocessed multichannel EEG signals. The target with the highest beamformer output is then selected by the decoder (maximum selection). In addition to this simple selection rule, we also investigated whether interactions between beamformer outputs could be employed to increase accuracy by combining the outputs for all targets into a feature vector and applying three common classification algorithms. The results show that the accuracy of spatiotemporal beamforming with maximum selection is at par with that of the classification algorithms and interactions between beamformer outputs do not further improve that accuracy.

Code-modulated visual evoked potentials using fast stimulus presentation and spatiotemporal beamformer decoding

When encoding visual targets using various lagged versions of a pseudorandom binary sequence of luminance changes, the EEG signal recorded over the viewer’s occipital pole exhibits so-called code-modulated visual evoked potentials (cVEPs), the phase lags of which can be tied to these targets. The cVEP paradigm has enjoyed interest in the brain-computer interfacing (BCI) community for the reported high information transfer rates (ITR, in bits/min). In this study, we introduce a novel decoding algorithm based on spatiotemporal beamforming, and show that this algorithm is able to accurately identify the gazed target. Especially for a small number of repetitions of the coding sequence, our beamforming approach significantly outperforms an optimised support vector machine (SVM)-based classifier, which is considered state-of-the-art in cVEP-based BCI. In addition to the traditional 60 Hz stimulus presentation rate for the coding sequence, we also explore the 120 Hz rate, and show that the latter enables faster communication, with a maximal median ITR of 172.87 bits/min. Finally, we also report on a transition effect in the EEG signal following the onset of the stimulus sequence, and recommend to exclude the first 150 ms of the trials from decoding when relying on a single presentation of the stimulus sequence.

A frequency recognition method based on multitaper spectral analysis and SNR estimation for SSVEP-based brain-computer interface

Over the past several years, steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have attracted wide attention in the field of BCIs research due to high information transfer rate, little user training, and applicability to the majority. In conventional recognition methods for training-free SSVEP-based BCIs, the energy difference between the frequencies of electroencephalogram (EEG) background noise is usually ignored, therefore, there is a significant variance among the recognition accuracy of different stimulus frequencies. In order to improve the performance of training-free SSVEP-based BCIs system and balance the accuracy of recognition between different stimulus frequencies, a recognition method based on multitaper spectral analysis and signal-to-noise ratio estimation (MTSA-SNR) is proposed in this paper. A 40-class SSVEP public benchmark SSVEP dataset recorded from 35 subjects was used to evaluate the performance of the proposed method. Under the condition of 2.25s data length, the accuracy of the three methods were 81.1% (MTSA-SNR), 74.5% (canonical correlation analysis, CCA) and 73.4% (multivariate synchronization index, MSI), and the corresponding ITRs were 101 bits/min (MTSA-SNR), 89 bits/min (CCA), 87 bits/min (MSI). In the low frequency range (8-9.8Hz), the average recognition accuracy of the three methods is 82.9% (MTSA-SNR), 82.0% (CCA), 83.3% (MSI). The average accuracy of the three methods was 78.6% (MTSA-SNR), 64.9% (CCA) and 61.8% (MSI) in the high frequency range (14-15.8Hz). According to the results, the proposed method can effectively improve the performance of training-free SSVEP-based BCI system, and balance the recognition accuracy between different stimulation frequencies.

Independent component analysis-based spatial filtering improves template-based SSVEP detection

This study proposes a new algorithm to detect steady-state visual evoked potentials (SSVEPs) based on a template-matching approach combined with independent component analysis (ICA)-based spatial filtering. In recent studies, the effectiveness of the template-based SSVEP detection has been demonstrated in a high-speed brain-computer interface (BCI). Since SSVEPs can be considered as electroencephalogram (EEG) signals generated from underlying brain sources independent from other activities and artifacts, ICA has great potential to enhance the signal-to-noise ratio (SNR) of SSVEPs by separating them from artifacts. This study proposes to apply the ICA-based spatial filters to test data and individual templates obtained by averaging training trials, and then to use the correlation coefficients between the filtered data and templates as features for SSVEP classification. This study applied the proposed method to a 40-class SSVEP dataset to evaluate its classification accuracy against those obtained by conventional canonical correlation analysis (CCA)- and extended CCA-based methods. The study results showed that the ICA-based method outperformed the other methods in terms of the classification accuracy. Furthermore, its computational time was comparable to the CCA-based method, and was much shorter than that of the extended CCA-based method.

Detecting Glaucoma With a Portable Brain-Computer Interface for Objective Assessment of Visual Function Loss

Importance

The current assessment of visual field loss in diseases such as glaucoma is affected by the subjectivity of patient responses and the lack of portability of standard perimeters.

Objective

To describe the development and initial validation of a portable brain-computer interface (BCI) for objectively assessing visual function loss.

Design, Setting, and Participants

This case-control study involved 62 eyes of 33 patients with glaucoma and 30 eyes of 17 healthy participants. Glaucoma was diagnosed based on a masked grading of optic disc stereophotographs. All participants underwent testing with a BCI device and standard automated perimetry (SAP) within 3 months. The BCI device integrates wearable, wireless, dry electroencephalogram and electrooculogram systems and a cellphone-based head-mounted display to enable the detection of multifocal steady state visual-evoked potentials associated with visual field stimulation. The performances of global and sectoral multifocal steady state visual-evoked potentials metrics to discriminate glaucomatous from healthy eyes were compared with global and sectoral SAP parameters. The repeatability of the BCI device measurements was assessed by collecting results of repeated testing in 20 eyes of 10 participants with glaucoma for 3 sessions of measurements separated by weekly intervals.

Main Outcomes and Measures

Receiver operating characteristic curves summarizing diagnostic accuracy. Intraclass correlation coefficients and coefficients of variation for assessing repeatability.

Results

Among the 33 participants with glaucoma, 19 (58%) were white, 12 (36%) were black, and 2 (6%) were Asian, while among the 17 participants with healthy eyes, 9 (53%) were white, 8 (47%) were black, and none were Asian. The receiver operating characteristic curve area for the global BCI multifocal steady state visual-evoked potentials parameter was 0.92 (95% CI, 0.86-0.96), which was larger than for SAP mean deviation (area under the curve, 0.81; 95% CI, 0.72-0.90), SAP mean sensitivity (area under the curve, 0.80; 95% CI, 0.69-0.88; P = .03), and SAP pattern standard deviation (area under the curve, 0.77; 95% CI, 0.66-0.87; P = .01). No statistically significant differences were seen for the sectoral measurements between the BCI and SAP. Intraclass coefficients for global and sectoral parameters ranged from 0.74 to 0.92, and mean coefficients of variation ranged from 3.03% to 7.45%.

Conclusions and Relevance

The BCI device may be useful for assessing the electrical brain responses associated with visual field stimulation. The device discriminated eyes with glaucomatous neuropathy from healthy eyes in a clinically based setting. Further studies should investigate the feasibility of the BCI device for home-based testing as well as for detecting visual function loss over time.

Phase-based SSVEPs for real-time control of a motorised bed

Brain-computer interface (BCI) systems have emerged as an augmentative technology that can provide a promising solution for individuals with motor dysfunctions and for the elderly who are experiencing muscle weakness. Steady-state visually evoked potentials (SSVEPs) are widely adopted in BCI systems due to their high speed and accuracy when compared to other BCI paradigms. In this paper, we apply combined magnitude and phase features for class discrimination in a real-time SSVEP-based BCI platform. In the proposed real-time system users gain control of a motorised bed system with seven motion commands and an idle state. Experimental results amongst eight participants demonstrate that the proposed real-time BCI system can successfully discriminate between different SSVEP signals achieving high information transfer rates (ITR) of 82.73 bits/min. The attractive features of the proposed system include noninvasive recording, simple electrode configuration, excellent BCI response and minimal training requirements.

A SSVEP Stimuli Encoding Method Using Trinary Frequency-Shift Keying Encoded SSVEP (TFSK-SSVEP)

SSVEP is a kind of BCI technology with advantage of high information transfer rate. However, due to its nature, frequencies could be used as stimuli are scarce. To solve such problem, a stimuli encoding method which encodes SSVEP signal using Frequency Shift–Keying (FSK) method is developed. In this method, each stimulus is controlled by a FSK signal which contains three different frequencies that represent “Bit 0,” “Bit 1” and “Bit 2” respectively. Different to common BFSK in digital communication, “Bit 0” and “Bit 1” composited the unique identifier of stimuli in binary bit stream form, while “Bit 2” indicates the ending of a stimuli encoding. EEG signal is acquired on channel Oz, O1, O2, Pz, P3, and P4, using ADS1299 at the sample rate of 250 SPS. Before original EEG signal is quadrature demodulated, it is detrended and then band-pass filtered using FFT-based FIR filtering to remove interference. Valid peak of the processed signal is acquired by calculating its derivative and converted into bit stream using window method. Theoretically, this coding method could implement at least 2ⁿ⁻¹ (n is the length of bit command) stimulus while keeping the ITR the same. This method is suitable to implement stimuli on a monitor and where the frequency and phase could be used to code stimuli is limited as well as implementing portable BCI devices which is not capable of performing complex calculations.

Decoding of responses to mixed frequency and phase coded visual stimuli using multiset canonical correlation analysis

Brain-computer interfacing (BCI) based on steady-state visual evoked potentials (SSVEPs) is one of the most practical BCIs because of its high recognition accuracies and little training of a user. Mixed frequency and phase coding which can implement a number of commands and achieve a high information transfer rate (ITR) has recently been gaining much attention. In order to implement mixed-coded SSVEP-BCI as a reliable interface, it is important to detect commands fast and accurately. This paper presents a novel method to recognize mixed-coded SSVEPs which achieves high performance. The method employs multiset canonical correlation analysis to obtain spatial filters which enhance SSVEP components. An experiment with a mixed-coded SSVEP-BCI was conducted to evaluate performance of the proposed method compared with the previous work. The experimental results showed that the proposed method achieved significantly higher command recognition accuracy and ITR than the state-of-the-art.

Applicability of SSVEP-based brain-computer interfaces for robot navigation in real environments

Brain-computer interfaces have been extensively studied and used in order to aid patients suffering from neuromuscular diseases to communicate and control the surrounding environment. Steady-state visual evoked potentials (SSVEP) constitute a very popular BCI stimulation protocol, due to their efficiency and quick response time. In this study, we developed a SSVEP-based BCI along with a low-cost custom radio-controlled robot-car providing live video feedback from a wireless camera mounted on the robot, serving as our testbed. Our goal was to quantitatively assess the applicability of SSVEPs in real time navigation in realistic environments using a pragmatic approach. In order to assess the additional fatigue that the camera video introduces, we designed a two-session experiment, a control one with no connection to the robot and, thus, no live camera feed, and a realistic one where the users could navigate the robot with the provision of front scenes, captured from the camera. Statistical tests revealed a significant decrease of the accuracy of the system during the realistic session that included live video, in comparison with the session that did not. The results suggest that the moving camera image sequence introduces an extra level of fatigue and/or distraction to the users.

Polychromatic SSVEP stimuli with subtle flickering adapted to brain-display interactions

OBJECTIVE

Interactive displays armed with natural user interfaces (NUIs) will likely lead the next breakthrough in consumer electronics, and brain-computer interfaces (BCIs) are often regarded as the ultimate NUI-enabling machines to respond to human emotions and mental states. Steady-state visual evoked potentials (SSVEPs) are a commonly used BCI modality due to the ease of detection and high information transfer rates. However, the presence of flickering stimuli may cause user discomfort and can even induce migraines and seizures. With the aim of designing visual stimuli that can be embedded into video images, this study developed a novel approach to induce detectable SSVEPs using a composition of red/green/blue flickering lights.

APPROACH

Based on the opponent theory of colour vision, this study used 32 Hz/40 Hz rectangular red-green or red-blue LED light pulses with a 50% duty cycle, balanced/equal luminance and 0°/180° phase shifts as the stimulating light sources and tested their efficacy in producing SSVEP responses with high signal-to-noise ratios (SNRs) while reducing the perceived flickering sensation.

MAIN RESULTS

The empirical results from ten healthy subjects showed that dual-colour lights flickering at 32 Hz/40 Hz with a 50% duty cycle and 180° phase shift achieved a greater than 90% detection accuracy with little or no flickering sensation.

SIGNIFICANCE

As a first step in developing an embedded SSVEP stimulus in commercial displays, this study provides a foundation for developing a combination of three primary colour flickering backlights with adjustable luminance proportions to create a subtle flickering polychromatic light that can elicit SSVEPs at the basic flickering frequency.

Decoding of top-down cognitive processing for SSVEP-controlled BMI

We present a fast and accurate non-invasive brain-machine interface (BMI) based on demodulating steady-state visual evoked potentials (SSVEPs) in electroencephalography (EEG). Our study reports an SSVEP-BMI that, for the first time, decodes primarily based on top-down and not bottom-up visual information processing. The experimental setup presents a grid-shaped flickering line array that the participants observe while intentionally attending to a subset of flickering lines representing the shape of a letter. While the flickering pixels stimulate the participant’s visual cortex uniformly with equal probability, the participant’s intention groups the strokes and thus perceives a ‘letter Gestalt’. We observed decoding accuracy of 35.81% (up to 65.83%) with a regularized linear discriminant analysis; on average 2.05-fold, and up to 3.77-fold greater than chance levels in multi-class classification. Compared to the EEG signals, an electrooculogram (EOG) did not significantly contribute to decoding accuracies. Further analysis reveals that the top-down SSVEP paradigm shows the most focalised activation pattern around occipital visual areas; Granger causality analysis consistently revealed prefrontal top-down control over early visual processing. Taken together, the present paradigm provides the first neurophysiological evidence for the top-down SSVEP BMI paradigm, which potentially enables multi-class intentional control of EEG-BMIs without using gaze-shifting.

Independent Component Decomposition of Human Somatosensory Evoked Potentials Recorded by Micro-Electrocorticography

High-density surface microelectrodes for electrocorticography (ECoG) have become more common in recent years for recording electrical signals from the cortex. With an acceptable invasiveness/signal fidelity trade-off and high spatial resolution, micro-ECoG is a promising tool to resolve fine task-related spatial-temporal dynamics. However, volume conduction - not a negligible phenomenon - is likely to frustrate efforts to obtain reliable and resolved signals from a sub-millimeter electrode array. To address this issue, we performed an independent component analysis (ICA) on micro-ECoG recordings of somatosensory-evoked potentials (SEPs) elicited by median nerve stimulation in three human patients undergoing brain surgery for tumor resection. Using well-described cortical responses in SEPs, we were able to validate our results showing that the array could segregate different functional units possessing unique, highly localized spatial distributions. The representation of signals through the root-mean-square (rms) maps and the signal-to-noise ratio (SNR) analysis emphasizes the advantages of adopting a source analysis approach on micro-ECoG recordings in order to obtain a clear picture of cortical activity. The implications are twofold: while on one side ICA may be used as a spatial-temporal filter extracting micro-signal components relevant to tasks for brain-computer interface (BCI) applications, it could also be adopted to accurately identify the sites of nonfunctional regions for clinical purposes.

Frequency- and Phase Encoded SSVEP Using Spatiotemporal Beamforming

In brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEPs) the number of selectable targets is rather limited when each target has its own stimulation frequency. One way to remedy this is by combining frequency- with phase encoding. We introduce a new multivariate spatiotemporal filter, based on Linearly Constrained Minimum Variance (LCMV) beamforming, for discriminating between frequency-phase encoded targets more accurately, even when using short signal lengths than with (extended) Canonical Correlation Analysis (CCA), which is traditionally posited for this stimulation paradigm.

Developing a one-channel BCI system using a dry claw-like electrode

An eight-class SSVEP-based BCI system was designed and demonstrated in this study. To minimize the complexity of the traditional equipment and operation, only one work electrode was used. The work electrode was fabricated in our laboratory and designed as a claw-like structure with a diameter of 15 mm, featuring 8 small fingers of 4mm length and 2 mm diameter, and the weight was only 0.1g. The structure and elasticity can help the fingers pass through the hair and contact the scalp when placed on head. The electrode was capable to collect evoked brain activities such as steady-state visual evoked potentials (SSVEPs). This study showed that although the amplitude and SNR of SSVEPs obtained from a dry claw electrode was relatively lower than that from a wet electrode, the difference was not significant. This study further implemented an eight-class SSVEP-based BCI system using a dry claw-like electrode. Three subjects participated in the experiment. Using infinite impulse response (IIR) filtering and a simplified threshold method based on fast Fourier transform (FFT), the average accuracy of the three participants was 89.3% using 4 sec-long SSVEPs, leading to an average information transfer rate (ITR) of 26.5 bits/min. The results suggested the ability of using a dry claw-like electrode to perform practical BCI applications.

BCI in practice

Brain-computer interfaces are systems that use signals recorded from the brain to enable communication and control applications for individuals who have impaired function. This technology has developed to the point that it is now being used by individuals who can actually benefit from it. However, there are several outstanding issues that prevent widespread use. These include the ease of obtaining high-quality recordings by home users, the speed, and accuracy of current devices and adapting applications to the needs of the user. In this chapter, we discuss some of these unsolved issues.

Recursive Bayesian Coding for BCIs

Brain-Computer Interfaces (BCIs) seek to infer some task symbol, a task relevant instruction, from brain symbols, classifiable physiological states. For example, in a motor imagery robot control task a user would indicate their choice from a dictionary of task symbols (rotate arm left, grasp, etc.) by selecting from a smaller dictionary of brain symbols (imagined left or right hand movements). We examine how a BCI infers a task symbol using selections of brain symbols. We offer a recursive Bayesian decision framework which incorporates context prior distributions (e.g., language model priors in spelling applications), accounts for varying brain symbol accuracy and is robust to single brain symbol query errors. This framework is paired with Maximum Mutual Information (MMI) coding which maximizes a generalization of ITR. Both are applicable to any discrete task and brain phenomena (e.g., P300, SSVEP, MI). To demonstrate the efficacy of our approach we perform SSVEP "Shuffle" Speller experiments and compare our recursive coding scheme with traditional decision tree methods including Huffman coding. MMI coding leverages the asymmetry of the classifier's mistakes across a particular user's SSVEP responses; in doing so it offers a 33% increase in letter accuracy though it is 13% slower in our experiment.

Asynchronous Detection of Trials Onset from Raw EEG Signals

Clinical processing of event-related potentials (ERPs) requires a precise synchrony between the stimulation and the acquisition units that are guaranteed by means of a physical link between them. This precise synchrony is needed since temporal misalignments during trial averaging can lead to high deviations of peak times, thus causing error in diagnosis or inefficiency in classification in brain-computer interfaces (BCIs). Out of the laboratory, mobile EEG systems and BCI headsets are not provided with the physical link, thus being inadequate for acquisition of ERPs. In this study, we propose a method for the asynchronous detection of trials onset from raw EEG without physical links. We validate it with a BCI application based on the dichotic listening task. The user goal was to attend the cued auditory message and to report three keywords contained in it while ignoring the other message. The BCI goal was to detect the attended message from the analysis of auditory ERPs. The rate of successful onset detection in both synchronous (using the real onset) and asynchronous (blind detection of trial onset from raw EEG) was 73% with a synchronization error of less than 1[Formula: see text]ms. The level of synchronization provided by this proposal would allow home-based acquisition of ERPs with low cost BCI headsets and any media player unit without physical links between them.

Stimulus Specificity of Brain-Computer Interfaces Based on Code Modulation Visual Evoked Potentials

A brain-computer interface (BCI) based on code modulated visual evoked potentials (c-VEP) is among the fastest BCIs that have ever been reported, but it has not yet been given a thorough study. In this study, a pseudorandom binary M sequence and its time lag sequences are utilized for modulation of different stimuli and template matching is adopted as the method for target recognition. Five experiments were devised to investigate the effect of stimulus specificity on target recognition and we made an effort to find the optimal stimulus parameters for size, color and proximity of the stimuli, length of modulation sequence and its lag between two adjacent stimuli. By changing the values of these parameters and measuring classification accuracy of the c-VEP BCI, an optimal value of each parameter can be attained. Experimental results of ten subjects showed that stimulus size of visual angle 3.8°, white, spatial proximity of visual angle 4.8° center to center apart, modulation sequence of length 63 bits and the lag of 4 bits between adjacent stimuli yield individually superior performance. These findings provide a basis for determining stimulus presentation of a high-performance c-VEP based BCI system.

An Online Brain-Computer Interface Based on SSVEPs Measured From Non-Hair-Bearing Areas

Steady state visual evoked potential (SSVEP)-based brain-computer interface (BCI) has gained a lot of attention due to its robustness and high information transfer rate (ITR). However, transitioning well-controlled laboratory-oriented BCI demonstrations to real-world applications poses severe challenges for this exciting field. For instance, conducting BCI experiments usually requires skilled technicians to abrade the area of skin underneath each electrode and apply an electrolytic gel or paste to acquire high-quality SSVEPs from hair-covered areas. Our previous proof-of-concept study has proposed an alternative approach that employed electroencephalographic signals collected from easily accessible non-hair-bearing areas including neck, behind the ears, and face to realize an SSVEP-based BCI. The study results showed that, with proper electrode placements and advanced signal-processing algorithms, the SSVEPs measured from non-hair-bearing areas in off-line SSVEP experiments could achieve comparable SNR to that obtained from the hair-bearing occipital areas. This study extended the previous work to systematically investigate the costs and benefits of non-hair SSVEPs. Furthermore, this study developed and evaluated an online BCI system based solely on non-hair EEG signals. A 12-target identification task was employed to quantitatively assess the performance of the online SSVEP-based BCI system. All subjects successfully completed the tasks using non-hair SSVEPs with 84.08 ± 15.60% averaged accuracy and 30.21 ± 10.61 bits/min averaged ITR. The empirical results of this study demonstrated the practicality of implementing an SSVEP-based BCI based on signals from non-hair-bearing areas, significantly improving the feasibility and practicality of real-world BCIs.

Incorporation of Inter-Subject Information to Improve the Accuracy of Subject-Specific P300 Classifiers

Although the inter-subject information has been demonstrated to be effective for a rapid calibration of the P300-based brain–computer interface (BCI), it has never been comprehensively tested to find if the incorporation of heterogeneous data could enhance the accuracy. This study aims to improve the subject-specific P300 classifier by adding other subject’s data. A classifier calibration strategy, weighted ensemble learning generic information (WELGI), was developed, in which elementary classifiers were constructed by using both the intra- and inter-subject information and then integrated into a strong classifier with a weight assessment. 55 subjects were recruited to spell 20 characters offline using the conventional P300-based BCI, i.e. the P300-speller. Four different metrics, the P300 accuracy and precision, the round accuracy, and the character accuracy, were performed for a comprehensive investigation. The results revealed that the classifier constructed on the training dataset in combination with adding other subject’s data was significantly superior to that without the inter-subject information. Therefore, the WELGI is an effective classifier calibration strategy which uses the inter-subject information to improve the accuracy of subject-specific P300 classifiers, and could also be applied to other BCI paradigms.

Reduction of Delay in Detecting Initial Dips from Functional Near-Infrared Spectroscopy Signals Using Vector-Based Phase Analysis

In this paper, we present a systematic method to reduce the time lag in detecting initial dips using a vector-based phase diagram and an autoregressive moving average with exogenous signals (ARMAX) model-based [Formula: see text]-step-ahead prediction algorithm. With functional near-infrared spectroscopy (fNIRS), signals related to mental arithmetic and right-hand clenching are acquired from the prefrontal and left primary motor cortices, respectively. The interrelationship between oxygenated hemoglobin, deoxygenated hemoglobin, total hemoglobin and cerebral oxygen exchange are related to initial dips. Specifically, a threshold value from the resting state hemodynamics is incorporated, as a decision criterion, into the vector-based phase diagram to determine the occurrence of initial dips. To further reduce the time lag, a [Formula: see text]-step-ahead prediction method is applied to predict the occurrence of the dips. A combination of the threshold criterion and the prediction method resulted in the delay time of about 0.9[Formula: see text]s. The results demonstrate that rapid detection of initial dip is possible and therefore can be used for real-time brain–computer interfacing.

EEG-Based Detection of Starting and Stopping During Gait Cycle

Walking is for humans an essential task in our daily life. However, there is a huge (and growing) number of people who have this ability diminished or are not able to walk due to motor disabilities. In this paper, a system to detect the start and the stop of the gait through electroencephalographic signals has been developed. The system has been designed in order to be applied in the future to control a lower limb exoskeleton to help stroke or spinal cord injured patients during the gait. The brain-machine interface (BMI) training has been optimized through a preliminary analysis using the brain information recorded during the experiments performed by three healthy subjects. Afterward, the system has been verified by other four healthy subjects and three patients in a real-time test. In both preliminary optimization analysis and real-time tests, the results obtained are very similar. The true positive rates are [Formula: see text] and [Formula: see text] respectively. Regarding the false positive per minute, the values are also very similar, decreasing from 2.66 in preliminary tests to 1.90 in real-time. Finally, the average latencies in the detection of the movement intentions are 794 and 798[Formula: see text]ms, preliminary and real-time tests respectively.