Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication

Alpha and SSVEP power outperform gamma power in capturing attentional modulation in human EEG

Attention typically reduces power in the alpha (8-12 Hz) band and increases power in gamma (>30 Hz) band in brain signals, as reported in macaque local field potential (LFP) and human electro/magneto-encephalogram (EEG/MEG) studies. In addition, EEG studies often use flickering stimuli that produce a specific measure called steady-state-visually-evoked-potential (SSVEP), whose power also increases with attention. However, effectiveness of these neural measures in capturing attentional modulation is unknown since stimuli and task paradigms vary widely across studies. In a recent macaque study, attentional modulation was more salient in the gamma band of the LFP, compared to alpha or SSVEP. To compare this with human EEG, we designed an orientation change detection task where we presented both static and counterphasing stimuli of matched difficulty levels to 26 subjects and compared attentional modulation of various measures under similar conditions. We report two main results. First, attentional modulation was comparable for SSVEP and alpha. Second, non-foveal stimuli produced weak gamma despite various stimulus optimizations and showed negligible attentional modulation although full-screen gratings showed robust gamma activity. Our results are useful for brain-machine-interfacing studies where suitable features are used for decoding attention, and also provide clues about spatial scales of neural mechanisms underlying attention.

Application of rapid invisible frequency tagging for brain computer interfaces

BACKGROUND

Brain-computer interfaces (BCI) based on steady-state visual evoked potentials (SSVEPs/SSVEFs) are among the most commonly used BCI systems. They require participants to covertly attend to visual objects flickering at specified frequencies. The attended location is decoded online by analysing the power of neuronal responses at the flicker frequency.

NEW METHOD

We implemented a novel rapid invisible frequency-tagging technique, utilizing a state-of-the-art projector with refresh rates of up to 1440 Hz. We flickered the luminance of visual objects at 56 and 60 Hz, which was invisible to participants but produced strong neuronal responses measurable with magnetoencephalography (MEG). The direction of covert attention, decoded from frequency-tagging responses, was used to control an online BCI PONG game.

RESULTS

Our results show that seven out of eight participants were able to play the pong game controlled by the frequency-tagging signal, with average accuracies exceeding 60 %. Importantly, participants were able to modulate the power of the frequency-tagging response within a 1-second interval, while only seven occipital sensors were required to reliably decode the neuronal response.

COMPARISON WITH EXISTING METHODS

In contrast to existing SSVEP-based BCI systems, rapid frequency-tagging does not produce a visible flicker. This extends the time-period participants can use it without fatigue, by avoiding distracting visual input. Furthermore, higher frequencies increase the temporal resolution of decoding, resulting in higher communication rates.

CONCLUSION

Using rapid invisible frequency-tagging opens new avenues for fundamental research and practical applications. In combination with novel optically pumped magnetometers (OPMs), it could facilitate the development of high-speed and mobile next-generation BCI systems.

The Full Informational Spectral Analysis for Auditory Steady-State Responses in Human Brain Using the Combination of Canonical Correlation Analysis and Holo-Hilbert Spectral Analysis

Auditory steady-state response (ASSR) is a translational biomarker for several neurological and psychiatric disorders, such as hearing loss, schizophrenia, bipolar disorder, autism, etc. The ASSR is sinusoidal electroencephalography (EEG)/magnetoencephalography (MEG) responses induced by periodically presented auditory stimuli. Traditional frequency analysis assumes ASSR is a stationary response, which can be analyzed using linear analysis approaches, such as Fourier analysis or Wavelet. However, recent studies have reported that the human steady-state responses are dynamic and can be modulated by the subject’s attention, wakefulness state, mental load, and mental fatigue. The amplitude modulations on the measured oscillatory responses can result in the spectral broadening or frequency splitting on the Fourier spectrum, owing to the trigonometric product-to-sum formula. Accordingly, in this study, we analyzed the human ASSR by the combination of canonical correlation analysis (CCA) and Holo-Hilbert spectral analysis (HHSA). The CCA was used to extract ASSR-related signal features, and the HHSA was used to decompose the extracted ASSR responses into amplitude modulation (AM) components and frequency modulation (FM) components, in which the FM frequency represents the fast-changing intra-mode frequency and the AM frequency represents the slow-changing inter-mode frequency. In this paper, we aimed to study the AM and FM spectra of ASSR responses in a 37 Hz steady-state auditory stimulation. Twenty-five healthy subjects were recruited for this study, and each subject was requested to participate in two auditory stimulation sessions, including one right-ear and one left-ear monaural steady-state auditory stimulation. With the HHSA, both the 37 Hz (fundamental frequency) and the 74 Hz (first harmonic frequency) auditory responses were successfully extracted. Examining the AM spectra, the 37 Hz and the 74 Hz auditory responses were modulated by distinct AM spectra, each with at least three composite frequencies. In contrast to the results of traditional Fourier spectra, frequency splitting was seen at 37 Hz, and a spectral peak was obscured at 74 Hz in Fourier spectra. The proposed method effectively corrects the frequency splitting problem resulting from time-varying amplitude changes. Our results have validated the HHSA as a useful tool for steady-state response (SSR) studies so that the misleading or wrong interpretation caused by amplitude modulation in the traditional Fourier spectrum can be avoided.

Optimising the classification of feature-based attention in frequency-tagged electroencephalography data

Brain-computer interfaces (BCIs) are a rapidly expanding field of study and require accurate and reliable real-time decoding of patterns of neural activity. These protocols often exploit selective attention, a neural mechanism that prioritises the sensory processing of task-relevant stimulus features (feature-based attention) or task-relevant spatial locations (spatial attention). Within the visual modality, attentional modulation of neural responses to different inputs is well indexed by steady-state visual evoked potentials (SSVEPs). These signals are reliably present in single-trial electroencephalography (EEG) data, are largely resilient to common EEG artifacts, and allow separation of neural responses to numerous concurrently presented visual stimuli. To date, efforts to use single-trial SSVEPs to classify visual attention for BCI control have largely focused on spatial attention rather than feature-based attention. Here, we present a dataset that allows for the development and benchmarking of algorithms to classify feature-based attention using single-trial EEG data. The dataset includes EEG and behavioural responses from 30 healthy human participants who performed a feature-based motion discrimination task on frequency tagged visual stimuli.

A BCI-Based Study on the Relationship Between the SSVEP and Retinal Eccentricity in Overt and Covert Attention

Covert attention aids us in monitoring the environment and optimizing performance in visual tasks. Past behavioral studies have shown that covert attention can enhance spatial resolution. However, electroencephalography (EEG) activity related to neural processing between central and peripheral vision has not been systematically investigated. Here, we conducted an EEG study with 25 subjects who performed covert attentional tasks at different retinal eccentricities ranging from 0.75° to 13.90°, as well as tasks involving overt attention and no attention. EEG signals were recorded with a single stimulus frequency to evoke steady-state visual evoked potentials (SSVEPs) for attention evaluation. We found that the SSVEP response in fixating at the attended location was generally negatively correlated with stimulus eccentricity as characterized by Euclidean distance or horizontal and vertical distance. Moreover, more pronounced characteristics of SSVEP analysis were also acquired in overt attention than in covert attention. Furthermore, offline classification of overt attention, covert attention, and no attention yielded an average accuracy of 91.42%. This work contributes to our understanding of the SSVEP representation of attention in humans and may also lead to brain-computer interfaces (BCIs) that allow people to communicate with choices simply by shifting their attention to them.

A Novel Approach to Decode Covert Spatial Attention Using SSVEP and Single-Frequency Phase-Coded Stimuli

This paper investigates for the first time the use of single-frequency phase-coded stimuli to detect covert visuo-spatial attention (CVSA) with steady-state visual evoked potentials (SSVEP). Two 15Hz pattern-onset stimulations were encoded with opposite phases and simultaneously presented on a LCD monitor. The effects of attending each stimulus on the amplitudes and phases of the evoked SSVEPs across the visual cortex are explored. A real-time CVSA classification experiment was simulated offline with 9 BCI-naive subjects, achieving an average classification accuracy of 88.4 ± 8% SE. Our results are, to our knowledge, the first report that CVSA can be decoded with SSVEP using single-frequency phase-coded stimuli. This opens opportunities for attention-tracking applications with largely increased number of targets.

Decoding the covert shift of spatial attention from electroencephalographic signals permits reliable control of a brain-computer interface

OBJECTIVE

One of the main goals of brain-computer interfaces (BCI) is to restore communication abilities in patients. BCIs often use event-related potentials (ERPs) like the P300 which signals the presence of a target in a stream of stimuli. The P300 and related approaches, however, are inherently limited, as they require many stimulus presentations to obtain a usable control signal. Many approaches depend on gaze direction to focus the target, which is also not a viable approach in many cases, because eye movements might be impaired in potential users. Here we report on a BCI that avoids both shortcomings by decoding spatial target information, independent of gaze shifts.

APPROACH

We present a new method to decode from the electroencephalogram (EEG) covert shifts of attention to one out of four targets simultaneously presented in the left and right visual field. The task is designed to evoke the N2pc component-a hemisphere lateralized response, elicited over the occipital scalp contralateral to the attended target. The decoding approach involves decoding of the N2pc based on data-driven estimation of spatial filters and a correlation measure.

MAIN RESULTS

Despite variability of decoding performance across subjects, 22 out of 24 subjects performed well above chance level. Six subjects even exceeded 80% (cross-validated: 89%) correct predictions in a four-class discrimination task. Hence, the single-trial N2pc proves to be a component that allows for reliable BCI control. An offline analysis of the EEG data with respect to their dependence on stimulation time and number of classes demonstrates that the present method is also a workable approach for two-class tasks.

SIGNIFICANCE

Our method extends the range of strategies for gaze-independent BCI control. The proposed decoding approach has the potential to be efficient in similar applications intended to decode ERPs.

Stress-Induced Effects in Resting EEG Spectra Predict the Performance of SSVEP-Based BCI

Most research in Brain-Computer-Interfaces (BCI) focuses on technologies to improve accuracy and speed. Little has been done on the effects of subject variability, both across individuals and within the same individual, on BCI performance. For example, stress, arousal, motivation, and fatigue can all affect the electroencephalogram (EEG) signals used by a BCI, which in turn impacts performance. Overcoming the impact of such user variability on BCI performance is an impending and inevitable challenge for routine applications of BCIs in the real world. To systematically explore the factors affecting BCI performance, this study embeds a Steady-State Visually Evoked Potential (SSVEP) based BCI into a "game with a purpose" (GWAP) to obtain data over significant lengths of time, under both high- and low-stress conditions. Ten healthy volunteers played a GWAP that resembles popular match-three games, such as Jewel Quest, Zoo Boom, or Candy Crush. We recorded the target search time, target search accuracy, and EEG signals during gameplay to investigate the impacts of stress on EEG signals and BCI performance. We used Canonical Correlation Analysis (CCA) to determine whether the subject had found and attended to the correct target. The experimental results show that SSVEP target-classification accuracy is reduced by stress. We also found a negative correlation between EEG spectra and the SNR of EEG in the frontal and occipital regions during gameplay, with a larger negative correlation for the high-stress conditions. Furthermore, CCA also showed that when the EEG alpha and theta power increased, the search accuracy decreased, and the spectral amplitude drop was more evident under the high-stress situation. These results provide new, valuable insights into research on how to improve the robustness of BCIs in real-world applications.

Too little, too late, and in the wrong place: Alpha band activity does not reflect an active mechanism of selective attention

Selective attention focuses visual processing on relevant stimuli in order to allow for adaptive behaviour despite an abundance of distracting information. It has been proposed that increases in alpha band (8-12 ​Hz) amplitude reflect an active mechanism for distractor suppression. If this were the case, increases in alpha band amplitude should be succeeded by a decrease in distractor processing. Surprisingly, this connection has not been tested directly; specifically, studies that have investigated changes in alpha band after attention-directing cues have not directly assessed the neuronal processing of distractors. We concurrently recorded alpha activity and steady-state visual evoked potentials (SSVEPs) to assess the processing of target and distractor stimuli. In two experiments, participants covertly shifted attention to one of two letter streams (left or right) to detect infrequent target letters 'X' while ignoring the other stream. In line with previous findings, alpha band amplitudes contralateral to the unattended location increased compared to a pre-cue baseline. However, there was no suppression of SSVEP amplitudes elicited by unattended stimuli, while there was a pronounced enhancement of SSVEPs elicited by attended stimuli. Furthermore, and crucially, changes in alpha band amplitude during attention shifts did not precede those in SSVEPs and hit rates in both experiments, indicating that changes in alpha band amplitudes are likely to be a consequence of attention shifts rather than the other way around. We conclude that these findings contradict the notion that alpha band activity reflects mechanisms that have a causal role in the allocation of selective attention.

Fuzzy ensemble system for SSVEP stimulation frequency detection using the MLR and MsetCCA

BACKGROUND

BCI systems based on steady-state visual evoked potentials (SSVEP) have formed an immense contribution to practical applications, due to their high recognition accuracy and ease of use. The MLR method has a better frequency recognition accuracy for short-term windows, and the MsetCCA method works more accurately in long-term windows.

NEW METHOD

The proposed fuzzy ensemble system can analyze the relevant SSVEP signals of each subject from 0.5 to 4 s windows with 0.5 s incremental steps. It is capable of taking decisions to improve the accuracy of SSVEP stimulation frequency recognition using the MLR and MsetCCA methods.

RESULTS

Our fuzzy system provides high-accuracy results for the stimulation frequency recognition in signals with the length of 1 s and more. Specifically, the average accuracy of 2 s windows has improved to 100 percent.

COMPARISON WITH EXISTING METHODS

The recognition accuracy of the presented system is always better than both MLR and MsetCCA methods.

CONCLUSION

One of the capabilities of fuzzy systems is that they can use human information and knowledge to build engineering systems. The fuzzy ensemble system can utilize various methods or classifiers simultaneously. The new system has proposed to combine multiple methods using the fuzzy ensemble, which encompasses the benefits of all the subsystems.

Electroencephalogram (EEG) Alpha Power as a Marker of Visuospatial Attention Orienting and Suppression in Normoxia and Hypoxia. An Exploratory Study

While electroencephalogram (EEG) alpha desynchronization has been related to anticipatory orienting of visuospatial attention, an increase in alpha power has been associated to its inhibition. A separate line of findings indicated that alpha is affected by a deficient oxygenation of the brain or hypoxia, although leaving unclear whether the latter increases or decreases alpha synchronization. Here, we carried out an exploratory study on these issues by monitoring attention alerting, orienting, and control networks functionality by means of EEG recorded both in normoxia and hypoxia in college students engaged in four attentional cue-target conditions induced by a redesigned Attention Network Test. Alpha power was computed through Fast Fourier Transform. Regardless of brain oxygenation condition, alpha desynchronization was the highest during exogenous, uncued orienting of spatial attention, the lowest during alerting but spatially unpredictable, cued exogenous orienting of attention, and of intermediate level during validly cued endogenous orienting of attention, no matter the motor response workload demanded by the latter, especially over the left hemisphere. Hypoxia induced an increase in alpha power over the right-sided occipital and parietal scalp areas independent of attention cueing and conflict conditions. All in all, these findings prove that attention orienting is undergirded by alpha desynchronization and that alpha right-sided synchronization in hypoxia might sub-serve either the effort to sustain attention over time or an overall suppression of attention networks functionality.

Group task-related component analysis (gTRCA): a multivariate method for inter-trial reproducibility and inter-subject similarity maximization for EEG data analysis

EEG is known to contain considerable inter-trial and inter-subject variability, which poses a challenge in any group-level EEG analyses. A true experimental effect must be reproducible even with variabilities in trials, sessions, and subjects. Extracting components that are reproducible across trials and subjects benefits both understanding common mechanisms in neural processing of cognitive functions and building robust brain-computer interfaces. This study extends our previous method (task-related component analysis, TRCA) by maximizing not only trial-by-trial reproducibility within single subjects but also similarity across a group of subjects, hence referred to as group TRCA (gTRCA). The problem of maximizing reproducibility of time series across trials and subjects is formulated as a generalized eigenvalue problem. We applied gTRCA to EEG data recorded from 35 subjects during a steady-state visual-evoked potential (SSVEP) experiment. The results revealed: (1) The group-representative data computed by gTRCA showed higher and consistent spectral peaks than other conventional methods; (2) Scalp maps obtained by gTRCA showed estimated source locations consistently within the occipital lobe; And (3) the high-dimensional features extracted by gTRCA are consistently mapped to a low-dimensional space. We conclude that gTRCA offers a framework for group-level EEG data analysis and brain-computer interfaces alternative in complement to grand averaging.

Dynamic time window mechanism for time synchronous VEP-based BCIs-Performance evaluation with a dictionary-supported BCI speller employing SSVEP and c-VEP

Brain-Computer Interfaces (BCIs) based on visual evoked potentials (VEPs) allow high communication speeds and accuracies. The fastest speeds can be achieved if targets are identified in a synchronous way (i.e., after a pre-set time period the system will produce a command output). The duration a target needs to be fixated on until the system classifies an output command affects the overall system performance. Hence, extracting a data window dedicated for the classification is of critical importance for VEP-based BCIs. Secondly, unintentional fixation on a target could easily lead to its selection. For the practical usability of BCI applications it is desirable to distinguish between intentional and unintentional fixations. This can be achieved by using threshold-based target identification methods. The study explores personalized dynamic classification time windows for threshold-based time synchronous VEP BCIs. The proposed techniques were tested employing the SSVEP and the c-VEP paradigm. Spelling performance was evaluated using an 8-target dictionary-supported BCI utilizing an n-gram word prediction model. The performance of twelve healthy participants was assessed with the information transfer rate (ITR) and accuracy. All participants completed sentence spelling tasks, reaching average accuracies of 94% and 96.3% for the c-VEP and the SSVEP paradigm, respectively. Average ITRs around 57 bpm were achieved for both paradigms.

Exploring Cognitive Flexibility With a Noninvasive BCI Using Simultaneous Steady-State Visual Evoked Potentials and Sensorimotor Rhythms

EEG-based brain-computer interface (BCI) technology creates non-biological pathways for conveying a user's mental intent solely through noninvasively measured neural signals. While optimizing the performance of a single task has long been the focus of BCI research, in order to translate this technology into everyday life, realistic situations, in which multiple tasks are performed simultaneously, must be investigated. In this paper, we explore the concept of cognitive flexibility, or multitasking, within the BCI framework by utilizing a 2-D cursor control task, using sensorimotor rhythms (SMRs), and a four-target visual attention task, using steady-state visual evoked potentials (SSVEPs), both individually and simultaneously. We found no significant difference between the accuracy of the tasks when executing them alone (SMR-57.9% ± 15.4% and SSVEP-59.0% ± 14.2%) and simultaneously (SMR-54.9% ± 17.2% and SSVEP-57.5% ± 15.4%). These modest decreases in performance were supported by similar, non-significant changes in the electrophysiology of the SSVEP and SMR signals. In this sense, we report that multiple BCI tasks can be performed simultaneously without a significant deterioration in performance; this finding will help drive these systems toward realistic daily use in which a user's cognition will need to be involved in multiple tasks at once.

Mental fatigue in central-field and peripheral-field steady-state visually evoked potential and its effects on event-related potential responses

The steady-state visually evoked potential (SSVEP) is a natural response of the brain to visual stimulation at specific frequencies and is used widely for electroencephalography-based brain–computer interface (BCI) systems. Although the SSVEP is useful for its high level of decoding accuracy, visual fatigue from the repetitive visual flickering is an unavoidable problem. In addition, hybrid BCI systems that combine the SSVEP with the event-related potential (ERP) have been proposed recently. These hybrid BCI systems would improve the decoding accuracy; however, the competing effect by simultaneous presentation of the visual stimulus could possibly supervene the signal in the hybrid system. Nevertheless, previous studies have not sufficiently reported these problems of visual fatigue with SSVEP stimuli or the competing effect in the SSVEP+ERP system. In this study, two different experiments were designed to explore our claims. The first experiment evaluated the visual fatigue level and decoding accuracy for the different types of SSVEP stimuli, which were the peripheral-field SSVEP (pSSVEP) and the central-field SSVEP (cSSVEP). We report that the pSSVEP could reduce the visual fatigue level by avoiding direct exposure of the eye-retina to the flickering visual stimulus, while also delivering a decoding accuracy comparable to that of cSSVEP. The second experiment was designed to examine the competing effect of the SSVEP stimuli on ERP performance and vice versa. To do this, the visual stimuli of ERP and SSVEP were presented simultaneously as part of the BCI speller layout. We found a clear competing effect wherein the evoked brain potentials were influenced by the SSVEP stimulus and the band power at the target frequencies was also decreased significantly by the ERP stimuli. Nevertheless, these competing effects did not lead to a significant loss in decoding accuracy; their features preserved sufficient information for discriminating a target class. Our work is the first to evaluate the visual fatigue and competing effect together, which should be considered when designing BCI applications. Furthermore, our findings suggest that the pSSVEP is a viable substitution for the cSSVEP because of its ability to reduce the level of visual fatigue while maintaining a minimal loss of decoding accuracy.

BrainNet: A Multi-Person Brain-to-Brain Interface for Direct Collaboration Between Brains

We present BrainNet which, to our knowledge, is the first multi-person non-invasive direct brain-to-brain interface for collaborative problem solving. The interface combines electroencephalography (EEG) to record brain signals and transcranial magnetic stimulation (TMS) to deliver information noninvasively to the brain. The interface allows three human subjects to collaborate and solve a task using direct brain-to-brain communication. Two of the three subjects are designated as "Senders" whose brain signals are decoded using real-time EEG data analysis. The decoding process extracts each Sender's decision about whether to rotate a block in a Tetris-like game before it is dropped to fill a line. The Senders' decisions are transmitted via the Internet to the brain of a third subject, the "Receiver," who cannot see the game screen. The Senders' decisions are delivered to the Receiver's brain via magnetic stimulation of the occipital cortex. The Receiver integrates the information received from the two Senders and uses an EEG interface to make a decision about either turning the block or keeping it in the same orientation. A second round of the game provides an additional chance for the Senders to evaluate the Receiver's decision and send feedback to the Receiver's brain, and for the Receiver to rectify a possible incorrect decision made in the first round. We evaluated the performance of BrainNet in terms of (1) Group-level performance during the game, (2) True/False positive rates of subjects' decisions, and (3) Mutual information between subjects. Five groups, each with three human subjects, successfully used BrainNet to perform the collaborative task, with an average accuracy of 81.25%. Furthermore, by varying the information reliability of the Senders by artificially injecting noise into one Sender's signal, we investigated how the Receiver learns to integrate noisy signals in order to make a correct decision. We found that like conventional social networks, BrainNet allows Receivers to learn to trust the Sender who is more reliable, in this case, based solely on the information transmitted directly to their brains. Our results point the way to future brain-to-brain interfaces that enable cooperative problem solving by humans using a "social network" of connected brains.

Multiple-command single-frequency SSVEP-based BCI system using flickering action video

BACKGROUND

The number of commands in a brain-computer interface (BCI) system is important. This study proposes a new BCI technique to increase the number of commands in a single BCI system without loss of accuracy.

NEW METHOD

We expected that a flickering action video with left and right elbow movements could simultaneously activate the different pattern of event-related desynchronization (ERD) according to the video contents (e.g., left or right) and steady-state visually evoked potential (SSVEP). The classification accuracy to discriminate left, right, and rest states was compared under the three following feature combinations: SSVEP power (19-21 Hz), Mu power (8-13 Hz), and simultaneous SSVEP and Mu power.

RESULTS

The SSVEP feature could discriminate the stimulus condition, regardless of left or right, from the rest condition, while the Mu feature discriminated left or right, but was relatively poor in discriminating stimulus from rest. However, combining the SSVEP and Mu features, which were evoked by the stimulus with a single frequency, showed superior performance for discriminating all the stimuli among rest, left, or right.

COMPARISON WITH THE EXISTING METHOD

The video contents could activate the ERD differently, and the flickering component increased its accuracy, such that it revealed a better performance to discriminate when considering together.

CONCLUSIONS

This paradigm showed possibility of increasing performance in terms of accuracy and number of commands with a single frequency by applying flickering action video paradigm and applicability to rehabilitation systems used by patients to facilitate their mirror neuron systems while training.

A conceptual space for EEG-based brain-computer interfaces

Brain-Computer Interfaces (BCIs) have become more and more popular these last years. Researchers use this technology for several types of applications, including attention and workload measures but also for the direct control of objects by the means of BCIs. In this work we present a first, multidimensional feature space for EEG-based BCI applications to help practitioners to characterize, compare and design systems, which use EEG-based BCIs. Our feature space contains 4 axes and 9 sub-axes and consists of 41 options in total as well as their different combinations. We presented the axes of our feature space and we positioned our feature space regarding the existing BCI and HCI taxonomies and we showed how our work integrates the past works, and/or complements them.

EEG-Based Brain-Computer Interfaces

Brain-computer interfaces (BCIs) provide a direct communication channel between human brain and output devices. Due to advantages such as non-invasiveness, ease of use, and low cost, electroencephalography (EEG) is the most popular method for current BCIs. This chapter gives an overview of the current EEG-based BCIs for the main purpose of communication and control. This chapter first provides a taxonomy of the EEG-based BCI systems by categorizing them into three major groups: (1) BCIs based on event-related potentials (ERPs), (2) BCIs based on sensorimotor rhythms, and (3) hybrid BCIs. Next, this chapter describes challenges and potential solutions in developing practical BCI systems toward high communication speed, convenient system use, and low user variation. Then this chapter briefly reviews both medical and non-medical applications of current BCIs. Finally, this chapter concludes with a summary of current stage and future perspectives of the EEG-based BCI technology.

Effects of an Integrated Neurofeedback System with Dry Electrodes: EEG Acquisition and Cognition Assessment

Electroencephalogram (EEG) neurofeedback improves cognitive capacity and behaviors by regulating brain activity, which can lead to cognitive enhancement in healthy people and better rehabilitation in patients. The increased use of EEG neurofeedback highlights the urgent need to reduce the discomfort and preparation time and increase the stability and simplicity of the system’s operation. Based on brain-computer interface technology and a multithreading design, we describe a neurofeedback system with an integrated design that incorporates wearable, multichannel, dry electrode EEG acquisition equipment and cognitive function assessment. Then, we evaluated the effectiveness of the system in a single-blind control experiment in healthy people, who increased the alpha frequency band power in a neurofeedback protocol. We found that upregulation of the alpha power density improved working memory following short-term training (only five training sessions in a week), while the attention network regulation may be related to other frequency band activities, such as theta and beta. Our integrated system will be an effective neurofeedback training and cognitive function assessment system for personal and clinical use.

An Adaptive SSVEP-Based Brain-Computer Interface to Compensate Fatigue-Induced Decline of Performance in Practical Application

Brain-computer interfaces based on steady-state visual evoked potentials are promising communication systems for people with speech and motor disabilities. However, reliable SSVEP response requires user's attention, which degrades over time due to significant eye-fatigue when low-frequency visual stimuli (5-15 Hz) are used. Previous studies have shown that eye-fatigue can be reduced using high-frequency flickering stimuli (>25 Hz). Here, it is quantitatively demonstrated that the performance of a high-frequency SSVEP BCI decreases over time, but this amount of decrease can be compensated effectively by using two proposed adaptive algorithms. This leaded to a robust alternative communication system for practical applications. The asynchronous spelling system implemented in this study uses a threshold-based version of LASSO algorithm for frequency recognition. In long online experiments, when participants typed a sentence with the BCI system for 16 times, accuracy of the system was close to its maximum along the experiment. However, regression analysis on typing speed of each sentence demonstrated a significant decrease in all 7 subjects ( ) when thresholds obtained from a calibration test were kept fixed over the experiment. In comparison, no significant change in typing speed was observed when the proposed adaptive algorithms were used. The analysis of variances revealed that the average typing speed of the last four sentences when using adaptive relational algorithm (8.7 char/min) was significantly higher than the tolerance-based algorithm (8.1 char/min) and significantly above 6 char/min when the fixed thresholds were used. Therefore, the relational algorithm proposed in this paper could successfully compensate for the effect of fatigue on performance of the SSVEP BCI system.

Decoding spatial attention with EEG and virtual acoustic space

Decoding spatial attention based on brain signals has wide applications in brain–computer interface (BCI). Previous BCI systems mostly relied on visual patterns or auditory stimulation (e.g., loudspeakers) to evoke synchronous brain signals. There would be difficulties to cover a large range of spatial locations with such a stimulation protocol. The present study explored the possibility of using virtual acoustic space and a visual‐auditory matching paradigm to overcome this issue. The technique has the flexibility of generating sound stimulation from virtually any spatial location. Brain signals of eight human subjects were obtained with a 32‐channel Electroencephalogram (EEG). Two amplitude‐modulated noise or speech sentences carrying distinct spatial information were presented concurrently. Each sound source was tagged with a unique modulation phase so that the phase of the recorded EEG signals indicated the sound being attended to. The phase‐tagged sound was further filtered with head‐related transfer functions to create the sense of virtual space. Subjects were required to pay attention to the sound source that best matched the location of a visual target. For all the subjects, the phase of a single sound could be accurately reflected over the majority of electrodes based on EEG responses of 90 s or less. The electrodes providing significant decoding performance on auditory attention were fewer and may require longer EEG responses. The reliability and efficiency of decoding with a single electrode varied with subjects. Overall, the virtual acoustic space protocol has the potential of being used in practical BCI systems.

Accurate Decoding of Short, Phase-Encoded SSVEPs

Four novel EEG signal features for discriminating phase-coded steady-state visual evoked potentials (SSVEPs) are presented, and their performance in view of target selection in an SSVEP-based brain-computer interfacing (BCI) is assessed. The novel features are based on phase estimation and correlations between target responses. The targets are decoded from the feature scores using the least squares support vector machine (LS-SVM) classifier, and it is shown that some of the proposed features compete with state-of-the-art classifiers when using short (0.5 s) EEG recordings in a binary classification setting.

Examining sensory ability, feature matching and assessment-based adaptation for a brain-computer interface using the steady-state visually evoked potential

PURPOSE

We investigated how overt visual attention and oculomotor control influence successful use of a visual feedback brain-computer interface (BCI) for accessing augmentative and alternative communication (AAC) devices in a heterogeneous population of individuals with profound neuromotor impairments. BCIs are often tested within a single patient population limiting generalization of results. This study focuses on examining individual sensory abilities with an eye toward possible interface adaptations to improve device performance.

METHODS

Five individuals with a range of neuromotor disorders participated in four-choice BCI control task involving the steady state visually evoked potential. The BCI graphical interface was designed to simulate a commercial AAC device to examine whether an integrated device could be used successfully by individuals with neuromotor impairment.

RESULTS

All participants were able to interact with the BCI and highest performance was found for participants able to employ an overt visual attention strategy. For participants with visual deficits to due to impaired oculomotor control, effective performance increased after accounting for mismatches between the graphical layout and participant visual capabilities.

CONCLUSION

As BCIs are translated from research environments to clinical applications, the assessment of BCI-related skills will help facilitate proper device selection and provide individuals who use BCI the greatest likelihood of immediate and long term communicative success. Overall, our results indicate that adaptations can be an effective strategy to reduce barriers and increase access to BCI technology. These efforts should be directed by comprehensive assessments for matching individuals to the most appropriate device to support their complex communication needs. Implications for Rehabilitation Brain computer interfaces using the steady state visually evoked potential can be integrated with an augmentative and alternative communication device to provide access to language and literacy for individuals with neuromotor impairment. Comprehensive assessments are needed to fully understand the sensory, motor, and cognitive abilities of individuals who may use brain-computer interfaces for proper feature matching as selection of the most appropriate device including optimization device layouts and control paradigms. Oculomotor impairments negatively impact brain-computer interfaces that use the steady state visually evoked potential, but modifications to place interface stimuli and communication items in the intact visual field can improve successful outcomes.

A novel method for the detection of VEP signals from frontal region

PURPOSE

This paper studies the feasibility of frontal positioning of electrode to detect electroencephalographic signals.

METHODS

The experiments were conducted using a board with light-emitting diodes (LEDs) designed to stimulate SSVEP and FVEP. The flashing frequencies were conducted at 15, 23, and 31 Hz. We used the Quick Amp brain wave amplifier to collect brain wave signals at a sampling rate of 1 kHz using a frequency filter band of 10-100 Hz.

RESULTS

We found that the energy power of VEP will gradually increase from Oz position to Fp2 position. We analyze the data, proving that the Fp2 position can also be used to collect VEP data.

CONCLUSIONS

Traditional measurements at the Oz location are limited because of the interference from human hair, and an additional electrode is required to detect eye movement and filter this electro-oculogram signal. Our proposed method can effectively acquire critical visually evoked potential and electro-oculogram signals without the electrode at the Oz location, which require extensive preparation of removing hair strands. We also reduced the number of electrodes used in receiving electroencephalographic signals. The frontal electrode positioning could be a remarkable breakthrough to design brain computer interface.

Decoding Saccadic Directions Using Epidural ECoG in Non-Human Primates

A brain-computer interface (BCI) can be used to restore some communication as an alternative interface for patients suffering from locked-in syndrome. However, most BCI systems are based on SSVEP, P300, or motor imagery, and a diversity of BCI protocols would be needed for various types of patients. In this paper, we trained the choice saccade (CS) task in 2 non-human primate monkeys and recorded the brain signal using an epidural electrocorticogram (eECoG) to predict eye movement direction. We successfully predicted the direction of the upcoming eye movement using a support vector machine (SVM) with the brain signals after the directional cue onset and before the saccade execution. The mean accuracies were 80% for 2 directions and 43% for 4 directions. We also quantified the spatial-spectro-temporal contribution ratio using SVM recursive feature elimination (RFE). The channels over the frontal eye field (FEF), supplementary eye field (SEF), and superior parietal lobule (SPL) area were dominantly used for classification. The α-band in the spectral domain and the time bins just after the directional cue onset and just before the saccadic execution were mainly useful for prediction. A saccade based BCI paradigm can be projected in the 2D space, and will hopefully provide an intuitive and convenient communication platform for users.

Frequency- and Phase Encoded SSVEP Using Spatiotemporal Beamforming

In brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEPs) the number of selectable targets is rather limited when each target has its own stimulation frequency. One way to remedy this is by combining frequency- with phase encoding. We introduce a new multivariate spatiotemporal filter, based on Linearly Constrained Minimum Variance (LCMV) beamforming, for discriminating between frequency-phase encoded targets more accurately, even when using short signal lengths than with (extended) Canonical Correlation Analysis (CCA), which is traditionally posited for this stimulation paradigm.

Towards a gaze-independent hybrid-BCI based on SSVEPs, alpha-band modulations and the P300

In recent years it has been shown to be possible to create a Brain Computer Interface (BCI) using non-invasive electroencephalographic (EEG) measurements of covert visual spatial attention. For example, that both Steady-State Visual Evoked Potentials (SSVEP) and parieto-occipital alpha band activity have been shown to be sensitive to covert attention and this has been exploited to provide simple communication control without the need for any physical movement. In this study, potential improvements in the speed and accuracy of such a BCI are investigated by exploring the possibility of incorporating a P300 task into an SSVEP covert attention paradigm. Should this be possible it would pave the way for a gaze-independent hybrid BCI based on three somewhat independent EEG signals. Within a well-established SSVEP-based attention paradigm we show that it is possible to make a binary classification of covert attention using just the P300 with an average accuracy of 71% across three subjects. We also validate previously published research by showing robust attention effects on the SSVEP and alpha band activity within this paradigm. In future work, it is hoped that by integrating the three signals into a hybrid BCI a significant improvement in performance will be forthcoming leading to an easily usable real time communication device for patients with severe disabilities such as Locked-In Syndrome (LIS).

Alpha neurofeedback training improves SSVEP-based BCI performance

OBJECTIVE

Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can provide relatively easy, reliable and high speed communication. However, the performance is still not satisfactory, especially in some users who are not able to generate strong enough SSVEP signals. This work aims to strengthen a user's SSVEP by alpha down-regulating neurofeedback training (NFT) and consequently improve the performance of the user in using SSVEP-based BCIs.

APPROACH

An experiment with two steps was designed and conducted. The first step was to investigate the relationship between the resting alpha activity and the SSVEP-based BCI performance, in order to determine the training parameter for the NFT. Then in the second step, half of the subjects with 'low' performance (i.e. BCI classification accuracy <80%) were randomly assigned to a NFT group to perform a real-time NFT, and the rest half to a non-NFT control group for comparison.

MAIN RESULTS

The first step revealed a significant negative correlation between the BCI performance and the individual alpha band (IAB) amplitudes in the eyes-open resting condition in a total of 33 subjects. In the second step, it was found that during the IAB down-regulating NFT, on average the subjects were able to successfully decrease their IAB amplitude over training sessions. More importantly, the NFT group showed an average increase of 16.5% in the SSVEP signal SNR (signal-to-noise ratio) and an average increase of 20.3% in the BCI classification accuracy, which was significant compared to the non-NFT control group.

SIGNIFICANCE

These findings indicate that the alpha down-regulating NFT can be used to improve the SSVEP signal quality and the subjects' performance in using SSVEP-based BCIs. It could be helpful to the SSVEP related studies and would contribute to more effective SSVEP-based BCI applications.

Driving steady-state visual evoked potentials at arbitrary frequencies using temporal interpolation of stimulus presentation

Background

Steady-state visual evoked potentials have been utilized widely in basic and applied research in recent years. These oscillatory responses of the visual cortex are elicited by flickering stimuli. They have the same fundamental frequency as the driving stimulus and are highly sensitive to manipulations of attention and stimulus properties. While standard computer monitors offer great flexibility in the choice of visual stimuli for driving SSVEPs, the frequencies that can be elicited are limited to integer divisors of the monitor’s refresh rate.

Results

To avoid this technical constraint, we devised an interpolation technique for stimulus presentation, with which SSVEPs can be elicited at arbitrary frequencies. We tested this technique with monitor refresh rates of 85 and 120 Hz. At a refresh rate of 85 Hz, interpolated presentation produced artifacts in the recorded spectrum in the form of additional peaks not located at the stimulated frequency or its harmonics. However, at a refresh rate of 120 Hz, these artifacts did not occur and the spectrum elicited by an interpolated flicker became indistinguishable from the spectrum obtained by non-interpolated presentation of the same frequency.

Conclusions

Our interpolation technique eliminates frequency limitations of the common non-interpolated presentation technique and has many possible applications for future research.

Evaluate the Feasibility of Using Frontal SSVEP to Implement an SSVEP-Based BCI in Young, Elderly and ALS Groups

This paper studies the amplitude-frequency characteristic of frontal steady-state visual evoked potential (SSVEP) and its feasibility as a control signal for brain computer interface (BCI). SSVEPs induced by different stimulation frequencies, from 13 ~ 31 Hz in 2 Hz steps, were measured in eight young subjects, eight elders and seven ALS patients. Each subject was requested to participate in a calibration study and an application study. The calibration study was designed to find the amplitude-frequency characteristics of SSVEPs recorded from Oz and Fpz positions, while the application study was designed to test the feasibility of using frontal SSVEP to control a two-command SSVEP-based BCI. The SSVEP amplitude was detected by an epoch-average process which enables artifact-contaminated epochs can be removed. The seven ALS patients were severely impaired, and four patients, who were incapable of completing our BCI task, were excluded from calculation of BCI performance. The averaged accuracies, command transfer intervals and information transfer rates in operating frontal SSVEP-based BCI were 96.1%, 3.43 s/command, and 14.42 bits/min in young subjects; 91.8%, 6.22 s/command, and 6.16 bits/min in elders; 81.2%, 12.14 s/command, and 1.51 bits/min in ALS patients, respectively. The frontal SSVEP could be an alternative choice to design SSVEP-based BCI.

Covert enaction at work: Recording the continuous movements of visuospatial attention to visible or imagined targets by means of Steady-State Visual Evoked Potentials (SSVEPs)

Whereas overt visuospatial attention is customarily measured with eye tracking, covert attention is assessed by various methods. Here we exploited Steady-State Visual Evoked Potentials (SSVEPs) - the oscillatory responses of the visual cortex to incoming flickering stimuli - to record the movements of covert visuospatial attention in a way operatively similar to eye tracking (attention tracking), which allowed us to compare motion observation and motion extrapolation with and without eye movements. Observers fixated a central dot and covertly tracked a target oscillating horizontally and sinusoidally. In the background, the left and the right halves of the screen flickered at two different frequencies, generating two SSVEPs in occipital regions whose size varied reciprocally as observers attended to the moving target. The two signals were combined into a single quantity that was modulated at the target frequency in a quasi-sinusoidal way, often clearly visible in single trials. The modulation continued almost unchanged when the target was switched off and observers mentally extrapolated its motion in imagery, and also when observers pointed their finger at the moving target during covert tracking, or imagined doing so. The amplitude of modulation during covert tracking was ∼25-30% of that measured when observers followed the target with their eyes. We used 4 electrodes in parieto-occipital areas, but similar results were achieved with a single electrode in Oz. In a second experiment we tested ramp and step motion. During overt tracking, SSVEPs were remarkably accurate, showing both saccadic-like and smooth pursuit-like modulations of cortical responsiveness, although during covert tracking the modulation deteriorated. Covert tracking was better with sinusoidal motion than ramp motion, and better with moving targets than stationary ones. The clear modulation of cortical responsiveness recorded during both overt and covert tracking, identical for motion observation and motion extrapolation, suggests to include covert attention movements in enactive theories of mental imagery.

Designing Better, Cost-Effective Brain–Computer Interfaces

Despite the increase in research interest in the brain–computer interface (BCI), there remains a general lack of understanding of, and even inattention to, human factors/ergonomics (HF/E) issues in BCI research and development. The goal of this article is to raise awareness of the importance of HF/E involvement in the emerging field of BCI technology by providing HF/E researchers with a brief guide on how to design and implement a cost-effective, steady-state visually evoked potential (SSVEP)–based BCI system. We also discuss how SSVEP BCI systems can be improved to accommodate users with special needs.

Developing stimulus presentation on mobile devices for a truly portable SSVEP-based BCI

This study integrates visual stimulus presentation and near real-time data processing on a mobile device (e.g. a Tablet or a cell-phone) to implement a steady-state visual evoked potentials (SSVEP)-based brain-computer interface (BCI). The goal of this study is to increase the practicability, portability and ubiquity of an SSVEP-based BCI for daily use. The accuracy of flickering frequencies on the mobile SSVEP BCI system was tested against that on a laptop/desktop used in our previous studies. This study then analyzed the power spectrum density of the electroencephalogram signals elicited by the visual stimuli rendered on the mobile BCIs. Finally, this study performed an online test with the Tablet-based BCI system and obtained an averaged information transfer rate of 33.87 bits/min in three subjects. The current integration leads to a truly practical and ubiquitous SSVEP BCI on mobile devices for real-life applications.

Broad-Band Visually Evoked Potentials: Re(con)volution in Brain-Computer Interfacing

Brain-Computer Interfaces (BCIs) allow users to control devices and communicate by using brain activity only. BCIs based on broad-band visual stimulation can outperform BCIs using other stimulation paradigms. Visual stimulation with pseudo-random bit-sequences evokes specific Broad-Band Visually Evoked Potentials (BBVEPs) that can be reliably used in BCI for high-speed communication in speller applications. In this study, we report a novel paradigm for a BBVEP-based BCI that utilizes a generative framework to predict responses to broad-band stimulation sequences. In this study we designed a BBVEP-based BCI using modulated Gold codes to mark cells in a visual speller BCI. We defined a linear generative model that decomposes full responses into overlapping single-flash responses. These single-flash responses are used to predict responses to novel stimulation sequences, which in turn serve as templates for classification. The linear generative model explains on average 50% and up to 66% of the variance of responses to both seen and unseen sequences. In an online experiment, 12 participants tested a 6 × 6 matrix speller BCI. On average, an online accuracy of 86% was reached with trial lengths of 3.21 seconds. This corresponds to an Information Transfer Rate of 48 bits per minute (approximately 9 symbols per minute). This study indicates the potential to model and predict responses to broad-band stimulation. These predicted responses are proven to be well-suited as templates for a BBVEP-based BCI, thereby enabling communication and control by brain activity only.

Sequence detection analysis based on canonical correlation for steady-state visual evoked potential brain computer interfaces

BACKGROUND

Steady-state visual evoked potential (SSVEP) has been widely applied to develop brain computer interface (BCI) systems. The essence of SSVEP recognition is to recognize the frequency component of target stimulus focused by a subject significantly present in EEG spectrum.

NEW METHOD

In this paper, a novel statistical approach based on sequence detection (SD) is proposed for improving the performance of SSVEP recognition. This method uses canonical correlation analysis (CCA) coefficients to observe SSVEP signal sequence. And then, a threshold strategy is utilized for SSVEP recognition.

RESULTS

The result showed the classification performance with the longer duration of time window achieved the higher accuracy for most subjects. And the average time costing per trial was lower than the predefined recognition time. It was implicated that our approach could improve the speed of BCI system in contrast to other methods. Comparison with existing method(s): In comparison with other resultful algorithms, experimental accuracy of SD approach was better than those using a widely used CCA-based method and two newly proposed algorithms, least absolute shrinkage and selection operator (LASSO) recognition model as well as multivariate synchronization index (MSI) method. Furthermore, the information transfer rate (ITR) obtained by SD approach was higher than those using other three methods for most participants.

CONCLUSIONS

These conclusions demonstrated that our proposed method was promising for a high-speed online BCI.