Extracting Stress-Related EEG Patterns From Pre-Sleep EEG for Forecasting Slow-Wave Sleep Deficiency

Sleep is vital to our daily activity. Lack of proper sleep can impair functionality and overall health. While stress is known for its detrimental impact on sleep quality, the precise effect of pre-sleep stress on subsequent sleep structure remains unknown. This study introduced a novel approach to study the pre-sleep stress effect on sleep structure, specifically slow-wave sleep (SWS) deficiency. To achieve this, we selected forehead resting EEG immediately before and upon sleep onset to extract stress-related neurological markers through power spectra and entropy analysis. These markers include beta/delta correlation, alpha asymmetry, fuzzy entropy (FuzzEn) and spectral entropy (SpEn). Fifteen subjects were included in this study. Our results showed that subjects lacking SWS often exhibited signs of stress in EEG, such as an increased beta/delta correlation, higher alpha asymmetry, and increased FuzzEn in frontal EEG. Conversely, individuals with ample SWS displayed a weak beta/delta correlation and reduced FuzzEn. Finally, we employed several supervised learning models and found that the selected neurological markers can predict subsequent SWS deficiency. Our investigation demonstrated that the classifiers could effectively predict varying levels of slow-wave sleep (SWS) from pre-sleep EEG segments, achieving a mean balanced accuracy surpassing 0.75. The SMOTE-Tomek resampling method could improve the performance to 0.77. This study suggests that stress-related neurological markers derived from pre-sleep EEG can effectively predict SWS deficiency. Such information can be integrated with existing sleep-improving techniques to provide a personalized sleep forecasting and improvement solution.

Activation network improves spatiotemporal modelling of human brain communication processes

Dynamic functional networks (DFN) have considerably advanced modelling of the brain communication processes. The prevailing implementation capitalizes on the system and network-level correlations between time series. However, this approach does not account for the continuous impact of non-dynamic dependencies within the statistical correlation, resulting in relatively stable connectivity patterns of DFN over time with limited sensitivity for communication dynamic between brain regions. Here, we propose an activation network framework based on the activity of functional connectivity (AFC) to extract new types of connectivity patterns during brain communication process. The AFC captures potential time-specific fluctuations associated with the brain communication processes by eliminating the non-dynamic dependency of the statistical correlation. In a simulation study, the positive correlation (r=0.966,p<0.001) between the extracted dynamic dependencies and the simulated "ground truth" validates the method's dynamic detection capability. Applying to autism spectrum disorders (ASD) and COVID-19 datasets, the proposed activation network extracts richer topological reorganization information, which is largely invisible to the DFN. Detailed, the activation network exhibits significant inter-regional connections between function-specific subnetworks and reconfigures more efficiently in the temporal dimension. Furthermore, the DFN fails to distinguish between patients and healthy controls. However, the proposed method reveals a significant decrease (p<0.05) in brain information processing abilities in patients. Finally, combining two types of networks successfully classifies ASD (83.636 % ± 11.969 %,mean±std) and COVID-19 (67.333 % ± 5.398 %). These findings suggest the proposed method could be a potential analytic framework for elucidating the neural mechanism of brain dynamics.

A data expansion technique based on training and testing sample to boost the detection of SSVEPs for brain-computer interfaces

Objective.Currently, steady-state visual evoked potentials (SSVEPs)-based brain-computer interfaces (BCIs) have achieved the highest interaction accuracy and speed among all BCI paradigms. However, its decoding efficacy depends deeply on the number of training samples, and the system performance would have a dramatic drop when the training dataset decreased to a small size. To date, no study has been reported to incorporate the unsupervised learning information from testing trails into the construction of supervised classification model, which is a potential way to mitigate the overfitting effect of limited samples.Approach.This study proposed a novel method for SSVEPs detection, i.e. cyclic shift trials (CSTs), which could combine unsupervised learning information from test trials and supervised learning information from train trials. Furthermore, since SSVEPs are time-locked and phase-locked to the onset of specific flashes, CST could also expand training samples on the basis of its regularity and periodicity. In order to verify the effectiveness of CST, we designed an online SSVEP-BCI system, and tested this system combined CST with two common classification algorithms, i.e. extended canonical correlation analysis and ensemble task-related component analysis.Main results.CST could significantly enhance the signal to noise ratios of SSVEPs and improve the performance of systems especially for the condition of few training samples and short stimulus time. The online information transfer rate could reach up to 236.19 bits min-1using 36 s calibration time of only one training sample for each category.Significance.The proposed CST method can take full advantages of supervised learning information from training samples and unsupervised learning information of testing samples. Furthermore, it is a data expansion technique, which can enhance the SSVEP characteristics and reduce dependence on sample size. Above all, CST is a promising method to improve the performance of SSVEP-based BCI without any additional experimental burden.

Almost free of calibration for SSVEP-based brain-computer interfaces

Objective. Steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) is a promising technology that can achieve high information transfer rate (ITR) with supervised algorithms such as ensemble task-related component analysis (eTRCA) and task-discriminant component analysis (TDCA). However, training individual models requires a tedious and time-consuming calibration process, which hinders the real-life use of SSVEP-BCIs. A recent data augmentation method, called source aliasing matrix estimation (SAME), can generate new EEG samples from a few calibration trials. But SAME does not exploit the information across stimuli as well as only reduces the number of calibration trials per command, so it still has some limitations.Approach. This study proposes an extended version of SAME, called multi-stimulus SAME (msSAME), which exploits the similarity of the aliasing matrix across frequencies to enhance the performance of SSVEP-BCI with insufficient calibration trials. We also propose a semi-supervised approach based on msSAME that can further reduce the number of SSVEP frequencies needed for calibration. We evaluate our method on two public datasets, Benchmark and BETA, and an online experiment.Main results. The results show that msSAME outperforms SAME for both eTRCA and TDCA on the public datasets. Moreover, the semi-supervised msSAME-based method achieves comparable performance to the fully calibrated methods and outperforms the conventional free-calibrated methods. Remarkably, our method only needs 24 s to calibrate 40 targets in the online experiment and achieves an average ITR of 213.8 bits min-1with a peak of 242.6 bits min-1.Significance. This study significantly reduces the calibration effort for individual SSVEP-BCIs, which is beneficial for developing practical plug-and-play SSVEP-BCIs.

Cross-dataset transfer learning for motor imagery signal classification via multi-task learning and pre-training

Objective.Deep learning (DL) models have been proven to be effective in decoding motor imagery (MI) signals in Electroencephalogram (EEG) data. However, DL models' success relies heavily on large amounts of training data, whereas EEG data collection is laborious and time-consuming. Recently, cross-dataset transfer learning has emerged as a promising approach to meet the data requirements of DL models. Nevertheless, transferring knowledge across datasets involving different MI tasks remains a significant challenge in cross-dataset transfer learning, limiting the full utilization of valuable data resources.

APPROACH

This study proposes a pre-training-based cross-dataset transfer learning method inspired by Hard Parameter Sharing in multi-task learning. Different datasets with distinct MI paradigms are considered as different tasks, classified with shared feature extraction layers and individual task-specific layers to allow cross-dataset classification with one unified model. Then, Pre-training and fine-tuning are employed to transfer knowledge across datasets. We also designed four fine-tuning schemes and conducted extensive experiments on them.

MAIN RESULTS

The results showed that compared to models without pre-training, models with pre-training achieved a maximum increase in accuracy of 7.76%. Moreover, when limited training data were available, the pre-training method significantly improved DL model's accuracy by 27.34% at most. The experiments also revealed that pre-trained models exhibit faster convergence and remarkable robustness. The training time per subject could be reduced by up to 102.83 s, and the variance of classification accuracy decreased by 75.22% at best.

SIGNIFICANCE

This study represents the first comprehensive investigation of the cross-dataset transfer learning method between two datasets with different MI tasks. The proposed pre-training method requires only minimal fine-tuning data when applying DL models to new MI paradigms, making MI-Brain-computer interface more practical and user-friendly.

Investigating EEG-based cross-session and cross-task vigilance estimation in BCI systems

Objective. The state of vigilance is crucial for effective performance in brain-computer interface (BCI) tasks, and therefore, it is essential to investigate vigilance levels in BCI tasks. Despite this, most studies have focused on vigilance levels in driving tasks rather than on BCI tasks, and the electroencephalogram (EEG) patterns of vigilance states in different BCI tasks remain unclear. This study aimed to identify similarities and differences in EEG patterns and performances of vigilance estimation in different BCI tasks and sessions.Approach.To achieve this, we built a steady-state visual evoked potential-based BCI system and a rapid serial visual presentation-based BCI system and recruited 18 participants to carry out four BCI experimental sessions over four days.Main results. Our findings demonstrate that specific neural patterns for high and low vigilance levels are relatively stable across sessions. Differential entropy features significantly differ between different vigilance levels in all frequency bands and between BCI tasks in the delta and theta frequency bands, with the theta frequency band features playing a critical role in vigilance estimation. Additionally, prefrontal, temporal, and occipital regions are more relevant to the vigilance state in BCI tasks. Our results suggest that cross-session vigilance estimation is more accurate than cross-task estimation.Significance.Our study clarifies the underlying mechanisms of vigilance state in two BCI tasks and provides a foundation for further research in vigilance estimation in BCI applications.

Diversity and Suitability of the State-of-the-Art Wearable and Wireless EEG Systems Review

Wireless electroencephalography (EEG) systems have been attracting increasing attention in recent times. Both the number of articles discussing wireless EEG and their proportion relative to general EEG publications have increased over years. These trends indicate that wireless EEG systems could be more accessible to researchers and the research community has recognized the potential of wireless EEG systems. To explore the development and diverse applications of wireless EEG systems, this review highlights the trends in wearable and wireless EEG systems over the past decade and compares the specifications and research applications of the major wireless systems marketed by 16 companies. For each product, five parameters (number of channels, sampling rate, cost, battery life, and resolution) were assessed for comparison. Currently, these wearable and portable wireless EEG systems have three main application areas: consumer, clinical, and research. To address this multitude of options, the article also discussed the thought process to find a suitable device that meets personalization and use cases specificities. These investigations suggest that low-price and convenience are key factors for consumer applications, wireless EEG systems with FDA or CE-certification may be more suitable for clinical settings, and devices that provide raw EEG data with high-density channels are important for laboratory research. This article presents an overview of the current state of the wireless EEG systems specifications and possible applications and serves as a guide point as it is expected that more influential and novel research will cyclically promote the development of such EEG systems.

Design and Test of Spatial Cognitive Training and Evaluation System based on Virtual Reality Head-mounted Display with EEG Recording

The field of spatial cognitive training and evaluation has rapidly evolved. However, the low learning motivation and engagement of the subjects hinder the widespread use of spatial cognitive training. This study designed a home-based spatial cognitive training and evaluation system (SCTES), which aimed to train subjects on spatial cognitive tasks for 20 days, and compared the brain activities before and after the training. This study also evaluated the feasibility of using a portable all-in-one prototype for cognitive training that combined a virtual reality (VR) head-mounted display with high-quality electroencephalogram (EEG) recording. During the course of training, the length of the navigation path and the distance between the starting position and the platform position revealed significant behavioral differences. In the testing sessions, the subjects showed significant behavioral differences in the time it took to complete the test task before and after training. After only four days of training, the subjects demonstrated significant differences in the Granger causality analysis (GCA) characteristics of brain regions in the δ, θ, α₁, β₂, and γ frequency bands of the EEG, as well as significant differences in the GCA of the EEG in the β₁, β₂, and γ frequency bands between the two test sessions. The proposed SCTES used a compact and all-in-one form factor to train and evaluate spatial cognition and collect EEG signals and behavioral data simultaneously. The recorded EEG data can be used to quantitatively assess the efficacy of spatial training in patients with spatial cognitive impairments.

Stimulus Design for Visual Evoked Potential Based Brain-Computer Interfaces

Visual stimuli design plays an important role in brain-computer interfaces (BCIs) based on visual evoked potentials (VEPs). Variations in stimulus parameters have been shown to affect both decoding accuracy and subjective perception experience, implying the need for a trade-off in design. In this study, we comprehensively and systematically compared various combinations of amplitude contrast and spectral content parameters in the stimulus design to quantify their impact on decoding performance and subject comfort. Specifically, three parameters were investigated: 1) contrast level, 2) temporal pattern (periodic steady-state or pseudo-random code-modulated), and 3) frequency range. We collected electroencephalogram (EEG) data and subjective perception ratings from ten subjects and evaluated the decoding accuracy and subject comfort rating for different combinations of the stimulus parameters. Our results indicate that while high-frequency steady-state VEP (SSVEP) stimuli were rated the most comfortable, they also had the lowest decoding accuracy. Conversely, low-frequency SSVEP stimuli were rated the least comfortable but had the highest decoding accuracy. Standard and high-frequency M-sequence code-modulated VEPs (c-VEPs) produced intermediates between the two. We observed a consistent trade-off relationship between decoding accuracy and subjective comfort level across all parameters. Based on our findings, we offer c-VEP as a preferable stimulus for achieving reliable decoding accuracy while maintaining a reasonable level of comfortability.

Modulation of rhythmic visual stimulation on left-right attentional asymmetry

The rhythmic visual stimulation (RVS)-induced oscillatory brain responses, namely steady-state visual evoked potentials (SSVEPs), have been widely used as a biomarker in studies of neural processing based on the assumption that they would not affect cognition. However, recent studies have suggested that the generation of SSVEPs might be attributed to neural entrainment and thus could impact brain functions. But their neural and behavioral effects are yet to be explored. No study has reported the SSVEP influence on functional cerebral asymmetry (FCA). We propose a novel lateralized visual discrimination paradigm to test the SSVEP effects on visuospatial selective attention by FCA analyses. Thirty-eight participants covertly shifted their attention to a target triangle appearing in either the lower-left or -right visual field (LVF or RVF), and judged its orientation. Meanwhile, participants were exposed to a series of task-independent RVSs at different frequencies, including 0 (no RVS), 10, 15, and 40-Hz. As a result, it showed that target discrimination accuracy and reaction time (RT) varied significantly across RVS frequency. Furthermore, attentional asymmetries differed for the 40-Hz condition relative to the 10-Hz condition as indexed by enhanced RT bias to the right visual field, and larger Pd EEG component for attentional suppression. Our results demonstrated that RVSs had frequency-specific effects on left-right attentional asymmetries in both behavior and neural activities. These findings provided new insights into the functional role of SSVEP on FCAs.

EEG-Based Brain-Computer Interfaces Are Vulnerable to Backdoor Attacks

Research and development of electroencephalogram (EEG) based brain-computer interfaces (BCIs) have advanced rapidly, partly due to deeper understanding of the brain and wide adoption of sophisticated machine learning approaches for decoding the EEG signals. However, recent studies have shown that machine learning algorithms are vulnerable to adversarial attacks. This paper proposes to use narrow period pulse for poisoning attack of EEG-based BCIs, which makes adversarial attacks much easier to implement. One can create dangerous backdoors in the machine learning model by injecting poisoning samples into the training set. Test samples with the backdoor key will then be classified into the target class specified by the attacker. What most distinguishes our approach from previous ones is that the backdoor key does not need to be synchronized with the EEG trials, making it very easy to implement. The effectiveness and robustness of the backdoor attack approach is demonstrated, highlighting a critical security concern for EEG-based BCIs and calling for urgent attention to address it.

Using EEG signals to assess workload during memory retrieval in a real-world scenario

OBJECTIVE

The Electroencephalogram (EEG) is gaining popularity as a physiological measure for neuroergonomics in human factor studies because it is objective, less prone to bias, and capable of assessing the dynamics of cognitive states. This study investigated the associations between memory workload and EEG during participants' typical office tasks on a single-monitor and dual-monitor arrangement. We expect a higher memory workload for the single-monitor arrangement.&#xD;&#xD;Approach: We designed an experiment that mimics the scenario of a subject performing some office work and examined whether the subjects experienced various levels of memory workload in two different office setups: 1) a single-monitor setup and 2) a dual-monitor&#xD;setup. We used EEG band power, mutual information, and coherence as features to train machine learning models to classify high versus low memory workload states.&#xD;&#xD;Main results: The study results showed that these characteristics exhibited significant differences that were consistent across all participants. We also verified the robustness and consistency of these EEG signatures in a different data set collected during a Sternberg task in a prior study.&#xD;&#xD;Significance: The study found the EEG correlates of memory workload across individuals, demonstrating the effectiveness of using EEG analysis in conducting real-world neuroergonomic studies.

Human Brain Dynamics and Coordination Reflect the Task Difficulty of Optical Image Relational Reasoning

Despite advances in neuroscience, the mechanisms by which human brain resolve optical image formation through relational reasoning remain unclear, particularly its relationships with task difficulty. Therefore, this study explores the underlying brain dynamics involved in optical image formation tasks at various difficulty levels, including those with a single convex lens and a single mirror. Compared to single convex lens relational reasoning with high task difficulty, the single mirror relational reasoning exhibited significantly higher response accuracy and shorter latency. As compared to single mirror tasks, single convex tasks exhibited greater frontal midline theta augmentation and right parietal alpha suppression during phase I and earlier phase II, and augmentation of frontal midline theta, right parietal-occipital alpha, and left mu alpha suppression during late phase II. Moreover, the frontal midline theta power in late phase II predicts the likelihood of solving single convex tasks the best, while the parietal alpha power in phase I is most predictive. In addition, frontal midline theta power exhibited stronger synchronization with right parietal alpha, right occipital alpha, and mu alpha power when solving single convex tasks than single mirror tasks. In summary, having stronger brain dynamics and coordination is vital for achieving optical image formation with greater difficulty.

Enhanced electroencephalography effective connectivity in frontal low-gamma band correlates of emotional regulation after mindfulness training

Practicing mindfulness, focusing attention on the internal and external experiences occurring in the present moment with open and nonjudgement stance, can lead to the development of emotional regulation skills. Yet, the effective connectivity of brain regions during mindfulness has been largely unexplored. Studies have shown that mindfulness practice promotes functional connectivity in practitioners, potentially due to improved emotional regulation abilities and increased connectivity in the lateral prefrontal areas. To examine the changes in effective connectivity due to mindfulness training, we analyzed electroencephalogram (EEG) signals taken before and after mindfulness training, focusing on training-related effective connectivity changes in the frontal area. The mindfulness training group participated in an 8-week mindfulness-based stress reduction (MBSR) program. The control group did not take part. Regardless of the specific mindfulness practice used, low-gamma band effective connectivity increased globally after the mindfulness training. High-beta band effective connectivity increased globally only during Breathing. Moreover, relatively higher outgoing effective connectivity strength was seen during Resting and Breathing and Body-scan. By analyzing the changes in outgoing and incoming connectivity edges, both F7 and F8 exhibited strong parietal connectivity during Resting and Breathing. Multiple regression analysis revealed that the changes in effective connectivity of the right lateral prefrontal area predicted mindfulness and emotional regulation abilities. These results partially support the theory that the lateral prefrontal areas have top-down modulatory control, as these areas have high outflow effective connectivity, implying that mindfulness training cultivates better emotional regulation.

A high-speed hybrid brain-computer interface with more than 200 targets

Objective. Brain-computer interfaces (BCIs) have recently made significant strides in expanding their instruction set, which has attracted wide attention from researchers. The number of targets and commands is a key indicator of how well BCIs can decode the brain's intentions. No studies have reported a BCI system with over 200 targets.Approach. This study developed the first high-speed BCI system with up to 216 targets that were encoded by a combination of electroencephalography features, including P300, motion visual evoked potential (mVEP), and steady-state visual evoked potential (SSVEP). Specifically, the hybrid BCI paradigm used the time-frequency division multiple access strategy to elaborately tag targets with P300 and mVEP of different time windows, along with SSVEP of different frequencies. The hybrid features were then decoded by task-discriminant component analysis and linear discriminant analysis. Ten subjects participated in the offline and online cued-guided spelling experiments. Other ten subjects took part in online free-spelling experiments.Main results.The offline results showed that the mVEP and P300 components were prominent in the central, parietal, and occipital regions, while the most distinct SSVEP feature was in the occipital region. The online cued-guided spelling and free-spelling results showed that the proposed BCI system achieved an average accuracy of 85.37% ± 7.49% and 86.00% ± 5.98% for the 216-target classification, resulting in an average information transfer rate (ITR) of 302.83 ± 39.20 bits min-1and 204.47 ± 37.56 bits min-1, respectively. Notably, the peak ITR could reach up to 367.83 bits min-1.Significance.This study developed the first high-speed BCI system with more than 200 targets, which holds promise for extending BCI's application scenarios.

Novel Moving Steady-State Visual Evoked Potential Stimulus to Assess Afferent and Efferent Dysfunction in Multiple Sclerosis

Afferent and efferent visual dysfunction are prominent features of multiple sclerosis (MS). Visual outcomes have been shown to be robust biomarkers of the overall disease state. Unfortunately, precise measurement of afferent and efferent function is typically limited to tertiary care facilities, which have the equipment and analytical capacity to make these measurements, and even then, only a few centers can accurately quantify both afferent and efferent dysfunction. These measurements are currently unavailable in acute care facilities (ER, hospital floors). We aimed to develop a moving multifocal steady-state visual evoked potential (mfSSVEP) stimulus to simultaneously assess afferent and efferent dysfunction in MS for application on a mobile platform. The brain-computer interface (BCI) platform consists of a head-mounted virtual-reality headset with electroencephalogram (EEG) and electrooculogram (EOG) sensors. To evaluate the platform, we recruited consecutive patients who met the 2017 MS McDonald diagnostic criteria and healthy controls for a pilot cross-sectional study. Nine MS patients (mean age 32.7 years, SD 4.33) and ten healthy controls (24.9 years, SD 7.2) completed the research protocol. The afferent measures based on mfSSVEPs showed a significant difference between the groups (signal-to-noise ratio of mfSSVEPs for controls: 2.50 ± 0.72 vs. MS: 2.04 ± 0.47) after controlling for age (p = 0.049). In addition, the moving stimulus successfully induced smooth pursuit movement that can be measured by the EOG signals. There was a trend for worse smooth pursuit tracking in cases vs. controls, but this did not reach nominal statistical significance in this small pilot sample. This study introduces a novel moving mfSSVEP stimulus for a BCI platform to evaluate neurologic visual function. The moving stimulus showed a reliable capability to assess both afferent and efferent visual functions simultaneously.

Enhancing Detection of Multi-Frequency-Modulated SSVEP Using Phase Difference Constrained Canonical Correlation Analysis

OBJECTIVE

Multi-frequency-modulated visual stimulation scheme has been shown effective for the steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) recently, especially in increasing the visual target number with less stimulus frequencies and mitigating the visual fatigue. However, the existing calibration-free recognition algorithms based on the traditional canonical correlation analysis (CCA) cannot provide the merited performance.

APPROACH

To improve the recognition performance, this study proposes a phase difference constrained CCA (pdCCA), which assumes that the multi-frequency-modulated SSVEPs share a common spatial filter over different frequencies and have a specified phase difference. Specifically, during the CCA computation, the phase differences of the spatially filtered SSVEPs are constrained using the temporal concatenation of the sine-cosine reference signals with the pre-defined initial phases.

MAIN RESULTS

We evaluate the performance of the proposed pdCCA-based method on three representative multi-frequency-modulated visual stimulation paradigms (i.e., based on the multi-frequency sequential coding, the dual-frequency, and the amplitude modulation). The evaluation results on four SSVEP datasets (Dataset Ia, Ib, II, and III) show that the pdCCA-based method can significantly outperform the current CCA method in terms of recognition accuracy. It improves the accuracy by 22.09% in Dataset Ia, 20.86% in Dataset Ib, 8.61% in Dataset II, and 25.85% in Dataset III.

SIGNIFICANCE

The pdCCA-based method, which actively controls the phase difference of the multi-frequency-modulated SSVEPs after spatial filtering, is a new calibration-free method for multi-frequency-modulated SSVEP-based BCIs.

[Research advances in non-invasive brain-computer interface control strategies]

Brain-computer interface (BCI) can establish a direct communications pathway between the human brain and the external devices, which is independent of peripheral nerves and muscles. Compared with invasive BCI, non-invasive BCI has the advantages of low cost, low risk, and ease of operation. In recent years, using non-invasive BCI technology to control devices has gradually evolved into a new type of human-computer interaction manner. Moreover, the control strategy for BCI is an essential component of this manner. First, this study introduced how the brain control techniques were developed and classified. Second, the basic characteristics of direct and shared control strategies were thoroughly explained. And then the benefits and drawbacks of these two strategies were compared and further analyzed. Finally, the development direction and application prospects for non-invasive brain control strategies were suggested.

IC-U-Net: A U-Net-based Denoising Autoencoder Using Mixtures of Independent Components for Automatic EEG Artifact Removal

Electroencephalography (EEG) signals are often contaminated with artifacts. It is imperative to develop a practical and reliable artifact removal method to prevent the misinterpretation of neural signals and the underperformance of brain-computer interfaces. Based on the U-Net architecture, we developed a new artifact removal model, IC-U-Net, for removing pervasive EEG artifacts and reconstructing brain signals. IC-U-Net was trained using mixtures of brain and non-brain components decomposed by independent component analysis. It uses an ensemble of loss functions to model complex signal fluctuations in EEG recordings. The effectiveness of the proposed method in recovering brain activities and removing various artifacts (e.g., eye blinks/movements, muscle activities, and line/channel noise) was demonstrated in a simulation study and four real-world EEG experiments. IC-U-Net can reconstruct a multi-channel EEG signal and is applicable to most artifact types, offering a promising end-to-end solution for automatically removing artifacts from EEG recordings. It also meets the increasing need to image natural brain dynamics in a mobile setting. The code and pre-trained IC-U-Net model are available at https://github.com/roseDwayane/AIEEG.

Stimulus-Stimulus Transfer Based on Time-Frequency-Joint Representation in SSVEP-Based BCIs

OBJECTIVE

Brain-computer interfaces (BCIs) based on steady-state visual evoked potential (SSVEP) require extensive and costly calibration to achieve high performance. Using transfer learning to re-use existing calibration data from old stimuli is a promising strategy, but finding commonalities in the SSVEP signals across different stimuli remains a challenge.

METHOD

This study presents a new perspective, namely time-frequency-joint representation, in which SSVEP signals corresponding to different stimuli can be synchronized, and thus can emphasize common components. According to this time-frequency-joint representation, an adaptive decomposition technique based on the multi-channel adaptive Fourier decomposition (MAFD) is proposed to adaptively decompose SSVEP signals of different stimuli simultaneously. Then, common components can be identified and transformed across stimuli.

RESULTS

A simulation study on public SSVEP datasets demonstrates that the proposed stimulus-stimulus transfer method has the ability to extract and transfer these common components across stimuli. By using calibration data from eight source stimuli, the proposed stimulus-stimulus transfer method can generate SSVEP templates of other 32 target stimuli. It boosts the stimulus-stimulus transfer based recognition method's ITR from 95.966 bits/min to 123.684 bits/min.

CONCLUSION

By extracting and transfer common components across stimuli in the proposed time-frequency-joint representation, the proposed stimulus-stimulus transfer method produces good classification performance without requiring calibration data of target stimuli.

SIGNIFICANCE

This study provides a synchronization standpoint to analyze and model SSVEP signals. In addition, the proposed stimulus-stimulus method shortens the calibration time and thus improve comfort, which could facilitate real-world applications of SSVEP-based BCIs.

Development of an Adaptive Artifact Subspace Reconstruction Based on Hebbian/Anti-Hebbian Learning Networks for Enhancing BCI Performance

Brain-computer interface (BCI) actively translates the brain signals into executable actions by establishing direct communication between the human brain and external devices. Recording brain activity through electroencephalography (EEG) is generally contaminated with both physiological and nonphysiological artifacts, which significantly hinders the BCI performance. Artifact subspace reconstruction (ASR) is a well-known statistical technique that automatically removes artifact components by determining the rejection threshold based on the initial reference EEG segment in multichannel EEG recordings. In real-world applications, the fixed threshold may limit the efficacy of the artifact correction, especially when the quality of the reference data is poor. This study proposes an adaptive online ASR technique by integrating the Hebbian/anti-Hebbian neural networks into the ASR algorithm, namely, principle subspace projection ASR (PSP-ASR) and principal subspace whitening ASR (PSW-ASR) that segmentwise self-organize the artifact subspace by updating the synaptic weights according to the Hebbian and anti-Hebbian learning rules. The effectiveness of the proposed algorithm is compared to the conventional ASR approaches on benchmark EEG dataset and three BCI frameworks, including steady-state visual evoked potential (SSVEP), rapid serial visual presentation (RSVP), and motor imagery (MI) by evaluating the root-mean-square error (RMSE), the signal-to-noise ratio (SNR), the Pearson correlation, and classification accuracy. The results demonstrated that the PSW-ASR algorithm effectively removed the EEG artifacts and retained the activity-specific brain signals compared to the PSP-ASR, standard ASR (Init-ASR), and moving-window ASR (MW-ASR) methods, thereby enhancing the SSVEP, RSVP, and MI BCI performances. Finally, our empirical results from the PSW-ASR algorithm suggested the choice of an aggressive cutoff range of c = 1-10 for activity-specific BCI applications and a moderate range of for the benchmark dataset and general BCI applications.

[A review of researches on decoding algorithms of steady-state visual evoked potentials]

Brain-computer interface (BCI) systems based on steady-state visual evoked potential (SSVEP) have become one of the major paradigms in BCI research due to their high signal-to-noise ratio and short training time required by users. Fast and accurate decoding of SSVEP features is a crucial step in SSVEP-BCI research. However, the current researches lack a systematic overview of SSVEP decoding algorithms and analyses of the connections and differences between them, so it is difficult for researchers to choose the optimum algorithm under different situations. To address this problem, this paper focuses on the progress of SSVEP decoding algorithms in recent years and divides them into two categories-trained and non-trained-based on whether training data are needed. This paper also explains the fundamental theories and application scopes of decoding algorithms such as canonical correlation analysis (CCA), task-related component analysis (TRCA) and the extended algorithms, concludes the commonly used strategies for processing decoding algorithms, and discusses the challenges and opportunities in this field in the end.

Unsupervised Learning of Brain State Dynamics during Emotion Imagination using High-Density EEG

This study applies adaptive mixture independent component analysis (AMICA) to learn a set of ICA models, each optimized by fitting a distributional model for each identified component process while maximizing component process independence within some subsets of time points of a multi-channel EEG dataset. Here, we applied 20-model AMICA decomposition to long-duration (1-2 hr), high-density (128-channel) EEG data recorded while participants used guided imagination to imagine situations stimulating the experience of 15 specified emotions. These decompositions tended to return models identifying spatiotemporal EEG patterns or states within single emotion imagination periods. Model probability transitions reflected time-courses of EEG dynamics during emotion imagination, which varied across emotions. Transitions between models accounting for imagined "grief" and "happiness" were more abrupt and better aligned with participant reports, while transitions for imagined "contentment" extended into adjoining "relaxation" periods. The spatial distributions of brain-localizable independent component processes (ICs) were more similar within participants (across emotions) than emotions (across participants). Across participants, brain regions with differences in IC spatial distributions (i.e., dipole density) between emotion imagination versus relaxation were identified in or near the left rostrolateral prefrontal, posterior cingulate cortex, right insula, bilateral sensorimotor, premotor, and associative visual cortex. No difference in dipole density was found between positive versus negative emotions. AMICA models of changes in high-density EEG dynamics may allow data-driven insights into brain dynamics during emotional experience, possibly enabling the improved performance of EEG-based emotion decoding and advancing our understanding of emotion.

Data augmentation of SSVEPs using source aliasing matrix estimation for brain-computer interfaces

OBJECTIVE

Currently, ensemble task-related component analysis (eTRCA) and task discriminative component analysis (TDCA) are the state-of-the-art algorithms for steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs). However, training the BCIs requires multiple calibration trials. With insufficient calibration data, the accuracy of the BCI will degrade, or even become invalid with only one calibration trial. However, collecting a large amount of electroencephalography (EEG) data for calibration is a time-consuming and laborious process, which hinders the practical use of eTRCA and TDCA.

METHODS

This study proposed a novel method, namely Source Aliasing Matrix Estimation (SAME), to augment the calibration data for SSVEP-BCIs. SAME could generate artificial EEG trials with the featured SSVEPs. Its effectiveness was evaluated using two public datasets (i.e., Benchmark, BETA).

RESULTS

When combined with SAME, both eTRCA and TDCA had significantly improved performance with a limited number of calibration data. Specifically, SAME increased the average accuracy of eTRCA and TDCA by about 12% and 3%, respectively, with as few as two calibration trials. Notably, SAME enabled eTRCA and TDCA to work well with a single calibration trial, achieving an average accuracy >90% for the Benchmark dataset and >70% for the BETA dataset with 1-second EEG.

CONCLUSION

SAME is an effective method for SSVEP-BCIs to augment the calibration data, thereby significantly enhancing the performance of eTRCA and TDCA.

SIGNIFICANCE

We propose a new data-augmentation method that is compatible with the state-of-the-art algorithms of SSVEP-based BCIs. It can significantly reduce the efforts required to calibrate SSVEP-BCIs, which is promising for the development of practical BCIs.

Online Adaptation Boosts SSVEP-Based BCI Performance

OBJECTIVE

A user-friendly steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) prefers no calibration for its target recognition algorithm, however, the existing calibration-free schemes perform still far behind their calibration-based counterparts. To tackle this issue, learning online from the subject's unlabeled data is investigated as a potential approach to boost the performance of the calibration-free SSVEP-based BCIs.

METHODS

An online adaptation scheme is developed to tune the spatial filters using the online unlabeled data from previous trials, and then developing the online adaptive canonical correlation analysis (OACCA) method.

RESULTS

A simulation study on two public SSVEP datasets (Dataset I and II) with a total of 105 subjects demonstrated that the proposed online adaptation scheme can boost the CCA's averaged information transfer rate (ITR) from 94.60 to 158.87 bits/min in Dataset I and from 85.80 to 123.91 bits/min in Dataset II. Furthermore, in our online experiment it boosted the CCA's ITR from 55.81 bits/min to 95.73 bits/min. More importantly, this online adaptation scheme can be easily combined with any spatial filtering-based algorithms to achieve online learning.

CONCLUSION

By online adaptation, the proposed OACCA performed much better than the calibration-free CCA, and comparable to the calibration-based algorithms.

SIGNIFICANCE

This work provides a general way for the SSVEP-based BCIs to learn online from unlabeled data and thus avoid calibration.

Resting-State EEG Signal for Major Depressive Disorder Detection: A Systematic Validation on a Large and Diverse Dataset

Major depressive disorder (MDD) is a global healthcare issue and one of the leading causes of disability. Machine learning combined with non-invasive electroencephalography (EEG) has recently been shown to have the potential to diagnose MDD. However, most of these studies analyzed small samples of participants recruited from a single source, raising serious concerns about the generalizability of these results in clinical practice. Thus, it has become critical to re-evaluate the efficacy of various common EEG features for MDD detection across large and diverse datasets. To address this issue, we collected resting-state EEG data from 400 participants across four medical centers and tested classification performance of four common EEG features: band power (BP), coherence, Higuchi's fractal dimension, and Katz's fractal dimension. Then, a sequential backward selection (SBS) method was used to determine the optimal subset. To overcome the large data variability due to an increased data size and multi-site EEG recordings, we introduced the conformal kernel (CK) transformation to further improve the MDD as compared with the healthy control (HC) classification performance of support vector machine (SVM). The results show that (1) coherence features account for 98% of the optimal feature subset; (2) the CK-SVM outperforms other classifiers such as K-nearest neighbors (K-NN), linear discriminant analysis (LDA), and SVM; (3) the combination of the optimal feature subset and CK-SVM achieves a high five-fold cross-validation accuracy of 91.07% on the training set (140 MDD and 140 HC) and 84.16% on the independent test set (60 MDD and 60 HC). The current results suggest that the coherence-based connectivity is a more reliable feature for achieving high and generalizable MDD detection performance in real-life clinical practice.

Near-zero phase-lag hyperscanning in a novel wireless EEG system

Objective. Hyperscanning is an emerging technology that concurrently scans the neural dynamics of multiple individuals to study interpersonal interactions. In particular, hyperscanning with electroencephalography (EEG) is increasingly popular owing to its mobility and its ability to allow studying social interactions in naturalistic settings at the millisecond scale.Approach.To align multiple EEG time series with sophisticated event markers in a single time domain, a precise and unified timestamp is required for stream synchronization. This study proposes a clock-synchronized method that uses a custom-made RJ45 cable to coordinate the sampling between wireless EEG amplifiers to prevent incorrect estimation of interbrain connectivity due to asynchronous sampling. In this method, analog-to-digital converters are driven by the same sampling clock. Additionally, two clock-synchronized amplifiers leverage additional radio frequency channels to keep the counter of their receiving dongles updated, which guarantees that binding event markers received by the dongle with the EEG time series have the correct timestamp.Main results.The results of two simulation experiments and one video gaming experiment reveal that the proposed method ensures synchronous sampling in a system with multiple EEG devices, achieving near-zero phase lag and negligible amplitude difference between the signals.Significance.According to all of the signal-similarity metrics, the suggested method is a promising option for wireless EEG hyperscanning and can be utilized to precisely assess the interbrain couplings underlying social-interaction behaviors.

Affective response to volitional input perturbations in obstacle avoidance and target tracking games

We present the use of two game-like tasks, Catnip and Dinorun, to explore affective responses to volitional control perturbations. We analyze behavioral and physiological measures with the self-assessment manikin (SAM), pupillometry, and electroencephalography (EEG) responses to provide intratrial emotional state as well as inter-trial correlates with selfreported survey responses. We find that subject gameplay characteristics significantly correlate with valence and dominance scores for both games, and that perturbations to the games produce a measurable decrease in response scores for Dinorun. During perturbation events, pupillometry analysis reveals considerable SAM-agnostic dilation, with stronger responses in more rigid trialized event structures. Furthermore, analyses of neural activity from central and parietal regions demonstrate significant measurable evoked responses to perturbed events across the majority of subjects for both games. By introducing perturbations, this set of experiments and analyses inform and enable further studies of affective responses to the loss of volitional control during engaging, game-like tasks.

Improving Transfer Performance of Deep Learning with Adaptive Batch Normalization for Brain-computer Interfaces. Annu Int Conf IEEE Eng Med Biol Soc 2021;2021:5800-5803. [PMID: 34892438 DOI: 10.1109/embc46164.2021.9629529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Recently, transfer learning and deep learning have been introduced to solve intra- and inter-subject variability problems in Brain-Computer Interfaces. However, the generalization ability of these BCIs is still to be further verified in a cross-dataset scenario. This study compared the transfer performance of manifold embedded knowledge transfer and pre-trained EEGNet with three preprocessing strategies. This study also introduced AdaBN for target domain adaptation. The results showed that EEGNet with Riemannian alignment and AdaBN could achieve the best transfer accuracy about 65.6% on the target dataset. This study may provide new insights into the design of transfer neural networks for BCIs by separating source and target batch normalization layers in the domain adaptation process.

Classification of left-versus right-hand motor imagery in stroke patients using supplementary data generated by CycleGAN

Acquiring Electroencephalography (EEG) data is often time-consuming, laborious, and costly, posing practical challenges to train powerful but data-demanding deep learning models. This study proposes a surrogate EEG data-generation system based on cycle-consistent adversarial networks (CycleGAN) that can expand the number of training data. This study used EEG2Image based on a modified S-transform (MST) to convert EEG data into EEG-topography. This method retains the frequency-domain characteristics and spatial information of the EEG signals. Then, the CycleGAN is used to learn and generate motor-imagery EEG data of stroke patients. From the visual inspection, there is no difference between the EEG topographies of the generated and original EEG data collected from the stroke patients. Finally, we used convolutional neural networks (CNN) to evaluate and analyze the generated EEG data. The experimental results show that the generated data effectively improved the classification accuracy.

Mindfulness Training Associated With Resting-State Electroencephalograms Dynamics in Novice Practitioners via Mindful Breathing and Body-Scan

Objectives: Mindfulness-based stress reduction has been proven to improve mental health and quality of life. This study examined how mindfulness training and various types of mindfulness practices altered brain activity. Methods: Specifically, the spectral powers of scalp electroencephalography of the mindfulness-based stress reduction (MBSR) group (n=17) who underwent an 8-week MBSR training-including mindful breathing and body-scan-were evaluated and compared with those of the waitlist controls (n=14). Results: Empirical results indicated that the post-intervention effect of MBSR significantly elevated the resting-state beta powers and reduced resting-state delta powers in both practices; such changes were not observed in the waitlist control. Compared with mindful breathing, body-scanning resulted in an overall decline in electroencephalograms (EEG) spectral powers at both delta and low-gamma bands among trained participants. Conclusion: Together with our preliminary data of expert mediators, the aforementioned spectral changes were salient after intervention, but mitigated along with expertise. Additionally, after receiving training, the MBSR group's mindfulness and emotion regulation levels improved significantly, which were correlated with the EEG spectral changes in the theta, alpha, and low-beta bands. The results supported that MBSR might function as a unique internal processing tool that involves increased vigilant capability and induces alterations similar to other cognitive training.

Correction to: Review of brain encoding and decoding mechanisms for EEG-based brain-computer interface

[This corrects the article DOI: 10.1007/s11571-021-09676-z.].

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment. Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications. EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas. In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years. Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals. Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications. Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.

Enhancing transfer performance across datasets for brain-computer interfaces using a combination of alignment strategies and adaptive batch normalization

Objective. Recently, transfer learning (TL) and deep learning (DL) have been introduced to solve intra- and inter-subject variability problems in brain-computer interfaces (BCIs). However, current TL and DL algorithms are usually validated within a single dataset, assuming that data of the test subjects are acquired under the same condition as that of training (source) subjects. This assumption is generally violated in practice because of different acquisition systems and experimental settings across studies and datasets. Thus, the generalization ability of these algorithms needs further validations in a cross-dataset scenario, which is closer to the actual situation. This study compared the transfer performance of pre-trained deep-learning models with different preprocessing strategies in a cross-dataset scenario.Approach. This study used four publicly available motor imagery datasets, each was successively selected as a source dataset, and the others were used as target datasets. EEGNet and ShallowConvNet with four preprocessing strategies, namely channel normalization, trial normalization, Euclidean alignment, and Riemannian alignment, were trained with the source dataset. The transfer performance of pre-trained models was validated on the target datasets. This study also used adaptive batch normalization (AdaBN) for reducing interval covariate shift across datasets. This study compared the transfer performance of using the four preprocessing strategies and that of a baseline approach based on manifold embedded knowledge transfer (MEKT). This study also explored the possibility and performance of fusing MEKT and EEGNet.Main results. The results show that DL models with alignment strategies had significantly better transfer performance than the other two preprocessing strategies. As an unsupervised domain adaptation method, AdaBN could also significantly improve the transfer performance of DL models. The transfer performance of DL models that combined AdaBN and alignment strategies significantly outperformed MEKT. Moreover, the generalizability of EEGNet models that combined AdaBN and alignment strategies could be further improved via the domain adaptation step in MEKT, achieving the best generalization ability among multiple datasets (BNCI2014001: 0.788, PhysionetMI: 0.679, Weibo2014: 0.753, Cho2017: 0.650).Significance. The combination of alignment strategies and AdaBN could easily improve the generalizability of DL models without fine-tuning. This study may provide new insights into the design of transfer neural networks for BCIs by separating source and target batch normalization layers in the domain adaptation process.

Review of brain encoding and decoding mechanisms for EEG-based brain-computer interface

A brain-computer interface (BCI) can connect humans and machines directly and has achieved successful applications in the past few decades. Many new BCI paradigms and algorithms have been developed in recent years. Therefore, it is necessary to review new progress in BCIs. This paper summarizes progress for EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms, which are two key elements of BCI systems. Encoding paradigms are grouped by their underlying neural meachanisms, namely sensory- and motor-related, vision-related, cognition-related and hybrid paradigms. Decoding algorithms are reviewed in four categories, namely decomposition algorithms, Riemannian geometry, deep learning and transfer learning. This review will provide a comprehensive overview of both modern primary paradigms and algorithms, making it helpful for those who are developing BCI systems.

Comparing the Differences in Brain Activities and Neural Comodulations Associated With Motion Sickness Between Drivers and Passengers

It is common to believe that passengers are more adversely affected by motion sickness than drivers. However, no study has compared passengers and drivers' neural activities and drivers experiencing motion sickness (MS). Therefore, this study attempts to explore brain dynamics in motion sickness among passengers and drivers. Eighteen volunteers participated in simulating the driving winding road experiment while their subjective motion sickness levels and electroencephalogram (EEG) signals were simultaneously recorded. Independent Component Analysis (ICA) was employed to isolate MS-related independent components (ICs) from EEG. Furthermore, comodulation analysis was applied to decompose spectra of interest ICs, related to MS, to find the specific spectra-related temporally independent modulators (IMs). The results showed that passengers' alpha band (8-12 Hz) power increased in correlation with the MS level in the parietal, occipital midline and left and right motor areas, and drivers' alpha band (8-12 Hz) power showed relatively smaller increases than those in the passenger. Further, the results also indicate that the enhanced activation of alpha IMs in the passenger than the driver is due to a higher degree of motion sickness. In conclusion, compared to the driver, the passenger experience more conflicts among multimodal sensory systems and demand neuro-physiological regulation.

Detection of fixation points using a small visual landmark for brain-computer interfaces

Objective.The speed of visual brain-computer interfaces (v-BCIs) has been greatly improved in recent years. However, the traditional v-BCI paradigms require users to directly gaze at the intensive flickering items, which would cause severe problems such as visual fatigue and excessive visual resource consumption in practical applications. Therefore, it is imperative to develop a user-friendly v-BCI.Approach.According to the retina-cortical relationship, this study developed a novel BCI paradigm to detect the fixation point of eyes using a small visual stimulus that subtended only 0.6° in visual angle and was out of the central visual field. Specifically, the visual stimulus was treated as a landmark to judge the eccentricity and polar angle of the fixation point. Sixteen different fixation points were selected around the visual landmark, i.e. different combinations of two eccentricities (2° and 4°) and eight polar angles (0,π4,π2,3π4,π,5π4,3π2and7π4). Twelve subjects participated in this study, and they were asked to gaze at one out of the 16 points for each trial. A multi-class discriminative canonical pattern matching (Multi-DCPM) algorithm was proposed to decode the user's fixation point.Main results.We found the visual stimulation landmark elicited different spatial event-related potential patterns for different fixation points. Multi-DCPM could achieve an average accuracy of 66.2% with a standard deviation of 15.8% for the classification of the sixteen fixation points, which was significantly higher than traditional algorithms (p⩽0.001). Experimental results of this study demonstrate the feasibility of using a small visual stimulus as a landmark to track the relative position of the fixation point.Significance.The proposed new paradigm provides a potential approach to alleviate the problem of irritating stimuli in v-BCIs, which can broaden the applications of BCIs.

[Classification algorithms of error-related potentials in brain-computer interface]

Error self-detection based on error-related potentials (ErrP) is promising to improve the practicability of brain-computer interface systems. But the single trial recognition of ErrP is still a challenge that hinters the development of this technology. To assess the performance of different algorithms on decoding ErrP, this paper test four kinds of linear discriminant analysis algorithms, two kinds of support vector machines, logistic regression, and discriminative canonical pattern matching (DCPM) on two open accessed datasets. All algorithms were evaluated by their classification accuracies and their generalization ability on different sizes of training sets. The study results show that DCPM has the best performance. This study shows a comprehensive comparison of different algorithms on ErrP classification, which could give guidance for the selection of ErrP algorithm.

Enhancement for P300-speller classification using multi-window discriminative canonical pattern matching

Objective.P300s are one of the most studied event-related potentials (ERPs), which have been widely used for brain-computer interfaces (BCIs). Thus, fast and accurate recognition of P300s is an important issue for BCI study. Recently, there emerges a lot of novel classification algorithms for P300-speller. Among them, discriminative canonical pattern matching (DCPM) has been proven to work effectively, in which discriminative spatial pattern (DSP) filter can significantly enhance the spatial features of P300s. However, the pattern of ERPs in space varies with time, which was not taken into consideration in the traditional DCPM algorithm.Approach.In this study, we developed an advanced version of DCPM, i.e. multi-window DCPM, which contained a series of time-dependent DSP filters to fine-tune the extraction of spatial ERP features. To verify its effectiveness, 25 subjects were recruited and they were asked to conduct the typical P300-speller experiment.Main results.As a result, multi-window DCPM achieved the character recognition accuracy of 91.84% with only five training characters, which was significantly better than the traditional DCPM algorithm. Furthermore, it was also compared with eight other popular methods, including SWLDA, SKLDA, STDA, BLDA, xDAWN, HDCA, sHDCA and EEGNet. The results showed multi-window DCPM preformed the best, especially using a small calibration dataset. The proposed algorithm was applied to the BCI Controlled Robot Contest of P300 paradigm in 2019 World Robot Conference, and won the first place.Significance.These results demonstrate that multi-window DCPM is a promising method for improving the performance and enhancing the practicability of P300-speller.

Low-Dimensional Subject Representation-Based Transfer Learning in EEG Decoding

Recently, the advances in passive brain-computer interfaces (BCIs) based on electroencephalogram (EEG) have shed light on real-world neuromonitoring technologies. However, human variability in the EEG activities hinders the development of practical applications of EEG-based BCI. To tackle this problem, many transfer-learning techniques perform supervised calibration. This kind of calibration approach requires task-relevant data, which is impractical in real-life scenarios such as drowsiness during driving. This study presents a transfer-learning framework for EEG decoding based on the low-dimensional representations of subjects learned from the pre-trial EEG. Tensor decomposition was applied to the pre-trial EEG of subjects to extract the underlying characteristics in subject, spatial, and spectral domains. Then, the proposed framework assessed the characteristics to obtain the low-dimensional subject representations such that the subjects with similar brain dynamics can be identified. This method can leverage the existing data from other users, and a small number of data from a rapid, non-task, unsupervised calibration from a new user to build an accurate BCI. Our results demonstrated that, in terms of prediction accuracy, the proposed low-dimensional subject representation-based transfer learning (LDSR-TL) framework outperformed the random selection, and the Riemannian manifold approach in cognitive-state tracking, while requiring fewer training data. The results can greatly improve the practicability, and usability of EEG-based BCI in the real world.

Learning Common Time-Frequency-Spatial Patterns for Motor Imagery Classification

The common spatial patterns (CSP) algorithm is the most popular spatial filtering method applied to extract electroencephalogram (EEG) features for motor imagery (MI) based brain-computer interface (BCI) systems. The effectiveness of the CSP algorithm depends on optimal selection of the frequency band and time window from the EEG. Many algorithms have been designed to optimize frequency band selection for CSP, while few algorithms seek to optimize the time window. This study proposes a novel framework, termed common time-frequency-spatial patterns (CTFSP), to extract sparse CSP features from multi-band filtered EEG data in multiple time windows. Specifically, the whole MI period is first segmented into multiple subseries using a sliding time window approach. Then, sparse CSP features are extracted from multiple frequency bands in each time window. Finally, multiple support vector machine (SVM) classifiers with the Radial Basis Function (RBF) kernel are trained to identify the MI tasks and the voting result of these classifiers determines the final output of the BCI. This study applies the proposed CTFSP algorithm to three public EEG datasets (BCI competition III dataset IVa, BCI competition III dataset IIIa, and BCI competition IV dataset 1) to validate its effectiveness, compared against several other state-of-the-art methods. The experimental results demonstrate that the proposed algorithm is a promising candidate for improving the performance of MI-BCI systems.

Retrosplenial Segregation Reflects the Navigation Load During Ambulatory Movement

Spatial navigation is a complex cognitive process based on vestibular, proprioceptive, and visualcues that are integrated and processed by an extensive network of brain areas. The retrosplenial complex (RSC) is an integral part of coordination and translation between spatial reference frames. Previous studies have demonstrated that the RSC is active during a spatial navigation tasks. The specifics of RSC activity under various navigation loads, however, are still not characterized. This study investigated the local information processed by the RSC under various navigation load conditions manipulated by the number of turns in the physical navigation setup. The results showed that the local information processed via the RSC, which was reflected by the segregation network, was higher when the number of turns increased, suggesting that RSC activity is associated with the navigation task load. The present findings shed light on how the brain processes spatial information in a physical navigation task.

The Current Research of Combining Multi-Modal Brain-Computer Interfaces With Virtual Reality

Combing brain-computer interfaces (BCI) and virtual reality (VR) is a novel technique in the field of medical rehabilitation and game entertainment. However, the limitations of BCI such as a limited number of action commands and low accuracy hinder the widespread use of BCI-VR. Recent studies have used hybrid BCIs that combine multiple BCI paradigms and/or the multi-modal biosensors to alleviate these issues, which may become the mainstream of BCIs in the future. The main purpose of this review is to discuss the current status of multi-modal BCI-VR. This study first reviewed the development of the BCI-VR, and explored the advantages and disadvantages of incorporating eye tracking, motor capture, and myoelectric sensing into the BCI-VR system. Then, this study discussed the development trend of the multi-modal BCI-VR, hoping to provide a pathway for further research in this field.

BCI-Based Rehabilitation on the Stroke in Sequela Stage

Background

Stroke is the leading cause of serious and long-term disability worldwide. Survivors may recover some motor functions after rehabilitation therapy. However, many stroke patients missed the best time period for recovery and entered into the sequela stage of chronic stroke.

Method

Studies have shown that motor imagery- (MI-) based brain-computer interface (BCI) has a positive effect on poststroke rehabilitation. This study used both virtual limbs and functional electrical stimulation (FES) as feedback to provide patients with a closed-loop sensorimotor integration for motor rehabilitation. An MI-based BCI system acquired, analyzed, and classified motor attempts from electroencephalogram (EEG) signals. The FES system would be activated if the BCI detected that the user was imagining wrist dorsiflexion on the instructed side of the body. Sixteen stroke patients in the sequela stage were randomly assigned to a BCI group and a control group. All of them participated in rehabilitation training for four weeks and were assessed by the Fugl-Meyer Assessment (FMA) of motor function.

Results

The average improvement score of the BCI group was 3.5, which was higher than that of the control group (0.9). The active EEG patterns of the four patients in the BCI group whose FMA scores increased gradually became centralized and shifted to sensorimotor areas and premotor areas throughout the study.

Conclusions

Study results showed evidence that patients in the BCI group achieved larger functional improvements than those in the control group and that the BCI-FES system is effective in restoring motor function to upper extremities in stroke patients. This study provides a more autonomous approach than traditional treatments used in stroke rehabilitation.

SSVEP-assisted RSVP Brain-Computer Interface paradigm for multi-target classification

Brain-Computer Interface (BCI) is actively involved in optimizing the communication medium between the human brain and external devices.

OBJECTIVE

Rapid serial visual presentation (RSVP) is a robust and highly efficient BCI technique in recognizing target objects but suffers from limited target selections. The BCI systems that combine steady-state visual evoked potential (SSVEP) and RSVP can mitigate this limitation and allow users to operate on multiple targets.

APPROACH

This study proposes a novel SSVEP-assisted RSVP BCI model to improve the performance of classifying the target/non-target objects in a multi-target scenario. In this paradigm, SSVEP stimuli helps in identifying the user's focus location and RSVP stimuli that elicits event-related potentials (ERPs) differentiate target and non-target objects.

MAIN RESULTS

The proposed model achieved an offline accuracy of 81.59% by using 12 electroencephalogram (EEG) channels and an online (real-time) accuracy of 78.10% when only 4 EEG channels are considered. Further, the biomarkers of physiological states are analyzed to assess the cognitive states (mental fatigue and user attention) of the participants based on resting theta and alpha band powers. The results indicate an inverse relationship between the BCI performance and the resting EEG power, validating that the subjects' performance is affected by physiological states for prolonged BCI tasks.

SIGNIFICANCE

Our findings demonstrate that the combination of SSVEP and RSVP stimuli improves the BCI performance and further enhances the possibility of performing multiple user command tasks, which are inevitable in real-world applications. Additionally, the cognitive state biomarkers discussed imply the need for an efficient and attractive experimental paradigm that reduces the physiological state disparities and provide enhanced BCI performance.

Boosting template-based SSVEP decoding by cross-domain transfer learning

OBJECTIVE

This study aims to establish a generalized transfer-learning framework for boosting the performance of steady-state visual evoked potential(SSVEP)-based brain-computer interfaces (BCIs) by leveraging cross-domain data transferring.

APPROACH

We enhanced the state-of-the-art template-based SSVEP decoding through incorporating a least-squares transformation (LST)-based transfer learning to leverage calibration data across multiple domains (sessions, subjects, and EEG montages).

MAIN RESULTS

Study results verified the efficacy of LST in obviating the variability of SSVEPs when transferring existing data across domains. Furthermore, the LST-based method achieved significantly higher SSVEP-decoding accuracy than the standard task-related component analysis (TRCA)-based method and the non-LST naive transfer-learning method.

SIGNIFICANCE

This study demonstrated the capability of the LST-based transfer learning to leverage existing data across subjects and/or devices with an in-depth investigation of its rationale and behavior in various circumstances. The proposed framework significantly improved the SSVEP decoding accuracy over the standard TRCA approach when calibration data are limited. Its performance in calibration reduction could facilitate plug-and-play SSVEP-based BCIs and further practical applications.