High-speed spelling with a noninvasive brain-computer interface

Filter banks guided correlational convolutional neural network for SSVEPs based BCI classification

Objective.In the field of steady-state visual evoked potential brain computer interfaces (SSVEP-BCIs) research, convolutional neural networks (CNNs) have gradually been proved to be an effective method. Whereas, majority works apply the frequency domain characteristics in long time window to train the network, thus lead to insufficient performance of those networks in short time window. Furthermore, only the frequency domain information for classification lacks of other task-related information.Approach.To address these issues, we propose a time-frequency domain generalized filter-bank convolutional neural network (FBCNN-G) to improve the SSVEP-BCIs classification performance. The network integrates multiple frequency information of electroencephalogram (EEG) with template and predefined prior of sine-cosine signals to perform feature extraction, which contains correlation analyses in both template and signal aspects. Then the classification is performed at the end of the network. In addition, the method proposes the use of filter banks divided into specific frequency bands as pre-filters in the network to fully consider the fundamental and harmonic frequency characteristics of the signal.Main results.The proposed FBCNN-G model is compared with other methods on the public dataset Benchmark. The results manifest that this model has higher accuracy of character recognition accuracy and information transfer rates in several time windows. Particularly, in the 0.2 s time window, the mean accuracy of the proposed method reaches62.02%±5.12%, indicating its superior performance.Significance.The proposed FBCNN-G model is critical for the exploitation of SSVEP-BCIs character recognition models.

Using the Cocktail Party Effect to Add the Coding Dimension of Auditory Event Related Potential Brain-Computer Interface

OBJECTIVE

The auditory event-related potential based brain-computer interface (aERP-BCI) is a classical paradigm of brain-computer communication. To improve the coding efficiency of aERP-BCI, this study proposes a method using two parallel voice channels to add the coding dimension based on the cocktail party effect.

METHODS

The novel paradigm used male and female voices to establish two parallel oddball sound stimulus sequences. In comparison, the baseline paradigm only presented male or female stimulus sequences. Both the double voice condition (DVC) and the single voice condition (SVC) paradigms carried out offline experiments and the DVC also carried out online experiment. Subsequently, the EEG signal and BCI operation results were compared and analyzed.

CONCLUSION

The cocktail party effect caused a significant difference in the EEG responses of non-target stimulus between the focused vocal channel and the ignored vocal channel under the DVC paradigm, and the focused and ignored channels achieved a recognition accuracy of 97.2%. The target recognition rate of DVC was 82.3%, with no significant difference compared with 85% of SVC while the information transfer rate (ITR) of DVC reaching 15.3 bits/min was significantly higher than that of SVC.

SIGNIFICANCE

The cocktail party effect improves the coding efficiency by adding parallel channels without reducing the target/non-target stimulus recognition in the focused vocal channel. This provides a novel direction for the performance improvement of aERP-BCI.

Optimization of stimulus properties for SSVEP-based BMI system with a heads-up display to control in-vehicle features

Over the years, the driver-vehicle interface has been improved, but interacting with in-vehicle features can still increase distraction and affect road safety. This study aims to introduce brain-machine interface (BMI)- based solution to potentially enhance road safety. To achieve this goal, we evaluated visual stimuli properties (SPs) for a steady state visually evoked potentials (SSVEP)-based BMI system. We used a heads-up display (HUD) as the primary screen to present icons for controlling in-vehicle functions such as music, temperature, settings, and navigation. We investigated the effect of various SPs on SSVEP detection performance including the duty cycle and signal-to-noise ratio of visual stimuli, the size, color, and frequency of the icons, and array configuration and location. The experiments were conducted with 10 volunteers and the signals were analyzed using the canonical correlation analysis (CCA), filter bank CCA (FBCCA), and power spectral density analysis (PSDA). Our experimental results suggest that stimuli with a green color, a duty cycle of 50%, presented at a central location, with a size of 36 cm2 elicit a significantly stronger SSVEP response and enhanced SSVEP detection time. We also observed that lower SNR stimuli significantly affect SSVEP detection performance. There was no statistically significant difference observed in SSVEP response between the use of an LCD monitor and a HUD.

Binocularly incongruent, multifrequency-coded SSVEP in VR: feasibility and characteristics

Objective.Steady-state visual evoked potentials (SSVEPs) in response to flickering stimuli are popular in brain-computer interfacing but their implementation in virtual reality (VR) offers new opportunities also for clinical applications. While traditional SSVEP target selection relies on single-frequency stimulation of both eyes simultaneously, further called congruent stimulation, recent studies attempted to improve the information transfer rate by using dual-frequency-coded SSVEP where each eye is presented with a stimulus flickering at a different frequency, further called incongruent stimulation. However, few studies have investigated incongruent multifrequency-coded SSVEP (MultiIncong-SSVEP).Approach.This paper reports on a systematical investigation of incongruent dual-, triple-, and quadruple-frequency-coded SSVEP for use in VR, several of which are entirely novel, and compares their performance with that of congruent dual-frequency-coded SSVEP.Main results.We were able to confirm the presence of a summation effect when comparing monocular- and binocular single-frequency congruent stimulation, and a suppression effect when comparing monocular- and binocular dual-frequency incongruent stimulation, as both tap into the binocular vision capabilities which, when hampered, could signal amblyopia.Significance.In sum, our findings not only evidence the potential of VR-based binocularly incongruent SSVEP but also underscore the importance of paradigm choice and decoder design to optimize system performance and user comfort.

[Visual object detection system based on augmented reality and steady-state visual evoked potential]

This study investigates a brain-computer interface (BCI) system based on an augmented reality (AR) environment and steady-state visual evoked potentials (SSVEP). The system is designed to facilitate the selection of real-world objects through visual gaze in real-life scenarios. By integrating object detection technology and AR technology, the system augmented real objects with visual enhancements, providing users with visual stimuli that induced corresponding brain signals. SSVEP technology was then utilized to interpret these brain signals and identify the objects that users focused on. Additionally, an adaptive dynamic time-window-based filter bank canonical correlation analysis was employed to rapidly parse the subjects' brain signals. Experimental results indicated that the system could effectively recognize SSVEP signals, achieving an average accuracy rate of 90.6% in visual target identification. This system extends the application of SSVEP signals to real-life scenarios, demonstrating feasibility and efficacy in assisting individuals with mobility impairments and physical disabilities in object selection tasks.

Human-centred physical neuromorphics with visual brain-computer interfaces

Steady-state visual evoked potentials (SSVEPs) are widely used for brain-computer interfaces (BCIs) as they provide a stable and efficient means to connect the computer to the brain with a simple flickering light. Previous studies focused on low-density frequency division multiplexing techniques, i.e. typically employing one or two light-modulation frequencies during a single flickering light stimulation. Here we show that it is possible to encode information in SSVEPs excited by high-density frequency division multiplexing, involving hundreds of frequencies. We then demonstrate the ability to transmit entire images from the computer to the brain/EEG read-out in relatively short times. High-density frequency multiplexing also allows to implement a photonic neural network utilizing SSVEPs, that is applied to simple classification tasks and exhibits promising scalability properties by connecting multiple brains in series. Our findings open up new possibilities for the field of neural interfaces, holding potential for various applications, including assistive technologies and cognitive enhancements, to further improve human-machine interactions.

Research on a soft saturation nonlinear SSVEP signal feature extraction algorithm

Brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEP) have received widespread attention due to their high information transmission rate, high accuracy, and rich instruction set. However, the performance of its identification methods strongly depends on the amount of calibration data for within-subject classification. Some studies use deep learning (DL) algorithms for inter-subject classification, which can reduce the calculation process, but there is still much room for improvement in performance compared with intra-subject classification. To solve these problems, an efficient SSVEP signal recognition deep learning network model e-SSVEPNet based on the soft saturation nonlinear module is proposed in this paper. The soft saturation nonlinear module uses a similar exponential calculation method for output when it is less than zero, improving robustness to noise. Under the conditions of the SSVEP data set, two sliding time window lengths (1 s and 0.5 s), and three training data sizes, this paper evaluates the proposed network model and compares it with other traditional and deep learning model baseline methods. The experimental results of the nonlinear module were classified and compared. A large number of experimental results show that the proposed network has the highest average accuracy of intra-subject classification on the SSVEP data set, improves the performance of SSVEP signal classification and recognition, and has higher decoding accuracy under short signals, so it has huge potential ability to realize high-speed SSVEP-based for BCI.

A Convolutional Neural Network for SSVEP Identification by Using a Few-Channel EEG

The application of wearable electroencephalogram (EEG) devices is growing in brain-computer interfaces (BCI) owing to their good wearability and portability. Compared with conventional devices, wearable devices typically support fewer EEG channels. Devices with few-channel EEGs have been proven to be available for steady-state visual evoked potential (SSVEP)-based BCI. However, fewer-channel EEGs can cause the BCI performance to decrease. To address this issue, an attention-based complex spectrum-convolutional neural network (atten-CCNN) is proposed in this study, which combines a CNN with a squeeze-and-excitation block and uses the spectrum of the EEG signal as the input. The proposed model was assessed on a wearable 40-class dataset and a public 12-class dataset under subject-independent and subject-dependent conditions. The results show that whether using a three-channel EEG or single-channel EEG for SSVEP identification, atten-CCNN outperformed the baseline models, indicating that the new model can effectively enhance the performance of SSVEP-BCI with few-channel EEGs. Therefore, this SSVEP identification algorithm based on a few-channel EEG is particularly suitable for use with wearable EEG devices.

A Dynamic Window Method Based on Reinforcement Learning for SSVEP Recognition

Steady-state visual evoked potential (SSVEP) is one of the most used brain-computer interface (BCI) paradigms. Conventional methods analyze SSVEPs at a fixed window length. Compared with these methods, dynamic window methods can achieve a higher information transfer rate (ITR) by selecting an appropriate window length. These methods dynamically evaluate the credibility of the result by linear discriminant analysis (LDA) or Bayesian estimation and extend the window length until credible results are obtained. However, the hypotheses introduced by LDA and Bayesian estimation may not align with the collected real-world SSVEPs, which leads to an inappropriate window length. To address the issue, we propose a novel dynamic window method based on reinforcement learning (RL). The proposed method optimizes the decision of whether to extend the window length based on the impact of decisions on the ITR, without additional hypotheses. The decision model can automatically learn a strategy that maximizes the ITR through trial and error. In addition, compared with traditional methods that manually extract features, the proposed method uses neural networks to automatically extract features for the dynamic selection of window length. Therefore, the proposed method can more accurately decide whether to extend the window length and select an appropriate window length. To verify the performance, we compared the novel method with other dynamic window methods on two public SSVEP datasets. The experimental results demonstrate that the novel method achieves the highest performance by using RL.

SSVEP-DAN: Cross-Domain Data Alignment for SSVEP-Based Brain-Computer Interfaces

Steady-state visual-evoked potential (SSVEP)-based brain-computer interfaces (BCIs) offer a non-invasive means of communication through high-speed speller systems. However, their efficiency is highly dependent on individual training data acquired during time-consuming calibration sessions. To address the challenge of data insufficiency in SSVEP-based BCIs, we introduce SSVEP-DAN, the first dedicated neural network model designed to align SSVEP data across different domains, encompassing various sessions, subjects, or devices. Our experimental results demonstrate the ability of SSVEP-DAN to transform existing source SSVEP data into supplementary calibration data. This results in a significant improvement in SSVEP decoding accuracy while reducing the calibration time. We envision SSVEP-DAN playing a crucial role in future applications of high-performance SSVEP-based BCIs. The source code for this work is available at: https://github.com/CECNL/SSVEP-DAN.

An SSVEP-based BCI with 112 targets using frequency spatial multiplexing

Objective.Brain-computer interface (BCI) systems with large directly accessible instruction sets are one of the difficulties in BCI research. Research to achieve high target resolution (⩾100) has not yet entered a rapid development stage, which contradicts the application requirements. Steady-state visual evoked potential (SSVEP) based BCIs have an advantage in terms of the number of targets, but the competitive mechanism between the target stimulus and its neighboring stimuli is a key challenge that prevents the target resolution from being improved significantly.Approach.In this paper, we reverse the competitive mechanism and propose a frequency spatial multiplexing method to produce more targets with limited frequencies. In the proposed paradigm, we replicated each flicker stimulus as a 2 × 2 matrix and arrange the matrices of all frequencies in a tiled fashion to form the interaction interface. With different arrangements, we designed and tested three example paradigms with different layouts. Further we designed a graph neural network that distinguishes between targets of the same frequency by recognizing the different electroencephalography (EEG) response distribution patterns evoked by each target and its neighboring targets.Main results.Extensive experiment studies employing eleven subjects have been performed to verify the validity of the proposed method. The average classification accuracies in the offline validation experiments for the three paradigms are 89.16%, 91.38%, and 87.90%, with information transfer rates (ITR) of 51.66, 53.96, and 50.55 bits/min, respectively.Significance.This study utilized the positional relationship between stimuli and did not circumvent the competing response problem. Therefore, other state-of-the-art methods focusing on enhancing the efficiency of SSVEP detection can be used as a basis for the present method to achieve very promising improvements.

Comparison of recognition methods for an asynchronous (un-cued) BCI system: an investigation with 40-class SSVEP dataset

Steady-state visual evoked potential (SSVEP)-based brain-computer Interface (BCI) has demonstrated the potential to manage multi-command targets to achieve high-speed communication. Recent studies on multi-class SSVEP-based BCI have focused on synchronous systems, which rely on predefined time and task indicators; thus, these systems that use passive approaches may be less suitable for practical applications. Asynchronous systems recognize the user's intention (whether or not the user is willing to use systems) from brain activity; then, after recognizing the user's willingness, they begin to operate by switching swiftly for real-time control. Consequently, various methodologies have been proposed to capture the user's intention. However, in-depth investigation of recognition methods in asynchronous BCI system is lacking. Thus, in this work, three recognition methods (power spectral density analysis, canonical correlation analysis (CCA), and support vector machine (SVM)) used widely in asynchronous SSVEP BCI systems were explored to compare their performance. Further, we categorized asynchronous systems into two approaches (1-stage and 2-stage) based upon the recognition process's design, and compared their performance. To do so, a 40-class SSVEP dataset collected from 40 subjects was introduced. Finally, we found that the CCA-based method in the 2-stage approach demonstrated statistically significantly higher performance with a sensitivity of 97.62 ± 02.06%, specificity of 76.50 ± 23.50%, and accuracy of 75.59 ± 10.09%. Thus, it is expected that the 2-stage approach together with CCA-based recognition and FB-CCA classification have good potential to be implemented in practical asynchronous SSVEP BCI systems.

Precise prediction of cerebrospinal fluid amyloid beta protein for early Alzheimer's disease detection using multimodal data

Alzheimer's disease (AD) constitutes a neurodegenerative disorder marked by a progressive decline in cognitive function and memory capacity. The accurate diagnosis of this condition predominantly relies on cerebrospinal fluid (CSF) markers, notwithstanding the associated burdens of pain and substantial financial costs endured by patients. This study encompasses subjects exhibiting varying degrees of cognitive impairment, encompassing individuals with subjective cognitive decline, mild cognitive impairment, and dementia, constituting a total sample size of 82 participants. The primary objective of this investigation is to explore the relationships among brain atrophy measurements derived from magnetic resonance imaging, atypical electroencephalography (EEG) patterns, behavioral assessment scales, and amyloid β-protein (Aβ) indicators. The findings of this research reveal that individuals displaying reduced Aβ1-42/Aβ-40 levels exhibit significant atrophy in the frontotemporal lobe, alongside irregularities in various parameters related to EEG frequency characteristics, signal complexity, inter-regional information exchange, and microstates. The study additionally endeavors to estimate Aβ1-42/Aβ-40 content through the application of a random forest algorithm, amalgamating structural data, electrophysiological features, and clinical scales, achieving a remarkable predictive precision of 91.6%. In summary, this study proposes a cost-effective methodology for acquiring CSF markers, thereby offering a valuable tool for the early detection of AD.

Visual tracking brain-computer interface

Brain-computer interfaces (BCIs) offer a way to interact with computers without relying on physical movements. Non-invasive electroencephalography-based visual BCIs, known for efficient speed and calibration ease, face limitations in continuous tasks due to discrete stimulus design and decoding methods. To achieve continuous control, we implemented a novel spatial encoding stimulus paradigm and devised a corresponding projection method to enable continuous modulation of decoded velocity. Subsequently, we conducted experiments involving 17 participants and achieved Fitt's information transfer rate (ITR) of 0.55 bps for the fixed tracking task and 0.37 bps for the random tracking task. The proposed BCI with a high Fitt's ITR was then integrated into two applications, including painting and gaming. In conclusion, this study proposed a visual BCI based-control method to go beyond discrete commands, allowing natural continuous control based on neural activity.

A calibration-free c-VEP based BCI employing narrow-band random sequences

Objective.Code-modulated visual evoked potential (c-VEP) based brain-computer interfaces (BCIs) exhibit high encoding efficiency. Nevertheless, the majority of c-VEP based BCIs necessitate an initial training or calibration session, particularly when the number of targets expands, which impedes the practicality. To address this predicament, this study introduces a calibration-free c-VEP based BCI employing narrow-band random sequences.Approach.For the encoding method, a series of random sequences were generated within a specific frequency band. The c-VEP signals were subsequently elicited through the application of on-type grid flashes that were modulated by these sequences. For the calibration-free decoding algorithm, filter-bank canonical correlation analysis (FBCCA) was utilized with the reference templates generated from the original sequences. Thirty-five subjects participated into an online BCI experiment. The performances of c-VEP based BCIs utilizing narrow-band random sequences with frequency bands of 15-25 Hz (NBRS-15) and 8-16 Hz (NBRS-8) were compared with that of a steady-state visual evoked potential (SSVEP) based BCI within a frequency range of 8-15.8 Hz.Main results.The offline analysis results demonstrated a substantial correlation between the c-VEPs and the original narrow-band random sequences. After parameter optimization, the calibration-free system employing the NBRS-15 frequency band achieved an average information transfer rate (ITR) of 78.56 ± 37.03 bits/min, which exhibited no significant difference compared to the performance of the SSVEP based system when utilizing FBCCA. The proposed system achieved an average ITR of 102.1 ± 57.59 bits/min in a simulation of a 1000-target BCI system.Significance.This study introduces a novel calibration-free c-VEP based BCI system employing narrow-band random sequences and shows great potential of the proposed system in achieving a large number of targets and high ITR.

Enhancing SSVEP-BCI Performance Under Fatigue State Using Dynamic Stopping Strategy

Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have emerged as a prominent technology due to their high information transfer rate, rapid calibration time, and robust signal-to-noise ratio. However, a critical challenge for practical applications is performance degradation caused by user fatigue during prolonged use. This work proposes novel methods to address this challenge by dynamically adjusting data acquisition length and updating detection models based on a fatigue-aware stopping strategy. Two 16-target SSVEP-BCIs were employed, one using low-frequency and the other using high-frequency stimulation. A self-recorded fatigue dataset from 24 subjects was utilized for extensive evaluation. A simulated online experiment demonstrated that the proposed methods outperform the conventional fixed stopping strategy in terms of classification accuracy, information transfer rate, and selection time, irrespective of stimulation frequency. These findings suggest that the proposed approach can significantly improve SSVEP-BCI performance under fatigue conditions, leading to superior performance during extended use.

Estimating and approaching the maximum information rate of noninvasive visual brain-computer interface

An essential priority of visual brain-computer interfaces (BCIs) is to enhance the information transfer rate (ITR) to achieve high-speed communication. Despite notable progress, noninvasive visual BCIs have encountered a plateau in ITRs, leaving it uncertain whether higher ITRs are achievable. In this study, we used information theory to study the characteristics and capacity of the visual-evoked channel, which leads us to investigate whether and how we can decode higher information rates in a visual BCI system. Using information theory, we estimate the upper and lower bounds of the information rate with the white noise (WN) stimulus. Consequently, we found out that the information rate is determined by the signal-to-noise ratio (SNR) in the frequency domain, which reflects the spectrum resources of the channel. Based on this discovery, we propose a broadband WN BCI by implementing stimuli on a broader frequency band than the steady-state visual evoked potentials (SSVEPs)-based BCI. Through validation, the broadband BCI outperforms the SSVEP BCI by an impressive 7 bps, setting a record of 50 bps. The integration of information theory and the decoding analysis presented in this study offers valuable insights applicable to general sensory-evoked BCIs, providing a potential direction of next-generation human-machine interaction systems.

A novel brain-controlled prosthetic hand method integrating AR-SSVEP augmentation, asynchronous control, and machine vision assistance

Background and objective

The brain-computer interface (BCI) system based on steady-state visual evoked potentials (SSVEP) is expected to help disabled patients achieve alternative prosthetic hand assistance. However, the existing study still has some shortcomings in interaction aspects such as stimulus paradigm and control logic. The purpose of this study is to innovate the visual stimulus paradigm and asynchronous decoding/control strategy by integrating augmented reality technology, and propose an asynchronous pattern recognition algorithm, thereby improving the interaction logic and practical application capabilities of the prosthetic hand with the BCI system.

Methods

An asynchronous visual stimulus paradigm based on an augmented reality (AR) interface was proposed in this paper, in which there were 8 control modes, including Grasp, Put down, Pinch, Point, Fist, Palm push, Hold pen, and Initial. According to the attentional orienting characteristics of the paradigm, a novel asynchronous pattern recognition algorithm that combines center extended canonical correlation analysis and support vector machine (Center-ECCA-SVM) was proposed. Then, this study proposed an intelligent BCI system switch based on a deep learning object detection algorithm (YOLOv4) to improve the level of user interaction. Finally, two experiments were designed to test the performance of the brain-controlled prosthetic hand system and its practical performance in real scenarios.

Results

Under the AR paradigm of this study, compared with the liquid crystal display (LCD) paradigm, the average SSVEP spectrum amplitude of multiple subjects increased by 17.41%, and the signal-noise ratio (SNR) increased by 3.52%. The average stimulus pattern recognition accuracy was 96.71 ± 3.91%, which was 2.62% higher than the LCD paradigm. Under the data analysis time of 2s, the Center-ECCA-SVM classifier obtained 94.66 ± 3.87% and 97.40 ± 2.78% asynchronous pattern recognition accuracy under the Normal metric and the Tolerant metric, respectively. And the YOLOv4-tiny model achieves a speed of 25.29fps and a 96.4% confidence in the prosthetic hand in real-time detection. Finally, the brain-controlled prosthetic hand helped the subjects to complete 4 kinds of daily life tasks in the real scene, and the time-consuming were all within an acceptable range, which verified the effectiveness and practicability of the system.

Conclusion

This research is based on improving the user interaction level of the prosthetic hand with the BCI system, and has made improvements in the SSVEP paradigm, asynchronous pattern recognition, interaction, and control logic. Furthermore, it also provides support for BCI areas for alternative prosthetic control, and movement disorder rehabilitation programs.

Effects of the presentation order of stimulations in sequential ERP/SSVEP Hybrid Brain-Computer Interface

Hybrid Brain-Computer Interface (hBCI) combines multiple neurophysiology modalities or paradigms to speed up the output of a single command or produce multiple ones simultaneously. Concurrent hBCIs that employ endogenous and exogenous paradigms are limited by the reduced set of possible commands. Conversely, the fusion of different exogenous visual evoked potentials demonstrated impressive performances; however, they suffer from limited portability. Yet, sequential hBCIs did not receive much attention mainly due to slower transfer rate and user fatigue during prolonged BCI use (Lorenz et al 2014 J. Neural Eng. 11 035007). Moreover, the crucial factors for optimizing the hybridization remain under-explored. In this paper, we test the feasibility of sequential Event Related-Potentials (ERP) and Steady-State Visual Evoked Potentials (SSVEP) hBCI and study the effect of stimulus order presentation between ERP-SSVEP and SSVEP-ERP for the control of directions and speed of powered wheelchairs or mobile robots with 15 commands. Exploiting the fast single trial face stimulus ERP, SSVEP and modern efficient convolutional neural networks, the configuration with SSVEP presented at first achieved significantly (p < 0.05) higher average accuracy rate with 76.39% ( ± 7.30 standard deviation) hybrid command accuracy and an average Information Transfer Rate (ITR) of 25.05 ( ± 5.32 standard deviation) bits per minute (bpm). The results of the study demonstrate the suitability of a sequential SSVEP-ERP hBCI with challenging dry electroencephalography (EEG) electrodes and low-compute capacity. Although it presents lower ITR than concurrent hBCIs, our system presents an alternative in small screen settings when the conditions for concurrent hBCIs are difficult to satisfy.

Optimizing Visual Stimulation Paradigms for User-Friendly SSVEP-Based BCIs

In steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems, traditional flickering stimulation patterns face challenges in achieving a trade-off in both BCI performance and visual comfort across various frequency bands. To investigate the optimal stimulation paradigms with high performance and high comfort for each frequency band, this study systematically compared the characteristics of SSVEP and user experience of different stimulation paradigms with a wide stimulation frequency range of 1-60 Hz. The findings suggest that, for a better balance between system performance and user experience, ON and OFF grid stimuli with a Weber contrast of 50% can be utilized as alternatives to traditional flickering stimulation paradigms in the frequency band of 1-25 Hz. In the 25-35 Hz range, uniform flicker stimuli with the same 50% contrast are more suitable. In the higher frequency band, traditional uniform flicker stimuli with a high 300% contrast are preferred. These results are significant for developing high performance and user-friendly SSVEP-based BCI systems.

EEG-based finger movement classification with intrinsic time-scale decomposition

Introduction

Brain-computer interfaces (BCIs) are systems that acquire the brain's electrical activity and provide control of external devices. Since electroencephalography (EEG) is the simplest non-invasive method to capture the brain's electrical activity, EEG-based BCIs are very popular designs. Aside from classifying the extremity movements, recent BCI studies have focused on the accurate coding of the finger movements on the same hand through their classification by employing machine learning techniques. State-of-the-art studies were interested in coding five finger movements by neglecting the brain's idle case (i.e., the state that brain is not performing any mental tasks). This may easily cause more false positives and degrade the classification performances dramatically, thus, the performance of BCIs. This study aims to propose a more realistic system to decode the movements of five fingers and the no mental task (NoMT) case from EEG signals.

Methods

In this study, a novel praxis for feature extraction is utilized. Using Proper Rotational Components (PRCs) computed through Intrinsic Time Scale Decomposition (ITD), which has been successfully applied in different biomedical signals recently, features for classification are extracted. Subsequently, these features were applied to the inputs of well-known classifiers and their different implementations to discriminate between these six classes. The highest classifier performances obtained in both subject-independent and subject-dependent cases were reported. In addition, the ANOVA-based feature selection was examined to determine whether statistically significant features have an impact on the classifier performances or not.

Results

As a result, the Ensemble Learning classifier achieved the highest accuracy of 55.0% among the tested classifiers, and ANOVA-based feature selection increases the performance of classifiers on five-finger movement determination in EEG-based BCI systems.

Discussion

When compared with similar studies, proposed praxis achieved a modest yet significant improvement in classification performance although the number of classes was incremented by one (i.e., NoMT).

Investigating critical brain area for EEG-based binocular color fusion and rivalry with EEGNet

Introduction

Binocular color fusion and rivalry are two specific phenomena in binocular vision, which could be used as experimental tools to study how the brain processes conflicting information. There is a lack of objective evaluation indexes to distinguish the fusion or rivalry for dichoptic color.

Methods

This paper introduced EEGNet to construct an EEG-based model for binocular color fusion and rivalry classification. We developed an EEG dataset from 10 subjects.

Results

By dividing the EEG data from five different brain areas to train the corresponding models, experimental results showed that: (1) the brain area represented by the back area had a large difference on EEG signals, the accuracy of model reached the highest of 81.98%, and more channels decreased the model performance; (2) there was a large effect of inter-subject variability, and the EEG-based recognition is still a very challenge across subjects; and (3) the statistics of EEG data are relatively stationary at different time for the same individual, the EEG-based recognition is highly reproducible for an individual.

Discussion

The critical channels for EEG-based binocular color fusion and rivalry could be meaningful for developing the brain computer interfaces (BCIs) based on color-related visual evoked potential (CVEP).

Multi-information improves the performance of CCA-based SSVEP classification

The target recognition algorithm based on canonical correlation analysis (CCA) has been widely used in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces. To reduce visual fatigue and improve the information transfer rate (ITR), how to improve the accuracy of algorithms within a short time window has become one of the main problems at present. There were filter bank CCA (FBCCA), individual template CCA (ITCCA), and temporally local CCA (TCCA), which improve the CCA algorithm from different aspects.This paper proposed to consider individual, frequency, and time information at the same time, so as to extract features more effectively. A comparison of the various methods was performed using benchmark dataset. Classification accuracy and ITR were used for performance evaluation. In the different extensions of CCA, the method incorporating the above three kinds of information simultaneously achieved the best performance within a short time window. This study explores the effect of using a variety of information to improve the CCA algorithm.

MetaBCI: An open-source platform for brain-computer interfaces

BACKGROUND

Recently, brain-computer interfaces (BCIs) have attracted worldwide attention for their great potential in clinical and real-life applications. To implement a complete BCI system, one must set up several links to translate the brain intent into computer commands. However, there is not an open-source software platform that can cover all links of the BCI chain.

METHOD

This study developed a one-stop open-source BCI software, namely MetaBCI, to facilitate the construction of a BCI system. MetaBCI is written in Python, and has the functions of stimulus presentation (Brainstim), data loading and processing (Brainda), and online information flow (Brainflow). This paper introduces the detailed information of MetaBCI and presents four typical application cases.

RESULTS

The results showed that MetaBCI was an extensible and feature-rich software platform for BCI research and application, which could effectively encode, decode, and feedback brain activities.

CONCLUSIONS

MetaBCI can greatly lower the BCI's technical threshold for BCI beginners and can save time and cost to build up a practical BCI system. The source code is available at https://github.com/TBC-TJU/MetaBCI, expecting new contributions from the BCI community.

Filter bank second-order underdamped stochastic resonance analysis for implementing a short-term high-speed SSVEP detection

OBJECTIVE

The progression of brain-computer interfaces (BCIs) has been propelled by breakthroughs in neuroscience, signal processing, and machine learning, marking it as a dynamic field of study over the past few decades. Nevertheless, the nonlinear and non-stationary characteristics of steady-state visual evoked potentials (SSVEPs), coupled with the incongruity between frequently employed linear techniques and nonlinear signal attributes, resulted in the subpar performance of mainstream non-training algorithms like canonical correlation analysis (CCA), multivariate synchronization index (MSI), and filter bank CCA (FBCCA) in short-term SSVEP detection.

METHODS

To tackle this problem, the novel fusions of common filter bank analysis, CCA dimensionality reduction methods, USSR models, and MSI recognition models are used in SSVEP signal recognition.

RESULTS

Unlike conventional linear techniques such as CCA, MSI, and FBCCA, the filter bank second-order underdamped stochastic resonance (FBUSSR) analysis demonstrates superior efficacy in the detection of short-term high-speed SSVEPs.

CONCLUSION

This research enlists 32 subjects and uses a public dataset to assess the proposed approach, and the experimental outcomes indicate that the non-training method can attain greater recognition precision and stability. Furthermore, under the conditions of the newly proposed fusion method and light stimulation, the USSR model exhibits the most optimal enhancement effect.

SIGNIFICANCE

The findings of this study underscore the expansive potential for the application of BCI systems in the realm of neuroscience and signal processing.

Advanced Electrode Technologies for Noninvasive Brain-Computer Interfaces

Brain-computer interfaces (BCIs) have garnered significant attention in recent years due to their potential applications in medical, assistive, and communication technologies. Building on this, noninvasive BCIs stand out as they provide a safe and user-friendly method for interacting with the human brain. In this work, we provide a comprehensive overview of the latest developments and advancements in material, design, and application of noninvasive BCIs electrode technology. We also explore the challenges and limitations currently faced by noninvasive BCI electrode technology and sketch out the technological roadmap from three dimensions: Materials and Design; Performances; Mode and Function. We aim to unite research efforts within the field of noninvasive BCI electrode technology, focusing on the consolidation of shared goals and fostering integrated development strategies among a diverse array of multidisciplinary researchers.

Using oscillatory and aperiodic neural activity features for identifying idle state in SSVEP-based BCIs reduces false triggers

Objective.In existing studies, rhythmic (oscillatory) components were used as main features to identify brain states, such as control and idle states, while non-rhythmic (aperiodic) components were ignored. Recent studies have shown that aperiodic (1/f) activity is functionally related to cognitive processes. It is not clear if aperiodic activity can distinguish brain states in asynchronous brain-computer interfaces (BCIs) to reduce false triggers. In this paper, we propose an asynchronous method based on the fusion of oscillatory and aperiodic features for steady-state visual evoked potential-based BCIs.Approach.The proposed method first evaluates the oscillatory and aperiodic components of control and idle states using irregular-resampling auto-spectral analysis. Oscillatory features are then extracted using the spectral power of fundamental, second-harmonic, and third-harmonic frequencies of the oscillatory component, and aperiodic features are extracted using the slope and intercept of the first-order polynomial of the spectral fit of the aperiodic component under a log-logarithmic axis. The process produces two types of feature pools (oscillatory, aperiodic features). Next, feature selection (dimensionality reduction) is applied to the feature pools by Bonferroni correctedp-values from two-way analysis of variance. Last, these spatial-specific statistically significant features are used as input for classification to identify the idle state.Mainresults.On a 7-target dataset from 15 subjects, the mix of oscillatory and aperiodic features achieved an average accuracy of 88.39% compared to 83.53% when using oscillatory features alone (4.86% improvement). The results demonstrated that the proposed idle state recognition method achieved enhanced performance by incorporating aperiodic features.Significance.Our results demonstrated that (1) aperiodic features were effective in recognizing idle states and (2) fusing features of oscillatory and aperiodic components enhanced classification performance by 4.86% compared to oscillatory features alone.

Reduced interocular suppression after inverse patching in anisometropic amblyopia

Purpose

Recent investigations observed substantial enhancements in binocular balance, visual acuity, and stereovision among older children and adults with amblyopia by patching the amblyopic eye (i.e., inverse patching) for 2 h daily over 2 months. Despite these promising findings, the precise neural mechanisms underlying inverse patching remain elusive. This study endeavors to delve deeper into the neural alterations induced by inverse patching, focusing on steady-state visual evoked potentials (SSVEPs). We specifically investigate the changes in SSVEPs following monocular deprivation of either the fellow eye or the amblyopic eye in older amblyopic children and adults.

Method

Ten participants (17.60 ± 2.03 years old; mean ± SEM), clinically diagnosed with anisometropic amblyopia, were recruited for this study. Each participant underwent a 120 min patching session on their fellow eye on the first day, followed by a similar session on their amblyopic eye on the second day. Baseline steady-state visual evoked potentials (SSVEPs) measurements were collected each day prior to patching, with post-patching SSVEPs measurements obtained immediately after the patching session. The experimental design incorporated a binocular rivalry paradigm, utilizing SSVEPs measurements.

Results

The results revealed that inverse patching induced a heightened influence on neural plasticity, manifesting in a reduction of interocular suppression from the fellow eye to the amblyopic eye. In contrast, patching the fellow eye demonstrated negligible effects on the visual cortex. Furthermore, alterations in interocular suppression subsequent to inverse patching exhibited a correlation with the visual acuity of the amblyopic eye.

Conclusion

Inverse patching emerges as a promising therapeutic avenue for adolescents and adults grappling with severe anisometropic amblyopia that proves refractory to conventional interventions. This innovative approach exhibits the potential to induce more robust neural plasticity within the visual cortex, thereby modulating neural interactions more effectively than traditional amblyopia treatments.

A data expansion technique based on training and testing sample to boost the detection of SSVEPs for brain-computer interfaces

Objective.Currently, steady-state visual evoked potentials (SSVEPs)-based brain-computer interfaces (BCIs) have achieved the highest interaction accuracy and speed among all BCI paradigms. However, its decoding efficacy depends deeply on the number of training samples, and the system performance would have a dramatic drop when the training dataset decreased to a small size. To date, no study has been reported to incorporate the unsupervised learning information from testing trails into the construction of supervised classification model, which is a potential way to mitigate the overfitting effect of limited samples.Approach.This study proposed a novel method for SSVEPs detection, i.e. cyclic shift trials (CSTs), which could combine unsupervised learning information from test trials and supervised learning information from train trials. Furthermore, since SSVEPs are time-locked and phase-locked to the onset of specific flashes, CST could also expand training samples on the basis of its regularity and periodicity. In order to verify the effectiveness of CST, we designed an online SSVEP-BCI system, and tested this system combined CST with two common classification algorithms, i.e. extended canonical correlation analysis and ensemble task-related component analysis.Main results.CST could significantly enhance the signal to noise ratios of SSVEPs and improve the performance of systems especially for the condition of few training samples and short stimulus time. The online information transfer rate could reach up to 236.19 bits min-1using 36 s calibration time of only one training sample for each category.Significance.The proposed CST method can take full advantages of supervised learning information from training samples and unsupervised learning information of testing samples. Furthermore, it is a data expansion technique, which can enhance the SSVEP characteristics and reduce dependence on sample size. Above all, CST is a promising method to improve the performance of SSVEP-based BCI without any additional experimental burden.

Almost free of calibration for SSVEP-based brain-computer interfaces

Objective. Steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) is a promising technology that can achieve high information transfer rate (ITR) with supervised algorithms such as ensemble task-related component analysis (eTRCA) and task-discriminant component analysis (TDCA). However, training individual models requires a tedious and time-consuming calibration process, which hinders the real-life use of SSVEP-BCIs. A recent data augmentation method, called source aliasing matrix estimation (SAME), can generate new EEG samples from a few calibration trials. But SAME does not exploit the information across stimuli as well as only reduces the number of calibration trials per command, so it still has some limitations.Approach. This study proposes an extended version of SAME, called multi-stimulus SAME (msSAME), which exploits the similarity of the aliasing matrix across frequencies to enhance the performance of SSVEP-BCI with insufficient calibration trials. We also propose a semi-supervised approach based on msSAME that can further reduce the number of SSVEP frequencies needed for calibration. We evaluate our method on two public datasets, Benchmark and BETA, and an online experiment.Main results. The results show that msSAME outperforms SAME for both eTRCA and TDCA on the public datasets. Moreover, the semi-supervised msSAME-based method achieves comparable performance to the fully calibrated methods and outperforms the conventional free-calibrated methods. Remarkably, our method only needs 24 s to calibrate 40 targets in the online experiment and achieves an average ITR of 213.8 bits min-1with a peak of 242.6 bits min-1.Significance. This study significantly reduces the calibration effort for individual SSVEP-BCIs, which is beneficial for developing practical plug-and-play SSVEP-BCIs.

EEG-Based Target Detection Using an RSVP Paradigm under Five Levels of Weak Hidden Conditions

Although target detection based on electroencephalogram (EEG) signals has been extensively investigated recently, EEG-based target detection under weak hidden conditions remains a problem. In this paper, we proposed a rapid serial visual presentation (RSVP) paradigm for target detection corresponding to five levels of weak hidden conditions quantitively based on the RGB color space. Eighteen subjects participated in the experiment, and the neural signatures, including P300 amplitude and latency, were investigated. Detection performance was evaluated under five levels of weak hidden conditions using the linear discrimination analysis and support vector machine classifiers on different channel sets. The experimental results showed that, compared with the benchmark condition, (1) the P300 amplitude significantly decreased (8.92 ± 1.24 μV versus 7.84 ± 1.40 μV, p = 0.021) and latency was significantly prolonged (582.39 ± 25.02 ms versus 643.83 ± 26.16 ms, p = 0.028) only under the weakest hidden condition, and (2) the detection accuracy decreased by less than 2% (75.04 ± 3.24% versus 73.35 ± 3.15%, p = 0.029) with a more than 90% reduction in channel number (62 channels versus 6 channels), determined using the proposed channel selection method under the weakest hidden condition. Our study can provide new insights into target detection under weak hidden conditions based on EEG signals with a rapid serial visual presentation paradigm. In addition, it may expand the application of brain-computer interfaces in EEG-based target detection areas.

Acoustoelectric brain imaging with different conductivities and acoustic distributions

Objective: Acoustoelectric brain imaging (AEBI) is a promising imaging method for mapping brain biological current densities with high spatiotemporal resolution. Currently, it is still challenging to achieve human AEBI with an unclear acoustoelectric (AE) signal response of medium characteristics, particularly in conductivity and acoustic distribution. This study introduces different conductivities and acoustic distributions into the AEBI experiment, and clarifies the response interaction between medium characteristics and AEBI performance to address these key challenges. Approach: AEBI with different conductivities is explored by the imaging experiment, potential measurement, and simulation on a pig's fat, muscle, and brain tissue. AEBI with different acoustic distributions is evaluated on the imaging experiment and acoustic field measurement through a deep and surface transmitting model built on a human skullcap and pig brain tissue. Main results: The results show that conductivity is not only inversely proportional to the AE signal amplitude but also leads to a higher AEBI spatial resolution as it increases. In addition, the current source and sulcus can be located simultaneously with a strong AE signal intensity. The transcranial focal zone enlargement, pressure attenuation in the deep-transmitting model, and ultrasound echo enhancement in the surface-transmitting model cause a reduced spatial resolution, FFT-SNR, and timing correlation of AEBI. Under the comprehensive effect of conductivity and acoustics, AEBI with skull finally shows reduced imaging performance for both models compared with no-skull AEBI. On the contrary, the AE signal amplitude decreases in the deep-transmitting model and increases in the surface-transmitting model. Significance: This study reveals the response interaction between medium characteristics and AEBI performance, and makes an essential step toward developing AEBI as a practical neuroimaging technique.

Improving Generalized Zero-Shot Learning SSVEP Classification Performance From Data-Efficient Perspective

Generalized zero-shot learning (GZSL) has significantly reduced the training requirements for steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs). Traditional methods require complete class data sets for training, but GZSL allows for only partial class data sets, dividing them into 'seen' (those with training data) and 'unseen' classes (those without training data). However, inefficient utilization of SSVEP data limits the accuracy and information transfer rate (ITR) of existing GZSL methods. To this end, we proposed a framework for more effective utilization of SSVEP data at three systematically combined levels: data acquisition, feature extraction, and decision-making. First, prevalent SSVEP-based BCIs overlook the inter-subject variance in visual latency and employ fixed sampling starting time (SST). We introduced a dynamic sampling starting time (DSST) strategy at the data acquisition level. This strategy uses the classification results on the validation set to find the optimal sampling starting time (OSST) for each subject. In addition, we developed a Transformer structure to capture the global information of input data for compensating the small receptive field of existing networks. The global receptive fields of the Transformer can adequately process the information from longer input sequences. For the decision-making level, we designed a classifier selection strategy that can automatically select the optimal classifier for the seen and unseen classes, respectively. We also proposed a training procedure to make the above solutions in conjunction with each other. Our method was validated on three public datasets and outperformed the state-of-the-art (SOTA) methods. Crucially, we also outperformed the representative methods that require training data for all classes.

Phase-Locked Time-Shift Data Augmentation Method for SSVEP Brain-Computer Interfaces

Steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs) have achieved an information transfer rate (ITR) of over 300 bits/min, but abundant training data is required. The performance of SSVEP algorithms deteriorates greatly under limited data, and the existing time-shift data augmentation method fails to improve it because the phase-locked requirement between training samples is violated. To address this issue, this study proposes a novel augmentation method, namely phase-locked time-shift (PLTS), for SSVEP-BCI. The similarity between epochs at different time moments was evaluated, and a unique time-shift step was calculated for each class to augment additional data epochs in each trial. The results showed that the PLTS significantly improved the classification performance of SSVEP algorithms on the BETA SSVEP datasets. Moreover, under the condition of one calibration block, by slightly prolonging the calibration duration (from 48 s to 51.5 s), the ITR increased from 40.88±4.54 bits/min to 122.61±7.05 bits/min with the PLTS. This study provides a new perspective on augmenting data epochs for training-based SSVEP-BCI, promotes the classification accuracy and ITR under limited training data, and thus facilitates the real-life applications of SSVEP-based brain spellers.

A novel untrained SSVEP-EEG feature enhancement method using canonical correlation analysis and underdamped second-order stochastic resonance

Objective

Compared with the light-flashing paradigm, the ring-shaped motion checkerboard patterns avoid uncomfortable flicker or brightness modulation, improving the practical interactivity of brain-computer interface (BCI) applications. However, due to fewer harmonic responses and more concentrated frequency energy elicited by the ring-shaped checkerboard patterns, the mainstream untrained algorithms such as canonical correlation analysis (CCA) and filter bank canonical correlation analysis (FBCCA) methods have poor recognition performance and low information transmission rate (ITR).

Methods

To address this issue, a novel untrained SSVEP-EEG feature enhancement method using CCA and underdamped second-order stochastic resonance (USSR) is proposed to extract electroencephalogram (EEG) features.

Results

In contrast to typical unsupervised dimensionality reduction methods such as common average reference (CAR), principal component analysis (PCA), multidimensional scaling (MDS), and locally linear embedding (LLE), CCA exhibits higher adaptability for SSVEP rhythm components.

Conclusion

This study recruits 42 subjects to evaluate the proposed method and experimental results show that the untrained method can achieve higher detection accuracy and robustness.

Significance

This untrained method provides the possibility of applying a nonlinear model from one-dimensional signals to multi-dimensional signals.

Cross Domain Correlation Maximization for Enhancing the Target Recognition of SSVEP-Based Brain-Computer Interfaces

The target recognition performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces can be significantly improved with a training-based approach. However, the training procedure is time consuming and often causes fatigue. Consequently, the number of training data should be limited, which may reduce the classification performance. Thus, how to improve classification accuracy without increasing the training time is crucial to SSVEP-based BCI system. This study proposes a transfer-related component analysis (TransRCA) method for addressing the above issue. In this method, the SSVEP-related components are extracted from a small number of training data of the current individual and combined with those extracted from a large number of existing training data of other individuals. The TransRCA method maximizes not only the inter-trial covariances between the source and target subjects, but also the correlation between the reference signals and SSVEP signals from the source and target subjects. The proposed method was validated on the SSVEP public Benchmark and BETA datasets, and the classification accuracy and information transmission rate of the ensemble version of the proposed TransRCA method were compared with those of the state-of-the-art eCCA, eTRCA, ttCCA, LSTeTRCA, and eIISMC methods on both datasets. The comparison results indicate that the proposed method provides a superior performance compared with these state-of-the-art methods, and thus has high potential for the development of a SSVEP-based brain-computer interface system with high classification performance that only uses a small number of training data.

A Real-Time Non-Implantation Bi-Directional Brain-Computer Interface Solution Without Stimulation Artifacts

The non-implantation bi-directional brain-computer interface (BCI) is a neural interface technology that enables direct two-way communication between the brain and the external world by both "reading" neural signals and "writing" stimulation patterns to the brain. This technology has vast potential applications, such as improving the quality of life for individuals with neurological and mental illnesses and even expanding the boundaries of human capabilities. Nonetheless, non-implantation bi-directional BCIs face challenges in generating real-time feedback and achieving compatibility between stimulation and recording. These issues arise due to the considerable overlap between electrical stimulation frequencies and electrophysiological recording frequencies, as well as the impediment caused by the skull to the interaction of external and internal currents. To address those challenges, this work proposes a novel solution that combines the temporal interference stimulation paradigm and minimally invasive skull modification. A longitudinal animal experiment has preliminarily validated the feasibility of the proposed method. In signal recording experiments, the average impedance of our scheme decreased by 4.59 kΩ , about 67%, compared to the conventional technique at 18 points. The peak-to-peak value of the Somatosensory Evoked Potential increased by 8%. Meanwhile, the signal-to-noise ratio of Steady-State Visual Evoked Potential increased by 5.13 dB, and its classification accuracy increased by 44%. The maximum bandwidth of the resting state rose by 63%. In electrical stimulation experiments, the signal-to-noise ratio of the low-frequency response evoked by our scheme rose by 8.04 dB, and no stimulation artifacts were generated. The experimental results show that signal quality in acquisition has significantly improved, and frequency-band isolation eliminates stimulation artifacts at the source. The acquisition and stimulation pathways are real-time compatible in this non-implantation bi-directional BCI solution, which can provide technical support and theoretical guidance for creating closed-loop adaptive systems coupled with particular application scenarios in the future.

EEG measurement for the effect of perceptual eye position and eye position training on comitant strabismus

One of the clinical features of comitant strabismus is that the deviation angles in the first and second eye positions are equal. However, there has been no report of consistency in the electroencephalography (EEG) signals between the 2 positions. In order to address this issue, we developed a new paradigm based on perceptual eye position. We collected steady-state visual evoked potentials (SSVEPs) signals and resting-state EEG data before and after the eye position training. We found that SSVEP signals could characterize the suppression effect and eye position effect of comitant strabismus, that is, the SSVEP response of the dominant eye was stronger than that of the strabismus eye in the first eye position but not in the second eye position. Perceptual eye position training could modulate the frequency band activities in the occipital and surrounding areas. The changes in the visual function of comitant strabismus after training could also be characterized by SSVEP. There was a correlation between intermodulation frequency, power of parietal electrodes, and perceptual eye position, indicating that EEG might be a potential indicator for evaluating strabismus visual function.

A Calibration-Free Hybrid BCI Speller System Based on High-Frequency SSVEP and sEMG

Hybrid brain-computer interface (hBCI) systems that combine steady-state visual evoked potential (SSVEP) and surface electromyography (sEMG) signals have attracted attention of researchers due to the advantage of exhibiting significantly improved system performance. However, almost all existing studies adopt low-frequency SSVEP to build hBCI. It produces much more visual fatigue than high-frequency SSVEP. Therefore, the current study attempts to build a hBCI based on high-frequency SSVEP and sEMG. With these two signals, this study designed and realized a 32-target hBCI speller system. Thirty-two targets were separated from the middle into two groups. Each side contained 16 sets of targets with different high-frequency visual stimuli (i.e., 31-34.75 Hz with an interval of 0.25 Hz). sEMG was utilized to choose the group and SSVEP was adopted to identify intra-group targets. The filter bank canonical correlation analysis (FBCCA) and the root mean square value (RMS) methods were used to identify signals. Therefore, the proposed system allowed users to operate it without system calibration. A total of 12 healthy subjects participated in online experiment, with an average accuracy of 93.52 ± 1.66% and the average information transfer rate (ITR) reached 93.50 ± 3.10 bits/min. Furthermore, 12 participants perfectly completed the free-spelling tasks. These results of the experiments indicated feasibility and practicality of the proposed hybrid BCI speller system.

SSVEP-based brain-computer interface for music using a low-density EEG system

In this paper, we present a bespoke brain-computer interface (BCI), which was developed for a person with severe motor-impairments, who was previously a Violinist, to allow performing and composing music at home. It uses steady-state visually evoked potential (SSVEP) and adopts a dry, low-density, and wireless electroencephalogram (EEG) headset. In this study, we investigated two parameters: (1) placement of the EEG headset and (2) inter-stimulus distance and found that the former significantly improved the information transfer rate (ITR). To analyze EEG, we adopted canonical correlation analysis (CCA) without weight-calibration. The BCI for musical performance realized a high ITR of 37.59 ± 9.86 bits min-1 and a mean accuracy of 88.89 ± 10.09%. The BCI for musical composition obtained an ITR of 14.91 ± 2.87 bits min-1 and a mean accuracy of 95.83 ± 6.97%. The BCI was successfully deployed to the person with severe motor-impairments. She regularly uses it for musical composition at home, demonstrating how BCIs can be translated from laboratories to real-world scenarios.

Effect of 3D paradigm synchronous motion for SSVEP-based hybrid BCI-VR system

A brain-computer interface (BCI) system and virtual reality (VR) are integrated as a more interactive hybrid system (BCI-VR) that allows the user to manipulate the car. A virtual scene in the VR system that is the same as the physical environment is built, and the object's movement can be observed in the VR scene. The four-class three-dimensional (3D) paradigm is designed and moves synchronously in virtual reality. The dynamic paradigm may affect their attention according to the experimenters' feedback. Fifteen subjects in our experiment steered the car according to a specified motion trajectory. According to our online experimental result, different motion trajectories of the paradigm have various effects on the system's performance, and training can mitigate this adverse effect. Moreover, the hybrid system using frequencies between 5 and 10 Hz indicates better performance than those using lower or higher stimulation frequencies. The experiment results show a maximum average accuracy of 0.956 and a maximum information transfer rate (ITR) of 41.033 bits/min. It suggests that a hybrid system provides a high-performance way of brain-computer interaction. This research could encourage more interesting applications involving BCI and VR technologies.

A high-frequency SSVEP-BCI system based on a 360 Hz refresh rate

Objective. Steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs) often struggle to balance user experience and system performance. To address this challenge, this study employed stimuli in the 55-62.8 Hz frequency range to implement a 40-target BCI speller that offered both high-performance and user-friendliness.Approach. This study proposed a method that presents stable multi-target stimuli on a monitor with a 360 Hz refresh rate. Real-time generation of stimulus matrix and stimulus rendering was used to ensure stable presentation while reducing the computational load. The 40 targets were encoded using the joint frequency and phase modulation method, offline and online BCI experiments were conducted on 16 subjects using the task discriminant component analysis algorithm for feature extraction and classification.Main results. The online BCI system achieved an average accuracy of 88.87% ± 3.05% and an information transfer rate of 51.83 ± 2.77 bits min-1under the low flickering perception condition.Significance. These findings suggest the feasibility and significant practical value of the proposed high-frequency SSVEP BCI system in advancing the visual BCI technology.

[Recognition of high-frequency steady-state visual evoked potential for brain-computer interface]

Coding with high-frequency stimuli could alleviate the visual fatigue of users generated by the brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP). It would improve the comfort and safety of the system and has promising applications. However, most of the current advanced SSVEP decoding algorithms were compared and verified on low-frequency SSVEP datasets, and their recognition performance on high-frequency SSVEPs was still unknown. To address the aforementioned issue, electroencephalogram (EEG) data from 20 subjects were collected utilizing a high-frequency SSVEP paradigm. Then, the state-of-the-art SSVEP algorithms were compared, including 2 canonical correlation analysis algorithms, 3 task-related component analysis algorithms, and 1 task discriminant component analysis algorithm. The results indicated that they all could effectively decode high-frequency SSVEPs. Besides, there were differences in the classification performance and algorithms' speed under different conditions. This paper provides a basis for the selection of algorithms for high-frequency SSVEP-BCI, demonstrating its potential utility in developing user-friendly BCI.

Cross-Subject Transfer Method Based on Domain Generalization for Facilitating Calibration of SSVEP-Based BCIs

In steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs), various spatial filtering methods based on individual calibration data have been proposed to alleviate the interference of spontaneous activities in SSVEP signals for enhancing the SSVEP detection performance. However, the time-consuming calibration session would increase the visual fatigue of subjects and reduce the usability of the BCI system. The key idea of this study is to propose a cross-subject transfer method based on domain generalization, which transfers the domain-invariant spatial filters and templates learned from source subjects to the target subject with no access to the EEG data from the target subject. The transferred spatial filters and templates are obtained by maximizing the intra- and inter-subject correlations using the SSVEP data corresponding to the target and its neighboring stimuli. For SSVEP detection of the target subject, four types of correlation coefficients are calculated to construct the feature vector. Experimental results estimated with three SSVEP datasets show that the proposed cross-subject transfer method improves the SSVEP detection performance compared to state-of-art methods. The satisfactory results demonstrate that the proposed method provides an effective transfer learning strategy requiring no tedious data collection process for new users, holding the potential of promoting practical applications of SSVEP-based BCI.

Shared Three-Dimensional Robotic Arm Control Based on Asynchronous BCI and Computer Vision

OBJECTIVE

A brain-computer interface (BCI) can be used to translate neuronal activity into commands to control external devices. However, using noninvasive BCI to control a robotic arm for movements in three-dimensional (3D) environments and accomplish complicated daily tasks, such as grasping and drinking, remains a challenge.

APPROACH

In this study, a shared robotic arm control system based on hybrid asynchronous BCI and computer vision was presented. The BCI model, which combines steady-state visual evoked potentials (SSVEPs) and blink-related electrooculography (EOG) signals, allows users to freely choose from fifteen commands in an asynchronous mode corresponding to robot actions in a 3D workspace and reach targets with a wide movement range, while computer vision can identify objects and assist a robotic arm in completing more precise tasks, such as grasping a target automatically.

RESULTS

Ten subjects participated in the experiments and achieved an average accuracy of more than 92% and a high trajectory efficiency for robot movement. All subjects were able to perform the reach-grasp-drink tasks successfully using the proposed shared control method, with fewer error commands and shorter completion time than with direct BCI control.

SIGNIFICANCE

Our results demonstrated the feasibility and efficiency of generating practical multidimensional control of an intuitive robotic arm by merging hybrid asynchronous BCI and computer vision-based recognition.

SSVEP unsupervised adaptive feature recognition method based on self-similarity of same-frequency signals

Introduction

As an important human-computer interaction technology, steady-state visual evoked potential (SSVEP) plays a key role in the application of brain computer interface (BCI) systems by accurately decoding SSVEP signals. Currently, the majority SSVEP feature recognition methods use a static classifier. However, electroencephalogram (EEG) signals are non-stationary and time-varying. Hence, an adaptive classification method would be an alternative option to a static classifier for tracking the changes in EEG feature distribution, as its parameters can be re-estimated and updated with the input of new EEG data.

Methods

In this study, an unsupervised adaptive classification algorithm is designed based on the self-similarity of same-frequency signals. The proposed classification algorithm saves the EEG data that has undergone feature recognition as a template signal in accordance with its estimated label, and the new testing signal is superimposed with the template signals at each stimulus frequency as the new test signals to be analyzed. With the continuous input of EEG data, the template signals are continuously updated.

Results

By comparing the classification accuracy of the original testing signal and the testing signal superimposed with the template signals, this study demonstrates the effectiveness of using the self-similarity of same-frequency signals in the adaptive classification algorithm. The experimental results also show that the longer the SSVEP-BCI system is used, the better the responses of users on SSVEP are, and the more significantly the adaptive classification algorithm performs in terms of feature recognition. The testing results of two public datasets show that the adaptive classification algorithm outperforms the static classification method in terms of feature recognition.

Discussion

The proposed adaptive classification algorithm can update the parameters with the input of new EEG data, which is of favorable impact for the accurate analysis of EEG data with time-varying characteristics.

BCI Control of a Robotic Arm Based on SSVEP With Moving Stimuli for Reach and Grasp Tasks

Brain-computer interface (BCI) provides a novel technology for patients and healthy human subjects to control a robotic arm. Currently, BCI control of a robotic arm to complete the reaching and grasping tasks in an unstructured environment is still challenging because the current BCI technology does not meet the requirement of manipulating a multi-degree robotic arm accurately and robustly. BCI based on steady-state visual evoked potential (SSVEP) could output a high information transfer rate; however, the conventional SSVEP paradigm failed to control a robotic arm to move continuously and accurately because the users have to switch their gaze between the flickering stimuli and the target frequently. This study proposed a novel SSVEP paradigm in which the flickering stimuli were attached to the robotic arm's gripper and moved with it. First, an offline experiment was designed to investigate the effects of moving flickering stimuli on the SSVEP's responses and decoding accuracy. After that, contrast experiments were conducted, and twelve subjects were recruited to participate in a robotic arm control experiment using both the paradigm one (P1, with moving flickering stimuli) and the paradigm two (P2, conventional fixed flickering stimuli) using a block randomization design to balance their sequences. Double blinks were used to trigger the grasping action asynchronously whenever the subjects were confident that the position of the robotic arm's gripper was accurate enough. Experimental results showed that the paradigm P1 with moving flickering stimuli provided a much better control performance than the conventional paradigm P2 in completing a reaching and grasping task in an unstructured environment. Subjects' subjective feedback scored by a NASA-TLX mental workload scale also corroborated the BCI control performance. The results of this study suggest that the proposed control interface based on SSVEP BCI provides a better solution for robotic arm control to complete the accurate reaching and grasping tasks.

A wearable group-synchronized EEG system for multi-subject brain-computer interfaces

Objective

The multi-subject brain-computer interface (mBCI) is becoming a key tool for the analysis of group behaviors. It is necessary to adopt a neural recording system for collaborative brain signal acquisition, which is usually in the form of a fixed wire.

Approach

In this study, we designed a wireless group-synchronized neural recording system that supports real-time mBCI and event-related potential (ERP) analysis. This system uses a wireless synchronizer to broadcast events to multiple wearable EEG amplifiers. The simultaneously received broadcast signals are marked in data packets to achieve real-time event correlation analysis of multiple targets in a group.

Main results

To evaluate the performance of the proposed real-time group-synchronized neural recording system, we conducted collaborative signal sampling on 10 wireless mBCI devices. The average signal correlation reached 99.8%, the amplitude of average noise was 0.87 μV, and the average common mode rejection ratio (CMRR) reached 109.02 dB. The minimum synchronization error is 237 μs. We also tested the system in real-time processing of the steady-state visual-evoked potential (SSVEP) ranging from 8 to 15.8 Hz. Under 40 target stimulators, with 2 s data length, the average information transfer rate (ITR) reached 150 ± 20 bits/min, and the highest reached 260 bits/min, which was comparable to the marketing leading EEG system (the average: 150 ± 15 bits/min; the highest: 280 bits/min). The accuracy of target recognition in 2 s was 98%, similar to that of the Synamps2 (99%), but a higher signal-to-noise ratio (SNR) of 5.08 dB was achieved. We designed a group EEG cognitive experiment; to verify, this system can be used in noisy settings.

Significance

The evaluation results revealed that the proposed real-time group-synchronized neural recording system is a high-performance tool for real-time mBCI research. It is an enabler for a wide range of future applications in collaborative intelligence, cognitive neurology, and rehabilitation.

Conformal in-ear bioelectronics for visual and auditory brain-computer interfaces

Brain-computer interfaces (BCIs) have attracted considerable attention in motor and language rehabilitation. Most devices use cap-based non-invasive, headband-based commercial products or microneedle-based invasive approaches, which are constrained for inconvenience, limited applications, inflammation risks and even irreversible damage to soft tissues. Here, we propose in-ear visual and auditory BCIs based on in-ear bioelectronics, named as SpiralE, which can adaptively expand and spiral along the auditory meatus under electrothermal actuation to ensure conformal contact. Participants achieve offline accuracies of 95% in 9-target steady state visual evoked potential (SSVEP) BCI classification and type target phrases successfully in a calibration-free 40-target online SSVEP speller experiment. Interestingly, in-ear SSVEPs exhibit significant 2nd harmonic tendencies, indicating that in-ear sensing may be complementary for studying harmonic spatial distributions in SSVEP studies. Moreover, natural speech auditory classification accuracy can reach 84% in cocktail party experiments. The SpiralE provides innovative concepts for designing 3D flexible bioelectronics and assists the development of biomedical engineering and neural monitoring.

Dataset Evaluation Method and Application for Performance Testing of SSVEP-BCI Decoding Algorithm

Steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems have been extensively researched over the past two decades, and multiple sets of standard datasets have been published and widely used. However, there are differences in sample distribution and collection equipment across different datasets, and there is a lack of a unified evaluation method. Most new SSVEP decoding algorithms are tested based on self-collected data or offline performance verification using one or two previous datasets, which can lead to performance differences when used in actual application scenarios. To address these issues, this paper proposed a SSVEP dataset evaluation method and analyzed six datasets with frequency and phase modulation paradigms to form an SSVEP algorithm evaluation dataset system. Finally, based on the above datasets, performance tests were carried out on the four existing SSVEP decoding algorithms. The findings reveal that the performance of the same algorithm varies significantly when tested on diverse datasets. Substantial performance variations were observed among subjects, ranging from the best-performing to the worst-performing. The above results demonstrate that the SSVEP dataset evaluation method can integrate six datasets to form a SSVEP algorithm performance testing dataset system. This system can test and verify the SSVEP decoding algorithm from different perspectives such as different subjects, different environments, and different equipment, which is helpful for the research of new SSVEP decoding algorithms and has significant reference value for other BCI application fields.