An MEG-based brain-computer interface (BCI)

Comparing MEG and EEG measurement set-ups for a brain-computer interface based on selective auditory attention

Auditory attention modulates auditory evoked responses to target vs. non-target sounds in electro- and magnetoencephalographic (EEG/MEG) recordings. Employing whole-scalp MEG recordings and offline classification algorithms has been shown to enable high accuracy in tracking the target of auditory attention. Here, we investigated the decrease in accuracy when moving from the whole-scalp MEG to lower channel count EEG recordings and when training the classifier only from the initial or middle part of the recording instead of extracting training trials throughout the recording. To this end, we recorded simultaneous MEG (306 channels) and EEG (64 channels) in 18 healthy volunteers while presented with concurrent streams of spoken "Yes"/"No" words and instructed to attend to one of them. We then trained support vector machine classifiers to predict the target of attention from unaveraged trials of MEG/EEG. Classifiers were trained on 204 MEG gradiometers or on EEG with 64, 30, nine or three channels with trials extracted randomly across or only from the beginning of the recording. The highest classification accuracy, 73.2% on average across the participants for one-second trials, was obtained with MEG when the training trials were randomly extracted throughout the recording. With EEG, the accuracy was 69%, 69%, 66%, and 61% when using 64, 30, nine, and three channels, respectively. When training the classifiers with the same amount of data but extracted only from the beginning of the recording, the accuracy dropped by 11%-units on average, causing the result from the three-channel EEG to fall below the chance level. The combination of five consecutive trials partially compensated for this drop such that it was one to 5%-units. Although moving from whole-scalp MEG to EEG reduces classification accuracy, usable auditory-attention-based brain-computer interfaces can be implemented with a small set of optimally placed EEG channels.

Neurophysiological data augmentation for EEG-fNIRS multimodal features based on a denoising diffusion probabilistic model

BACKGROUND AND OBJECTIVE

The hybrid brain computer interfaces (BCI) combining electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) have attracted extensive attention for overcoming the decoding limitations of the single-modality BCI. With the deepening application of deep learning approaches in BCI systems, its significant performance improvement has become apparent. However, the scarcity of brain signal data limits the performance of deep learning models.

METHODS

In this study, we proposed an EEG-fNIRS data augmentation framework based on the combination of denoising diffusion probabilistic model (DDPM) and adding Gaussian noise (EFDA-CDG), for enhancing the performance of hybrid BCI systems. Firstly, we unified the temporal and spatial dimensions of EEG and fNIRS by manually extracting features and spatial mapping interpolation to create EEG-fNIRS joint distribution samples. Then, the DDPM generative model was combined with the traditional method of adding Gaussian noise to provide richer training data for the classifier. Finally, we constructed a classification module that applies EEG feature attention and fNIRS terrain attention to improve classification accuracy.

RESULTS

In order to evaluate the effectiveness of EFDA-CDG framework, experiments were conducted and fully validated on three publicly available databases and one self-collected database. In the context of a participant-dependent training approach, our method achieves accuracy rates of 82.02% for motor imagery, 91.93% for mental arithmetic, and 90.54% for n-back tasks on public databases. Additionally, our method boasts an accuracy rate of 97.82% for drug addiction discrimination task on the self-collected database.

CONCLUSIONS

EFDA-CDG framework successfully facilitates data augmentation, thereby enhancing the performance of EEG-fNIRS hybrid BCI systems.

Invasive Brain-Computer Interface for Communication: A Scoping Review

BACKGROUND

The rapid expansion of the brain-computer interface for patients with neurological deficits has garnered significant interest, and for patients, it provides an additional route where conventional rehabilitation has its limits. This has particularly been the case for patients who lose the ability to communicate. Circumventing neural injuries by recording from the intact cortex and subcortex has the potential to allow patients to communicate and restore self-expression. Discoveries over the last 10-15 years have been possible through advancements in technology, neuroscience, and computing. By examining studies involving intracranial brain-computer interfaces that aim to restore communication, we aimed to explore the advances made and explore where the technology is heading.

METHODS

For this scoping review, we systematically searched PubMed and OVID Embase. After processing the articles, the search yielded 41 articles that we included in this review.

RESULTS

The articles predominantly assessed patients who had either suffered from amyotrophic lateral sclerosis, cervical cord injury, or brainstem stroke, resulting in tetraplegia and, in some cases, difficulty speaking. Of the intracranial implants, ten had ALS, six had brainstem stroke, and thirteen had a spinal cord injury. Stereoelectroencephalography was also used, but the results, whilst promising, are still in their infancy. Studies involving patients who were moving cursors on a screen could improve the speed of movement by optimising the interface and utilising better decoding methods. In recent years, intracortical devices have been successfully used for accurate speech-to-text and speech-to-audio decoding in patients who are unable to speak.

CONCLUSIONS

Here, we summarise the progress made by BCIs used for communication. Speech decoding directly from the cortex can provide a novel therapeutic method to restore full, embodied communication to patients suffering from tetraplegia who otherwise cannot communicate.

Recognition of brain activities via graph-based long short-term memory-convolutional neural network

Introduction

Human brain activities are always difficult to recognize due to its diversity and susceptibility to disturbance. With its unique capability of measuring brain activities, magnetoencephalography (MEG), as a high temporal and spatial resolution neuroimaging technique, has been used to identify multi-task brain activities. Accurately and robustly classifying motor imagery (MI) and cognitive imagery (CI) from MEG signals is a significant challenge in the field of brain-computer interface (BCI).

Methods

In this study, a graph-based long short-term memory-convolutional neural network (GLCNet) is proposed to classify the brain activities in MI and CI tasks. It was characterized by implementing three modules of graph convolutional network (GCN), spatial convolution and long short-term memory (LSTM) to effectively extract time-frequency-spatial features simultaneously. For performance evaluation, our method was compared with six benchmark algorithms of FBCSP, FBCNet, EEGNet, DeepConvNets, Shallow ConvNet and MEGNet on two public datasets of MEG-BCI and BCI competition IV dataset 3.

Results

The results demonstrated that the proposed GLCNet outperformed other models with the average accuracies of 78.65% and 65.8% for two classification and four classification on the MEG-BCI dataset, respectively.

Discussion

It was concluded that the GLCNet enhanced the model's adaptability in handling individual variability with robust performance. This would contribute to the exploration of brain activates in neuroscience.

Repairbads: An automatic and adaptive method to repair bad channels and segments for OPM-MEG

The optically pumped magnetometer (OPM) based magnetoencephalography (MEG) system offers advantages such as flexible layout and wearability. However, the position instability or jitter of OPM sensors can result in bad channels and segments, which significantly impede subsequent preprocessing and analysis. Most common methods directly reject or interpolate to repair these bad channels and segments. Direct rejection leads to data loss, and when the number of sensors is limited, interpolation using neighboring sensors can cause significant signal distortion and cannot repair bad segments present in all channels. Therefore, most existing methods are unsuitable for OPM-MEG systems with fewer channels. We introduce an automatic bad segments and bad channels repair method for OPM-MEG, called Repairbads. This method aims to repair all bad data and reduce signal distortion, especially capable of automatically repairing bad segments present in all channels simultaneously. Repairbads employs Riemannian Potato combined with joint decorrelation to project out artifact components, achieving automatic bad segment repair. Then, an adaptive algorithm is used to segment the signal into relatively stable noise data chunks, and the source-estimate-utilizing noise-discarding algorithm is applied to each chunk to achieve automatic bad channel repair. We compared the performance of Repairbads with the Autoreject method on both simulated and real auditory evoked data, using five evaluation metrics for quantitative assessment. The results demonstrate that Repairbads consistently outperforms across all five metrics. In both simulated and real OPM-MEG data, Repairbads shows better performance than current state-of-the-art methods, reliably repairing bad data with minimal distortion. The automation of this method significantly reduces the burden of manual inspection, promoting the automated processing and clinical application of OPM-MEG.

Coherence-based channel selection and Riemannian geometry features for magnetoencephalography decoding

Magnetoencephalography (MEG) records the extremely weak magnetic fields on the surface of the scalp through highly sensitive sensors. Multi-channel MEG data provide higher spatial and temporal resolution when measuring brain activities, and can be applied for brain-computer interfaces as well. However, a large number of channels leads to high computational complexity and can potentially impact decoding accuracy. To improve the accuracy of MEG decoding, this paper proposes a new coherence-based channel selection method that effectively identifies task-relevant channels, reducing the presence of noisy and redundant information. Riemannian geometry is then used to extract effective features from selected channels of MEG data. Finally, MEG decoding is achieved by training a support vector machine classifier with the Radial Basis Function kernel. Experiments were conducted on two public MEG datasets to validate the effectiveness of the proposed method. The results from Dataset 1 show that Riemannian geometry achieves higher classification accuracy (compared to common spatial patterns and power spectral density) in the single-subject visual decoding task. Moreover, coherence-based channel selection significantly (P = 0.0002) outperforms the use of all channels. Moving on to Dataset 2, the results reveal that coherence-based channel selection is also significantly (P <0.0001) superior to all channels and channels around C3 and C4 in cross-session mental imagery decoding tasks. Additionally, the proposed method outperforms state-of-the-art methods in motor imagery tasks.

A class alignment network based on self-attention for cross-subject EEG classification

Due to the inherent variability in EEG signals across different individuals, domain adaptation and adversarial learning strategies are being progressively utilized to develop subject-specific classification models by leveraging data from other subjects. These approaches primarily focus on domain alignment and tend to overlook the critical task-specific class boundaries. This oversight can result in weak correlation between the extracted features and categories. To address these challenges, we propose a novel model that uses the known information from multiple subjects to bolster EEG classification for an individual subject through adversarial learning strategies. Our method begins by extracting both shallow and attention-driven deep features from EEG signals. Subsequently, we employ a class discriminator to encourage the same-class features from different domains to converge while ensuring that the different-class features diverge. This is achieved using our proposed discrimination loss function, which is designed to minimize the feature distance for samples of the same class across different domains while maximizing it for those from different classes. Additionally, our model incorporates two parallel classifiers that are harmonious yet distinct and jointly contribute to decision-making. Extensive testing on two publicly available EEG datasets has validated our model's efficacy and superiority.

Feasibility of Stereo EEG Based Brain Computer Interfacing in An Adult and Pediatric Cohort

Introduction

Stereoelectroencephalography (sEEG) is a mesoscale intracranial monitoring method which records from the brain volumetrically with depth electrodes. Implementation of sEEG in BCI has not been well-described across a diverse patient cohort.

Methods

Across eighteen subjects, channels with high frequency broadband (HFB, 65-115Hz) power increases during hand, tongue, or foot movements during a motor screening task were provided real-time feedback based on these HFB power changes to control a cursor on a screen.

Results

Seventeen subjects established successful control of the overt motor BCI, but only nine were able to control imagery BCI with ≥ 80% accuracy. In successful imagery BCI, HFB power in the two target conditions separated into distinct subpopulations, which appear to engage unique subnetworks of the motor cortex compared to cued movement or imagery alone.

Conclusion

sEEG-based motor BCI utilizing overt movement and kinesthetic imagery is robust across patient ages and cortical regions with substantial differences in learning proficiency between real or imagined movement.

Estimating and approaching the maximum information rate of noninvasive visual brain-computer interface

An essential priority of visual brain-computer interfaces (BCIs) is to enhance the information transfer rate (ITR) to achieve high-speed communication. Despite notable progress, noninvasive visual BCIs have encountered a plateau in ITRs, leaving it uncertain whether higher ITRs are achievable. In this study, we used information theory to study the characteristics and capacity of the visual-evoked channel, which leads us to investigate whether and how we can decode higher information rates in a visual BCI system. Using information theory, we estimate the upper and lower bounds of the information rate with the white noise (WN) stimulus. Consequently, we found out that the information rate is determined by the signal-to-noise ratio (SNR) in the frequency domain, which reflects the spectrum resources of the channel. Based on this discovery, we propose a broadband WN BCI by implementing stimuli on a broader frequency band than the steady-state visual evoked potentials (SSVEPs)-based BCI. Through validation, the broadband BCI outperforms the SSVEP BCI by an impressive 7 bps, setting a record of 50 bps. The integration of information theory and the decoding analysis presented in this study offers valuable insights applicable to general sensory-evoked BCIs, providing a potential direction of next-generation human-machine interaction systems.

Brain-computer interface paradigms and neural coding

Brain signal patterns generated in the central nervous system of brain-computer interface (BCI) users are closely related to BCI paradigms and neural coding. In BCI systems, BCI paradigms and neural coding are critical elements for BCI research. However, so far there have been few references that clearly and systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding. Therefore, these contents are expounded and the existing main BCI paradigms and neural coding are introduced in the review. Finally, the challenges and future research directions of BCI paradigm and neural coding were discussed, including user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance. It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.

Multimodal Neuroimaging Computing: Basics and Applications in Neurosurgery

In neurosurgery, multimodal neuroimaging computing plays a critical role by providing a comprehensive and detailed understanding of the brain and its function. This integrated approach can unlock deeper insights into complex neurological diseases, as well as providing a big picture for image-guided neurosurgery and precision medicine. In this chapter, we will introduce the recent updates of neuroimaging techniques, their applications in neurosurgery scenarios, the difficulties of data processing and computing, and potential future perspectives.

Cybersecurity in neural interfaces: Survey and future trends

With the joint advancement in areas such as pervasive neural data sensing, neural computing, neuromodulation and artificial intelligence, neural interface has become a promising technology facilitating both the closed-loop neurorehabilitation for neurologically impaired patients and the intelligent man-machine interactions for general application purposes. However, although neural interface has been widely studied, few previous studies focused on the cybersecurity issues in related applications. In this survey, we systematically investigated possible cybersecurity risks in neural interfaces, together with potential solutions to these problems. Importantly, our survey considers interfacing techniques on both central nervous systems (i.e., brain-computer interfaces) and peripheral nervous systems (i.e., general neural interfaces), covering diverse neural modalities such as electroencephalography, electromyography and more. Moreover, our survey is organized on three different levels: (1) the data level, which mainly focuses on the privacy leakage issue via attacking and analyzing neural database of users; (2) the permission level, which mainly focuses on the prospects and risks to directly use real time neural signals as biometrics for continuous and unobtrusive user identity verification; and (3) the model level, which mainly focuses on adversarial attacks and defenses on both the forward neural decoding models (e.g. via machine learning) and the backward feedback implementation models (e.g. via neuromodulation and stimulation). This is the first study to systematically investigate cybersecurity risks and possible solutions in neural interfaces which covers both central and peripheral nervous systems, and considers multiple different levels to provide a complete picture of this issue.

It takes two (seconds): decreasing encoding time for two-choice functional near-infrared spectroscopy brain-computer interface communication

Significance

Brain-computer interfaces (BCIs) can provide severely motor-impaired patients with a motor-independent communication channel. Functional near-infrared spectroscopy (fNIRS) constitutes a promising BCI-input modality given its high mobility, safety, user comfort, cost-efficiency, and relatively low motion sensitivity.

Aim

The present study aimed at developing an efficient and convenient two-choice fNIRS communication BCI by implementing a relatively short encoding time (2 s), considerably increasing communication speed, and decreasing the cognitive load of BCI users.

Approach

To encode binary answers to 10 biographical questions, 10 healthy adults repeatedly performed a combined motor-speech imagery task within 2 different time windows guided by auditory instructions. Each answer-encoding run consisted of 10 trials. Answers were decoded during the ongoing experiment from the time course of the individually identified most-informative fNIRS channel-by-chromophore combination.

Results

The answers of participants were decoded online with an accuracy of 85.8% (run-based group mean). Post-hoc analysis yielded an average single-trial accuracy of 68.1%. Analysis of the effect of number of trial repetitions showed that the best information-transfer rate could be obtained by combining four encoding trials.

Conclusions

The study demonstrates that an encoding time as short as 2 s can enable immediate, efficient, and convenient fNIRS-BCI communication.

SSVEP-based brain-computer interface for music using a low-density EEG system

In this paper, we present a bespoke brain-computer interface (BCI), which was developed for a person with severe motor-impairments, who was previously a Violinist, to allow performing and composing music at home. It uses steady-state visually evoked potential (SSVEP) and adopts a dry, low-density, and wireless electroencephalogram (EEG) headset. In this study, we investigated two parameters: (1) placement of the EEG headset and (2) inter-stimulus distance and found that the former significantly improved the information transfer rate (ITR). To analyze EEG, we adopted canonical correlation analysis (CCA) without weight-calibration. The BCI for musical performance realized a high ITR of 37.59 ± 9.86 bits min-1 and a mean accuracy of 88.89 ± 10.09%. The BCI for musical composition obtained an ITR of 14.91 ± 2.87 bits min-1 and a mean accuracy of 95.83 ± 6.97%. The BCI was successfully deployed to the person with severe motor-impairments. She regularly uses it for musical composition at home, demonstrating how BCIs can be translated from laboratories to real-world scenarios.

Hybrid brain/neural interface and autonomous vision-guided whole-arm exoskeleton control to perform activities of daily living (ADLs)

BACKGROUND

The aging of the population and the progressive increase of life expectancy in developed countries is leading to a high incidence of age-related cerebrovascular diseases, which affect people's motor and cognitive capabilities and might result in the loss of arm and hand functions. Such conditions have a detrimental impact on people's quality of life. Assistive robots have been developed to help people with motor or cognitive disabilities to perform activities of daily living (ADLs) independently. Most of the robotic systems for assisting on ADLs proposed in the state of the art are mainly external manipulators and exoskeletal devices. The main objective of this study is to compare the performance of an hybrid EEG/EOG interface to perform ADLs when the user is controlling an exoskeleton rather than using an external manipulator.

METHODS

Ten impaired participants (5 males and 5 females, mean age 52 ± 16 years) were instructed to use both systems to perform a drinking task and a pouring task comprising multiple subtasks. For each device, two modes of operation were studied: synchronous mode (the user received a visual cue indicating the sub-tasks to be performed at each time) and asynchronous mode (the user started and finished each of the sub-tasks independently). Fluent control was assumed when the time for successful initializations ranged below 3 s and a reliable control in case it remained below 5 s. NASA-TLX questionnaire was used to evaluate the task workload. For the trials involving the use of the exoskeleton, a custom Likert-Scale questionnaire was used to evaluate the user's experience in terms of perceived comfort, safety, and reliability.

RESULTS

All participants were able to control both systems fluently and reliably. However, results suggest better performances of the exoskeleton over the external manipulator (75% successful initializations remain below 3 s in case of the exoskeleton and bellow 5s in case of the external manipulator).

CONCLUSIONS

Although the results of our study in terms of fluency and reliability of EEG control suggest better performances of the exoskeleton over the external manipulator, such results cannot be considered conclusive, due to the heterogeneity of the population under test and the relatively limited number of participants.

Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities

Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.

Classification of Right/Left Hand Motor Imagery by Effective Connectivity Based on Transfer Entropy in Electroencephalogram Signal

Introduction

The right and left-hand motor imagery (MI) analysis based on the electroencephalogram (EEG) signal can directly link the central nervous system to a computer or a device. This study aims to identify a set of robust and nonlinear effective brain connectivity features quantified by transfer entropy (TE) to characterize the relationship between brain regions from EEG signals and create a hierarchical feature selection and classification for discrimination of right and lefthand MI tasks.

Methods

TE is calculated among EEG channels as the distinctive, effective connectivity features. TE is a model-free method that can measure nonlinear effective connectivity and analyze multivariate dependent directed information flow among neural EEG channels. Then four feature subset selection methods namely relief-F, Fisher, Laplacian, and local learningbased clustering (LLCFS) algorithms are used to choose the most significant effective connectivity features and reduce redundant information. Finally, support vector machine (SVM) and linear discriminant analysis (LDA) methods are used for classification.

Results

Results show that the best performance in 29 healthy subjects and 60 trials is achieved using the TE method via the Relief-F algorithm as feature selection and support vector machine (SVM) classification with 91.02% accuracy.

Conclusion

The TE index and a hierarchical feature selection and classification can be useful for the discrimination of right- and left-hand MI tasks from multichannel EEG signals.

Highlights

Effective connectivity features were extracted from electroencephalogram (EEG) to analyze relationships between regions.Four feature selection methods used to select most significant effective features.Support vector machine (SVM) used for discrimination of right and left hand motor imagery (MI) task.

Plain Language Summary

In this study, we investigated brain activity using effective connectivity during MI task based on EEG signals. The motor imagery task can accomplish the same goal as motor execution, since they are both activated by the same brain area. Transfer entropy, coherence, and Granger casualty were employed to extract the features. Differential patterns of activity between the left vs. right MI task showed activity around the motor area rather than other areas. In order to reduce redundant information and select the most significant effective connectivity features, four feature subset selection algorithms are used: Relief-F, Fisher, Laplacian, and learning-based clustering feature selection (LLCFS). Then, support vector machine (SVM) and linear discriminant analysis (LDA) are used to classify left and right hand MI task. Comparison of three different connectivity methods showed that TE index had the highest classification accuracy, and could be useful for the discrimination of right and left hand MI tasks from multichannel EEG signals.

An online hybrid BCI combining SSVEP and EOG-based eye movements

Hybrid brain-computer interface (hBCI) refers to a system composed of a single-modality BCI and another system. In this paper, we propose an online hybrid BCI combining steady-state visual evoked potential (SSVEP) and eye movements to improve the performance of BCI systems. Twenty buttons corresponding to 20 characters are evenly distributed in the five regions of the GUI and flash at the same time to arouse SSVEP. At the end of the flash, the buttons in the four regions move in different directions, and the subject continues to stare at the target with eyes to generate the corresponding eye movements. The CCA method and FBCCA method were used to detect SSVEP, and the electrooculography (EOG) waveform was used to detect eye movements. Based on the EOG features, this paper proposes a decision-making method based on SSVEP and EOG, which can further improve the performance of the hybrid BCI system. Ten healthy students took part in our experiment, and the average accuracy and information transfer rate of the system were 94.75% and 108.63 bits/min, respectively.

Riemannian distance based channel selection and feature extraction combining discriminative time-frequency bands and Riemannian tangent space for MI-BCIs

OBJECTIVE

Motor imagery-based brain computer interfaces (MI-BCIs) have been widely researched because they do not demand external stimuli and have a high degree of maneuverability. In most scenarios, superabundant selected channels, fixed time windows, and frequency bands would certainly affect the performance of MI-BCIs due to the neurophysiological diversities between different individuals. In this study, we tended to effectively use the Riemannian geometry of spatial covariance matrix to extract more robust features and thus enhance the decoding efficiency.

APPROACH

First, we propose a Riemannian distance-based EEG channel selection method (RDCS), which preliminarily reduces the information redundancy in the first stage. Second, we extract discriminative Riemannian Tangent Space features of EEG signals of selected channels from the most discriminant time-frequency bands (DTFRTS) to further enhance decoding accuracy for MI-BCIs. Finally, we trained a support vector machine (SVM) model with a linear kernel to classify our extracted discriminative Riemannian features and evaluated our proposed method using publicly available BCI Competition Ⅳ dataset Ⅰ (DS1) and Competition Ⅲ dataset Ⅲa (DS2).

MAIN RESULTS

The experimental results showed that the average classification accuracy with the selected 10-channel EEG signals of our method is 88.1% and 91.6% in DS1 and DS2 respectively. The average improvements are 24.3% & 27.1% on DS1 and 4.4% & 14.2% on DS2 for 10 & 20 selected channels, respectively.

SIGNIFICANCE

These results showed that our proposed method is a promising candidate for performance improvement of MI-BCIs.

Transfer learning for motor imagery based brain-computer interfaces: A tutorial

A brain-computer interface (BCI) enables a user to communicate directly with an external device, e.g., a computer, using brain signals. It can be used to research, map, assist, augment, or repair human cognitive or sensory-motor functions. A closed-loop BCI system performs signal acquisition, temporal filtering, spatial filtering, feature engineering and classification, before sending out the control signal to an external device. Transfer learning (TL) has been widely used in motor imagery (MI) based BCIs to reduce the calibration effort for a new subject, greatly increasing their utility. This tutorial describes how TL can be considered in as many components of a BCI system as possible, and introduces a complete TL pipeline for MI-based BCIs. Examples on two MI datasets demonstrated the advantages of considering TL in multiple components of MI-based BCIs. Especially, integrating data alignment and sophisticated TL approaches can significantly improve the classification performance, and hence greatly reduces the calibration effort.

A CNN-Based Deep Learning Approach for SSVEP Detection Targeting Binaural Ear-EEG

This paper discusses a machine learning approach for detecting SSVEP at both ears with minimal channels. SSVEP is a robust EEG signal suitable for many BCI applications. It is strong at the visual cortex around the occipital area, but the SNR gets worse when detected from other areas of the head. To make use of SSVEP measured around the ears following the ear-EEG concept, especially for practical binaural implementation, we propose a CNN structure coupled with regressed softmax outputs to improve accuracy. Evaluating on a public dataset, we studied classification performance for both subject-dependent and subject-independent trainings. It was found that with the proposed structure using a group training approach, a 69.21% accuracy was achievable. An ITR of 6.42 bit/min given 63.49 % accuracy was recorded while only monitoring data from T7 and T8. This represents a 12.47% improvement from a single ear implementation and illustrates potential of the approach to enhance performance for practical implementation of wearable EEG.

Evaluation of Real-Time Endogenous Brain-Computer Interface Developed Using Ear-Electroencephalography

While previous studies have demonstrated the feasibility of using ear-electroencephalography (ear-EEG) for the development of brain-computer interfaces (BCIs), most of them have been performed using exogenous paradigms in offline environments. To verify the reliable feasibility of constructing ear-EEG-based BCIs, the feasibility of using ear-EEG should be further demonstrated using another BCI paradigm, namely the endogenous paradigm, in real-time online environments. Exogenous and endogenous BCIs are to use the EEG evoked by external stimuli and induced by self-modulation, respectively. In this study, we investigated whether an endogenous ear-EEG-based BCI with reasonable performance can be implemented in online environments that mimic real-world scenarios. To this end, we used three different mental tasks, i.e., mental arithmetic, word association, and mental singing, and performed BCI experiments with fourteen subjects on three different days to investigate not only the reliability of a real-time endogenous ear-EEG-based BCI, but also its test-retest reliability. The mean online classification accuracy was almost 70%, which was equivalent to a marginal accuracy for a practical two-class BCI (70%), demonstrating the feasibility of using ear-EEG for the development of real-time endogenous BCIs, but further studies should follow to improve its performance enough to be used for practical ear-EEG-based BCI applications.

Classification of Event-Related Potentials with Regularized Spatiotemporal LCMV Beamforming

The usability of EEG-based visual brain–computer interfaces (BCIs) based on event-related potentials (ERPs) benefits from reducing the calibration time before BCI operation. Linear decoding models, such as the spatiotemporal beamformer model, yield state-of-the-art accuracy. Although the training time of this model is generally low, it can require a substantial amount of training data to reach functional performance. Hence, BCI calibration sessions should be sufficiently long to provide enough training data. This work introduces two regularized estimators for the beamformer weights. The first estimator uses cross-validated L2-regularization. The second estimator exploits prior information about the structure of the EEG by assuming Kronecker–Toeplitz-structured covariance. The performances of these estimators are validated and compared with the original spatiotemporal beamformer and a Riemannian-geometry-based decoder using a BCI dataset with P300-paradigm recordings for 21 subjects. Our results show that the introduced estimators are well-conditioned in the presence of limited training data and improve ERP classification accuracy for unseen data. Additionally, we show that structured regularization results in lower training times and memory usage, and a more interpretable classification model.

FGANet: fNIRS-guided Attention Network for Hybrid EEG-fNIRS Brain-Computer Interfaces

Non-invasive brain-computer interfaces (BCIs) have been widely used for neural decoding, linking neural signals to control devices. Hybrid BCI systems using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have received significant attention for overcoming the limitations of EEG- and fNIRS-standalone BCI systems. However, most hybrid EEG-fNIRS BCI studies have focused on late fusion because of discrepancies in their temporal resolutions and recording locations. Despite the enhanced performance of hybrid BCIs, late fusion methods have difficulty in extracting correlated features in both EEG and fNIRS signals. Therefore, in this study, we proposed a deep learning-based early fusion structure, which combines two signals before the fully-connected layer, called the fNIRS-guided attention network (FGANet). First, 1D EEG and fNIRS signals were converted into 3D EEG and fNIRS tensors to spatially align EEG and fNIRS signals at the same time point. The proposed fNIRS-guided attention layer extracted a joint representation of EEG and fNIRS tensors based on neurovascular coupling, in which the spatially important regions were identified from fNIRS signals, and detailed neural patterns were extracted from EEG signals. Finally, the final prediction was obtained by weighting the sum of the prediction scores of the EEG and fNIRS-guided attention features to alleviate performance degradation owing to delayed fNIRS response. In the experimental results, the FGANet significantly outperformed the EEG-standalone network. Furthermore, the FGANet has 4.0% and 2.7% higher accuracy than the state-of-the-art algorithms in mental arithmetic and motor imagery tasks, respectively.

Influence of the Number of Channels and Classification Algorithm on the Performance Robustness to Electrode Shift in Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces

There remains an active investigation on elevating the classification accuracy and information transfer rate of brain-computer interfaces based on steady-state visual evoked potential. However, it has often been ignored that the performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can be affected through the minor displacement of the electrodes from their optimal locations in practical applications because of the mislocation of electrodes and/or concurrent use of electroencephalography (EEG) devices with external devices, such as virtual reality headsets. In this study, we evaluated the performance robustness of SSVEP-based BCIs with respect to the changes in electrode locations for various channel configurations and classification algorithms. Our experiments involved 21 participants, where EEG signals were recorded from the scalp electrodes densely attached to the occipital area of the participants. The classification accuracies for all the possible cases of electrode location shifts for various channel configurations (1–3 channels) were calculated using five training-free SSVEP classification algorithms, i.e., the canonical correlation analysis (CCA), extended CCA, filter bank CCA, multivariate synchronization index (MSI), and extended MSI (EMSI). Then, the performances of the BCIs were evaluated using two measures, i.e., the average classification accuracy (ACA) across the electrode shifts and robustness to the electrode shift (RES). Our results showed that the ACA increased with an increase in the number of channels regardless of the algorithm. However, the RES was enhanced with an increase in the number of channels only when MSI and EMSI were employed. While both ACA and RES values for the five algorithms were similar under the single-channel condition, both ACA and RES values for MSI and EMSI were higher than those of the other algorithms under the multichannel (i.e., two or three electrodes) conditions. In addition, EMSI outperformed MSI when comparing the ACA and RES values under the multichannel conditions. In conclusion, our results suggested that the use of multichannel configuration and employment of EMSI could make the performance of SSVEP-based BCIs more robust to the electrode shift from the optimal locations.

Brain–Computer Interfacing Using Functional Near-Infrared Spectroscopy (fNIRS)

Functional Near-Infrared Spectroscopy (fNIRS) is a wearable optical spectroscopy system originally developed for continuous and non-invasive monitoring of brain function by measuring blood oxygen concentration. Recent advancements in brain–computer interfacing allow us to control the neuron function of the brain by combining it with fNIRS to regulate cognitive function. In this review manuscript, we provide information regarding current advancement in fNIRS and how it provides advantages in developing brain–computer interfacing to enable neuron function. We also briefly discuss about how we can use this technology for further applications.

Crossing time windows optimization based on mutual information for hybrid BCI

Hybrid EEG-fNIRS brain-computer interface (HBCI) is widely employed to enhance BCI performance. EEG and fNIRS signals are combined to increase the dimensionality of the information. Time windows are used to select EEG and fNIRS singles synchronously. However, it ignores that specific modal signals have their own characteristics, when the task is stimulated, the information between the modalities will mismatch at the moment, which has a significant impact on the classification performance. Here we propose a novel crossing time windows optimization for mental arithmetic (MA) based BCI. The EEG and fNIRS signals were segmented separately by sliding time windows. Then crossing time windows (CTW) were combined with each one segment from EEG and fNIRS selected independently. Furthermore, EEG and fNIRS features were extracted using Filter Bank Common Spatial Pattern (FBCSP) and statistical methods from each sample. Mutual information was calculated for FBCSP and statistical features to characterize the discrimination of crossing time windows, and the optimal window would be selected based on the largest mutual information. Finally, a sparse structured framework of Fisher Lasso feature selection (FLFS) was designed to select the joint features, and conventional Linear Discriminant Analysis (LDA) was employed to perform classification. We used proposed method for a MA dataset. The classification accuracy of the proposed method is 92.52 ± 5.38% and higher than other methods, which shows the rationality and superiority of the proposed method.

Practical real-time MEG-based neural interfacing with optically pumped magnetometers

BACKGROUND

Brain-computer interfaces decode intentions directly from the human brain with the aim to restore lost functionality, control external devices or augment daily experiences. To combine optimal performance with wide applicability, high-quality brain signals should be captured non-invasively. Magnetoencephalography (MEG) is a potent candidate but currently requires costly and confining recording hardware. The recently developed optically pumped magnetometers (OPMs) promise to overcome this limitation, but are currently untested in the context of neural interfacing.

RESULTS

In this work, we show that OPM-MEG allows robust single-trial analysis which we exploited in a real-time 'mind-spelling' application yielding an average accuracy of 97.7%.

CONCLUSIONS

This shows that OPM-MEG can be used to exploit neuro-magnetic brain responses in a practical and flexible manner, and opens up new avenues for a wide range of new neural interface applications in the future.

Extended Signal-Space Separation Method for Improved Interference Suppression in MEG

Objective:

Magnetoencephalography (MEG) signals typically reflect a mixture of neuromagnetic fields, subject-related artifacts, external interference and sensor noise. Even inside a magnetically shielded room, external interference can be significantly stronger than brain signals. Methods such as signal-space projection (SSP) and signal-space separation (SSS) have been developed to suppress this residual interference, but their performance might not be sufficient in cases of strong interference or when the sources of interference change over time.

Methods:

Here we suggest a new method, extended signal-space separation (eSSS), which combines a physical model of the magnetic fields (as in SSS) with a statistical description of the interference (as in SSP). We demonstrate the performance of this method via simulations and experimental MEG data.

Results:

The eSSS method clearly outperforms SSS and SSP in interference suppression regardless of the extent of a priori information available on the interference sources. We also show that the method does not cause location or amplitude bias in dipole modeling.

Conclusion:

Our eSSS method provides better data quality than SSP or SSS and can be readily combined with other SSS-based methods, such as spatiotemporal SSS or head movement compensation. Thus, eSSS extends and complements the interference suppression techniques currently available for MEG.

Significance:

Due to its ability to suppress external interference to the level of sensor noise, eSSS can facilitate single-trial data analysis, exemplified in automated analysis of epileptic data. Such an enhanced suppression is especially important in environments with large interference fields.

A magnetoencephalography dataset for motor and cognitive imagery-based brain-computer interface

Recent advancements in magnetoencephalography (MEG)-based brain-computer interfaces (BCIs) have shown great potential. However, the performance of current MEG-BCI systems is still inadequate and one of the main reasons for this is the unavailability of open-source MEG-BCI datasets. MEG systems are expensive and hence MEG datasets are not readily available for researchers to develop effective and efficient BCI-related signal processing algorithms. In this work, we release a 306-channel MEG-BCI data recorded at 1KHz sampling frequency during four mental imagery tasks (i.e. hand imagery, feet imagery, subtraction imagery, and word generation imagery). The dataset contains two sessions of MEG recordings performed on separate days from 17 healthy participants using a typical BCI imagery paradigm. The current dataset will be the only publicly available MEG imagery BCI dataset as per our knowledge. The dataset can be used by the scientific community towards the development of novel pattern recognition machine learning methods to detect brain activities related to motor imagery and cognitive imagery tasks using MEG signals.

CNN-based classification of fNIRS signals in motor imagery BCI system

Objective. Development of a brain-computer interface (BCI) requires classification of brain neural activities to different states. Functional near-infrared spectroscopy (fNIRS) can measure the brain activities and has great potential for BCI. In recent years, a large number of classification algorithms have been proposed, in which deep learning methods, especially convolutional neural network (CNN) methods are successful. fNIRS signal has typical time series properties, we combined fNIRS data and kinds of CNN-based time series classification (TSC) methods to classify BCI task.Approach. In this study, participants were recruited for a left and right hand motor imagery experiment and the cerebral neural activities were recorded by fNIRS equipment (FOIRE-3000). TSC methods are used to distinguish the brain activities when imagining the left or right hand. We have tested the overall person, single person and overall person with single-channel classification results, and these methods achieved excellent classification results. We also compared the CNN-based TSC methods with traditional classification methods such as support vector machine.Main results. Experiments showed that the CNN-based methods have significant advantages in classification accuracy: the CNN-based methods have achieved remarkable results in the classification of left-handed and right-handed imagination tasks, reaching 98.6% accuracy on overall person, 100% accuracy on single person, and in the single-channel classification an accuracy of 80.1% has been achieved with the best-performing channel.Significance. These results suggest that using the CNN-based TSC methods can significantly improve the BCI performance and also lay the foundation for the miniaturization and portability of training rehabilitation equipment.

Brain-Computer Interfaces in Neurorecovery and Neurorehabilitation

Recent advances in brain-computer interface technology to restore and rehabilitate neurologic function aim to enable persons with disabling neurologic conditions to communicate, interact with the environment, and achieve other key activities of daily living and personal goals. Here we evaluate the principles, benefits, challenges, and future directions of brain-computer interfaces in the context of neurorehabilitation. We then explore the clinical translation of these technologies and propose an approach to facilitate implementation of brain-computer interfaces for persons with neurologic disease.

Subject-Independent Functional Near-Infrared Spectroscopy-Based Brain-Computer Interfaces Based on Convolutional Neural Networks

Functional near-infrared spectroscopy (fNIRS) has attracted increasing attention in the field of brain–computer interfaces (BCIs) owing to their advantages such as non-invasiveness, user safety, affordability, and portability. However, fNIRS signals are highly subject-specific and have low test-retest reliability. Therefore, individual calibration sessions need to be employed before each use of fNIRS-based BCI to achieve a sufficiently high performance for practical BCI applications. In this study, we propose a novel deep convolutional neural network (CNN)-based approach for implementing a subject-independent fNIRS-based BCI. A total of 18 participants performed the fNIRS-based BCI experiments, where the main goal of the experiments was to distinguish a mental arithmetic task from an idle state task. Leave-one-subject-out cross-validation was employed to evaluate the average classification accuracy of the proposed subject-independent fNIRS-based BCI. As a result, the average classification accuracy of the proposed method was reported to be 71.20 ± 8.74%, which was higher than the threshold accuracy for effective BCI communication (70%) as well as that obtained using conventional shrinkage linear discriminant analysis (65.74 ± 7.68%). To achieve a classification accuracy comparable to that of the proposed subject-independent fNIRS-based BCI, 24 training trials (of approximately 12 min) were necessary for the traditional subject-dependent fNIRS-based BCI. It is expected that our CNN-based approach would reduce the necessity of long-term individual calibration sessions, thereby enhancing the practicality of fNIRS-based BCIs significantly.

Progress in Brain Computer Interface: Challenges and Opportunities

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

Decoding Three Different Preference Levels of Consumers Using Convolutional Neural Network: A Functional Near-Infrared Spectroscopy Study

This study decodes consumers' preference levels using a convolutional neural network (CNN) in neuromarketing. The classification accuracy in neuromarketing is a critical factor in evaluating the intentions of the consumers. Functional near-infrared spectroscopy (fNIRS) is utilized as a neuroimaging modality to measure the cerebral hemodynamic responses. In this study, a specific decoding structure, called CNN-based fNIRS-data analysis, was designed to achieve a high classification accuracy. Compared to other methods, the automated characteristics, constant training of the dataset, and learning efficiency of the proposed method are the main advantages. The experimental procedure required eight healthy participants (four female and four male) to view commercial advertisement videos of different durations (15, 30, and 60 s). The cerebral hemodynamic responses of the participants were measured. To compare the preference classification performances, CNN was utilized to extract the most common features, including the mean, peak, variance, kurtosis, and skewness. Considering three video durations, the average classification accuracies of 15, 30, and 60 s videos were 84.3, 87.9, and 86.4%, respectively. Among them, the classification accuracy of 87.9% for 30 s videos was the highest. The average classification accuracies of three preferences in females and males were 86.2 and 86.3%, respectively, showing no difference in each group. By comparing the classification performances in three different combinations (like vs. so-so, like vs. dislike, and so-so vs. dislike) between two groups, male participants were observed to have targeted preferences for commercial advertising, and the classification performance 88.4% between "like" vs. "dislike" out of three categories was the highest. Finally, pairwise classification performance are shown as follows: For female, 86.1% (like vs. so-so), 87.4% (like vs. dislike), 85.2% (so-so vs. dislike), and for male 85.7, 88.4, 85.1%, respectively.

Assessing impact of channel selection on decoding of motor and cognitive imagery from MEG data

OBJECTIVE

Magnetoencephalography (MEG) based brain-computer interface (BCI) involves a large number of sensors allowing better spatiotemporal resolution for assessing brain activity patterns. There have been many efforts to develop BCI using MEG with high accuracy, though an increase in the number of channels (NoC) means an increase in computational complexity. However, not all sensors necessarily contribute significantly to an increase in classification accuracy (CA) and specifically in the case of MEG-based BCI no channel selection methodology has been performed. Therefore, this study investigates the effect of channel selection on the performance of MEG-based BCI.

APPROACH

MEG data were recorded for two sessions from 15 healthy participants performing motor imagery, cognitive imagery and a mixed imagery task pair using a unique paradigm. Performance of four state-of-the-art channel selection methods (i.e. Class-Correlation, ReliefF, Random Forest, and Infinite Latent Feature Selection were applied across six binary tasks in three different frequency bands) were evaluated in this study on two state-of-the-art features, i.e. bandpower and common spatial pattern (CSP).

MAIN RESULTS

All four methods provided a statistically significant increase in CA compared to a baseline method using all gradiometer sensors, i.e. 204 channels with band-power features from alpha (8-12 Hz), beta (13-30 Hz), or broadband (α + β) (8-30 Hz). It is also observed that the alpha frequency band performed better than the beta and broadband frequency bands. The performance of the beta band gave the lowest CA compared with the other two bands. Channel selection improved accuracy irrespective of feature types. Moreover, all the methods reduced the NoC significantly, from 204 to a range of 1-25, using bandpower as a feature and from 15 to 105 for CSP. The optimal channel number also varied not only in each session but also for each participant. Reducing the NoC will help to decrease the computational cost and maintain numerical stability in cases of low trial numbers.

SIGNIFICANCE

The study showed significant improvement in performance of MEG-BCI with channel selection irrespective of feature type and hence can be successfully applied for BCI applications.

Temperament Predictors of Motor Imagery Control in BCI

Abstract. The aim of this study was to verify if selected temperament traits may be useful as predictors of motor imagery brain-computer interface (BCI) performance. In our experiment, 40 BCI-naive subjects were instructed to imagine the movement of clenching his/her right or left hand, in accordance with the visual cue. The activity of sensorimotor rhythms (SMR) (8–30 Hz) was measured by electroencephalography (EEG) and transformed into the information transfer rate (ITR) after feature selection and classification. All subjects also completed a self-assessment questionnaire for the determination of their temperament profile, comprising the following traits: Briskness, Perseveration, Sensory Sensitivity, Emotional Reactivity, Endurance, and Activity. We found significant correlations between ITR performance and Endurance (EN) and Perseveration (PE) scores. This effect was also visible in a topography of SMR desynchronization patterns, in groups with different results in EN and PE scales. Finally, a predictive model of motor imagery BCI control based on temperament traits was proposed. We interpret this finding as empirical support for an influence of basic, relatively stable personality traits on BCI control via the performance of the motor imagery task. Moreover, the implication of these results on the design of future brain-computer interfaces was discussed.

Subject-Specific feature selection for near infrared spectroscopy based brain-computer interfaces

BACKGROUND AND OBJECTIVE

Brain-computer interfaces (BCIs) enable people to control an external device by analyzing the brain's neural activity. Functional near-infrared spectroscopy (fNIRS), which is an emerging optical imaging technique, is frequently used in non-invasive BCIs. Determining the subject-specific features is an important concern in enhancing the classification accuracy as well as reducing the complexity of fNIRS based BCI systems. In this study, the effectiveness of subject-specific feature selection on classification accuracy of fNIRS signals is examined.

METHODS

In order to determine the subject-specific optimal feature subsets, stepwise regression analysis based on sequential feature selection (SWR-SFS) and ReliefF methods were employed. Feature selection is applied on time-domain features of fNIRS signals such as mean, slope, peak, skewness and kurtosis values of signals. Linear discriminant analysis, k nearest neighborhood and support vector machines are employed to evaluate the performance of the selected feature subsets. The proposed techniques are validated on benchmark motor imagery (MI) and mental arithmetic (MA) based fNIRS datasets collected from 29 healthy subjects.

RESULTS

Both SWR-SFS and reliefF feature selection methods have significantly improved the classification accuracy. However, the best results (88.67% (HbR) and 86.43% (HbO) for MA dataset and 77.01% (HbR) and 71.32% (HbO) for MI dataset) were achieved using SWR-SFS while feature selection provided extremely high feature reduction rates (89.50% (HbR) and 93.99% (HbO) for MA dataset and 94.04% (HbR) and 97.73% (HbO) for MI dataset).

CONCLUSIONS

The results of the study indicate that employing feature selection improves both MA and MI-based fNIRS signals classification performance significantly.

Using Transcranial Direct Current Stimulation to Augment the Effect of Motor Imagery-Assisted Brain-Computer Interface Training in Chronic Stroke Patients-Cortical Reorganization Considerations

Introduction: Transcranial direct current stimulation (tDCS) has been shown to modulate cortical plasticity, enhance motor learning and post-stroke upper extremity motor recovery. It has also been demonstrated to facilitate activation of brain-computer interface (BCI) in stroke patients. We had previously demonstrated that BCI-assisted motor imagery (MI-BCI) can improve upper extremity impairment in chronic stroke participants. This study was carried out to investigate the effects of priming with tDCS prior to MI-BCI training in chronic stroke patients with moderate to severe upper extremity paresis and to investigate the cortical activity changes associated with training. Methods: This is a double-blinded randomized clinical trial. Participants were randomized to receive 10 sessions of 20-min 1 mA tDCS or sham-tDCS before MI-BCI, with the anode applied to the ipsilesional, and the cathode to the contralesional primary motor cortex (M1). Upper extremity sub-scale of the Fugl-Meyer Assessment (UE-FM) and corticospinal excitability measured by transcranial magnetic stimulation (TMS) were assessed before, after and 4 weeks after intervention. Results: Ten participants received real tDCS and nine received sham tDCS. UE-FM improved significantly in both groups after intervention. Of those with unrecordable motor evoked potential (MEP-) to the ipsilesional M1, significant improvement in UE-FM was found in the real-tDCS group, but not in the sham group. Resting motor threshold (RMT) of ipsilesional M1 decreased significantly after intervention in the real-tDCS group. Short intra-cortical inhibition (SICI) in the contralesional M1 was reduced significantly following intervention in the sham group. Correlation was found between baseline UE-FM score and changes in the contralesional SICI for all, as well as between changes in UE-FM and changes in contralesional RMT in the MEP- group. Conclusion: MI-BCI improved the motor function of the stroke-affected arm in chronic stroke patients with moderate to severe impairment. tDCS did not confer overall additional benefit although there was a trend toward greater benefit. Cortical activity changes in the contralesional M1 associated with functional improvement suggests a possible role for the contralesional M1 in stroke recovery in more severely affected patients. This has important implications in designing neuromodulatory interventions for future studies and tailoring treatment. Clinical Trial Registration: The study was registered at https://clinicaltrials.gov (NCT01897025).

A comprehensive assessment of Brain Computer Interfaces: Recent trends and challenges

BACKGROUND

An uninterrupted channel of communication and control between the human brain and electronic processing units has led to an increased use of Brain Computer Interfaces (BCIs). This article attempts to present an all-encompassing review on BCI and the scientific advancements associated with it. The ultimate goal of this review is to provide a general overview of the BCI technology and to shed light on different aspects of BCIs. This review also underscores the applications, practical challenges and opportunities associated with BCI technology, which can be used to accelerate future developments in this field.

METHODS

This review is based on a systematic literature search for tracking down the relevant research annals and proceedings. Using a methodical search strategy, the search was carried out across major technical databases. The retrieved records were screened for their relevance and a total of 369 research chronicles were engulfed in this review based on the inclusion criteria.

RESULTS

This review describes the present scenario and recent advancements in BCI technology. It also identifies several application areas of BCI technology. This comprehensive review provides evidence that, while we are getting ever closer, significant challenges still exist for the development of BCIs that can seamlessly integrate with the user's biological system.

CONCLUSION

The findings of this review confirm the importance of BCI technology in various applications. It is concluded that BCI technology, still in its sprouting phase, requires significant explorations for further development.

Enhanced Performance of a Brain Switch by Simultaneous Use of EEG and NIRS Data for Asynchronous Brain-Computer Interface

Previous studies have shown the superior performance of hybrid electroencephalography (EEG)/ near-infrared spectroscopy (NIRS) brain-computer interfaces (BCIs). However, it has been veiled whether the use of a hybrid EEG/NIRS modality can provide better performance for a brain switch that can detect the onset of the intention to turn on a BCI. In this study, we developed such a hybrid EEG/NIRS brain switch and compared its performance with single modality EEG- and NIRS-based brain switch respectively, in terms of true positive rate (TPR), false positive rate (FPR), onset detection time (ODT), and information transfer rate (ITR). In an offline analysis, the performance of a hybrid EEG/NIRS brain switch was significantly improved over that of EEG- and NIRS-based brain switches in general, and in particular a significantly lower FPR was observed for the hybrid EEG/NIRS brain switch. A pseudo-online analysis was additionally performed to confirm the feasibility of implementing an online BCI system with our hybrid EEG/NIRS brain switch. The overall trend of pseudo-online analysis results generally coincided with that of the offline analysis results. No significant difference in all performance measures was also found between offline and pseudo online analysis schemes when the amount of training data was same, with one exception for the ITRs of an EEG brain switch. These offline and pseudo-online results demonstrate that a hybrid EEG/NIRS brain switch can be used to provide a better onset detection performance than that of a single neuroimaging modality.

Error Correction Regression Framework for Enhancing the Decoding Accuracies of Ear-EEG Brain-Computer Interfaces

Ear-electroencephalography (EEG) is a promising tool for practical brain-computer interface (BCI) applications because it is more unobtrusive, comfortable, and mobile than a typical scalp-EEG system. However, an ear-EEG has a natural constraint of electrode location (e.g., limited in or around the ear) for acquiring informative brain signals sufficiently. Achieving reliable performance of ear-EEG in specific BCI paradigms that do not utilize brain signals on the temporal lobe around the ear is difficult. For example, steady-state visual evoked potentials (SSVEPs), which are mainly generated in the occipital area, have a significantly attenuated and distorted amplitude in ear-EEG. Therefore, preserving the high level of decoding accuracy is challenging and essential for SSVEP BCI based on ear-EEG. In this paper, we first investigate linear and nonlinear regression methods to increase the decoding accuracy of ear-EEG regarding SSVEP paradigm by utilizing the estimated target EEG signals on the occipital area. Then, we investigate an ensemble method to consider the prediction variability of the regression methods. Finally, we propose an error correction regression (ECR) framework to reduce the prediction errors by adding an additional nonlinear regression process (i.e., kernel ridge regression). We evaluate the ECR framework in terms of single session, session-to-session transfer, and subject-transfer decoding. We also validate the online decoding ability of the proposed framework with a short-time window size. The average accuracies are observed to be 91.11±9.14%, 90.52±8.67%, 86.96±12.13%, and 78.79±12.59%. This paper demonstrates that SSVEP BCI based on ear-EEG can achieve reliable performance with the proposed ECR framework.

Brain-Computer Interface-Based Humanoid Control: A Review

A Brain-Computer Interface (BCI) acts as a communication mechanism using brain signals to control external devices. The generation of such signals is sometimes independent of the nervous system, such as in Passive BCI. This is majorly beneficial for those who have severe motor disabilities. Traditional BCI systems have been dependent only on brain signals recorded using Electroencephalography (EEG) and have used a rule-based translation algorithm to generate control commands. However, the recent use of multi-sensor data fusion and machine learning-based translation algorithms has improved the accuracy of such systems. This paper discusses various BCI applications such as tele-presence, grasping of objects, navigation, etc. that use multi-sensor fusion and machine learning to control a humanoid robot to perform a desired task. The paper also includes a review of the methods and system design used in the discussed applications.

Brain-Based Binary Communication Using Spatiotemporal Features of fNIRS Responses

“Locked-in” patients lose their ability to communicate naturally due to motor system dysfunction. Brain-computer interfacing offers a solution for their inability to communicate by enabling motor-independent communication. Straightforward and convenient in-session communication is essential in clinical environments. The present study introduces a functional near-infrared spectroscopy (fNIRS)-based binary communication paradigm that requires limited preparation time and merely nine optodes. Eighteen healthy participants performed two mental imagery tasks, mental drawing and spatial navigation, to answer yes/no questions during one of two auditorily cued time windows. Each of the six questions was answered five times, resulting in five trials per answer. This communication paradigm thus combines both spatial (two different mental imagery tasks, here mental drawing for “yes” and spatial navigation for “no”) and temporal (distinct time windows for encoding a “yes” and “no” answer) fNIRS signal features for information encoding. Participants’ answers were decoded in simulated real-time using general linear model analysis. Joint analysis of all five encoding trials resulted in an average accuracy of 66.67 and 58.33% using the oxygenated (HbO) and deoxygenated (HbR) hemoglobin signal respectively. For half of the participants, an accuracy of 83.33% or higher was reached using either the HbO signal or the HbR signal. For four participants, effective communication with 100% accuracy was achieved using either the HbO or HbR signal. An explorative analysis investigated the differentiability of the two mental tasks based solely on spatial fNIRS signal features. Using multivariate pattern analysis (MVPA) group single-trial accuracies of 58.33% (using 20 training trials per task) and 60.56% (using 40 training trials per task) could be obtained. Combining the five trials per run using a majority voting approach heightened these MVPA accuracies to 62.04 and 75%. Additionally, an fNIRS suitability questionnaire capturing participants’ physical features was administered to explore its predictive value for evaluating general data quality. Obtained questionnaire scores correlated significantly (r = -0.499) with the signal-to-noise of the raw light intensities. While more work is needed to further increase decoding accuracy, this study shows the potential of answer encoding using spatiotemporal fNIRS signal features or spatial fNIRS signal features only.

Effects of MEG-based neurofeedback for hand rehabilitation after tetraplegia: preliminary findings in cortical modulations and grip strength

OBJECTIVE

Neurofeedback (NF) trains people to volitionally modulate their cortical activity to affect a behavioral outcome. We evaluated the feasibility of using NF to improve hand function after chronic cervical-level spinal cord injury (SCI) using biologically-relevant visual feedback of motor-related brain activity and an intuitive control scheme.

APPROACH

The NF system acquired magnetoencephalography (MEG) data in real-time to provide feedback of event-related desynchronization (ERD) measured over the sensorimotor cortex during attempted hand grasping. During brain control, stronger ERD resulting from attempted grasping drove the virtual hand towards a more closed grasp, while less ERD drove the hand more open.

MAIN RESULTS

Eight individuals with partial or complete hand impairment due to chronic SCI controlled the NF to perform a grasping task that increased in difficulty as the participants achieved success. During their first NF session, participants achieved an average success rate of 63.7 ± 6.4% (chance level of 13.9%). After as few as one intervention session, four of the seven individuals evaluated for ERD changes had significantly strengthened ERD and three of the four participants with measurable grip strength prior to NF had increased grip strength. Interestingly, both individuals who participated in a longer-term study (i.e. >8 NF sessions) had improved grip strength and significantly strengthened ERD.

SIGNIFICANCE

This study demonstrates that MEG-based NF training can change brain activity in individuals with hand impairment due to SCI and has the potential to induce acute changes in grip strength. Future studies will evaluate whether neuroplasticity induced with long term NF can improve hand function for those with moderate impairment.

Indoor Simulated Training Environment for Brain-Controlled Wheelchair Based on Steady-State Visual Evoked Potentials

Brain-controlled wheelchair (BCW) has the potential to improve the quality of life for people with motor disabilities. A lot of training is necessary for users to learn and improve BCW control ability and the performances of BCW control are crucial for patients in daily use. In consideration of safety and efficiency, an indoor simulated training environment is built up in this paper to improve the performance of BCW control. The indoor simulated environment mainly realizes BCW implementation, simulated training scenario setup, path planning and recommendation, simulated operation, and scoring. And the BCW is based on steady-state visual evoked potentials (SSVEP) and the filter bank canonical correlation analysis (FBCCA) is used to analyze the electroencephalography (EEG). Five tasks include individual accuracy, simple linear path, obstacles avoidance, comprehensive steering scenarios, and evaluation task are designed, 10 healthy subjects were recruited and carried out the 7-days training experiment to assess the performance of the training environment. Scoring and command-consuming are conducted to evaluate the improvement before and after training. The results indicate that the average accuracy is 93.55% and improves from 91.05% in the first stage to 96.05% in the second stage (p = 0.001). Meanwhile, the average score increases from 79.88 in the first session to 96.66 in the last session and tend to be stable (p < 0.001). The average number of commands and collisions to complete the tasks decreases significantly with or without the approximate shortest path (p < 0.001). These results show that the performance of subjects in BCW control achieves improvement and verify the feasibility and effectiveness of the proposed simulated training environment.

Development of a Brain-Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography

Asynchronous brain-computer interfaces (BCIs) based on electroencephalography (EEG) generally suffer from poor performance in terms of classification accuracy and false-positive rate (FPR). Thus, BCI toggle switches based on electrooculogram (EOG) signals were developed to toggle on/off synchronous BCI systems. The conventional BCI toggle switches exhibit fast responses with high accuracy; however, they have a high FPR or cannot be applied to patients with oculomotor impairments. To circumvent these issues, we developed a novel BCI toggle switch that users can employ to toggle on or off synchronous BCIs by holding their breath for a few seconds. Two states-normal breath and breath holding-were classified using a linear discriminant analysis with features extracted from the respiration-modulated photoplethysmography (PPG) signals. A real-time BCI toggle switch was implemented with calibration data trained with only 1-min PPG data. We evaluated the performance of our PPG switch by combining it with a steady-state visual evoked potential-based BCI system that was designed to control four external devices, with regard to the true-positive rate and FPR. The parameters of the PPG switch were optimized through an offline experiment with five subjects, and the performance of the switch system was evaluated in an online experiment with seven subjects. All the participants successfully turned on the BCI by holding their breath for approximately 10 s (100% accuracy), and the switch system exhibited a very low FPR of 0.02 false operations per minute, which is the lowest FPR reported thus far. All participants could successfully control external devices in the synchronous BCI mode. Our results demonstrated that the proposed PPG-based BCI toggle switch can be used to implement practical BCIs.

Brain-computer interfaces for communication

Locked-in syndrome (LIS) is characterized by an inability to move or speak in the presence of intact cognition and can be caused by brainstem trauma or neuromuscular disease. Quality of life (QoL) in LIS is strongly impaired by the inability to communicate, which cannot always be remedied by traditional augmentative and alternative communication (AAC) solutions if residual muscle activity is insufficient to control the AAC device. Brain-computer interfaces (BCIs) may offer a solution by employing the person's neural signals instead of relying on muscle activity. Here, we review the latest communication BCI research using noninvasive signal acquisition approaches (electroencephalography, functional magnetic resonance imaging, functional near-infrared spectroscopy) and subdural and intracortical implanted electrodes, and we discuss current efforts to translate research knowledge into usable BCI-enabled communication solutions that aim to improve the QoL of individuals with LIS.