Endogenous Sensory Discrimination and Selection by a Fast Brain Switch for a High Transfer Rate Brain-Computer Interface

Considerations and discussions on the clear definition and definite scope of brain-computer interfaces

Brain-computer interface (BCI) is a revolutionizing human-computer interaction with potential applications in both medical and non-medical fields, emerging as a cutting-edge and trending research direction. Increasing numbers of groups are engaging in BCI research and development. However, in recent years, there has been some confusion regarding BCI, including misleading and hyped propaganda about BCI, and even non-BCI technologies being labeled as BCI. Therefore, a clear definition and a definite scope for BCI are thoroughly considered and discussed in the paper, based on the existing definitions of BCI, including the six key or essential components of BCI. In the review, different from previous definitions of BCI, BCI paradigms and neural coding are explicitly included in the clear definition of BCI provided, and the BCI user (the brain) is clearly identified as a key component of the BCI system. Different people may have different viewpoints on the definition and scope of BCI, as well as some related issues, which are discussed in the article. This review argues that a clear definition and definite scope of BCI will benefit future research and commercial applications. It is hoped that this review will reduce some of the confusion surrounding BCI and promote sustainable development in this field.

A model-based brain switch via periodic motor imagery modulation for asynchronous brain-computer interfaces

Objective.Brain switches provide a tangible solution to asynchronized brain-computer interface, which decodes user intention without a pre-programmed structure. However, most brain switches based on electroencephalography signals have high false positive rates (FPRs), resulting in less practicality. This research aims to improve the operating mode and usability of the brain switch.Approach.Here, we propose a novel virtual physical model-based brain switch that leverages periodic active modulation. An optimization problem of minimizing the triggering time subject to a required FPR is formulated, numerical and analytical approximate solutions are obtained based on the model.Main results.Our motor imagery (MI)-based brain switch can reach 0.8FP/h FPR with a median triggering time of 58 s. We evaluated the proposed brain switch during online device control, and their average FPRs substantially outperformed the conventional brain switches in the literature. We further improved the proposed brain switch with the Common Spatial Pattern (CSP) and optimization method. An average FPR of 0.3 FPs/h was obtained for the MI-CSP-based brain switch, and the average triggering time improved to 21.6 s.Significance.This study provides a new approach that could significantly reduce the brain switch's FPR to less than 1 Fps/h, which was less than 10% of the FPR (decreasing by more than a magnitude of order) by other endogenous methods, and the reaction time was comparable to the state-of-the-art approaches. This represents a significant advancement over the current non-invasive asynchronous BCI and will open widespread avenues for translating BCI towards clinical applications.

A Systematic Review of Virtual Reality and Robot Therapy as Recent Rehabilitation Technologies Using EEG-Brain-Computer Interface Based on Movement-Related Cortical Potentials

To enhance the treatment of motor function impairment, patients' brain signals for self-control as an external tool may be an extraordinarily hopeful option. For the past 10 years, researchers and clinicians in the brain-computer interface (BCI) field have been using movement-related cortical potential (MRCP) as a control signal in neurorehabilitation applications to induce plasticity by monitoring the intention of action and feedback. Here, we reviewed the research on robot therapy (RT) and virtual reality (VR)-MRCP-based BCI rehabilitation technologies as recent advancements in human healthcare. A list of 18 full-text studies suitable for qualitative review out of 322 articles published between 2000 and 2022 was identified based on inclusion and exclusion criteria. We used PRISMA guidelines for the systematic review, while the PEDro scale was used for quality evaluation. Bibliometric analysis was conducted using the VOSviewer software to identify the relationship and trends of key items. In this review, 4 studies used VR-MRCP, while 14 used RT-MRCP-based BCI neurorehabilitation approaches. The total number of subjects in all identified studies was 107, whereby 4.375 ± 6.3627 were patient subjects and 6.5455 ± 3.0855 were healthy subjects. The type of electrodes, the epoch, classifiers, and the performance information that are being used in the RT- and VR-MRCP-based BCI rehabilitation application are provided in this review. Furthermore, this review also describes the challenges facing this field, solutions, and future directions of these smart human health rehabilitation technologies. By key items relationship and trends analysis, we found that motor control, rehabilitation, and upper limb are important key items in the MRCP-based BCI field. Despite the potential of these rehabilitation technologies, there is a great scarcity of literature related to RT and VR-MRCP-based BCI. However, the information on these rehabilitation methods can be beneficial in developing RT and VR-MRCP-based BCI rehabilitation devices to induce brain plasticity and restore motor impairment. Therefore, this review will provide the basis and references of the MRCP-based BCI used in rehabilitation applications for further clinical and research development.

Deep Convolutional Neural Network for EEG-Based Motor Decoding

Brain-machine interfaces (BMIs) have been applied as a pattern recognition system for neuromodulation and neurorehabilitation. Decoding brain signals (e.g., EEG) with high accuracy is a prerequisite to building a reliable and practical BMI. This study presents a deep convolutional neural network (CNN) for EEG-based motor decoding. Both upper-limb and lower-limb motor imagery were detected from this end-to-end learning with four datasets. An average classification accuracy of 93.36 ± 1.68% was yielded on the four datasets. We compared the proposed approach with two other models, i.e., multilayer perceptron and the state-of-the-art framework with common spatial patterns and support vector machine. We observed that the performance of the CNN-based framework was significantly better than the other two models. Feature visualization was further conducted to evaluate the discriminative channels employed for the decoding. We showed the feasibility of the proposed architecture to decode motor imagery from raw EEG data without manually designed features. With the advances in the fields of computer vision and speech recognition, deep learning can not only boost the EEG decoding performance but also help us gain more insight from the data, which may further broaden the knowledge of neuroscience for brain mapping.

Time-Distributed Attention Network for EEG-based Motor Imagery Decoding from the Same Limb

A brain-computer interface (BCI) based on motor imagery (MI) from the same limb can provide an intuitive control pathway but has received limited attention. It is still a challenge to classify multiple MI tasks from the same limb. The goal of this study is to propose a novel decoding method to classify the MI tasks of four joints of the same upper limb and the resting state. EEG signals were collected from 20 participants. A time-distributed attention network (TD-Atten) was proposed to adaptively assign different weights to different classes and frequency bands of the input multiband Common Spatial Pattern (CSP) features. The long short-term memory (LSTM) and dense layers were then used to learn sequential information from the reweight features and perform the classification. Our proposed method outperformed other baseline and deep learning-based methods and obtained the accuracies of 46.8% in the 5-class scenario and 53.4% in the 4-class scenario. The visualization results of attention weights indicated that the proposed framework can adaptively pay attention to alpha-band related features in MI tasks, which was consistent with the analysis of brain activation patterns. These results demonstrated the feasibility and interpretability of the attention mechanism in MI decoding and the potential of this fine MI paradigm to be applied for the control of a robotic arm or a neural prosthesis.

Participant-specific classifier tuning increases the performance of hand movement detection from EEG in patients with amyotrophic lateral sclerosis

Objective.Brain-computer interface (BCI) systems can be employed to provide motor and communication assistance to patients suffering from neuromuscular diseases, such as amyotrophic lateral sclerosis (ALS). Movement related cortical potentials (MRCPs), which are naturally generated during movement execution, can be used to implement a BCI triggered by motor attempts. Such BCI could assist impaired motor functions of ALS patients during disease progression, and facilitate the training for the generation of reliable MRCPs. The training aspect is relevant to establish a communication channel in the late stage of the disease. Therefore, the aim of this study was to investigate the possibility of detecting MRCPs associated to movement intention in ALS patients with different levels of disease progression from slight to complete paralysis.Approach.Electroencephalography signals were recorded from nine channels in 30 ALS patients at various stages of the disease while they performed or attempted to perform hand movements timed to a visual cue. The movement detection was implemented using offline classification between movement and rest phase. Temporal and spectral features were extracted using 500 ms sliding windows with 50% overlap. The detection was tested for each individual channel and two surrogate channels by performing feature selection followed by classification using linear and non-linear support vector machine and linear discriminant analysis.Main results.The results demonstrated that the detection performance was high in all patients (accuracy 80.5 ± 5.6%) but that the classification parameters (channel, features and classifier) leading to the best performance varied greatly across patients. When the same channel and classifier were used for all patients (participant-generic analysis), the performance significantly decreased (accuracy 74 ± 8.3%).Significance.The present study demonstrates that to maximize the detection of brain waves across ALS patients at different stages of the disease, the classification pipeline should be tuned to each patient individually.

Online control of an assistive active glove by slow cortical signals in patients with amyotrophic lateral sclerosis

Objective.A brain-computer interface (BCI) allows users to control external devices using brain signals that can be recorded non-invasively via electroencephalography (EEG). Movement related cortical potentials (MRCPs) are an attractive option for BCI control since they arise naturally during movement execution and imagination, and therefore, do not require an extensive training. This study tested the feasibility of online detection of reaching and grasping using MRCPs for the application in patients suffering from amyotrophic lateral sclerosis (ALS).Approach.A BCI system was developed to trigger closing of a soft assistive glove by detecting a reaching movement. The custom-made software application included data collection, a novel method for collecting the input data for classifier training from the offline recordings based on a sliding window approach, and online control of the glove. Eight healthy subjects and two ALS patients were recruited to test the developed BCI system. They performed assessment blocks without the glove active (NG), in which the movement detection was indicated by a sound feedback, and blocks (G) in which the glove was controlled by the BCI system. The true positive rate (TPR) and the positive predictive value (PPV) were adopted as the outcome measures. Correlation analysis between forehead EEG detecting ocular artifacts and sensorimotor area EEG was conducted to confirm the validity of the results.Main results.The overall median TPR and PPV were >0.75 for online BCI control, in both healthy individuals and patients, with no significant difference across the blocks (NG versus G).Significance.The results demonstrate that cortical activity during reaching can be detected and used to control an external system with a limited amount of training data (30 trials). The developed BCI system can be used to provide grasping assistance to ALS patients.

Learning Common Time-Frequency-Spatial Patterns for Motor Imagery Classification

The common spatial patterns (CSP) algorithm is the most popular spatial filtering method applied to extract electroencephalogram (EEG) features for motor imagery (MI) based brain-computer interface (BCI) systems. The effectiveness of the CSP algorithm depends on optimal selection of the frequency band and time window from the EEG. Many algorithms have been designed to optimize frequency band selection for CSP, while few algorithms seek to optimize the time window. This study proposes a novel framework, termed common time-frequency-spatial patterns (CTFSP), to extract sparse CSP features from multi-band filtered EEG data in multiple time windows. Specifically, the whole MI period is first segmented into multiple subseries using a sliding time window approach. Then, sparse CSP features are extracted from multiple frequency bands in each time window. Finally, multiple support vector machine (SVM) classifiers with the Radial Basis Function (RBF) kernel are trained to identify the MI tasks and the voting result of these classifiers determines the final output of the BCI. This study applies the proposed CTFSP algorithm to three public EEG datasets (BCI competition III dataset IVa, BCI competition III dataset IIIa, and BCI competition IV dataset 1) to validate its effectiveness, compared against several other state-of-the-art methods. The experimental results demonstrate that the proposed algorithm is a promising candidate for improving the performance of MI-BCI systems.

Enhanced Performance of a Brain Switch by Simultaneous Use of EEG and NIRS Data for Asynchronous Brain-Computer Interface

Previous studies have shown the superior performance of hybrid electroencephalography (EEG)/ near-infrared spectroscopy (NIRS) brain-computer interfaces (BCIs). However, it has been veiled whether the use of a hybrid EEG/NIRS modality can provide better performance for a brain switch that can detect the onset of the intention to turn on a BCI. In this study, we developed such a hybrid EEG/NIRS brain switch and compared its performance with single modality EEG- and NIRS-based brain switch respectively, in terms of true positive rate (TPR), false positive rate (FPR), onset detection time (ODT), and information transfer rate (ITR). In an offline analysis, the performance of a hybrid EEG/NIRS brain switch was significantly improved over that of EEG- and NIRS-based brain switches in general, and in particular a significantly lower FPR was observed for the hybrid EEG/NIRS brain switch. A pseudo-online analysis was additionally performed to confirm the feasibility of implementing an online BCI system with our hybrid EEG/NIRS brain switch. The overall trend of pseudo-online analysis results generally coincided with that of the offline analysis results. No significant difference in all performance measures was also found between offline and pseudo online analysis schemes when the amount of training data was same, with one exception for the ITRs of an EEG brain switch. These offline and pseudo-online results demonstrate that a hybrid EEG/NIRS brain switch can be used to provide a better onset detection performance than that of a single neuroimaging modality.

Asynchronous Prediction of Human Gait Intention in a Pseudo Online Paradigm Using Wavelet Transform

Prediction of human voluntary gait intention is a very significant task to ensure direct cortical control of real-life assistive technologies for locomotion rehabilitation. Neurophysiological studies provide that human voluntary gait intention is represented by slow DC potentials and power shifts in specific frequency ranges of brain wave, which can be detected 1.5- 2 seconds before the actual onset. The goal of this study was to determine whether it is possible to reliably detect the intention of voluntary gait 'starting' and 'stopping' intention before it takes place. A computational algorithm was designed to implement asynchronous prediction of gait intention in an offline and pseudo-online environment using support vector machine. Six healthy subjects participated in the study and performed self- paced voluntary gait cycles. A combination of advanced wavelet transform algorithms resulted in 88.23± 1.59% accuracy, 85.42± 4.03% sensitivity and 90.24± 2.78% specificity for intention of start detection and 87.04± 1.72% accuracy, 82.69± 4.13% sensitivity and 89.59± 3.04% specificity for intention to stop walking in offline testing. Additionally, the wavelet transform methods accompanied with threshold regulation and majority voting algorithm resulted in a True Positive Rate of 85.5± 5.0% and 81.2± 3.3% for 'start' and 'stop' prediction with 6.8± 0.7 and 9.4± 1.0 False Positives per Minute respectively in pseudo online testing. The average detection latencies were -1002 ± 603 ms and -943 ± 603 ms, respectively, for 'start' and 'stop' prediction. The study provides promising outcomes in terms of TPR, FP/min, and detection latency, which suggests that human voluntary gait intention can be predicted before the onset of movement.

Brain-Switches for Asynchronous Brain–Computer Interfaces: A Systematic Review

A brain–computer interface (BCI) has been extensively studied to develop a novel communication system for disabled people using their brain activities. An asynchronous BCI system is more realistic and practical than a synchronous BCI system, in that, BCI commands can be generated whenever the user wants. However, the relatively low performance of an asynchronous BCI system is problematic because redundant BCI commands are required to correct false-positive operations. To significantly reduce the number of false-positive operations of an asynchronous BCI system, a two-step approach has been proposed using a brain-switch that first determines whether the user wants to use an asynchronous BCI system before the operation of the asynchronous BCI system. This study presents a systematic review of the state-of-the-art brain-switch techniques and future research directions. To this end, we reviewed brain-switch research articles published from 2000 to 2019 in terms of their (a) neuroimaging modality, (b) paradigm, (c) operation algorithm, and (d) performance.

A Fast Brain Switch Based on Multi-Class Code-Modulated VEPs

To realize asynchronous control of a brain-computer interface (BCI) system, a fast brain switch with low false positive rate (FPR) is required. This paper proposed a brain switch based on code-modulated visual-evoked potential (c-VEP), in which seven 8-bit pseudorandom codes were used to modulate the electroencephalogram (EEG) signal. This study optimized and demonstrated the control strategy through an offline and an online experiments. By decoding the brain state continuously with the task-related component analysis (TRCA) algorithm, the brain switch achieved an average reaction time (RT) of 1.72 seconds and an average idle time of 183.53 seconds without false positive events in the online experiment.

Development of a Brain-Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography

Asynchronous brain-computer interfaces (BCIs) based on electroencephalography (EEG) generally suffer from poor performance in terms of classification accuracy and false-positive rate (FPR). Thus, BCI toggle switches based on electrooculogram (EOG) signals were developed to toggle on/off synchronous BCI systems. The conventional BCI toggle switches exhibit fast responses with high accuracy; however, they have a high FPR or cannot be applied to patients with oculomotor impairments. To circumvent these issues, we developed a novel BCI toggle switch that users can employ to toggle on or off synchronous BCIs by holding their breath for a few seconds. Two states-normal breath and breath holding-were classified using a linear discriminant analysis with features extracted from the respiration-modulated photoplethysmography (PPG) signals. A real-time BCI toggle switch was implemented with calibration data trained with only 1-min PPG data. We evaluated the performance of our PPG switch by combining it with a steady-state visual evoked potential-based BCI system that was designed to control four external devices, with regard to the true-positive rate and FPR. The parameters of the PPG switch were optimized through an offline experiment with five subjects, and the performance of the switch system was evaluated in an online experiment with seven subjects. All the participants successfully turned on the BCI by holding their breath for approximately 10 s (100% accuracy), and the switch system exhibited a very low FPR of 0.02 false operations per minute, which is the lowest FPR reported thus far. All participants could successfully control external devices in the synchronous BCI mode. Our results demonstrated that the proposed PPG-based BCI toggle switch can be used to implement practical BCIs.

Continuous 2D control via state-machine triggered by endogenous sensory discrimination and a fast brain switch

OBJECTIVE

Brain computer interfacing (BCI) is a promising method to control assistive systems for patients with severe disabilities. Recently, we have presented a novel BCI approach that combines an electrotactile menu and a brain switch, which allows the user to trigger many commands robustly and efficiently. However, the commands are timed to periodic tactile cues and this may challenge online control. In the present study, therefore, we implemented and evaluated a novel approach for online closed-loop control using the proposed BCI.

APPROACH

Eleven healthy subjects used the novel method to move a cursor in a 2D space. To assure robust control with properly timed commands, the BCI was integrated within a state machine allowing the subject to start the cursor movement in the selected direction and asynchronously stop the cursor. The brain switch was controlled using motor execution (ME) or imagery (MI) and the menu implemented four (straight movements) or eight commands (straight and diagonal movements).

MAIN RESULTS

The results showed a high completion rate of a target hitting task (~97% and ~92% for ME and MI, respectively), with a small number of collisions, when four-channel control was used. There was no significant difference in outcome measures between MI and ME, and performance was similar for four and eight commands.

SIGNIFICANCE

These results demonstrate that the novel state-based scheme driven by a robust BCI can be successfully utilized for online control. Therefore, it can be an attractive solution for providing the user an online-control interface with many commands, which is difficult to achieve using classic BCI solutions.

An EEG Experimental Study Evaluating the Performance of Texas Instruments ADS1299

Texas Instruments ADS1299 is an attractive choice for low cost electroencephalography (EEG) devices owing to its low power consumption and low input referred noise. To date, there have been no rigorous evaluations of its performance. In this EEG experimental study we evaluated the performance of the ADS1299 against a high quality laboratory-based system. Two self-paced lower limb motor tasks were performed by 22 healthy participants. Recorded power across delta, theta, alpha, and beta EEG bands, the power ratio across the motor tasks, pre-movement noise, and signal-to-noise ratio were obtained for evaluation. The amplitude and time of the negative peak in the movement-related cortical potentials (MRCPs) extracted from the EEG data were also obtained. Using linear mixed models, no statistically significant differences (p > 0.05) were found in any of these measures across the two systems. These findings were further supported by evaluation of cosine similarity, waveform differences, and topographic maps. There were statistically significant differences in MRCPs across the motor tasks in both systems. We conclude that the performance of the ADS1299 in combination with wet Ag/AgCl electrodes is analogous to that of a laboratory-based system in a low frequency (<40 Hz) EEG recording.

EEG-Based Lower-Limb Movement Onset Decoding: Continuous Classification and Asynchronous Detection

Brain-machine interfaces have been used to incorporate the user intention to trigger robotic devices by decoding movement onset from electroencephalography. Active neural participation is crucial to promote brain plasticity thus to enhance the opportunity of motor recovery. This paper presents the decoding of lower-limb movement-related cortical potentials with continuous classification and asynchronous detection. We executed experiments in a customized gait trainer, where 10 healthy subjects performed self-initiated ankle plantar flexion. We further analyzed the features, evaluated the impact of the limb side, and compared the proposed framework with other typical decoding methods. No significant differences were observed between the left and right legs in terms of neural signatures of movement and classification performance. We obtained a higher true positive rate, lower false positives, and comparable latencies with respect to the existing online detection methods. This paper demonstrates the feasibility of the proposed framework to build a closed-loop gait trainer. Potential applications include gait training neurorehabilitation in clinical trials.