Brain computer interface: control signals review

Analysis and recognition of a novel experimental paradigm for musical emotion brain-computer interface

Musical emotions have received increasing attention over the years. To better recognize the emotions by brain-computer interface (BCI), the random music-playing and sequential music-playing experimental paradigms are proposed and compared in this paper. Two experimental paradigms consist of three positive pieces, three neutral pieces and three negative pieces of music. Ten subjects participate in two experimental paradigms. The features of electroencephalography (EEG) signals are firstly analyzed in the time, frequency and spatial domains. To improve the effect of emotion recognition, a recognition model is proposed with the optimal channels selecting by Pearson's correlation coefficient, and the feature fusion combining differential entropy and wavelet packet energy. According to the analysis results, the features of sequential music-playing experimental paradigm are more different among three emotions. The classification results of sequential music-playing experimental paradigm are also better, and its average results of positive, neutral and negative emotions are 78.53%, 72.81% and 77.35%, respectively. The more obvious the changes of EEG induced by the emotions, the higher the classification accuracy will be. After analyzing two experimental paradigms, a better way for music to induce the emotions can be explored. Therefore, our research offers a novel perspective on affective BCIs.

Application of artificial hibernation technology in acute brain injury

Controlling intracranial pressure, nerve cell regeneration, and microenvironment regulation are the key issues in reducing mortality and disability in acute brain injury. There is currently a lack of effective treatment methods. Hibernation has the characteristics of low temperature, low metabolism, and hibernation rhythm, as well as protective effects on the nervous, cardiovascular, and motor systems. Artificial hibernation technology is a new technology that can effectively treat acute brain injury by altering the body's metabolism, lowering the body's core temperature, and allowing the body to enter a state similar to hibernation. This review introduces artificial hibernation technology, including mild hypothermia treatment technology, central nervous system regulation technology, and artificial hibernation-inducer technology. Upon summarizing the relevant research on artificial hibernation technology in acute brain injury, the research results show that artificial hibernation technology has neuroprotective, anti-inflammatory, and oxidative stress-resistance effects, indicating that it has therapeutic significance in acute brain injury. Furthermore, artificial hibernation technology can alleviate the damage of ischemic stroke, traumatic brain injury, cerebral hemorrhage, cerebral infarction, and other diseases, providing new strategies for treating acute brain injury. However, artificial hibernation technology is currently in its infancy and has some complications, such as electrolyte imbalance and coagulation disorders, which limit its use. Further research is needed for its clinical application.

A Dynamic Window Method Based on Reinforcement Learning for SSVEP Recognition

Steady-state visual evoked potential (SSVEP) is one of the most used brain-computer interface (BCI) paradigms. Conventional methods analyze SSVEPs at a fixed window length. Compared with these methods, dynamic window methods can achieve a higher information transfer rate (ITR) by selecting an appropriate window length. These methods dynamically evaluate the credibility of the result by linear discriminant analysis (LDA) or Bayesian estimation and extend the window length until credible results are obtained. However, the hypotheses introduced by LDA and Bayesian estimation may not align with the collected real-world SSVEPs, which leads to an inappropriate window length. To address the issue, we propose a novel dynamic window method based on reinforcement learning (RL). The proposed method optimizes the decision of whether to extend the window length based on the impact of decisions on the ITR, without additional hypotheses. The decision model can automatically learn a strategy that maximizes the ITR through trial and error. In addition, compared with traditional methods that manually extract features, the proposed method uses neural networks to automatically extract features for the dynamic selection of window length. Therefore, the proposed method can more accurately decide whether to extend the window length and select an appropriate window length. To verify the performance, we compared the novel method with other dynamic window methods on two public SSVEP datasets. The experimental results demonstrate that the novel method achieves the highest performance by using RL.

Bridging Minds and Machines: The Recent Advances of Brain-Computer Interfaces in Neurological and Neurosurgical Applications

Brain-computer interfaces (BCIs), a remarkable technological advancement in neurology and neurosurgery, mark a significant leap since the inception of electroencephalography in 1924. These interfaces effectively convert central nervous system signals into commands for external devices, offering revolutionary benefits to patients with severe communication and motor impairments due to a myriad of neurological conditions like stroke, spinal cord injuries, and neurodegenerative disorders. BCIs enable these individuals to communicate and interact with their environment, using their brain signals to operate interfaces for communication and environmental control. This technology is especially crucial for those completely locked in, providing a communication lifeline where other methods fall short. The advantages of BCIs are profound, offering autonomy and an improved quality of life for patients with severe disabilities. They allow for direct interaction with various devices and prostheses, bypassing damaged or nonfunctional neural pathways. However, challenges persist, including the complexity of accurately interpreting brain signals, the need for individual calibration, and ensuring reliable, long-term use. Additionally, ethical considerations arise regarding autonomy, consent, and the potential for dependence on technology. Despite these challenges, BCIs represent a transformative development in neurotechnology, promising enhanced patient outcomes and a deeper understanding of brain-machine interfaces.

Magnetoencephalography Decoding Transfer Approach: From Deep Learning Models to Intrinsically Interpretable Models

When decoding neuroelectrophysiological signals represented by Magnetoencephalography (MEG), deep learning models generally achieve high predictive performance but lack the ability to interpret their predicted results. This limitation prevents them from meeting the essential requirements of reliability and ethical-legal considerations in practical applications. In contrast, intrinsically interpretable models, such as decision trees, possess self-evident interpretability while typically sacrificing accuracy. To effectively combine the respective advantages of both deep learning and intrinsically interpretable models, an MEG transfer approach through feature attribution-based knowledge distillation is pioneered, which transforms deep models (teacher) into highly accurate intrinsically interpretable models (student). The resulting models provide not only intrinsic interpretability but also high predictive performance, besides serving as an excellent approximate proxy to understand the inner workings of deep models. In the proposed approach, post-hoc feature knowledge derived from post-hoc interpretable algorithms, specifically feature attribution maps, is introduced into knowledge distillation for the first time. By guiding intrinsically interpretable models to assimilate this knowledge, the transfer of MEG decoding information from deep models to intrinsically interpretable models is implemented. Experimental results demonstrate that the proposed approach outperforms the benchmark knowledge distillation algorithms. This approach successfully improves the prediction accuracy of Soft Decision Tree by a maximum of 8.28%, reaching almost equivalent or even superior performance to deep teacher models. Furthermore, the model-agnostic nature of this approach offers broad application potential.

A calibration-free c-VEP based BCI employing narrow-band random sequences

Objective.Code-modulated visual evoked potential (c-VEP) based brain-computer interfaces (BCIs) exhibit high encoding efficiency. Nevertheless, the majority of c-VEP based BCIs necessitate an initial training or calibration session, particularly when the number of targets expands, which impedes the practicality. To address this predicament, this study introduces a calibration-free c-VEP based BCI employing narrow-band random sequences.Approach.For the encoding method, a series of random sequences were generated within a specific frequency band. The c-VEP signals were subsequently elicited through the application of on-type grid flashes that were modulated by these sequences. For the calibration-free decoding algorithm, filter-bank canonical correlation analysis (FBCCA) was utilized with the reference templates generated from the original sequences. Thirty-five subjects participated into an online BCI experiment. The performances of c-VEP based BCIs utilizing narrow-band random sequences with frequency bands of 15-25 Hz (NBRS-15) and 8-16 Hz (NBRS-8) were compared with that of a steady-state visual evoked potential (SSVEP) based BCI within a frequency range of 8-15.8 Hz.Main results.The offline analysis results demonstrated a substantial correlation between the c-VEPs and the original narrow-band random sequences. After parameter optimization, the calibration-free system employing the NBRS-15 frequency band achieved an average information transfer rate (ITR) of 78.56 ± 37.03 bits/min, which exhibited no significant difference compared to the performance of the SSVEP based system when utilizing FBCCA. The proposed system achieved an average ITR of 102.1 ± 57.59 bits/min in a simulation of a 1000-target BCI system.Significance.This study introduces a novel calibration-free c-VEP based BCI system employing narrow-band random sequences and shows great potential of the proposed system in achieving a large number of targets and high ITR.

Fractal Dimension as a discriminative feature for high accuracy classification in motor imagery EEG-based brain-computer interface

BACKGROUND AND OBJECTIVE

The brain-computer interface (BCI) technology acquires human brain electrical signals, which can be effectively and successfully used to control external devices, potentially supporting subjects suffering from motor impairments in the interaction with the environment. To this aim, BCI systems must correctly decode and interpret neurophysiological signals reflecting the intention of the subjects to move. Therefore, the accurate classification of single events in motor tasks represents a fundamental challenge in ensuring efficient communication and control between users and BCIs. Movement-associated changes in electroencephalographic (EEG) sensorimotor rhythms, such as event-related desynchronization (ERD), are well-known features of discriminating motor tasks. Fractal dimension (FD) can be used to evaluate the complexity and self-similarity in brain signals, potentially providing complementary information to frequency-based signal features.

METHODS

In the present work, we introduce FD as a novel feature for subject-independent event classification, and we test several machine learning (ML) models in behavioural tasks of motor imagery (MI) and motor execution (ME).

RESULTS

Our results show that FD improves the classification accuracy of ML compared to ERD. Furthermore, unilateral hand movements have higher classification accuracy than bilateral movements in both MI and ME tasks.

CONCLUSIONS

These results provide further insights into subject-independent event classification in BCI systems and demonstrate the potential of FD as a discriminative feature for EEG signals.

Global brain analysis of minor hallucinations in Parkinson's disease using EEG and MRI data

Introduction

Visual hallucination is a prevalent psychiatric disorder characterized by the occurrence of false visual perceptions due to misinterpretation in the brain. Individuals with Parkinson's disease often experience both minor and complex visual hallucinations. The underlying mechanism of complex visual hallucinations in Parkinson's patients is commonly attributed to dysfunction in the visual pathway and attention network. However, there is limited research on the mechanism of minor hallucinations.

Methods

To address this gap, we conducted an experiment involving 13 Parkinson's patients with minor hallucinations, 13 Parkinson's patients without hallucinations, and 13 healthy elderly individuals. We collected and analyzed EEG and MRI data. Furthermore, we utilized EEG data from abnormal brain regions to train a machine learning model to determine whether the abnormal EEG data were associated with minor hallucinations.

Results

Our findings revealed that Parkinson's patients with minor hallucinations exhibited excessive activation of cortical excitability, an imbalanced interaction between the attention network and the default network, and disruption in the connection between these networks. These findings is similar to the mechanism observed in complex visual hallucinations. The visual reconstruction of one patient experiencing hallucinations yields results that differ from those observed in subjects without such symptoms.

Discussion

The visual reconstruction results demonstrated significant differences between Parkinson's patients with hallucinations and healthy subjects. This suggests that visual reconstruction techniques may offer a means of evaluating hallucinations.

Beta bursts question the ruling power for brain-computer interfaces

Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.

fNIRS-EEG BCIs for Motor Rehabilitation: A Review

Motor impairment has a profound impact on a significant number of individuals, leading to a substantial demand for rehabilitation services. Through brain-computer interfaces (BCIs), people with severe motor disabilities could have improved communication with others and control appropriately designed robotic prosthetics, so as to (at least partially) restore their motor abilities. BCI plays a pivotal role in promoting smoother communication and interactions between individuals with motor impairments and others. Moreover, they enable the direct control of assistive devices through brain signals. In particular, their most significant potential lies in the realm of motor rehabilitation, where BCIs can offer real-time feedback to assist users in their training and continuously monitor the brain's state throughout the entire rehabilitation process. Hybridization of different brain-sensing modalities, especially functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), has shown great potential in the creation of BCIs for rehabilitating the motor-impaired populations. EEG, as a well-established methodology, can be combined with fNIRS to compensate for the inherent disadvantages and achieve higher temporal and spatial resolution. This paper reviews the recent works in hybrid fNIRS-EEG BCIs for motor rehabilitation, emphasizing the methodologies that utilized motor imagery. An overview of the BCI system and its key components was introduced, followed by an introduction to various devices, strengths and weaknesses of different signal processing techniques, and applications in neuroscience and clinical contexts. The review concludes by discussing the possible challenges and opportunities for future development.

Eye State Detection Using Frequency Features from 1 or 2-Channel EEG

Brain-computer interfaces (BCIs) establish a direct communication channel between the human brain and external devices. Among various methods, electroencephalography (EEG) stands out as the most popular choice for BCI design due to its non-invasiveness, ease of use, and cost-effectiveness. This paper aims to present and compare the accuracy and robustness of an EEG system employing one or two channels. We present both hardware and algorithms for the detection of open and closed eyes. Firstly, we utilize a low-cost hardware device to capture EEG activity from one or two channels. Next, we apply the discrete Fourier transform to analyze the signals in the frequency domain, extracting features from each channel. For classification, we test various well-known techniques, including Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), Decision Tree (DT), or Logistic Regression (LR). To evaluate the system, we conduct experiments, acquiring signals associated with open and closed eyes, and compare the performance between one and two channels. The results demonstrate that employing a system with two channels and using SVM, DT, or LR classifiers enhances robustness compared to a single-channel setup and allows us to achieve an accuracy percentage greater than 95% for both eye states.

Influence of spatial frequency in visual stimuli for cVEP-based BCIs: evaluation of performance and user experience

Code-modulated visual evoked potentials (c-VEPs) are an innovative control signal utilized in brain-computer interfaces (BCIs) with promising performance. Prior studies on steady-state visual evoked potentials (SSVEPs) have indicated that the spatial frequency of checkerboard-like stimuli influences both performance and user experience. Spatial frequency refers to the dimensions of the individual squares comprising the visual stimulus, quantified in cycles (i.e., number of black-white squares pairs) per degree of visual angle. However, the specific effects of this parameter on c-VEP-based BCIs remain unexplored. Therefore, the objective of this study is to investigate the role of spatial frequency of checkerboard-like visual stimuli in a c-VEP-based BCI. Sixteen participants evaluated selection matrices with eight spatial frequencies: C001 (0 c/°, 1×1 squares), C002 (0.15 c/°, 2×2 squares), C004 (0.3 c/°, 4×4 squares), C008 (0.6 c/°, 8×8 squares), C016 (1.2 c/°, 16×16 squares), C032 (2.4 c/°, 32×32 squares), C064 (4.79 c/°, 64×64 squares), and C128 (9.58 c/°, 128×128 squares). These conditions were tested in an online spelling task, which consisted of 18 trials each conducted on a 3×3 command interface. In addition to accuracy and information transfer rate (ITR), subjective measures regarding comfort, ocular irritation, and satisfaction were collected. Significant differences in performance and comfort were observed based on different stimulus spatial frequencies. Although all conditions achieved mean accuracy over 95% after 2.1 s of trial duration, C016 stood out in terms user experience. The proposed condition not only achieved a mean accuracy of 96.53% and 164.54 bits/min with a trial duration of 1.05s, but also was reported to be significantly more comfortable than the traditional C001 stimulus. Since both features are key for BCI development, higher spatial frequencies than the classical black-to-white stimulus might be more adequate for c-VEP systems. Hence, we assert that the spatial frequency should be carefully considered in the development of future applications for c-VEP-based BCIs.

Multi-Band CNN With Band-Dependent Kernels and Amalgamated Cross Entropy Loss for Motor Imagery Classification

In this paper, we present a novel MI classification method based on multi-band convolutional neural network (CNN) with band-dependent kernel sizes, named MBK-CNN, to improve classification performance, by resolving the subject dependency issue of the widely used CNN-based approaches due to the kernel size optimization problem. The proposed structure exploits the frequency diversity of the EEG signals and simultaneously resolves the subject dependent kernel size issue. EEG signal is decomposed into overlapping multi-band and passed through multiple CNNs (termed 'branch-CNNs') with different kernel sizes to generate frequency dependent features, which are combined by a simple weighted sum. In contrast to the existing works where single-band multi-branch CNNs with different kernel sizes are used to resolve the subject dependency issue, a unique kernel size per frequency band is used. To prevent possible overfitting induced by a weighted sum, each branch-CNN is additionally trained by tentative cross entropy loss while overall network is optimized by the end-to-end cross entropy loss, which is named amalgamated cross entropy loss. In addition, we further propose multi-band CNN with enhanced spatial diversity, named MBK-LR-CNN, by replacing each branch-CNN with several sub branch-CNNs applied for channel subsets (termed 'local region') to improve the classification performance. We evaluated the performance of the proposed methods, MBK-CNN and MBK-LR-CNN, on publicly available datasets, BCI Competition IV dataset 2a and High Gamma Dataset. The experimental results confirm the performance improvement of the proposed methods compared to the currently existing MI classification methods.

Multi-Class Classification of Upper Limb Movements With Filter Bank Task-Related Component Analysis

The classification of limb movements can provide with control commands in non-invasive brain-computer interface. Previous studies on the classification of limb movements have focused on the classification of left/right limbs; however, the classification of different types of upper limb movements has often been ignored despite that it provides more active-evoked control commands in the brain-computer interface. Nevertheless, few machine learning method can be used as the state-of-the-art method in the multi-class classification of limb movements. This work focuses on the multi-class classification of upper limb movements and proposes the multi-class filter bank task-related component analysis (mFBTRCA) method, which consists of three steps: spatial filtering, similarity measuring and filter bank selection. The spatial filter, namely the task-related component analysis, is first used to remove noise from EEG signals. The canonical correlation measures the similarity of the spatial-filtered signals and is used for feature extraction. The correlation features are extracted from multiple low-frequency filter banks. The minimum-redundancy maximum-relevance selects the essential features from all the correlation features, and finally, the support vector machine is used to classify the selected features. The proposed method compared against previously used models is evaluated using two datasets. mFBTRCA achieved a classification accuracy of 0.4193 ± 0.0780 (7 classes) and 0.4032 ± 0.0714 (5 classes), respectively, which improves on the best accuracies achieved using the compared methods (0.3590 ± 0.0645 and 0.3159 ± 0.0736, respectively). The proposed method is expected to provide more control commands in the applications of non-invasive brain-computer interfaces.

A case for hybrid BCIs: combining optical and electrical modalities improves accuracy

Near-infrared spectroscopy (NIRS) is a promising research tool that found its way into the field of brain-computer interfacing (BCI). BCI is crucially dependent on maximized usability thus demanding lightweight, compact, and low-cost hardware. We designed, built, and validated a hybrid BCI system incorporating one optical and two electrical modalities ameliorating usability issues. The novel hardware consisted of a NIRS device integrated with an electroencephalography (EEG) system that used two different types of electrodes: Regular gelled gold disk electrodes and tri-polar concentric ring electrodes (TCRE). BCI experiments with 16 volunteers implemented a two-dimensional motor imagery paradigm in off- and online sessions. Various non-canonical signal processing methods were used to extract and classify useful features from EEG, tEEG (EEG through TCRE electrodes), and NIRS. Our analysis demonstrated evidence of improvement in classification accuracy when using the TCRE electrodes compared to disk electrodes and the NIRS system. Based on our synchronous hybrid recording system, we could show that the combination of NIRS-EEG-tEEG performed significantly better than either single modality only.

Somatosensory Event-Related Potential as an Electrophysiological Correlate of Endogenous Spatial Tactile Attention: Prospects for Electrotactile Brain-Computer Interface for Sensory Training

Tactile attention tasks are used in the diagnosis and treatment of neurological and sensory processing disorders, while somatosensory event-related potentials (ERP) measured by electroencephalography (EEG) are used as neural correlates of attention processes. Brain-computer interface (BCI) technology provides an opportunity for the training of mental task execution via providing online feedback based on ERP measures. Our recent work introduced a novel electrotactile BCI for sensory training, based on somatosensory ERP; however, no previous studies have addressed specific somatosensory ERP morphological features as measures of sustained endogenous spatial tactile attention in the context of BCI control. Here we show the morphology of somatosensory ERP responses induced by a novel task introduced within our electrotactile BCI platform i.e., the sustained endogenous spatial electrotactile attention task. By applying pulsed electrical stimuli to the two proximal stimulation hotspots at the user's forearm, stimulating sequentially the mixed branches of radial and median nerves with equal probability of stimuli occurrence, we successfully recorded somatosensory ERPs for both stimulation locations, in the attended and unattended conditions. Waveforms of somatosensory ERP responses for both mixed nerve branches showed similar morphology in line with previous reports on somatosensory ERP components obtained by stimulation of exclusively sensory nerves. Moreover, we found statistically significant increases in ERP amplitude on several components, at both stimulation hotspots, while sustained endogenous spatial electrotactile attention task is performed. Our results revealed the existence of general ERP windows of interest and signal features that can be used to detect sustained endogenous tactile attention and classify between spatial attention locations in 11 healthy subjects. The current results show that features of N140, P3a and P3b somatosensory ERP components are the most prominent global markers of sustained spatial electrotactile attention, over all subjects, within our novel electrotactile BCI task/paradigm, and this work proposes the features of those components as markers of sustained endogenous spatial tactile attention in online BCI control. Immediate implications of this work are the possible improvement of online BCI control within our novel electrotactile BCI system, while these finding can be used for other tactile BCI applications in the diagnosis and treatment of neurological disorders by employing mixed nerve somatosensory ERPs and sustained endogenous electrotactile attention task as control paradigms.

Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities

Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.

Neurotechnology and international security: Predicting commercial and military adoption of brain-computer interfaces (BCIs) in the United States and China

In the past decade, international actors have launched "brain projects" or "brain initiatives." One of the emerging technologies enabled by these publicly funded programs is brain-computer interfaces (BCIs), which are devices that allow communication between the brain and external devices like a prosthetic arm or a keyboard. BCIs are poised to have significant impacts on public health, society, and national security. This research presents the first analytical framework that attempts to predict the dissemination of neurotechnologies to both the commercial and military sectors in the United States and China. While China started its project later with less funding, we find that it has other advantages that make earlier adoption more likely. We also articulate national security risks implicit in later adoption, including the inability to set international ethical and legal norms for BCI use, especially in wartime operating environments, and data privacy risks for citizens who use technology developed by foreign actors.

A spatial-temporal linear feature learning algorithm for P300-based brain-computer interfaces

Speller brain-computer interface (BCI) systems can help neuromuscular disorders patients write their thoughts by using the electroencephalogram (EEG) signals by just focusing on the speller tasks. For practical speller-based BCI systems, the P300 event-related brain potential is measured by using the EEG signal. In this paper, we design a robust machine-learning algorithm for P300 target detection. The novel spatial-temporal linear feature learning (STLFL) algorithm is proposed to extract high-level P300 features. The STLFL method is a modified linear discriminant analysis technique focusing on the spatial-temporal aspects of information extraction. A new P300 detection structure is then proposed based on the combination of the novel STLFL feature extraction and discriminative restricted Boltzmann machine (DRBM) for the classification approach (STLFL + DRBM). The effectiveness of the proposed technique is evaluated using two state-of-the-art P300 BCI datasets. Across the two available databases, we show that in terms of average target recognition accuracy and standard deviation values, the proposed STLFL + DRBM method outperforms traditional methods by 33.5, 78.5, 93.5, and 98.5% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition III datasets II and by 71.3, 100, 100, and 100% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition II datasets II and by 67.5 ± 4, 84.2 ± 2.5, 93.5 ± 1, 96.3 ± 1, and 98.4 ± 0.5% for rapid serial visual presentation (RSVP) based dataset in repetitions 1-5. The method has some advantages over the existing variants including its efficiency, robustness with a small number of training samples, and a high ability to create discriminative features between classes.

Application of multi-task transfer learning: The combination of EA and optimized subband regularized CSP to classification of 8-channel EEG signals with small dataset

Introduction

The volume conduction effect and high dimensional characteristics triggered by the excessive number of channels of EEG cap-acquired signals in BCI systems can increase the difficulty of classifying EEG signals and the lead time of signal acquisition. We aim to combine transfer learning to decode EEG signals in the few-channel case, improve the classification performance of the motor imagery BCI system across subject cases, reduce the cost of signal acquisition performed by the BCI system, and improve the usefulness of the system.

Methods

Dataset2a from BCI CompetitionIV(2008) was used as Dataset1, and our team's self-collected dataset was used as Dataset2. Dataset1 acquired EEG signals from 9 subjects using a 22-channel device with a sampling frequency of 250 Hz. Dataset2 acquired EEG signals from 10 healthy subjects (8 males and 2 females; age distribution between 21-30 years old; mean age 25 years old) using an 8-channel system with a sampling frequency of 1000 Hz. We introduced EA in the data preprocessing process to reduce the signal differences between subjects and proposed VFB-RCSP in combination with RCSP and FBCSP to optimize the effect of feature extraction.

Results

Experiments were conducted on Dataset1 with EEG data containing only 8 channels and achieved an accuracy of 78.01 and a kappa coefficient of 0.54. The accuracy exceeded most of the other methods proposed in recent years, even though the number of channels used was significantly reduced. On Dataset 2, an accuracy of 59.77 and a Kappa coefficient of 0.34 were achieved, which is a significant improvement compared to other poorly improved classical protocols.

Discussion

Our work effectively improves the classification of few-channel EEG data. It overcomes the dependence of existing algorithms on the number of channels, the number of samples, and the frequency band, which is significant for reducing the complexity of BCI models and improving the user-friendliness of BCI systems.

A Sparse Representation Classification Scheme for the Recognition of Affective and Cognitive Brain Processes in Neuromarketing

In this work, we propose a novel framework to recognize the cognitive and affective processes of the brain during neuromarketing-based stimuli using EEG signals. The most crucial component of our approach is the proposed classification algorithm that is based on a sparse representation classification scheme. The basic assumption of our approach is that EEG features from a cognitive or affective process lie on a linear subspace. Hence, a test brain signal can be represented as a linear (or weighted) combination of brain signals from all classes in the training set. The class membership of the brain signals is determined by adopting the Sparse Bayesian Framework with graph-based priors over the weights of linear combination. Furthermore, the classification rule is constructed by using the residuals of linear combination. The experiments on a publicly available neuromarketing EEG dataset demonstrate the usefulness of our approach. For the two classification tasks offered by the employed dataset, namely affective state recognition and cognitive state recognition, the proposed classification scheme manages to achieve a higher classification accuracy compared to the baseline and state-of-the art methods (more than 8% improvement in classification accuracy).

A transfer learning-based feedback training motivates the performance of SMR-BCI

Objective. Feedback training is a practical approach to brain-computer interface (BCI) end-users learning to modulate their sensorimotor rhythms (SMRs). BCI self-regulation learning has been shown to be influenced by subjective psychological factors, such as motivation. However, few studies have taken into account the users' self-motivation as additional guidance for the cognitive process involved in BCI learning. In this study we tested a transfer learning (TL) feedback method designed to increase self-motivation by providing information about past performance.Approach. Electroencephalography (EEG) signals from the previous runs were affine transformed and displayed as points on the screen, along with the newly recorded EEG signals in the current run, giving the subjects a context for self-motivation. Subjects were asked to separate the feedback points for the current run under the display of the separability of prior training. We conducted a between-subject feedback training experiment, in which 24 healthy SMR-BCI naive subjects were trained to imagine left- and right-hand movements. The participants were provided with either TL feedback or typical cursor-bar (CB) feedback (control condition), for three sessions on separate days.Main results. The behavioral results showed an increased challenge and stable mastery confidence, suggesting that subjects' motivation grew as the feedback training went on. The EEG results showed favorable overall training effects with TL feedback in terms of the class distinctiveness and EEG discriminancy. Performance was 28.5% higher in the third session than in the first. About 41.7% of the subjects were 'learners' including not only low-performance subjects, but also good-performance subjects who might be affected by the ceiling effect. Subjects were able to control BCI with TL feedback with a higher performance of 60.5% during the last session compared to CB feedback.Significance. The present study demonstrated that the proposed TL feedback method boosted psychological engagement through the self-motivated context, and further allowed subjects to modulate SMR effectively. The proposed TL feedback method also provided an alternative to typical CB feedback.

Identifying Thematics in a Brain-Computer Interface Research

This umbrella review is motivated to understand the shift in research themes on brain-computer interfacing (BCI) and it determined that a shift away from themes that focus on medical advancement and system development to applications that included education, marketing, gaming, safety, and security has occurred. The background of this review examined aspects of BCI categorisation, neuroimaging methods, brain control signal classification, applications, and ethics. The specific area of BCI software and hardware development was not examined. A search using One Search was undertaken and 92 BCI reviews were selected for inclusion. Publication demographics indicate the average number of authors on review papers considered was 4.2 ± 1.8. The results also indicate a rapid increase in the number of BCI reviews from 2003, with only three reviews before that period, two in 1972, and one in 1996. While BCI authors were predominantly Euro-American in early reviews, this shifted to a more global authorship, which China dominated by 2020-2022. The review revealed six disciplines associated with BCI systems: life sciences and biomedicine (n = 42), neurosciences and neurology (n = 35), and rehabilitation (n = 20); (2) the second domain centred on the theme of functionality: computer science (n = 20), engineering (n = 28) and technology (n = 38). There was a thematic shift from understanding brain function and modes of interfacing BCI systems to more applied research novel areas of research-identified surround artificial intelligence, including machine learning, pre-processing, and deep learning. As BCI systems become more invasive in the lives of "normal" individuals, it is expected that there will be a refocus and thematic shift towards increased research into ethical issues and the need for legal oversight in BCI application.

Novel electrotactile brain-computer interface with somatosensory event-related potential based control

Objective

A brain computer interface (BCI) allows users to control external devices using non-invasive brain recordings, such as electroencephalography (EEG). We developed and tested a novel electrotactile BCI prototype based on somatosensory event-related potentials (sERP) as control signals, paired with a tactile attention task as a control paradigm.

Approach

A novel electrotactile BCI comprises commercial EEG device, an electrical stimulator and custom software for EEG recordings, electrical stimulation control, synchronization between devices, signal processing, feature extraction, selection, and classification. We tested a novel BCI control paradigm based on tactile attention on a sensation at a target stimulation location on the forearm. Tactile stimuli were electrical pulses delivered at two proximal locations on the user's forearm for stimulating branches of radial and median nerves, with equal probability of the target and distractor stimuli occurrence, unlike in any other ERP-based BCI design. We proposed a compact electrical stimulation electrodes configuration for delivering electrotactile stimuli (target and distractor) using 2 stimulation channels and 3 stimulation electrodes. We tested the feasibility of a single EEG channel BCI control, to determine pseudo-online BCI performance, in ten healthy subjects. For optimizing the BCI performance we compared the results for two classifiers, sERP averaging approaches, and novel dedicated feature extraction/selection methods via cross-validation procedures.

Main results

We achieved a single EEG channel BCI classification accuracy in the range of 75.1 to 88.1% for all subjects. We have established an optimal combination of: single trial averaging to obtain sERP, feature extraction/selection methods and classification approach.

Significance

The obtained results demonstrate that a novel electrotactile BCI paradigm with equal probability of attended (target) and unattended (distractor) stimuli and proximal stimulation sites is feasible. This method may be used to drive restorative BCIs for sensory retraining in stroke or brain injury, or assistive BCIs for communication in severely disabled users.

A novel ANN adaptive Riemannian-based kernel classification for motor imagery

More recently, a number of studies show the interest of the use of the Riemannian geometry in EEG classification. The idea is to exploit the EEG covariance matrices, instead of the raw EEG data, and use the Riemannian geometry to directly classify these matrices. This paper presents a novel Artificial Neural Network approach based on an Adaptive Riemannian Kernel, named ARK-ANN, to classify Electroencephalographic (EEG) motor imaging signals in the context of Brain Computer Interface (BCI). A multilayer perceptron is used to classify the covariance matrices of Motor Imagery (MI) signals employing an adaptive optimization of the testing set. The contribution of a geodesic filter is also assessed for the ANN and the original method which uses an SVM classifier. The results demonstrate that the ARK-ANN performs better than the other methods and the geodesic filter gives slightly better results in the ARK-SVM, considered here as the reference method, in the case of inter-subject classification (accuracy of 87.4% and 86% for ARK-ANN and ARK-SVM, respectively). Regarding the cross-subject classification, the proposed method gives an accuracy of 77.3% and increases the precision by 8.2% in comparison to the SVM based method.

Effect of Stimulus Size in a Visual ERP-Based BCI under RSVP

Rapid serial visual presentation (RSVP) is currently one of the most suitable paradigms for use with a visual brain-computer interface based on event-related potentials (ERP-BCI) by patients with a lack of ocular motility. However, gaze-independent paradigms have not been studied as closely as gaze-dependent ones, and variables such as the sizes of the stimuli presented have not yet been explored under RSVP. Hence, the aim of the present work is to assess whether stimulus size has an impact on ERP-BCI performance under the RSVP paradigm. Twelve participants tested the ERP-BCI under RSVP using three different stimulus sizes: small (0.1 × 0.1 cm), medium (1.9 × 1.8 cm), and large (20.05 × 19.9 cm) at 60 cm. The results showed significant differences in accuracy between the conditions; the larger the stimulus, the better the accuracy obtained. It was also shown that these differences were not due to incorrect perception of the stimuli since there was no effect from the size in a perceptual discrimination task. The present work therefore shows that stimulus size has an impact on the performance of an ERP-BCI under RSVP. This finding should be considered by future ERP-BCI proposals aimed at users who need gaze-independent systems.

A Review of Brain Activity and EEG-Based Brain-Computer Interfaces for Rehabilitation Application

Patients with severe CNS injuries struggle primarily with their sensorimotor function and communication with the outside world. There is an urgent need for advanced neural rehabilitation and intelligent interaction technology to provide help for patients with nerve injuries. Recent studies have established the brain-computer interface (BCI) in order to provide patients with appropriate interaction methods or more intelligent rehabilitation training. This paper reviews the most recent research on brain-computer-interface-based non-invasive rehabilitation systems. Various endogenous and exogenous methods, advantages, limitations, and challenges are discussed and proposed. In addition, the paper discusses the communication between the various brain-computer interface modes used between severely paralyzed and locked patients and the surrounding environment, particularly the brain-computer interaction system utilizing exogenous (induced) EEG signals (such as P300 and SSVEP). This discussion reveals with an examination of the interface for collecting EEG signals, EEG components, and signal postprocessing. Furthermore, the paper describes the development of natural interaction strategies, with a focus on signal acquisition, data processing, pattern recognition algorithms, and control techniques.

A Novel Method for Constructing EEG Large-Scale Cortical Dynamical Functional Network Connectivity (dFNC): WTCS

As a kind of biological network, the brain network conduces to understanding the mystery of high-efficiency information processing in the brain, which will provide instructions to develop efficient brain-like neural networks. Large-scale dynamical functional network connectivity (dFNC) provides a more context-sensitive, dynamical, and straightforward sight at a higher network level. Nevertheless, dFNC analysis needs good enough resolution in both temporal and spatial domains, and the construction of dFNC needs to capture the time-varying correlations between two multivariate time series with unmatched spatial dimensions. Effective methods still lack. With well-developed source imaging techniques, electroencephalogram (EEG) has the potential to possess both high temporal and spatial resolutions. Therefore, we proposed to construct the EEG large-scale cortical dFNC based on brain atlas to probe the subtle dynamic activities in the brain and developed a novel method, that is, wavelet coherence-S estimator (WTCS), to assess the dynamic couplings among functional subnetworks with different spatial dimensions. The simulation study demonstrated its robustness and availability of applying to dFNC. The application in real EEG data revealed the appealing "Primary peak" and "P3-like peak" in dFNC network properties and meaningful evolutions in dFNC network topology for P300. Our study brings new insights for probing brain activities at a more dynamical and higher hierarchical level and pushing forward the development of brain-inspired artificial neural networks. The proposed WTCS not only benefits the dFNC studies but also gives a new solution to capture the time-varying couplings between the multivariate time series that is often encountered in signal processing disciplines.

Improving pre-movement pattern detection with filter bank selection

Objective. Pre-movement decoding plays an important role in detecting the onsets of actions using low-frequency electroencephalography (EEG) signals before the movement of an upper limb. In this work, a binary classification method is proposed between two different states.Approach. The proposed method, referred to as filter bank standard task-related component analysis (FBTRCA), is to incorporate filter bank selection into the standard task-related component analysis (STRCA) method. In FBTRCA, the EEG signals are first divided into multiple sub-bands which start at specific fixed frequencies and end frequencies that follow in an arithmetic sequence. The STRCA method is then applied to the EEG signals in these bands to extract CCPs. The minimum redundancy maximum relevance feature selection method is used to select essential features from these correlation patterns in all sub-bands. Finally, the selected features are classified using the binary support vector machine classifier. A convolutional neural network (CNN) is an alternative approach to select canonical correlation patterns.Main Results. Three methods were evaluated using EEG signals in the time window from 2 s before the movement onset to 1 s after the movement onset. In the binary classification between a movement state and the resting state, the FBTRCA achieved an average accuracy of 0.8968 ± 0.0847 while the accuracies of STRCA and CNN were 0.8228 ± 0.1149 and 0.8828 ± 0.0917, respectively. In the binary classification between two actions, the accuracies of STRCA, CNN, and FBTRCA were 0.6611 ± 0.1432, 0.6993 ± 0.1271, 0.7178 ± 0.1274, respectively. Feature selection using filter banks, as in FBTRCA, produces comparable results to STRCA.Significance. The proposed method provides a way to select filter banks in pre-movement decoding, and thus it improves the classification performance. The improved pre-movement decoding of single upper limb movements is expected to provide people with severe motor disabilities with a more natural, non-invasive control of their external devices.

Exploration of Brain-Computer Interaction for Supporting Children's Attention Training: A Multimodal Design Based on Attention Network and Gamification Design

Recent developments in brain-computer interface (BCI) technology have shown great potential in terms of estimating users' mental state and supporting children's attention training. However, existing training tasks are relatively simple and lack a reliable task-generation process. Moreover, the training experience has not been deeply studied, and the empirical validation of the training effect is still insufficient. This study thusly proposed a BCI training system for children's attention improvement. In particular, to achieve a systematic training process, the attention network was referred to generate the training games for alerting, orienting and executive attentions, and to improve the training experience and adherence, the gamification design theory was introduced to derive attractive training tasks. A preliminary experiment was conducted to set and modify the training parameters. Subsequently, a series of contrasting user experiments were organized to examine the impact of BCI training. To test the training effect of the proposed system, a hypothesis-testing approach was adopted. The results revealed that the proposed BCI gamification attention training system can significantly improve the participants' attention behaviors and concentration ability. Moreover, an immersive, inspiring and smooth training process can be created, and a pleasant user experience can be achieved. Generally, this work is promising in terms of providing a valuable reference for related practices, especially for how to generate BCI attention training tasks using attention networks and how to improve training adherence by integrating multimodal gamification elements.

Subject adaptation convolutional neural network for EEG-based motor imagery classification

Objective.Deep transfer learning has been widely used to address the nonstationarity of electroencephalogram (EEG) data during motor imagery (MI) classification. However, previous deep learning approaches suffer from limited classification accuracy because the temporal and spatial features cannot be effectively extracted.Approach.Here, we propose a novel end-to-end deep subject adaptation convolutional neural network (SACNN) to handle the problem of EEG-based MI classification. Our proposed model jointly optimizes three modules, i.e. a feature extractor, a classifier, and a subject adapter. Specifically, the feature extractor simultaneously extracts the temporal and spatial features from the raw EEG data using a parallel multiscale convolution network. In addition, we design a subject adapter to reduce the feature distribution shift between the source and target subjects by using the maximum mean discrepancy. By minimizing the classification loss and the distribution discrepancy, the model is able to extract the temporal-spatial features to the prediction of a new subject.Main results.Extensive experiments are carried out on three EEG-based MI datasets, i.e. brain-computer interface (BCI) competition IV dataset IIb, BCI competition III dataset IVa, and BCI competition IV dataset I, and the average accuracy reaches to 86.42%, 81.71% and 79.35% on the three datasets respectively. Furthermore, the statistical analysis also indicates the significant performance improvement of SACNN.Significance.This paper reveals the importance of the temporal-spatial features on EEG-based MI classification task. Our proposed SACNN model can make fully use of the temporal-spatial information to achieve the purpose.

Multi-source joint domain adaptation for cross-subject and cross-session emotion recognition from electroencephalography

As an important component to promote the development of affective brain–computer interfaces, the study of emotion recognition based on electroencephalography (EEG) has encountered a difficult challenge; the distribution of EEG data changes among different subjects and at different time periods. Domain adaptation methods can effectively alleviate the generalization problem of EEG emotion recognition models. However, most of them treat multiple source domains, with significantly different distributions, as one single source domain, and only adapt the cross-domain marginal distribution while ignoring the joint distribution difference between the domains. To gain the advantages of multiple source distributions, and better match the distributions of the source and target domains, this paper proposes a novel multi-source joint domain adaptation (MSJDA) network. We first map all domains to a shared feature space and then align the joint distributions of the further extracted private representations and the corresponding classification predictions for each pair of source and target domains. Extensive cross-subject and cross-session experiments on the benchmark dataset, SEED, demonstrate the effectiveness of the proposed model, where more significant classification results are obtained on the more difficult cross-subject emotion recognition task.

Clustered event related spectral perturbation (ERSP) feature in right hand motor imagery classification

A technology that allows humans to interact with machines more directly and efficiently would be desirable. Research on brain-computer interfaces (BCIs) provides the possibility for computers to understand human thoughts in a straightforward manner thereby facilitating communication. As a branch of BCI research, motor imagery (MI) techniques analyze the brain signals and help people in many aspects such as rehabilitation, clinical applications, entertainment, and system controlling. In this study, an imagery experiment consisting of four kinds of right-hand movements (gripping, opening, pronation, and supination) was designed. Then a novel feature, namely, clustered feature was proposed based on the event-related spectral perturbation (ERSP) calculated from the EEG signal. Based on the selected features, two classical classifiers (support vector machine and linear discriminant classifier) were trained, achieving an acceptable accurate result (80%, on average).

A L1 normalization enhanced dynamic window method for SSVEP-based BCIs

BACKGROUND

Filter bank canonical correlation analysis (FBCCA) has been widely applied to detect the frequency components of steady-state visual evoked potential (SSVEP). FBCCA with dynamic window (FBCCA-DW) is recently proposed to improve its performance. FBCCA-DW adaptively chooses a proper window length based on the signal-to-noise ratio (SNR) of SSVEP signals. It takes the output of FBCCA to evaluate the SNR of SSVEP signals, by using the softmax function and cost function. In practice, SSVEP signals always contain task-unrelated electroencephalogram (EEG), which degrades the SSVEP task. When the power of task-unrelated EEG changes, there would be an offset in the output of FBCCA. However, due to the insensitivity of softmax function to the offset, the SNR in FBCCA-DW ignores the interference of the task-unrelated EEG. Therefore, FBCCA-DW will analyze SSVEP signals at an inappropriate window length.

NEW METHOD

To solve the issue, we replace the softmax function with the L1 normalization, which could respond a reasonable SNR to the offset. Since the proposed method takes task-unrelated EEG into account, it could choose a more appropriate window length.

RESULTS

We comprehensively validate the proposed method on three publicly available SSVEP datasets. The results indicate that the proposed method could improve the performance significantly.

COMPARISON WITH EXISTING METHODS

The proposed method outperforms FBCCA and FBCCA-DW in terms of information transfer rate (ITR).

CONCLUSIONS

The proposed method enhances the correlation between the window length and the credibility of the recognition result. It shows its potential for practical applications in complex environments.

Single channel approach for filtering electroencephalographic signals strongly contaminated with facial electromyography

Eliminating facial electromyographic (EMG) signal from the electroencephalogram (EEG) is crucial for the accuracy of applications such as brain computer interfaces (BCIs) and brain functionality measurement. Facial electromyography typically corrupts the electroencephalogram. Although it is possible to find in the literature a number of multi-channel approaches for filtering corrupted EEG, studies employing single-channel approaches are scarce. In this context, this study proposed a single-channel method for attenuating facial EMG noise from contaminated EEG. The architecture of the method allows for the evaluation and incorporation of multiple decomposition and adaptive filtering techniques. The decomposition method was responsible for generating EEG or EMG reference signals for the adaptive filtering stage. In this study, the decomposition techniques CiSSA, EMD, EEMD, EMD-PCA, SSA, and Wavelet were evaluated. The adaptive filtering methods RLS, Wiener, LMS, and NLMS were investigated. A time and frequency domain set of features were estimated from experimental signals to evaluate the performance of the single channel method. This set of characteristics permitted the characterization of the contamination of distinct facial muscles, namely Masseter, Frontalis, Zygomatic, Orbicularis Oris, and Orbicularis Oculi. Data were collected from ten healthy subjects executing an experimental protocol that introduced the necessary variability to evaluate the filtering performance. The largest level of contamination was produced by the Masseter muscle, as determined by statistical analysis of the set of features and visualization of topological maps. Regarding the decomposition method, the SSA method allowed for the generation of more suitable reference signals, whereas the RLS and NLMS methods were more suitable when the reference signal was derived from the EEG. In addition, the LMS and RLS methods were more appropriate when the reference signal was the EMG. This study has a number of practical implications, including the use of filtering techniques to reduce EEG contamination caused by the activation of facial muscles required by distinct types of studies. All the developed code, including examples, is available to facilitate a more accurate reproduction and improvement of the results of this study.

A data-driven machine learning approach for brain-computer interfaces targeting lower limb neuroprosthetics

Prosthetic devices that replace a lost limb have become increasingly performant in recent years. Recent advances in both software and hardware allow for the decoding of electroencephalogram (EEG) signals to improve the control of active prostheses with brain-computer interfaces (BCI). Most BCI research is focused on the upper body. Although BCI research for the lower extremities has increased in recent years, there are still gaps in our knowledge of the neural patterns associated with lower limb movement. Therefore, the main objective of this study is to show the feasibility of decoding lower limb movements from EEG data recordings. The second aim is to investigate whether well-known neuroplastic adaptations in individuals with an amputation have an influence on decoding performance. To address this, we collected data from multiple individuals with lower limb amputation and a matched able-bodied control group. Using these data, we trained and evaluated common BCI methods that have already been proven effective for upper limb BCI. With an average test decoding accuracy of 84% for both groups, our results show that it is possible to discriminate different lower extremity movements using EEG data with good accuracy. There are no significant differences (p = 0.99) in the decoding performance of these movements between healthy subjects and subjects with lower extremity amputation. These results show the feasibility of using BCI for lower limb prosthesis control and indicate that decoding performance is not influenced by neuroplasticity-induced differences between the two groups.

Evaluation of Machine Learning Algorithms for Classification of EEG Signals

In brain–computer interfaces (BCIs), it is crucial to process brain signals to improve the accuracy of the classification of motor movements. Machine learning (ML) algorithms such as artificial neural networks (ANNs), linear discriminant analysis (LDA), decision tree (D.T.), K-nearest neighbor (KNN), naive Bayes (N.B.), and support vector machine (SVM) have made significant progress in classification issues. This paper aims to present a signal processing analysis of electroencephalographic (EEG) signals among different feature extraction techniques to train selected classification algorithms to classify signals related to motor movements. The motor movements considered are related to the left hand, right hand, both fists, feet, and relaxation, making this a multiclass problem. In this study, nine ML algorithms were trained with a dataset created by the feature extraction of EEG signals.The EEG signals of 30 Physionet subjects were used to create a dataset related to movement. We used electrodes C3, C1, CZ, C2, and C4 according to the standard 10-10 placement. Then, we extracted the epochs of the EEG signals and applied tone, amplitude levels, and statistical techniques to obtain the set of features. LabVIEW™2015 version custom applications were used for reading the EEG signals; for channel selection, noise filtering, band selection, and feature extraction operations; and for creating the dataset. MATLAB 2021a was used for training, testing, and evaluating the performance metrics of the ML algorithms. In this study, the model of Medium-ANN achieved the best performance, with an AUC average of 0.9998, Cohen’s Kappa coefficient of 0.9552, a Matthews correlation coefficient of 0.9819, and a loss of 0.0147. These findings suggest the applicability of our approach to different scenarios, such as implementing robotic prostheses, where the use of superficial features is an acceptable option when resources are limited, as in embedded systems or edge computing devices.

Sequence-to-Sequence Voice Reconstruction for Silent Speech in a Tonal Language

Silent speech decoding (SSD), based on articulatory neuromuscular activities, has become a prevalent task of brain–computer interfaces (BCIs) in recent years. Many works have been devoted to decoding surface electromyography (sEMG) from articulatory neuromuscular activities. However, restoring silent speech in tonal languages such as Mandarin Chinese is still difficult. This paper proposes an optimized sequence-to-sequence (Seq2Seq) approach to synthesize voice from the sEMG-based silent speech. We extract duration information to regulate the sEMG-based silent speech using the audio length. Then, we provide a deep-learning model with an encoder–decoder structure and a state-of-the-art vocoder to generate the audio waveform. Experiments based on six Mandarin Chinese speakers demonstrate that the proposed model can successfully decode silent speech in Mandarin Chinese and achieve a character error rate (CER) of 6.41% on average with human evaluation.

A Lightweight Framework for Perception Analysis Based on Multimodal Cognition-Aware Computing

The VUCA environment challenged neuropsychological research conducted in conventional laboratories. Researchers expected to perform complex multimodal testing tasks in natural, open, and non-laboratory settings. However, for most neuropsychological scientists, the independent construction of a multimodal laboratory in a VUCA environment, such as a construction site, was a significant and comprehensive technological challenge. This study presents a generalized lightweight framework for perception analysis based on multimodal cognition-aware computing, which provided practical updated strategies and technological guidelines for neuromanagement and automation. A real-life test experiment on a construction site was provided to illustrate the feasibility and superiority of the method. The study aimed to fill a technology gap in the application of multimodal physiological and neuropsychological techniques in an open VUCA environment. Meanwhile, it enabled the researchers to improve their systematic technological capabilities and reduce the threshold and trial-and-error costs of experiments to conform to the new trend of VUCA.

Manual 3D Control of an Assistive Robotic Manipulator Using Alpha Rhythms and an Auditory Menu: A Proof-of-Concept

Brain–Computer Interfaces (BCIs) have been regarded as potential tools for individuals with severe motor disabilities, such as those with amyotrophic lateral sclerosis, that render interfaces that rely on movement unusable. This study aims to develop a dependent BCI system for manual end-point control of a robotic arm. A proof-of-concept system was devised using parieto-occipital alpha wave modulation and a cyclic menu with auditory cues. Users choose a movement to be executed and asynchronously stop said action when necessary. Tolerance intervals allowed users to cancel or confirm actions. Eight able-bodied subjects used the system to perform a pick-and-place task. To investigate the potential learning effects, the experiment was conducted twice over the course of two consecutive days. Subjects obtained satisfactory completion rates (84.0 ± 15.0% and 74.4 ± 34.5% for the first and second day, respectively) and high path efficiency (88.9 ± 11.7% and 92.2 ± 9.6%). Subjects took on average 439.7 ± 203.3 s to complete each task, but the robot was only in motion 10% of the time. There was no significant difference in performance between both days. The developed control scheme provided users with intuitive control, but a considerable amount of time is spent waiting for the right target (auditory cue). Implementing other brain signals may increase its speed.

Optimising the classification of feature-based attention in frequency-tagged electroencephalography data

Brain-computer interfaces (BCIs) are a rapidly expanding field of study and require accurate and reliable real-time decoding of patterns of neural activity. These protocols often exploit selective attention, a neural mechanism that prioritises the sensory processing of task-relevant stimulus features (feature-based attention) or task-relevant spatial locations (spatial attention). Within the visual modality, attentional modulation of neural responses to different inputs is well indexed by steady-state visual evoked potentials (SSVEPs). These signals are reliably present in single-trial electroencephalography (EEG) data, are largely resilient to common EEG artifacts, and allow separation of neural responses to numerous concurrently presented visual stimuli. To date, efforts to use single-trial SSVEPs to classify visual attention for BCI control have largely focused on spatial attention rather than feature-based attention. Here, we present a dataset that allows for the development and benchmarking of algorithms to classify feature-based attention using single-trial EEG data. The dataset includes EEG and behavioural responses from 30 healthy human participants who performed a feature-based motion discrimination task on frequency tagged visual stimuli.

How to successfully classify EEG in motor imagery BCI: a metrological analysis of the state of the art

Objective. Processing strategies are analysed with respect to the classification of electroencephalographic signals related to brain-computer interfaces based on motor imagery. A review of literature is carried out to understand the achievements in motor imagery classification, the most promising trends, and the challenges in replicating these results. Main focus is placed on performance by means of a rigorous metrological analysis carried out in compliance with the international vocabulary of metrology. Hence, classification accuracy and its uncertainty are considered, as well as repeatability and reproducibility.Approach. The paper works included in the review concern the classification of electroencephalographic signals in motor-imagery- based brain-computer interfaces. Article search was carried out in accordance with the PRISMA standard and 89 studies were included.Main results. Statistically-based analyses show that brain-inspired approaches are increasingly proposed, and that these are particularly successful in discriminating against multiple classes. Notably, many proposals involve convolutional neural networks. Instead, classical machine learning approaches are still effective for binary classifications. Many proposals combine common spatial pattern, least absolute shrinkage and selection operator, and support vector machines. Regarding reported classification accuracies, performance above the upper quartile is in the 85 % to 100 % range for the binary case and in the 83 % to 93 % range for multi-class one. Associated uncertainties are up to 6 % while repeatability for a predetermined dataset is up to 8 %. Reproducibility assessment was instead prevented by lack of standardization in experiments.Significance. By relying on the analysed studies, the reader is guided towards the development of a successful processing strategy as a crucial part of a brain-computer interface. Moreover, it is suggested that future studies should extend these approaches on data from more subjects and with custom experiments, even by investigating online operation. This would also enable the quantification of results reproducibility.

Evaluating Steady-State Visually Evoked Potentials-Based Brain-Computer Interface System Using Wavelet Features and Various Machine Learning Methods

Steady-state visual evoked potentials (SSVEPs) have been designated to be appropriate and are in use in many areas such as clinical neuroscience, cognitive science, and engineering. SSVEPs have become popular recently, due to their advantages including high bit rate, simple system structure and short training time. To design SSVEP-based BCI system, signal processing methods appropriate to the signal structure should be applied. One of the most appropriate signal processing methods of these non-stationary signals is the Wavelet Transform. In this study, we investigated both the effect of choosing a mother wavelet function and the most successful combination of classifier algorithm, wavelet features, and frequency pairs assigned to BCI commands. SSVEP signals that were recorded at seven different stimulus frequencies (6–6.5 – 7 – 7.5 – 8.2 – 9.3 – 10 Hz) were used in this study. A total of 115 features were extracted from time, frequency, and time-frequency domains. These features were classified by a total of seven different classification processes. Classification evaluation was presented with the 5-fold cross-validation method and accuracy values. According to the results, (I) the most successful wavelet function was Haar wavelet, (II) the most successful classifier was Ensemble Learning, (III) using the feature vector consisting of energy, entropy, and variance features yielded higher accuracy than using one of these features alone, and (IV) the highest performances were obtained in the frequency pairs with “6–10”, “6.5–10”, “7–10”, and “7.5–10” Hz.

Evaluating the Different Stages of Parkinson's Disease Using Electroencephalography With Holo-Hilbert Spectral Analysis

Electroencephalography (EEG) can reveal the abnormalities of dopaminergic subcortico-cortical circuits in patients with Parkinson's disease (PD). However, conventional time-frequency analysis of EEG signals cannot fully reveal the non-linear processes of neural activities and interactions. A novel Holo-Hilbert Spectral Analysis (HHSA) was applied to reveal non-linear features of resting state EEG in 99 PD patients and 59 healthy controls (HCs). PD patients demonstrated a reduction of β bands in frontal and central regions, and reduction of γ bands in central, parietal, and temporal regions. Compared with early-stage PD patients, late-stage PD patients demonstrated reduction of β bands in the posterior central region, and increased θ and δ2 bands in the left parietal region. θ and β bands in all brain regions were positively correlated with Hamilton depression rating scale scores. Machine learning algorithms using three prioritized HHSA features demonstrated "Bag" with the best accuracy of 0.90, followed by "LogitBoost" with an accuracy of 0.89. Our findings strengthen the application of HHSA to reveal high-dimensional frequency features in EEG signals of PD patients. The EEG characteristics extracted by HHSA are important markers for the identification of depression severity and diagnosis of PD.

3D Input Convolutional Neural Network for SSVEP Classification in Design of Brain Computer Interface for Patient User

This research was aimed at presenting performance of 3-dimensional input convolutional neural networks for steady-state visual evoked potential classification in a wireless EEG-based brain-computer interface system. Overall performance of a brain-computer interface system depends on information transfer rate. Parameters such as signal classification accuracy rate, signal stimulator structure, and user task completion time affect information transfer rate. In this study, we used 3 types of signal classification methods that are 1-dimensional, 2-dimensional, and 3-dimensional input convolutional neural network. According to online experiment with using 3-dimensional input convolutional neural network, we reached average classification accuracy rate and average information transfer rate as 93.75% and 58.35 bit/min, respectively. This both results significantly higher than the other methods that we used in experiments. Moreover, user task completion time was reduced with using 3-dimensional input convolutional neural network. Our proposed method is novel and state-of-art model for steady-state visual evoked potential classification.