Brain–machine interfaces: past, present and future

Considerations and discussions on the clear definition and definite scope of brain-computer interfaces

Brain-computer interface (BCI) is a revolutionizing human-computer interaction with potential applications in both medical and non-medical fields, emerging as a cutting-edge and trending research direction. Increasing numbers of groups are engaging in BCI research and development. However, in recent years, there has been some confusion regarding BCI, including misleading and hyped propaganda about BCI, and even non-BCI technologies being labeled as BCI. Therefore, a clear definition and a definite scope for BCI are thoroughly considered and discussed in the paper, based on the existing definitions of BCI, including the six key or essential components of BCI. In the review, different from previous definitions of BCI, BCI paradigms and neural coding are explicitly included in the clear definition of BCI provided, and the BCI user (the brain) is clearly identified as a key component of the BCI system. Different people may have different viewpoints on the definition and scope of BCI, as well as some related issues, which are discussed in the article. This review argues that a clear definition and definite scope of BCI will benefit future research and commercial applications. It is hoped that this review will reduce some of the confusion surrounding BCI and promote sustainable development in this field.

A hierarchical recursive feature elimination algorithm to develop brain computer interface application of user behavior for statistical reasoning and decision making

BACKGROUND

With the aid of a brain computer interface (BCI), users can communicate and receive signals wirelessly or over wired connections to operate smart devices. A BCI classifier's architecture is quite difficult since numerous elements should be combined. These elements are made up of brain signals, which also include high levels of weak sounds that could provide reliable participant recordings of daily activities. We must use computer vision techniques to create a model in order to control those information. The high dimension and volume of signals present the classification classifier with its primary obstacles.

NEW METHOD

Due to this, we extracted and classified the brain activity in this study, and we also presented a novel hierarchical recursive feature elimination method that we refer to as HRFE embracing noisy additions. HRFE makes a variety of categorization suggestions to eliminate bias in classifying BCI systems of different types. We put the HRFE to the test on two BCI signal data sets-specifically, dataset I and BCI contests III-using shallow and deep convolution network classification techniques. Just a grid of assets is used to create electrocorticography (ECoG) signals on the contralateral (right) motor cortex, and these signals are recorded in the BCI contests III database.

RESULTS

Using ECoG signals, we choose the top 20 features that have the biggest effects on distortion and classification selection.

COMPARISON WITH EXISTING METHODS

The simulation findings show that HRFE has a significant computer vision enhancement when compared to comparable feature selection methods in the literature, particularly for ECoG signal, which achieves about 93% reliability.

Exploring the frontier: Transformer-based models in EEG signal analysis for brain-computer interfaces

This review systematically explores the application of transformer-based models in EEG signal processing and brain-computer interface (BCI) development, with a distinct focus on ensuring methodological rigour and adhering to empirical validations within the existing literature. By examining various transformer architectures, such as the Temporal Spatial Transformer Network (TSTN) and EEG Conformer, this review delineates their capabilities in mitigating challenges intrinsic to EEG data, such as noise and artifacts, and their subsequent implications on decoding and classification accuracies across disparate mental tasks. The analytical scope extends to a meticulous examination of attention mechanisms within transformer models, delineating their role in illuminating critical temporal and spatial EEG features and facilitating interpretability in model decision-making processes. The discourse additionally encapsulates emerging works that substantiate the efficacy of transformer models in noise reduction of EEG signals and diversifying applications beyond the conventional motor imagery paradigm. Furthermore, this review elucidates evident gaps and propounds exploratory avenues in the applications of pre-trained transformers in EEG analysis and the potential expansion into real-time and multi-task BCI applications. Collectively, this review distils extant knowledge, navigates through the empirical findings, and puts forward a structured synthesis, thereby serving as a conduit for informed future research endeavours in transformer-enhanced, EEG-based BCI systems.

Cracking modes and force dynamics in the insertion of neural probes into hydrogel brain phantom

Objective. The insertion of penetrating neural probes into the brain is crucial for advancing neuroscience, yet it involves various inherent risks. Prototype probes are typically inserted into hydrogel-based brain phantoms and the mechanical responses are analyzed in order to inform the insertion mechanics duringin vivoimplantation. However, the underlying mechanism of the insertion dynamics of neural probes in hydrogel brain phantoms, particularly the phenomenon of cracking, remains insufficiently understood. This knowledge gap leads to misinterpretations and discrepancies when comparing results obtained from phantom studies to those observed under thein vivoconditions. This study aims to elucidate the impact of probe sharpness and dimensions on the cracking mechanisms and insertion dynamics characterized during the insertion of probes in hydrogel phantoms.Approach. The insertion of dummy probes with different shank shapes defined by the tip angle, width, and thickness is systematically studied. The insertion-induced cracks in the transparent hydrogel were accentuated by an immiscible dye, tracked byin situimaging, and the corresponding insertion force was recorded. Three-dimensional finite element analysis models were developed to obtain the contact stress between the probe tip and the phantom.Main results. The findings reveal a dual pattern: for sharp, slender probes, the insertion forces remain consistently low during the insertion process, owing to continuously propagating straight cracks that align with the insertion direction. In contrast, blunt, thick probes induce large forces that increase rapidly with escalating insertion depth, mainly due to the formation of branched crack with a conical cracking surface, and the subsequent internal compression. This interpretation challenges the traditional understanding that neglects the difference in the cracking modes and regards increased frictional force as the sole factor contributing to higher insertion forces. The critical probe sharpness factors separating straight and branched cracking is identified experimentally, and a preliminary explanation of the transition between the two cracking modes is derived from three-dimensional finite element analysis.Significance. This study presents, for the first time, the mechanism underlying two distinct cracking modes during the insertion of neural probes into hydrogel brain phantoms. The correlations between the cracking modes and the insertion force dynamics, as well as the effects of the probe sharpness were established, offering insights into the design of neural probes via phantom studies and informing future investigations into cracking phenomena in brain tissue during probe implantations.

Persistent spiking activity in neuromorphic circuits incorporating post-inhibitory rebound excitation

Objective. This study introduces a novel approach for integrating the post-inhibitory rebound excitation (PIRE) phenomenon into a neuronal circuit. Excitatory and inhibitory synapses are designed to establish a connection between two hardware neurons, effectively forming a network. The model demonstrates the occurrence of PIRE under strong inhibitory input. Emphasizing the significance of incorporating PIRE in neuromorphic circuits, the study showcases generation of persistent activity within cyclic and recurrent spiking neuronal networks.Approach. The neuronal and synaptic circuits are designed and simulated in Cadence Virtuoso using TSMC 180 nm technology. The operating mechanism of the PIRE phenomenon integrated into a hardware neuron is discussed. The proposed circuit encompasses several parameters for effectively controlling multiple electrophysiological features of a neuron.Main results. The neuronal circuit has been tuned to match the response of a biological neuron. The efficiency of this circuit is evaluated by computing the average power dissipation and energy consumption per spike through simulation. The sustained firing of neural spikes is observed till 1.7 s using the two neuronal networks.Significance. Persistent activity has significant implications for various cognitive functions such as working memory, decision-making, and attention. Therefore, hardware implementation of these functions will require our PIRE-integrated model. Energy-efficient neuromorphic systems are useful in many artificial intelligence applications, including human-machine interaction, IoT devices, autonomous systems, and brain-computer interfaces.

Decoding N400m Evoked Component: A Tutorial on Multivariate Pattern Analysis for OP-MEG Data

Multivariate pattern analysis (MVPA) has played an extensive role in interpreting brain activity, which has been applied in studies with modalities such as functional Magnetic Resonance Imaging (fMRI), Magnetoencephalography (MEG) and Electroencephalography (EEG). The advent of wearable MEG systems based on optically pumped magnetometers (OPMs), i.e., OP-MEG, has broadened the application of bio-magnetism in the realm of neuroscience. Nonetheless, it also raises challenges in temporal decoding analysis due to the unique attributes of OP-MEG itself. The efficacy of decoding performance utilizing multimodal fusion, such as MEG-EEG, also remains to be elucidated. In this regard, we investigated the impact of several factors, such as processing methods, models and modalities, on the decoding outcomes of OP-MEG. Our findings indicate that the number of averaged trials, dimensionality reduction (DR) methods, and the number of cross-validation folds significantly affect the decoding performance of OP-MEG data. Additionally, decoding results vary across modalities and fusion strategy. In contrast, decoder type, resampling frequency, and sliding window length exert marginal effects. Furthermore, we introduced mutual information (MI) to investigate how information loss due to OP-MEG data processing affect decoding accuracy. Our study offers insights for linear decoding research using OP-MEG and expand its application in the fields of cognitive neuroscience.

Machine to brain: facial expression recognition using brain machine generative adversarial networks

The human brain can effectively perform Facial Expression Recognition (FER) with a few samples by utilizing its cognitive ability. However, unlike the human brain, even the well-trained deep neural network is data-dependent and lacks cognitive ability. To tackle this challenge, this paper proposes a novel framework, Brain Machine Generative Adversarial Networks (BM-GAN), which utilizes the concept of brain's cognitive ability to guide a Convolutional Neural Network to generate LIKE-electroencephalograph (EEG) features. More specifically, we firstly obtain EEG signals triggered from facial emotion images, then we adopt BM-GAN to carry out the mutual generation of image visual features and EEG cognitive features. BM-GAN intends to use the cognitive knowledge learnt from EEG signals to instruct the model to perceive LIKE-EEG features. Thereby, BM-GAN has a superior performance for FER like the human brain. The proposed model consists of VisualNet, EEGNet, and BM-GAN. More specifically, VisualNet can obtain image visual features from facial emotion images and EEGNet can obtain EEG cognitive features from EEG signals. Subsequently, the BM-GAN completes the mutual generation of image visual features and EEG cognitive features. Finally, the predicted LIKE-EEG features of test images are used for FER. After learning, without the participation of the EEG signals, an average classification accuracy of 96.6 % is obtained on Chinese Facial Affective Picture System dataset using LIKE-EEG features for FER. Experiments demonstrate that the proposed method can produce an excellent performance for FER.

Event detection and classification from multimodal time series with application to neural data

The detection of events in time-series data is a common signal-processing problem. When the data can be modeled as a known template signal with an unknown delay in Gaussian noise, detection of the template signal can be done with a traditional matched filter. However, in many applications, the event of interest is represented in multimodal data consisting of both Gaussian and point-process time series. Neuroscience experiments, for example, can simultaneously record multimodal neural signals such as local field potentials (LFPs), which can be modeled as Gaussian, and neuronal spikes, which can be modeled as point processes. Currently, no method exists for event detection from such multimodal data, and as such our objective in this work is to develop a method to meet this need. Here we address this challenge by developing the multimodal event detector (MED) algorithm which simultaneously estimates event times and classes. To do this, we write a multimodal likelihood function for Gaussian and point-process observations and derive the associated maximum likelihood estimator of simultaneous event times and classes. We additionally introduce a cross-modal scaling parameter to account for model mismatch in real datasets. We validate this method in extensive simulations as well as in a neural spike-LFP dataset recorded during an eye-movement task, where the events of interest are eye movements with unknown times and directions. We show that the MED can successfully detect eye movement onset and classify eye movement direction. Further, the MED successfully combines information across data modalities, with multimodal performance exceeding unimodal performance. This method can facilitate applications such as the discovery of latent events in multimodal neural population activity and the development of brain-computer interfaces for naturalistic settings without constrained tasks or prior knowledge of event times.

Comparison of recognition methods for an asynchronous (un-cued) BCI system: an investigation with 40-class SSVEP dataset

Steady-state visual evoked potential (SSVEP)-based brain-computer Interface (BCI) has demonstrated the potential to manage multi-command targets to achieve high-speed communication. Recent studies on multi-class SSVEP-based BCI have focused on synchronous systems, which rely on predefined time and task indicators; thus, these systems that use passive approaches may be less suitable for practical applications. Asynchronous systems recognize the user's intention (whether or not the user is willing to use systems) from brain activity; then, after recognizing the user's willingness, they begin to operate by switching swiftly for real-time control. Consequently, various methodologies have been proposed to capture the user's intention. However, in-depth investigation of recognition methods in asynchronous BCI system is lacking. Thus, in this work, three recognition methods (power spectral density analysis, canonical correlation analysis (CCA), and support vector machine (SVM)) used widely in asynchronous SSVEP BCI systems were explored to compare their performance. Further, we categorized asynchronous systems into two approaches (1-stage and 2-stage) based upon the recognition process's design, and compared their performance. To do so, a 40-class SSVEP dataset collected from 40 subjects was introduced. Finally, we found that the CCA-based method in the 2-stage approach demonstrated statistically significantly higher performance with a sensitivity of 97.62 ± 02.06%, specificity of 76.50 ± 23.50%, and accuracy of 75.59 ± 10.09%. Thus, it is expected that the 2-stage approach together with CCA-based recognition and FB-CCA classification have good potential to be implemented in practical asynchronous SSVEP BCI systems.

Applying the IEEE BRAIN neuroethics framework to intra-cortical brain-computer interfaces

Objective. Brain-computer interfaces (BCIs) are neuroprosthetic devices that allow for direct interaction between brains and machines. These types of neurotechnologies have recently experienced a strong drive in research and development, given, in part, that they promise to restore motor and communication abilities in individuals experiencing severe paralysis. While a rich literature analyzes the ethical, legal, and sociocultural implications (ELSCI) of these novel neurotechnologies, engineers, clinicians and BCI practitioners often do not have enough exposure to these topics.Approach. Here, we present the IEEE Neuroethics Framework, an international, multiyear, iterative initiative aimed at developing a robust, accessible set of considerations for diverse stakeholders.Main results. Using the framework, we provide practical examples of ELSCI considerations for BCI neurotechnologies. We focus on invasive technologies, and in particular, devices that are implanted intra-cortically for medical research applications.Significance. We demonstrate the utility of our framework in exposing a wide range of implications across different intra-cortical BCI technology modalities and conclude with recommendations on how to utilize this knowledge in the development and application of ethical guidelines for BCI neurotechnologies.

EEG rhythm separation and time-frequency analysis of fast multivariate empirical mode decomposition for motor imagery BCI

Motor imagery electroencephalogram (EEG) is widely employed in brain-computer interface (BCI) systems. As a time-frequency analysis method for nonlinear and non-stationary signals, multivariate empirical mode decomposition (MEMD) and its noise-assisted version (NA-MEMD) has been widely used in the preprocessing step of BCI systems for separating EEG rhythms corresponding to specific brain activities. However, when applied to multichannel EEG signals, MEMD or NA-MEMD often demonstrate low robustness to noise and high computational complexity. To address these issues, we have explored the advantages of our recently proposed fast multivariate empirical mode decomposition (FMEMD) and its noise-assisted version (NA-FMEMD) for analyzing motor imagery data. We emphasize that FMEMD enables a more accurate estimation of EEG frequency information and exhibits a more noise-robust decomposition performance with improved computational efficiency. Comparative analysis with MEMD on simulation data and real-world EEG validates the above assertions. The joint average frequency measure is employed to automatically select intrinsic mode functions that correspond to specific frequency bands. Thus, FMEMD-based classification architecture is proposed. Using FMEMD as a preprocessing algorithm instead of MEMD can improve the classification accuracy by 2.3% on the BCI Competition IV dataset. On the Physiobank Motor/Mental Imagery dataset and BCI Competition IV Dataset 2a, FMEMD-based architecture also attained a comparable performance to complex algorithms. The results indicate that FMEMD proficiently extracts feature information from small benchmark datasets while mitigating dimensionality constraints resulting from computational complexity. Hence, FMEMD or NA-FMEMD can be a powerful time-frequency preprocessing method for BCI.

Exploring EEG-based motor imagery decoding: a dual approach using spatial features and spectro-spatial Deep Learning model IFNet

Introduction

In recent years, the decoding of motor imagery (MI) from electroencephalography (EEG) signals has become a focus of research for brain-machine interfaces (BMIs) and neurorehabilitation. However, EEG signals present challenges due to their non-stationarity and the substantial presence of noise commonly found in recordings, making it difficult to design highly effective decoding algorithms. These algorithms are vital for controlling devices in neurorehabilitation tasks, as they activate the patient's motor cortex and contribute to their recovery.

Methods

This study proposes a novel approach for decoding MI during pedalling tasks using EEG signals. A widespread approach is based on feature extraction using Common Spatial Patterns (CSP) followed by a linear discriminant analysis (LDA) as a classifier. The first approach covered in this work aims to investigate the efficacy of a task-discriminative feature extraction method based on CSP filter and LDA classifier. Additionally, the second alternative hypothesis explores the potential of a spectro-spatial Convolutional Neural Network (CNN) to further enhance the performance of the first approach. The proposed CNN architecture combines a preprocessing pipeline based on filter banks in the frequency domain with a convolutional neural network for spectro-temporal and spectro-spatial feature extraction.

Results and discussion

To evaluate the approaches and their advantages and disadvantages, EEG data has been recorded from several able-bodied users while pedalling in a cycle ergometer in order to train motor imagery decoding models. The results show levels of accuracy up to 80% in some cases. The CNN approach shows greater accuracy despite higher instability.

Power-to-Noise Optimization in the Design of Neural Recording Amplifier Based on Current Scaling, Source Degeneration Resistor, and Current Reuse

This article presents the design of a low-power, low-noise neural signal amplifier for neural recording. The structure reduces the current consumption of the amplifier through current scaling technology and lowers the input-referred noise of the amplifier by combining a source degeneration resistor and current reuse technologies. The amplifier was fabricated using a 0.18 μm CMOS MS RF G process. The results show the front-end amplifier exhibits a measured mid-band gain of 40 dB/46 dB and a bandwidth ranging from 0.54 Hz to 6.1 kHz; the amplifier's input-referred noise was measured to be 3.1 μVrms, consuming a current of 3.8 μA at a supply voltage of 1.8 V, with a Noise Efficiency Factor (NEF) of 2.97. The single amplifier's active silicon area is 0.082 mm2.

Spike sorting in the presence of stimulation artifacts: a dynamical control systems approach

Objective. Bi-directional electronic neural interfaces, capable of both electrical recording and stimulation, communicate with the nervous system to permit precise calibration of electrical inputs by capturing the evoked neural responses. However, one significant challenge is that stimulation artifacts often mask the actual neural signals. To address this issue, we introduce a novel approach that employs dynamical control systems to detect and decipher electrically evoked neural activity despite the presence of electrical artifacts.Approach. Our proposed method leverages the unique spatiotemporal patterns of neural activity and electrical artifacts to distinguish and identify individual neural spikes. We designed distinctive dynamical models for both the stimulation artifact and each neuron observed during spontaneous neural activity. We can estimate which neurons were active by analyzing the recorded voltage responses across multiple electrodes post-stimulation. This technique also allows us to exclude signals from electrodes heavily affected by stimulation artifacts, such as the stimulating electrode itself, yet still accurately differentiate between evoked spikes and electrical artifacts.Main results. We applied our method to high-density multi-electrode recordings from the primate retina in anex vivosetup, using a grid of 512 electrodes. Through repeated electrical stimulations at varying amplitudes, we were able to construct activation curves for each neuron. The curves obtained with our method closely resembled those derived from manual spike sorting. Additionally, the stimulation thresholds we estimated strongly agreed with those determined through manual analysis, demonstrating high reliability (R2=0.951for human 1 andR2=0.944for human 2).Significance. Our method can effectively separate evoked neural spikes from stimulation artifacts by exploiting the distinct spatiotemporal propagation patterns captured by a dense, large-scale multi-electrode array. This technique holds promise for future applications in real-time closed-loop stimulation systems and for managing multi-channel stimulation strategies.

The decoder design and performance comparative analysis for closed-loop brain-machine interface system

Brain-machine interface (BMI) can convert electroencephalography signals (EEGs) into the control instructions of external devices, and the key of control performance is the accuracy and efficiency of decoder. However, the performance of different decoders obtaining control instructions from complex and variable EEG signals is very different and irregular in the different neural information transfer model. Aiming at this problem, the off-line and on-line performance of eight decoders based on the improved single-joint information transmission (SJIT) model is compared and analyzed in this paper, which can provide a theoretical guidance for decoder design. Firstly, in order to avoid the different types of neural activities in the decoding process on the decoder performance, eight decoders based on the improved SJIT model are designed. And then the off-line decoding performance of these decoders is tested and compared. Secondly, a closed-loop BMI system which combining by the designed decoder and the random forest encoder based on the improved SJIT model is constructed. Finally, based on the constructed closed-loop BMI system, the on-line decoding performance of decoders is compared and analyzed. The results show that the LSTM-based decoder has better on-line decoding performance than others in the improved SJIT model.

Flexible Dry Electrode Based on a Wrinkled Surface That Uses Carbon Nanotube/Polymer Composites for Recording Electroencephalograms

Electroencephalography (EEG) captures minute electrical signals emanating from the brain. These signals are vulnerable to interference from external noise and dynamic artifacts; hence, accurately recording such signals is challenging. Although dry electrodes are convenient, their signals are of limited quality; consequently, wet electrodes are predominantly used in EEG. Therefore, developing dry electrodes for accurately and stably recording EEG signals is crucial. In this study, we developed flexible dry electrodes using polydimethylsiloxane (PDMS)/carbon-nanotube (CNT) composites with isotropically wrinkled surfaces that effectively combine the advantages of wet and dry electrodes. Adjusting the PDMS crosslinker ratio led to good adhesion, resulting in a highly adhesive CNT/PDMS composite with a low Young's modulus that exhibited excellent electrical and mechanical properties owing to its ability to conformally contact skin. The isotropically wrinkled surface also effectively controls dynamic artifacts during EEG signal detection and ensures accurate signal analysis. The results of this study demonstrate that dry electrodes based on flexible CNT/PDMS composites and corrugated structures can outperform wet electrodes. The introduction of such electrodes is expected to enable the accurate analysis and monitoring of EEG signals in various scenarios, including clinical trials.

Biohybrid tongue based on hypothalamic neuronal network-on-a-chip for real-time blood glucose sensing and assessment

The expression of sweet receptors in the hypothalamus has been implicated in energy homeostasis control and the pathogenesis of obesity and diabetes. However, the exact mechanism by which hypothalamic glucose-sensing neurons function remains unclear. Conventional detection methods, such as fiber photometry, optogenetics, brain-machine interfaces, patch clamp and calcium imaging, pose limitations for real-time glucose perception due to their complexity, cytotoxicity and so on. Therefore, this study proposes a biohybrid tongue based on hypothalamic neuronal network (HNN)-on-a-chip coupling with microelectrode array (MEA) for real-time glucose perception. Hypothalamic neuronal cultures were cultivated on a two-dimensional "brain-on-chip" device, enabling the formation of neuronal networks and electrophysiological signal detection. Additionally, we investigated the endogenous expression of sweet taste receptors (T1R2/T1R3) in hypothalamic neuronal cells, providing the basis for the biohybrid tongue based on HNN-on-a-chip's sweetness detection capabilities. The spike signal response to sucrose and glucose stimulation was detected, and concentration-dependent responses were explored with glucose concentrations ranging from 0.01 mM to 8 mM. MEAs allow for real-time recordings, enabling the observation of dynamic changes in neuronal responses to glucose fluctuations over time. The biohybrid tongue based on HNN-on-a-chip can measure various parameters, including spike frequency and amplitude, providing insights into neuronal firing patterns and excitability. Moreover, hypothalamic glucoregulatory neurons that sense and respond to changes in blood glucose was identified, including glucose-excited neurons (GE-Neurons) and glucose-inhibited neurons (GI-Neurons). The detection range for GE-Neurons spans from 0.4 to 6 mM, while GI-Neurons demonstrate sensitivity within the range of 1-8 mM. And the glucose detection limit was firmly established at 0.01 mM. Through non-linear regression analysis, the IC50 for GI-Neurons' spike firing was determined to be 4.18 mM. In conclusion, the biohybrid tongue based on HNN-on-a-chip offers a valuable in vitro tool for studying hypothalamic neurons, elucidating glucose sensing mechanisms, and understanding hypothalamic neuronal function.

A data expansion technique based on training and testing sample to boost the detection of SSVEPs for brain-computer interfaces

Objective.Currently, steady-state visual evoked potentials (SSVEPs)-based brain-computer interfaces (BCIs) have achieved the highest interaction accuracy and speed among all BCI paradigms. However, its decoding efficacy depends deeply on the number of training samples, and the system performance would have a dramatic drop when the training dataset decreased to a small size. To date, no study has been reported to incorporate the unsupervised learning information from testing trails into the construction of supervised classification model, which is a potential way to mitigate the overfitting effect of limited samples.Approach.This study proposed a novel method for SSVEPs detection, i.e. cyclic shift trials (CSTs), which could combine unsupervised learning information from test trials and supervised learning information from train trials. Furthermore, since SSVEPs are time-locked and phase-locked to the onset of specific flashes, CST could also expand training samples on the basis of its regularity and periodicity. In order to verify the effectiveness of CST, we designed an online SSVEP-BCI system, and tested this system combined CST with two common classification algorithms, i.e. extended canonical correlation analysis and ensemble task-related component analysis.Main results.CST could significantly enhance the signal to noise ratios of SSVEPs and improve the performance of systems especially for the condition of few training samples and short stimulus time. The online information transfer rate could reach up to 236.19 bits min-1using 36 s calibration time of only one training sample for each category.Significance.The proposed CST method can take full advantages of supervised learning information from training samples and unsupervised learning information of testing samples. Furthermore, it is a data expansion technique, which can enhance the SSVEP characteristics and reduce dependence on sample size. Above all, CST is a promising method to improve the performance of SSVEP-based BCI without any additional experimental burden.

Boosting brain-computer interfaces with functional electrical stimulation: potential applications in people with locked-in syndrome

Individuals with a locked-in state live with severe whole-body paralysis that limits their ability to communicate with family and loved ones. Recent advances in brain-computer interface (BCI) technology have presented a potential alternative for these people to communicate by detecting neural activity associated with attempted hand or speech movements and translating the decoded intended movements to a control signal for a computer. A technique that could potentially enrich the communication capacity of BCIs is functional electrical stimulation (FES) of paralyzed limbs and face to restore body and facial movements of paralyzed individuals, allowing to add body language and facial expression to communication BCI utterances. Here, we review the current state of the art of existing BCI and FES work in people with paralysis of body and face and propose that a combined BCI-FES approach, which has already proved successful in several applications in stroke and spinal cord injury, can provide a novel promising mode of communication for locked-in individuals.

Creating an AI-Enhanced Morse Code Translation System Based on Images for People with Severe Disabilities

(1) Background: Patients with severe physical impairments (spinal cord injury, cerebral palsy, amyotrophic lateral sclerosis) often have limited mobility due to physical limitations, and may even be bedridden all day long, losing the ability to take care of themselves. In more severe cases, the ability to speak may even be lost, making even basic communication very difficult. (2) Methods: This research will design a set of image-assistive communication equipment based on artificial intelligence to solve communication problems of daily needs. Using artificial intelligence for facial positioning, and facial-motion-recognition-generated Morse code, and then translating it into readable characters or commands, it allows users to control computer software by themselves and communicate through wireless networks or a Bluetooth protocol to control environment peripherals. (3) Results: In this study, 23 human-typed data sets were subjected to recognition using fuzzy algorithms. The average recognition rates for expert-generated data and data input by individuals with disabilities were 99.83% and 98.6%, respectively. (4) Conclusions: Through this system, users can express their thoughts and needs through their facial movements, thereby improving their quality of life and having an independent living space. Moreover, the system can be used without touching external switches, greatly improving convenience and safety.

A Combined Virtual Electrode-Based ESA and CNN Method for MI-EEG Signal Feature Extraction and Classification

A Brain-Computer Interface (BCI) is a medium for communication between the human brain and computers, which does not rely on other human neural tissues, but only decodes Electroencephalography (EEG) signals and converts them into commands to control external devices. Motor Imagery (MI) is an important BCI paradigm that generates a spontaneous EEG signal without external stimulation by imagining limb movements to strengthen the brain's compensatory function, and it has a promising future in the field of computer-aided diagnosis and rehabilitation technology for brain diseases. However, there are a series of technical difficulties in the research of motor imagery-based brain-computer interface (MI-BCI) systems, such as: large individual differences in subjects and poor performance of the cross-subject classification model; a low signal-to-noise ratio of EEG signals and poor classification accuracy; and the poor online performance of the MI-BCI system. To address the above problems, this paper proposed a combined virtual electrode-based EEG Source Analysis (ESA) and Convolutional Neural Network (CNN) method for MI-EEG signal feature extraction and classification. The outcomes reveal that the online MI-BCI system developed based on this method can improve the decoding ability of multi-task MI-EEG after training, it can learn generalized features from multiple subjects in cross-subject experiments and has some adaptability to the individual differences of new subjects, and it can decode the EEG intent online and realize the brain control function of the intelligent cart, which provides a new idea for the research of an online MI-BCI system.

Advancing the interfacing performances of chronically implantable neural probes in the era of CMOS neuroelectronics

Tissue penetrating microelectrode neural probes can record electrophysiological brain signals at resolutions down to single neurons, making them invaluable tools for neuroscience research and Brain-Computer-Interfaces (BCIs). The known gradual decrease of their electrical interfacing performances in chronic settings, however, remains a major challenge. A key factor leading to such decay is Foreign Body Reaction (FBR), which is the cascade of biological responses that occurs in the brain in the presence of a tissue damaging artificial device. Interestingly, the recent adoption of Complementary Metal Oxide Semiconductor (CMOS) technology to realize implantable neural probes capable of monitoring hundreds to thousands of neurons simultaneously, may open new opportunities to face the FBR challenge. Indeed, this shift from passive Micro Electro-Mechanical Systems (MEMS) to active CMOS neural probe technologies creates important, yet unexplored, opportunities to tune probe features such as the mechanical properties of the probe, its layout, size, and surface physicochemical properties, to minimize tissue damage and consequently FBR. Here, we will first review relevant literature on FBR to provide a better understanding of the processes and sources underlying this tissue response. Methods to assess FBR will be described, including conventional approaches based on the imaging of biomarkers, and more recent transcriptomics technologies. Then, we will consider emerging opportunities offered by the features of CMOS probes. Finally, we will describe a prototypical neural probe that may meet the needs for advancing clinical BCIs, and we propose axial insertion force as a potential metric to assess the influence of probe features on acute tissue damage and to control the implantation procedure to minimize iatrogenic injury and subsequent FBR.

Kernel Reinforcement Learning-Assisted Adaptive Decoder Facilitates Stable and Continuous Brain Control Tasks

Brain-Machine Interfaces (BMIs) assist paralyzed people to brain control (BC) the neuro-prosthesis continuously moving in space. During the BC process, the subject imagines the movement of the real limb and adapts the brain activity according to the sensory feedback. The neural adaptation in the closed-loop control results in complex and changing brain signals. Simultaneously, the decoder interprets the time-varying functional mapping between neural activity and continuous trajectory. It is crucial and challenging to accurately and adaptively track the mapping to help the subject accomplish the BC task with a stable performance. Existing Kalman Filter (KF) based decoders achieve continuous trajectory control by linearly interpreting neural firing observations into self-evolving prosthetic states. However, the linear neural-state mapping might not accurately reflect the movement intention of the subject. In this paper, we propose a novel method that allows subjects to achieve continuous brain control efficiently and stably. The proposed method incorporates a kernel reinforcement learning method into a state-observation model to decode the nonlinearly neural observation into a continuous trajectory state. The state transition function ensures the continuity of the prosthetic state. The kernel reinforcement learning allows the quick adaptation of the nonlinear neural-movement mapping during the BC process. The proposed method is tested in an online brain control reaching task for rats. Compared with KF, our method achieved more successful trials, faster response time, shorter inter-trial time, and remained stable over days. These results demonstrate that the proposed method is an efficient tool to assist subjects in brain control tasks.

Modified Neuropixels probes for recording human neurophysiology in the operating room

Neuropixels are silicon-based electrophysiology-recording probes with high channel count and recording-site density. These probes offer a turnkey platform for measuring neural activity with single-cell resolution and at a scale that is beyond the capabilities of current clinically approved devices. Our team demonstrated the first-in-human use of these probes during resection surgery for epilepsy or tumors and deep brain stimulation electrode placement in patients with Parkinson's disease. Here, we provide a better understanding of the capabilities and challenges of using Neuropixels as a research tool to study human neurophysiology, with the hope that this information may inform future efforts toward regulatory approval of Neuropixels probes as research devices. In perioperative procedures, the major concerns are the initial sterility of the device, maintaining a sterile field during surgery, having multiple referencing and grounding schemes available to de-noise recordings (if necessary), protecting the silicon probe from accidental contact before insertion and obtaining high-quality action potential and local field potential recordings. The research team ensures that the device is fully operational while coordinating with the surgical team to remove sources of electrical noise that could otherwise substantially affect the signals recorded by the sensitive hardware. Prior preparation using the equipment and training in human clinical research and working in operating rooms maximize effective communication within and between the teams, ensuring high recording quality and minimizing the time added to the surgery. The perioperative procedure requires ~4 h, and the entire protocol requires multiple weeks.

Adaptive Neuroplasticity in Brain Injury Recovery: Strategies and Insights

This review addresses the relationship between neuroplasticity and recovery from brain damage. Neuroplasticity's ability to adapt becomes crucial since brain injuries frequently result in severe impairments. We begin by describing the fundamentals of neuroplasticity and how it relates to rehabilitation. Examining different forms of brain injuries and their neurological effects highlights the complex difficulties in rehabilitation. By revealing cellular processes, we shed light on synaptic adaptability following damage. Our study of synaptic plasticity digs into axonal sprouting, dendritic remodeling, and the balance of long-term potentiation. These processes depict neural resilience amid change. Then, after damage, we investigate immediate and slow neuroplastic alterations, separating reorganizations that are adaptive from those that are maladaptive. As we go on to rehabilitation, we evaluate techniques that use neuroplasticity's potential. These methods take advantage of the brain's plasticity for healing, from virtual reality and brain-computer interfaces to constraint-induced movement therapy. Ethics and individualized neurorehabilitation are explored. We scrutinize the promise of combination therapy and the difficulties in putting new knowledge into clinical practice. In conclusion, this analysis highlights neuroplasticity's critical role in brain injury recovery, providing sophisticated approaches to improve life after damage.

Brain-Machine Coupled Learning Method for Facial Emotion Recognition

Neural network models of machine learning have shown promising prospects for visual tasks, such as facial emotion recognition (FER). However, the generalization of the model trained from a dataset with a few samples is limited. Unlike the machine, the human brain can effectively realize the required information from a few samples to complete the visual tasks. To learn the generalization ability of the brain, in this article, we propose a novel brain-machine coupled learning method for facial emotion recognition to let the neural network learn the visual knowledge of the machine and cognitive knowledge of the brain simultaneously. The proposed method utilizes visual images and electroencephalogram (EEG) signals to couple training the models in the visual and cognitive domains. Each domain model consists of two types of interactive channels, common and private. Since the EEG signals can reflect brain activity, the cognitive process of the brain is decoded by a model following reverse engineering. Decoding the EEG signals induced by the facial emotion images, the common channel in the visual domain can approach the cognitive process in the cognitive domain. Moreover, the knowledge specific to each domain is found in each private channel using an adversarial strategy. After learning, without the participation of the EEG signals, only the concatenation of both channels in the visual domain is used to classify facial emotion images based on the visual knowledge of the machine and the cognitive knowledge learned from the brain. Experiments demonstrate that the proposed method can produce excellent performance on several public datasets. Further experiments show that the proposed method trained from the EEG signals has good generalization ability on new datasets and can be applied to other network models, illustrating the potential for practical applications.

Conducting polymer-based nanostructured materials for brain-machine interfaces

As scientists discovered that raw neurological signals could translate into bioelectric information, brain-machine interfaces (BMI) for experimental and clinical studies have experienced massive growth. Developing suitable materials for bioelectronic devices to be used for real-time recording and data digitalizing has three important necessitates which should be covered. Biocompatibility, electrical conductivity, and having mechanical properties similar to soft brain tissue to decrease mechanical mismatch should be adopted for all materials. In this review, inorganic nanoparticles and intrinsically conducting polymers are discussed to impart electrical conductivity to systems, where soft materials such as hydrogels can offer reliable mechanical properties and a biocompatible substrate. Interpenetrating hydrogel networks offer more mechanical stability and provide a path for incorporating polymers with desired properties into one strong network. Promising fabrication methods, like electrospinning and additive manufacturing, allow scientists to customize designs for each application and reach the maximum potential for the system. In the near future, it is desired to fabricate biohybrid conducting polymer-based interfaces loaded with cells, giving the opportunity for simultaneous stimulation and regeneration. Developing multi-modal BMIs, Using artificial intelligence and machine learning to design advanced materials are among the future goals for this field. This article is categorized under: Therapeutic Approaches and Drug Discovery > Nanomedicine for Neurological Disease.

The Effect of Feedback Modality When Learning a Novel Wrist Sensorimotor Transformation Through a Body-Machine Interface

Body-Machine Interfaces (BoMIs) are promising assistive and rehabilitative tools for helping individuals with impaired motor abilities regain independence. When operating a BoMI, the user has to learn a novel sensorimotor transformation between the movement of certain body parts and the output of the device. In this study, we investigated how different feedback modalities impacted learning to operate a BoMI. Forty-seven able-bodied participants learned to control the velocity of a 1D cursor using the 3D rotation of their dominant wrist to reach as many targets as possible in a given amount of time. The map was designed to maximize cursor speed for movements around a predefined axis of wrist rotation. We compared the user's performance and control efficiency under three feedback modalities: i) visual feedback of the cursor position, ii) proprioceptive feedback of the cursor position delivered by a wrist manipulandum, iii) both i) and ii). We found that visual feedback led to a greater number of targets reached than proprioceptive feedback alone. Conversely, proprioceptive feedback yielded greater alignment between the axis of rotation of the wrist and the optimal axis represented by the map. These results suggest that proprioceptive feedback may be preferable over visual feedback when information about intrinsic task components, i.e. joint configurations, is of interest as in rehabilitative interventions aiming to promote more effective learning strategies.

Decoding numeracy and literacy in the human brain: insights from MEG and MVPA

Numbers and letters are the fundamental building blocks of our everyday social interactions. Previous studies have focused on determining the cortical pathways shaped by numeracy and literacy in the human brain, partially supporting the hypothesis of distinct perceptual neural circuits involved in the visual processing of the two categories. In this study, we aim to investigate the temporal dynamics for number and letter processing. We present magnetoencephalography (MEG) data from two experiments (N = 25 each). In the first experiment, single numbers, letters, and their respective false fonts (false numbers and false letters) were presented, whereas, in the second experiment, numbers, letters, and their respective false fonts were presented as a string of characters. We used multivariate pattern analysis techniques (time-resolved decoding and temporal generalization), testing the strong hypothesis that the neural correlates supporting letter and number processing can be logistically classified as categorically separate. Our results show a very early dissociation (~ 100 ms) between numbers, and letters when compared to false fonts. Number processing can be dissociated with similar accuracy when presented as isolated items or strings of characters, while letter processing shows dissociable classification accuracy for single items compared to strings. These findings reinforce the evidence indicating that early visual processing can be differently shaped by the experience with numbers and letters; this dissociation is stronger for strings compared to single items, thus showing that combinatorial mechanisms for numbers and letters could be categorically distinguished and influence early visual processing.

Partial maximum correntropy regression for robust electrocorticography decoding

The Partial Least Square Regression (PLSR) method has shown admirable competence for predicting continuous variables from inter-correlated electrocorticography signals in the brain-computer interface. However, PLSR is essentially formulated with the least square criterion, thus, being considerably prone to the performance deterioration caused by the brain recording noises. To address this problem, this study aims to propose a new robust variant for PLSR. To this end, the maximum correntropy criterion (MCC) is utilized to propose a new robust implementation of PLSR, called Partial Maximum Correntropy Regression (PMCR). The half-quadratic optimization is utilized to calculate the robust projectors for the dimensionality reduction, and the regression coefficients are optimized by a fixed-point optimization method. The proposed PMCR is evaluated with a synthetic example and a public electrocorticography dataset under three performance indicators. For the synthetic example, PMCR realized better prediction results compared with the other existing methods. PMCR could also abstract valid information with a limited number of decomposition factors in a noisy regression scenario. For the electrocorticography dataset, PMCR achieved superior decoding performance in most cases, and also realized the minimal neurophysiological pattern deterioration with the interference of the noises. The experimental results demonstrate that, the proposed PMCR could outperform the existing methods in a noisy, inter-correlated, and high-dimensional decoding task. PMCR could alleviate the performance degradation caused by the adverse noises and ameliorate the electrocorticography decoding robustness for the brain-computer interface.

Machine-Learning Methods for Speech and Handwriting Detection Using Neural Signals: A Review

Brain-Computer Interfaces (BCIs) have become increasingly popular in recent years due to their potential applications in diverse fields, ranging from the medical sector (people with motor and/or communication disabilities), cognitive training, gaming, and Augmented Reality/Virtual Reality (AR/VR), among other areas. BCI which can decode and recognize neural signals involved in speech and handwriting has the potential to greatly assist individuals with severe motor impairments in their communication and interaction needs. Innovative and cutting-edge advancements in this field have the potential to develop a highly accessible and interactive communication platform for these people. The purpose of this review paper is to analyze the existing research on handwriting and speech recognition from neural signals. So that the new researchers who are interested in this field can gain thorough knowledge in this research area. The current research on neural signal-based recognition of handwriting and speech has been categorized into two main types: invasive and non-invasive studies. We have examined the latest papers on converting speech-activity-based neural signals and handwriting-activity-based neural signals into text data. The methods of extracting data from the brain have also been discussed in this review. Additionally, this review includes a brief summary of the datasets, preprocessing techniques, and methods used in these studies, which were published between 2014 and 2022. This review aims to provide a comprehensive summary of the methodologies used in the current literature on neural signal-based recognition of handwriting and speech. In essence, this article is intended to serve as a valuable resource for future researchers who wish to investigate neural signal-based machine-learning methods in their work.

A novel visual brain-computer interfaces paradigm based on evoked related potentials evoked by weak and small number of stimuli

Introduction

Traditional visual Brain-Computer Interfaces (v-BCIs) usually use large-size stimuli to attract more attention from users and then elicit more distinct and robust EEG responses, which would cause visual fatigue and limit the length of use of the system. On the contrary, small-size stimuli always need multiple and repeated stimulus to code more instructions and increase separability among each code. These common v-BCIs paradigms can cause problems such as redundant coding, long calibration time, and visual fatigue.

Methods

To address these problems, this study presented a novel v-BCI paradigm using weak and small number of stimuli, and realized a nine-instruction v-BCI system that controlled by only three tiny stimuli. Each of these stimuli were located between instructions, occupied area with eccentricities subtended 0.4°, and flashed in the row-column paradigm. The weak stimuli around each instruction would evoke specific evoked related potentials (ERPs), and a template-matching method based on discriminative spatial pattern (DSP) was employed to recognize these ERPs containing the intention of users. Nine subjects participated in the offline and online experiments using this novel paradigm.

Results

The average accuracy of the offline experiment was 93.46% and the online average information transfer rate (ITR) was 120.95 bits/min. Notably, the highest online ITR achieved 177.5 bits/min.

Discussion

These results demonstrate the feasibility of using a weak and small number of stimuli to implement a friendly v-BCI. Furthermore, the proposed novel paradigm achieved higher ITR than traditional ones using ERPs as the controlled signal, which showed its superior performance and may have great potential of being widely used in various fields.

Probing inter-areal computations with a cellular resolution two-photon holographic mesoscope

Brain computation depends on intricately connected yet highly distributed neural networks. Due to the absence of the requisite technologies, causally testing fundamental hypotheses on the nature of inter-areal processing have remained largely out-of-each. Here we developed the first two photon holographic mesoscope, a system capable of simultaneously reading and writing neural activity patterns with single cell resolution across large regions of the brain. We demonstrate the precise photo-activation of spatial and temporal sequences of neurons in one brain area while reading out the downstream effect in several other regions. Investigators can use this new platform to understand feed-forward and feed-back processing in distributed neural circuits with single cell precision for the first time.

Toward higher-performance bionic limbs for wider clinical use

Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.

Chronic Stability of Local Field Potentials Using Amorphous Silicon Carbide Microelectrode Arrays Implanted in the Rat Motor Cortex

Implantable microelectrode arrays (MEAs) enable the recording of electrical activity of cortical neurons, allowing the development of brain-machine interfaces. However, MEAs show reduced recording capabilities under chronic conditions, prompting the development of novel MEAs that can improve long-term performance. Conventional planar, silicon-based devices and ultra-thin amorphous silicon carbide (a-SiC) MEAs were implanted in the motor cortex of female Sprague-Dawley rats, and weekly anesthetized recordings were made for 16 weeks after implantation. The spectral density and bandpower between 1 and 500 Hz of recordings were compared over the implantation period for both device types. Initially, the bandpower of the a-SiC devices and standard MEAs was comparable. However, the standard MEAs showed a consistent decline in both bandpower and power spectral density throughout the 16 weeks post-implantation, whereas the a-SiC MEAs showed substantially more stable performance. These differences in bandpower and spectral density between standard and a-SiC MEAs were statistically significant from week 6 post-implantation until the end of the study at 16 weeks. These results support the use of ultra-thin a-SiC MEAs to develop chronic, reliable brain-machine interfaces.

Connectivity Analysis in EEG Data: A Tutorial Review of the State of the Art and Emerging Trends

Understanding how different areas of the human brain communicate with each other is a crucial issue in neuroscience. The concepts of structural, functional and effective connectivity have been widely exploited to describe the human connectome, consisting of brain networks, their structural connections and functional interactions. Despite high-spatial-resolution imaging techniques such as functional magnetic resonance imaging (fMRI) being widely used to map this complex network of multiple interactions, electroencephalographic (EEG) recordings claim high temporal resolution and are thus perfectly suitable to describe either spatially distributed and temporally dynamic patterns of neural activation and connectivity. In this work, we provide a technical account and a categorization of the most-used data-driven approaches to assess brain-functional connectivity, intended as the study of the statistical dependencies between the recorded EEG signals. Different pairwise and multivariate, as well as directed and non-directed connectivity metrics are discussed with a pros-cons approach, in the time, frequency, and information-theoretic domains. The establishment of conceptual and mathematical relationships between metrics from these three frameworks, and the discussion of novel methodological approaches, will allow the reader to go deep into the problem of inferring functional connectivity in complex networks. Furthermore, emerging trends for the description of extended forms of connectivity (e.g., high-order interactions) are also discussed, along with graph-theory tools exploring the topological properties of the network of connections provided by the proposed metrics. Applications to EEG data are reviewed. In addition, the importance of source localization, and the impacts of signal acquisition and pre-processing techniques (e.g., filtering, source localization, and artifact rejection) on the connectivity estimates are recognized and discussed. By going through this review, the reader could delve deeply into the entire process of EEG pre-processing and analysis for the study of brain functional connectivity and learning, thereby exploiting novel methodologies and approaches to the problem of inferring connectivity within complex networks.

Improving the accuracy of decoding monkey brain-machine interface data by estimating the state of unobserved cell assemblies

BACKGROUND

The brain-machine interface is a technology that has been used for improving the quality of life of individuals with physical disabilities and also healthy individuals. It is important to improve the methods used for decoding the brain-machine interface data as the accuracy and speed of movements achieved using the existing technology are not comparable to the normal body.

COMPARISON WITH THE EXISTING METHOD

Decoding of brain-machine interface data using the proposed method resulted in improved decoding accuracy compared to the existing method.

CONCLUSIONS

The results demonstrated the usefulness of cell assembly state estimation method for decoding the brain-machine interface data.

NEW METHOD

We incorporated a novel method of estimating cell assembly states using spike trains with the existing decoding method that used only firing rate data. Synaptic connectivity pattern was used as feature values in addition to firing rate. Publicly available monkey brain-machine interface datasets were used in the study.

RESULTS

As long as the decoding was successful, the root mean square error of the proposed method was significantly smaller than the existing method. Artificial neural netowork-based decoding method resulted in more stable decoding, and also improved the decoding accuracy due to incorporation of synaptic connectivity pattern.

Grade-control outdoor turning flight of robo-pigeon with quantitative stimulus parameters

Introduction

The robo-pigeon using homing pigeons as a motion carrier has great potential in search and rescue operations due to its superior weight-bearing capacity and sustained flight capabilities. However, before deploying such robo-pigeons, it is necessary to establish a safe, stable, and long-term effective neuro-electrical stimulation interface and quantify the motion responses to various stimuli.

Methods

In this study, we investigated the effects of stimulation variables such as stimulation frequency (SF), stimulation duration (SD), and inter-stimulus interval (ISI) on the turning flight control of robo-pigeons outdoors, and evaluated the efficiency and accuracy of turning flight behavior accordingly.

Results

The results showed that the turning angle can be significantly controlled by appropriately increasing SF and SD. Increasing ISI can significantly control the turning radius of robotic pigeons. The success rate of turning flight control decreases significantly when the stimulation parameters exceed SF > 100 Hz or SD > 5 s. Thus, the robo-pigeon's turning angle from 15 to 55° and turning radius from 25 to 135 m could be controlled in a graded manner by selecting varying stimulus variables.

Discussion

These findings can be used to optimize the stimulation strategy of robo-pigeons to achieve precise control of their turning flight behavior outdoors. The results also suggest that robo-pigeons have potential for use in search and rescue operations where precise control of flight behavior is required.

Genetic algorithm based ensemble system using MLR and MsetCCA methods for SSVEP frequency recognition

BCI systems provide a direct communication channel between the human and the machine using brain signals. Among the various methods of steady-state visual evoked potential (SSVEP) stimulation frequency detection, multiple linear regression (MLR), and multiset canonical correlation analysis (MsetCCA) methods have achieved high accurate results in recent studies. The purpose of this study is to utilize both approaches and benefit from them using a genetic algorithm (GA). This algorithm leads to high-performance optimization due to its large number of regulatory parameters. Signal analysis was performed for the windows with 0.5 to 4 s duration length and with 0.5-second incremental steps. In this paper, we were able to achieve 100% accuracy of recognition for 2-second time-windows using the genetic algorithm to optimally ensemble SSVEP stimulation frequency detection methods. The accuracy of the proposed system indicates a significant improvement in detection compared to either MLR or MsetCCA alone and indicates that the ensemble system is correctly optimized using the genetic algorithm. Genetic algorithm is one of the most widely used algorithms because of its high regulatory parameters leading to its high flexibility. The improvement in detection of the proposed system is due to the use of the strengths of both two methods, and the optimal choice of the system response to visual stimuli.

Comparative study of attention-related features on attention monitoring systems with a single EEG channel

The easy-to-use attention monitoring systems usually detect the participant's attentional status via processing electroencephalogram (EEG) data recorded from a single FPz channel. But due to the influence of noises and artifacts, the attention-monitoring performance needs to be further improved to suit different individuals and devices. This paper compared the attention-related features extracted using four state-of-the-art methods including delta/beta1 (D/B1), α + β + δ + θ + R, entropy and optimized complex network (OCN). The classification performance was evaluated using receiver operating characteristic (ROC) curves and area under the ROC curves (AUC) on two EEG data acquisition devices, i.e., a BrainAmp device with high precision and a Sichiray device with low cost, respectively. Considering the varied performance on different individuals and devices, this paper proposed a novel Mutual information-based feature fusion (MIFF) method, selecting the optimal combinations of the attention-related features for classification, to enhance the attention detection performance. The experimental results showed that the proposed MIFF method outperformed the state-of-the-art methods regardless of data length on both devices. Especially, the proposed method with data length of 2.5 s achieved an average AUC of 0.8505 on the low-cost Sichiray device, which is 56.08 % higher than that of D/B1, 27.28 % higher than that of α + β + δ + θ + R, 17.42 % higher than that of entropy, and 15.48 % higher than that of OCN.

Hybrid mental tasks based human computer interface via integration of pronunciation and motor imagery

Objective.Among the existing active brain-computer interfaces (BCI), the motor imagination (MI) is widely used. To operate the MI BCI effectively, subjects need to carry out trainings on corresponding imagining tasks. Here, we studied how to reduce the discomfort and fatigue of active BCI imaginary tasks and the inability to concentrate on them while improving the accuracy.Approach.This paper proposes a hybrid BCI composed of MI and pronunciation imagination (PI). The electroencephalogram signals of ten subjects are recognized by the adaptive Riemannian distance classification and the improved frequency selective filter-bank Common Spatial Pattern recognition.Main results.The results show that under the new paradigm with the combination of MI and PI, the recognition accuracy is higher than the MI alone. The highest recognition rate of the proposed hybrid system can reach more than 90%. Furthermore, through the subjects' scoring results of the operation difficulty, it is concluded that the designed hybrid paradigm is more operable than the traditional BCI paradigm.Significance.The separable tasks in the active BCI are limited and the accuracy needs to be improved. The new hybrid paradigm proposed by us improves the accuracy and operability of the active BCI system, providing a new possibility for the research direction of the active BCI.

Brain-Computer Interface using neural network and temporal-spectral features

Brain-Computer Interfaces (BCIs) are increasingly useful for control. Such BCIs can be used to assist individuals who lost mobility or control over their limbs, for recreational purposes such as gaming or semi-autonomous driving, or as an interface toward man-machine integration. Thus far, the performance of algorithms used for thought decoding has been limited. We show that by extracting temporal and spectral features from electroencephalography (EEG) signals and, following, using deep learning neural network to classify those features, one can significantly improve the performance of BCIs in predicting which motor action was imagined by a subject. Our movement prediction algorithm uses Sequential Backward Selection technique to jointly choose temporal and spectral features and a radial basis function neural network for the classification. The method shows an average performance increase of 3.50% compared to state-of-the-art benchmark algorithms. Using two popular public datasets our algorithm reaches 90.08% accuracy (compared to an average benchmark of 79.99%) on the first dataset and 88.74% (average benchmark: 82.01%) on the second dataset. Given the high variability within- and across-subjects in EEG-based action decoding, we suggest that using features from multiple modalities along with neural network classification protocol is likely to increase the performance of BCIs across various tasks.

A multi-modal brain-computer interface based on threshold discrimination and its application in wheelchair control

In this study, we propose a novel multi-modal brain-computer interface (BCI) system based on the threshold discrimination, which is proposed for the first time to distinguish between SSVEP and MI potentials. The system combines these two heterogeneous signals to increase the number of control commands and improve the performance of asynchronous control of external devices. In this research, an electric wheelchair is controlled as an example. The user can continuously control the wheelchair to turn left/right through motion imagination (MI) by imagining left/right-hand movement and generate another 6 commands for the wheelchair control by focusing on the SSVEP stimulation panel. Ten subjects participated in a MI training session and eight of them completed a mobile obstacle-avoidance experiment in a complex environment requesting high control accuracy for successful manipulation. Comparing with the single-modal BCI-controlled wheelchair system, the results demonstrate that the proposed multi-modal method is effective by providing more satisfactory control accuracy, and show the potential of BCI-controlled systems to be applied in complex daily tasks.

An EEG-based asynchronous MI-BCI system to reduce false positives with a small number of channels for neurorehabilitation: A pilot study

Many studies have used motor imagery-based brain–computer interface (MI-BCI) systems for stroke rehabilitation to induce brain plasticity. However, they mainly focused on detecting motor imagery but did not consider the effect of false positive (FP) detection. The FP could be a threat to patients with stroke as it can induce wrong-directed brain plasticity that would result in adverse effects. In this study, we proposed a rehabilitative MI-BCI system that focuses on rejecting the FP. To this end, we first identified numerous electroencephalogram (EEG) signals as the causes of the FP, and based on the characteristics of the signals, we designed a novel two-phase classifier using a small number of EEG channels, including the source of the FP. Through experiments with eight healthy participants and nine patients with stroke, our proposed MI-BCI system showed 71.76% selectivity and 13.70% FP rate by using only four EEG channels in the patient group with stroke. Moreover, our system can compensate for day-to-day variations for prolonged session intervals by recalibration. The results suggest that our proposed system, a practical approach for the clinical setting, could improve the therapeutic effect of MI-BCI by reducing the adverse effect of the FP.

Brain Decoding of Multiple Subjects for Estimating Visual Information Based on a Probabilistic Generative Model

Brain decoding is a process of decoding human cognitive contents from brain activities. However, improving the accuracy of brain decoding remains difficult due to the unique characteristics of the brain, such as the small sample size and high dimensionality of brain activities. Therefore, this paper proposes a method that effectively uses multi-subject brain activities to improve brain decoding accuracy. Specifically, we distinguish between the shared information common to multi-subject brain activities and the individual information based on each subject's brain activities, and both types of information are used to decode human visual cognition. Both types of information are extracted as features belonging to a latent space using a probabilistic generative model. In the experiment, an publicly available dataset and five subjects were used, and the estimation accuracy was validated on the basis of a confidence score ranging from 0 to 1, and a large value indicates superiority. The proposed method achieved a confidence score of 0.867 for the best subject and an average of 0.813 for the five subjects, which was the best compared to other methods. The experimental results show that the proposed method can accurately decode visual cognition compared with other existing methods in which the shared information is not distinguished from the individual information.

Boosting the Evoked Response of Brain to Enhance the Reference Signals of CCA Method

Brain-computer interface (BCI) systems can be used to communicate and express desires from people with severe nervous system damage. Among BCI systems based on evoked responses, steady state visual evoked potential (SSVEP) responses are the most widely used. Canonical correlation analysis (CCA)-based methods have been widely used in SSVEP-based online BCIs due to their low computation and high speed, and many methods have been introduced to improve the results. In this research, a method for constructing reference signals used in CCA based on the amplified evoked response of brain is introduced. In the proposed method, after removing the latency in the training signals, to construct reference signals, multilayer perceptron neural networks of the fitting type are used instead of the usual sine/cosine signals. The results show the success of this method in boosting the evoked responses of brain. The detection accuracy in 100-second time windows was 100%, and the information transfer rate in the same period was 240 bits per minute. Making reference signals similar to the recorded electroencephalogram allowed us to make more similarities in the CCA between the signals under consideration, and the reference signals, and to dramatically improve the results.

Basal ganglia-cortical connectivity underlies self-regulation of brain oscillations in humans

Brain-computer interfaces provide an artificial link by which the brain can directly interact with the environment. To achieve fine brain-computer interface control, participants must modulate the patterns of the cortical oscillations generated from the motor and somatosensory cortices. However, it remains unclear how humans regulate cortical oscillations, the controllability of which substantially varies across individuals. Here, we performed simultaneous electroencephalography (to assess brain-computer interface control) and functional magnetic resonance imaging (to measure brain activity) in healthy participants. Self-regulation of cortical oscillations induced activity in the basal ganglia-cortical network and the neurofeedback control network. Successful self-regulation correlated with striatal activity in the basal ganglia-cortical network, through which patterns of cortical oscillations were likely modulated. Moreover, basal ganglia-cortical network and neurofeedback control network connectivity correlated with strong and weak self-regulation, respectively. The findings indicate that the basal ganglia-cortical network is important for self-regulation, the understanding of which should help advance brain-computer interface technology.

Simultaneous fMRI-EEG in 26 healthy participants indicate that the basal ganglia cortical network and the neurofeedback control network play different roles in self-regulation, providing further insight into the neural correlates for brain-machine interface control and feedback.

Cluster Kernel Reinforcement Learning-based Kalman Filter for Three-Lever Discrimination Task in Brain-Machine Interface

Brain-Machine Interface (BMI) translates paralyzed people's neural activity into control commands of the prosthesis so that their lost motor functions could be restored. The neural activities represent brain states that change continuously over time which brings the challenge to the online decoder. Reinforcement Learning (RL) has the advantage to construct the dynamic neural-kinematic mapping during the interaction. However, existing RL decoders output discrete actions as a classification problem and cannot provide continuous estimation. Previous work has combined Kalman Filter (KF) with RL for BMI, which achieves a continuous motor state estimation. However, this method adopts a neural network structure, which might get stuck in local optimum and cannot provide an efficient online update for the neural-kinematic mapping. In this paper, we propose a Cluster Kernel Reinforcement Learning-based Kalman Filter (CKRL-based KF) to avoid the local optimum problem for online neural-kinematic updating. The neural patterns are projected into Reproducing Kernel Hilbert Space (RKHS), which builds a universal approximation to guarantee the global optimum. We compare our proposed algorithm with the existing method on rat data collected during a brain control three-lever discrimination task. Our preliminary results show that the proposed method has a higher trial accuracy with lower variance across data segments, which shows its potential to improve the performance for online BMI control. Clinical Relevance- This paper provides a more stable decoding method for adaptive and continuous neural decoding. It is promising for clinical applications in BMI.

Exploring EEG spectral and temporal dynamics underlying a hand grasp movement

For brain-computer interfaces, resolving the differences between pre-movement and movement requires decoding neural ensemble activity in the motor cortex's functional regions and behavioural patterns. Here, we explored the underlying neural activity and mechanisms concerning a grasped motor task by recording electroencephalography (EEG) signals during the execution of hand movements in healthy subjects. The grasped movement included different tasks; reaching the target, grasping the target, lifting the object upwards, and moving the object in the left or right directions. 163 trials of EEG data were acquired from 30 healthy participants who performed the grasped movement tasks. Rhythmic EEG activity was analysed during the premovement (alert task) condition and compared against grasped movement tasks while the arm was moved towards the left or right directions. The short positive to negative deflection that initiated around -0.5ms as a wave before the onset of movement cue can be used as a potential biomarker to differentiate movement initiation and movement. A rebound increment of 14% of beta oscillations and 26% gamma oscillations in the central regions was observed and could be used to distinguish pre-movement and grasped movement tasks. Comparing movement initiation to grasp showed a decrease of 10% in beta oscillations and 13% in gamma oscillations, and there was a rebound increment 4% beta and 3% gamma from grasp to grasped movement. We also investigated the combination MRCPs and spectral estimates of α, β, and γ oscillations as features for machine learning classifiers that could categorize movement conditions. Support vector machines with 3rd order polynomial kernel yielded 70% accuracy. Pruning the ranked features to 5 leaf nodes reduced the error rate by 16%. For decoding grasped movement and in the context of BCI applications, this study identifies potential biomarkers, including the spatio-temporal characteristics of MRCPs, spectral information, and choice of classifiers for optimally distinguishing initiation and grasped movement.