Unsupervised neural decoding for concurrent and continuous multi-finger force prediction

Reliable prediction of multi-finger forces is crucial for neural-machine interfaces. Various neural decoding methods have progressed substantially for accurate motor output predictions. However, most neural decoding methods are performed in a supervised manner, i.e., the finger forces are needed for model training, which may not be suitable in certain contexts, especially in scenarios involving individuals with an arm amputation. To address this issue, we developed an unsupervised neural decoding approach to predict multi-finger forces using spinal motoneuron firing information. We acquired high-density surface electromyogram (sEMG) signals of the finger extensor muscle when subjects performed single-finger and multi-finger tasks of isometric extensions. We first extracted motor units (MUs) from sEMG signals of the single-finger tasks. Because of inevitable finger muscle co-activation, MUs controlling the non-targeted fingers can also be recruited. To ensure an accurate finger force prediction, these MUs need to be teased out. To this end, we clustered the decomposed MUs based on inter-MU distances measured by the dynamic time warping technique, and we then labeled the MUs using the mean firing rate or the firing rate phase amplitude. We merged the clustered MUs related to the same target finger and assigned weights based on the consistency of the MUs being retained. As a result, compared with the supervised neural decoding approach and the conventional sEMG amplitude approach, our new approach can achieve a higher R2 (0.77 ± 0.036 vs. 0.71 ± 0.11 vs. 0.61 ± 0.09) and a lower root mean square error (5.16 ± 0.58 %MVC vs. 5.88 ± 1.34 %MVC vs. 7.56 ± 1.60 %MVC). Our findings can pave the way for the development of accurate and robust neural-machine interfaces, which can significantly enhance the experience during human-robotic hand interactions in diverse contexts.

Beta bursts question the ruling power for brain-computer interfaces

Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.

Variance characteristic preserving common spatial pattern for motor imagery BCI

Introduction

The common spatial patterns (CSP) algorithm is the most popular technique for extracting electroencephalogram (EEG) features in motor imagery based brain-computer interface (BCI) systems. CSP algorithm embeds the dimensionality of multichannel EEG data to extract features of motor imagery tasks. Most previous studies focused on the optimization of the time domain and the spectrum domain of EEG signal to improve the effectiveness of CSP, whereas ignoring the constraint on the projected feature space.

Methods

This study proposed a variance characteristic preserving CSP (VPCSP) that is modified by a regularization item based on graph theory. Specifically, we calculated the loss of abnormalities of the projected data while preserving the variance characteristic locally. Then the loss could be rewritten as a matrix with the introduction of the Laplace matrix, which turned it into a generalized eigenvalue problem equivalent to CSP. This study evaluated the proposed method on two public EEG datasets from the BCI competition. The modified method could extract robust and distinguishable features that provided higher classification performance. Experimental results showed that the proposed regularization improved the effectiveness of CSP significantly and achieved superior performance compared with reported modified CSP algorithms significantly.

Results

The classification accuracy of the proposed method achieved 87.88 %, 90.07 %, and 76.06 % on public dataset IV part I, III part IVa and the self-collected dataset, respectively. Comparative experiments are conducted on two public datasets and one self-collected dataset. Results showed that the proposed method outperformed the reported algorithm.

Discussion

The proposed method can extract robust features to increase the performance of BCI systems. And the proposal still has expandability. These results show that our proposal is a promising candidate for the performance improvement of MI-BCI.

Implementation of artificial intelligence and machine learning-based methods in brain-computer interaction

Brain-computer interfaces are used for direct two-way communication between the human brain and the computer. Brain signals contain valuable information about the mental state and brain activity of the examined subject. However, due to their non-stationarity and susceptibility to various types of interference, their processing, analysis and interpretation are challenging. For these reasons, the research in the field of brain-computer interfaces is focused on the implementation of artificial intelligence, especially in five main areas: calibration, noise suppression, communication, mental condition estimation, and motor imagery. The use of algorithms based on artificial intelligence and machine learning has proven to be very promising in these application domains, especially due to their ability to predict and learn from previous experience. Therefore, their implementation within medical technologies can contribute to more accurate information about the mental state of subjects, alleviate the consequences of serious diseases or improve the quality of life of disabled patients.

Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities

Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.

Arithmetic Optimization with RetinaNet Model for Motor Imagery Classification on Brain Computer Interface

Brain Computer Interface (BCI) technology commonly used to enable communication for the person with movement disability. It allows the person to communicate and control assistive robots by the use of electroencephalogram (EEG) or other brain signals. Though several approaches have been available in the literature for learning EEG signal feature, the deep learning (DL) models need to further explore for generating novel representation of EEG features and accomplish enhanced outcomes for MI classification. With this motivation, this study designs an arithmetic optimization with RetinaNet based deep learning model for MI classification (AORNDL-MIC) technique on BCIs. The proposed AORNDL-MIC technique initially exploits Multiscale Principal Component Analysis (MSPCA) approach for the EEG signal denoising and Continuous Wavelet Transform (CWT) is exploited for the transformation of 1D-EEG signal into 2D time-frequency amplitude representation, which enables to utilize the DL model via transfer learning approach. In addition, the DL based RetinaNet is applied for extracting of feature vectors from the EEG signal which are then classified with the help of ID3 classifier. In order to optimize the classification efficiency of the AORNDL-MIC technique, arithmetical optimization algorithm (AOA) is employed for hyperparameter tuning of the RetinaNet. The experimental analysis of the AORNDL-MIC algorithm on the benchmark data sets reported its promising performance over the recent state of art methodologies.

An Impending Paradigm Shift in Motor Imagery Based Brain-Computer Interfaces

The development of reliable assistive devices for patients that suffer from motor impairments following central nervous system lesions remains a major challenge in the field of non-invasive Brain-Computer Interfaces (BCIs). These approaches are predominated by electroencephalography and rely on advanced signal processing and machine learning methods to extract neural correlates of motor activity. However, despite tremendous and still ongoing efforts, their value as effective clinical tools remains limited. We advocate that a rather overlooked research avenue lies in efforts to question neurophysiological markers traditionally targeted in non-invasive motor BCIs. We propose an alternative approach grounded by recent fundamental advances in non-invasive neurophysiology, specifically subject-specific feature extraction of sensorimotor bursts of activity recorded via (possibly magnetoencephalography-optimized) electroencephalography. This path holds promise in overcoming a significant proportion of existing limitations, and could foster the wider adoption of online BCIs in rehabilitation protocols.

Brain Computer Interfaces for Assisted Communication in Paralysis and Quality of Life

The rapid evolution of Brain-Computer Interface (BCI) technology and the exponential growth of BCI literature during the past 20 years is a consequence of increasing computational power and the achievements of statistical learning theory and machine learning since the 1960s. Despite this rapid scientific progress, the range of successful clinical and societal applications remained limited, with some notable exceptions in the rehabilitation of chronic stroke and first steps towards BCI-based assisted verbal communication in paralysis. In this contribution, we focus on the effects of noninvasive and invasive BCI-based verbal communication on the quality of life (QoL) of patients with amyotrophic lateral sclerosis (ALS) in the locked-in state (LIS) and the completely locked-in state (CLIS). Despite a substantial lack of replicated scientific data, this paper complements the existing methodological knowledge and focuses future investigators' attention on (1) Social determinants of QoL and (2) Brain reorganization and behavior. While it is not documented in controlled studies that the good QoL in these patients is a consequence of BCI-based neurorehabilitation, the proposed determinants of QoL might become the theoretical background needed to develop clinically more useful BCI systems and to evaluate the effects of BCI-based communication on QoL for advanced ALS patients and other forms of severe paralysis.

Data-Driven Approach to the Analysis of Real-Time FMRI Neurofeedback Data: Disorder-Specific Brain Synchrony in PTSD

Brain-computer interfaces (BCIs) can be used in real-time fMRI neurofeedback (rtfMRI NF) investigations to provide feedback on brain activity to enable voluntary regulation of the blood-oxygen-level dependent (BOLD) signal from localized brain regions. However, the temporal pattern of successful self-regulation is dynamic and complex. In particular, the general linear model (GLM) assumes fixed temporal model functions and misses other dynamics. We propose a novel data-driven analyses approach for rtfMRI NF using intersubject covariance (ISC) analysis. The potential of ISC was examined in a reanalysis of data from 21 healthy individuals and nine patients with post-traumatic stress-disorder (PTSD) performing up-regulation of the anterior cingulate cortex (ACC). ISC in the PTSD group differed from healthy controls in a network including the right inferior frontal gyrus (IFG). In both cohorts, ISC decreased throughout the experiment indicating the development of individual regulation strategies. ISC analyses are a promising approach to reveal novel information on the mechanisms involved in voluntary self-regulation of brain signals and thus extend the results from GLM-based methods. ISC enables a novel set of research questions that can guide future neurofeedback and neuroimaging investigations.

Multi-scale neural decoding and analysis

Objective.

Complex spatiotemporal neural activity encodes rich information related to behavior and cognition. Conventional research has focused on neural activity acquired using one of many different measurement modalities, each of which provides useful but incomplete assessment of the neural code. Multi-modal techniques can overcome tradeoffs in the spatial and temporal resolution of a single modality to reveal deeper and more comprehensive understanding of system-level neural mechanisms. Uncovering multi-scale dynamics is essential for a mechanistic understanding of brain function and for harnessing neuroscientific insights to develop more effective clinical treatment.

Approach.

We discuss conventional methodologies used for characterizing neural activity at different scales and review contemporary examples of how these approaches have been combined. Then we present our case for integrating activity across multiple scales to benefit from the combined strengths of each approach and elucidate a more holistic understanding of neural processes.

Main results.

We examine various combinations of neural activity at different scales and analytical techniques that can be used to integrate or illuminate information across scales, as well the technologies that enable such exciting studies. We conclude with challenges facing future multi-scale studies, and a discussion of the power and potential of these approaches.

Significance.

This roadmap will lead the readers toward a broad range of multi-scale neural decoding techniques and their benefits over single-modality analyses. This Review article highlights the importance of multi-scale analyses for systematically interrogating complex spatiotemporal mechanisms underlying cognition and behavior.

Feature Selection Combining Filter and Wrapper Methods for Motor-Imagery Based Brain-Computer Interfaces

Motor imagery (MI) based brain-computer interfaces help patients with movement disorders to regain the ability to control external devices. Common spatial pattern (CSP) is a popular algorithm for feature extraction in decoding MI tasks. However, due to noise and nonstationarity in electroencephalography (EEG), it is not optimal to combine the corresponding features obtained from the traditional CSP algorithm. In this paper, we designed a novel CSP feature selection framework that combines the filter method and the wrapper method. We first evaluated the importance of every CSP feature by the infinite latent feature selection method. Meanwhile, we calculated Wasserstein distance between feature distributions of the same feature under different tasks. Then, we redefined the importance of every CSP feature based on two indicators mentioned above, which eliminates half of CSP features to create a new CSP feature subspace according to the new importance indicator. At last, we designed the improved binary gravitational search algorithm (IBGSA) by rebuilding its transfer function and applied IBGSA on the new CSP feature subspace to find the optimal feature set. To validate the proposed method, we conducted experiments on three public BCI datasets and performed a numerical analysis of the proposed algorithm for MI classification. The accuracies were comparable to those reported in related studies and the presented model outperformed other methods in literature on the same underlying data.

Galvanic Vestibular Stimulation-Based Prediction Error Decoding and Channel Optimization

A significant problem in brain-computer interface (BCI) research is decoding - obtaining required information from very weak noisy electroencephalograph signals and extracting considerable information from limited data. Traditional intention decoding methods, which obtain information from induced or spontaneous brain activity, have shortcomings in terms of performance, computational expense and usage burden. Here, a new methodology called prediction error decoding was used for motor imagery (MI) detection and compared with direct intention decoding. Galvanic vestibular stimulation (GVS) was used to induce subliminal sensory feedback between the forehead and mastoids without any burden. Prediction errors were generated between the GVS-induced sensory feedback and the MI direction. The corresponding prediction error decoding of the front/back MI task was validated. A test decoding accuracy of 77.83-78.86% (median) was achieved during GVS for every 100[Formula: see text]ms interval. A nonzero weight parameter-based channel screening (WPS) method was proposed to select channels individually and commonly during GVS. When the WPS common-selected mode was compared with the WPS individual-selected mode and a classical channel selection method based on correlation coefficients (CCS), a satisfactory decoding performance of the selected channels was observed. The results indicated the positive impact of measuring common specific channels of the BCI.

Optimization of Model Training Based on Iterative Minimum Covariance Determinant In Motor-Imagery BCI

The common spatial patterns (CSP) algorithm is one of the most frequently used and effective spatial filtering methods for extracting relevant features for use in motor imagery brain-computer interfaces (MI-BCIs). However, the inherent defect of the traditional CSP algorithm is that it is highly sensitive to potential outliers, which adversely affects its performance in practical applications. In this work, we propose a novel feature optimization and outlier detection method for the CSP algorithm. Specifically, we use the minimum covariance determinant (MCD) to detect and remove outliers in the dataset, then we use the Fisher score to evaluate and select features. In addition, in order to prevent the emergence of new outliers, we propose an iterative minimum covariance determinant (IMCD) algorithm. We evaluate our proposed algorithm in terms of iteration times, classification accuracy and feature distribution using two BCI competition datasets. The experimental results show that the average classification performance of our proposed method is 12% and 22.9% higher than that of the traditional CSP method in two datasets ([Formula: see text]), and our proposed method obtains better performance in comparison with other competing methods. The results show that our method improves the performance of MI-BCI systems.

Decision-Feedback Stages Revealed by Hidden Markov Modeling of EEG

Decision response and feedback in gambling are interrelated. Different decisions lead to different ranges of feedback, which in turn influences subsequent decisions. However, the mechanism underlying the continuous decision-feedback process is still left unveiled. To fulfill this gap, we applied the hidden Markov model (HMM) to the gambling electroencephalogram (EEG) data to characterize the dynamics of this process. Furthermore, we explored the differences between distinct decision responses (i.e. choose large or small bets) or distinct feedback (i.e. win or loss outcomes) in corresponding phases. We demonstrated that the processing stages in decision-feedback process including strategy adjustment and visual information processing can be characterized by distinct brain networks. Moreover, time-varying networks showed, after decision response, large bet recruited more resources from right frontal and right center cortices while small bet was more related to the activation of the left frontal lobe. Concerning feedback, networks of win feedback showed a strong right frontal and right center pattern, while an information flow originating from the left frontal lobe to the middle frontal lobe was observed in loss feedback. Taken together, these findings shed light on general principles of natural decision-feedback and may contribute to the design of biologically inspired, participant-independent decision-feedback systems.

Phase/Amplitude Synchronization of Brain Signals During Motor Imagery BCI Tasks

In the last decade, functional connectivity (FC) has been increasingly adopted based on its ability to capture statistical dependencies between multivariate brain signals. However, the role of FC in the context of brain-computer interface applications is still poorly understood. To address this gap in knowledge, we considered a group of 20 healthy subjects during an EEG-based hand motor imagery (MI) task. We studied two well-established FC estimators, i.e. spectral- and imaginary-coherence, and we investigated how they were modulated by the MI task. We characterized the resulting FC networks by extracting the strength of connectivity of each EEG sensor and we compared the discriminant power with respect to standard power spectrum features. At the group level, results showed that while spectral-coherence based network features were increasing in the sensorimotor areas, those based on imaginary-coherence were significantly decreasing. We demonstrated that this opposite, but complementary, behavior was respectively determined by the increase in amplitude and phase synchronization between the brain signals. At the individual level, we eventually assessed the potential of these network connectivity features in a simple off-line classification scenario. Taken together, our results provide fresh insights into the oscillatory mechanisms subserving brain network changes during MI and offer new perspectives to improve BCI performance.

Detecting Connected Consciousness During Propofol-Induced Anesthesia Using EEG Based Brain Decoding

Connected consciousness refers to the state when external stimuli can enter into the stream of our consciousness experience. Emerging evidence suggests that although patients may not respond behaviorally to external stimuli during anesthesia, they may be aware of their surroundings. In this work, we investigated whether EEG based brain decoding could be used for detecting connected consciousness in the absence of behavioral responses during propofol infusion. A total of 14 subjects participated in our experiment. Subjects were asked to discriminate two types of auditory stimuli with a finger press during an ultraslow propofol infusion. We trained an EEG based brain decoding model using data collected in the awakened state using the same auditory stimuli and tested the model on data collected during the propofol infusion. The model provided a correct classification rate (CCR) of [Formula: see text]% when subjects were able to respond to the stimuli during the propofol infusion. The CCR dropped to [Formula: see text]% when subjects ceased responding and further decreased to [Formula: see text]% when we increased the propofol concentration by another 0.2 [Formula: see text]g/ml. After terminating the propofol infusion, we observed that the CCR rebounded to [Formula: see text]% before the subjects regained consciousness. With the classification results, we provided evidence that loss of consciousness is a gradual process and may progress from full consciousness to connected consciousness and then to disconnected consciousness.

Towards a mechanistic approach for the development of non-invasive brain-computer interfaces for motor rehabilitation

Brain-computer interfaces (BCIs) designed for motor rehabilitation use brain signals associated with motor-processing states to guide neuroplastic changes in a state-dependent manner. These technologies are uniquely positioned to induce targeted and functionally relevant plastic changes in the human motor nervous system. However, while several studies have shown that BCI-based neuromodulation interventions may improve motor function in patients with lesions in the central nervous system, the neurophysiological structures and processes targeted with the BCI interventions have not been identified. In this review, we first summarize current knowledge of the changes in the central nervous system associated with learning new motor skills. Then, we propose a classification of current BCI paradigms for plasticity induction and motor rehabilitation based on the expected neural plastic changes promoted. This classification proposes four paradigms based on two criteria: the plasticity induction methods and the brain states targeted. The existing evidence regarding the brain circuits and processes targeted with these different BCIs is discussed in detail. The proposed classification aims to serve as a starting point for future studies trying to elucidate the underlying plastic changes following BCI interventions.

Decoding Saccade Intention From Primate Prefrontal Cortical Local Field Potentials Using Spectral, Spatial, and Temporal Dimensionality Reduction

Most invasive Brain Computer Interfaces (iBCIs) use spike and Local Field Potentials (LFPs) from the motor or parietal cortices to decode movement intentions. It has been debated whether harvesting signals from other brain areas that encode global cognitive variables, such as the allocation of attention and eye movement goals in a variety of spatial reference frames, may improve the outcome of iBCIs. Here, we explore the ability of LFP signals, sampled from the lateral prefrontal cortex (LPFC) of macaque monkeys, to encode eye-movement intention during the pre-movement fixation period of a delayed saccade task. We use spectral dimensionality reduction to examine the spatiotemporal properties of the extracted non-rhythmic broadband activity and explore its usefulness in decoding saccade goals. The dynamics of the broadband signal in low spatial dimensions across the pre-movement fixation period uncovered saccade target separation; its discriminative potential was confirmed using support vector machine classifications. These findings reveal that broadband LFP from the LPFC can be used to decode intended saccade target location during pre-movement periods. We further provide a general workflow that can be implemented in iBCIs and it is relatively robust to the loss of spikes in individual electrodes.

Decoding Multiple Sound-Categories in the Auditory Cortex by Neural Networks: An fNIRS Study

This study aims to decode the hemodynamic responses (HRs) evoked by multiple sound-categories using functional near-infrared spectroscopy (fNIRS). The six different sounds were given as stimuli (English, non-English, annoying, nature, music, and gunshot). The oxy-hemoglobin (HbO) concentration changes are measured in both hemispheres of the auditory cortex while 18 healthy subjects listen to 10-s blocks of six sound-categories. Long short-term memory (LSTM) networks were used as a classifier. The classification accuracy was 20.38 ± 4.63% with six class classification. Though LSTM networks' performance was a little higher than chance levels, it is noteworthy that we could classify the data subject-wise without feature selections.

Brain-Computer Interfaces in Neurorecovery and Neurorehabilitation

Recent advances in brain-computer interface technology to restore and rehabilitate neurologic function aim to enable persons with disabling neurologic conditions to communicate, interact with the environment, and achieve other key activities of daily living and personal goals. Here we evaluate the principles, benefits, challenges, and future directions of brain-computer interfaces in the context of neurorehabilitation. We then explore the clinical translation of these technologies and propose an approach to facilitate implementation of brain-computer interfaces for persons with neurologic disease.

BCI learning induces core-periphery reorganization in M/EEG multiplex brain networks

Brain-computer interfaces (BCIs) constitute a promising tool for communication and control. However, mastering non-invasive closed-loop systems remains a learned skill that is difficult to develop for a non-negligible proportion of users. The involved learning process induces neural changes associated with a brain network reorganization that remains poorly understood. To address this inter-subject variability, we adopted a multilayer approach to integrate brain network properties from electroencephalographic (EEG) and magnetoencephalographic (MEG) data resulting from a four-session BCI training program followed by a group of healthy subjects. Our method gives access to the contribution of each layer to multilayer network that tends to be equal with time. We show that regardless the chosen modality, a progressive increase in the integration of somatosensory areas in the α band was paralleled by a decrease of the integration of visual processing and working memory areas in the β band. Notably, only brain network properties in multilayer network correlated with future BCI scores in the α2 band: positively in somatosensory and decision-making related areas and negatively in associative areas. Our findings cast new light on neural processes underlying BCI training. Integrating multimodal brain network properties provides new information that correlates with behavioral performance and could be considered as a potential marker of BCI learning.

Progress in Brain Computer Interface: Challenges and Opportunities

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

Concurrent Prediction of Finger Forces Based on Source Separation and Classification of Neuron Discharge Information

A reliable neural-machine interface is essential for humans to intuitively interact with advanced robotic hands in an unconstrained environment. Existing neural decoding approaches utilize either discrete hand gesture-based pattern recognition or continuous force decoding with one finger at a time. We developed a neural decoding technique that allowed continuous and concurrent prediction of forces of different fingers based on spinal motoneuron firing information. High-density skin-surface electromyogram (HD-EMG) signals of finger extensor muscle were recorded, while human participants produced isometric flexion forces in a dexterous manner (i.e. produced varying forces using either a single finger or multiple fingers concurrently). Motoneuron firing information was extracted from the EMG signals using a blind source separation technique, and each identified neuron was further classified to be associated with a given finger. The forces of individual fingers were then predicted concurrently by utilizing the corresponding motoneuron pool firing frequency of individual fingers. Compared with conventional approaches, our technique led to better prediction performances, i.e. a higher correlation ([Formula: see text] versus [Formula: see text]), a lower prediction error ([Formula: see text]% MVC versus [Formula: see text]% MVC), and a higher accuracy in finger state (rest/active) prediction ([Formula: see text]% versus [Formula: see text]%). Our decoding method demonstrated the possibility of classifying motoneurons for different fingers, which significantly alleviated the cross-talk issue of EMG recordings from neighboring hand muscles, and allowed the decoding of finger forces individually and concurrently. The outcomes offered a robust neural-machine interface that could allow users to intuitively control robotic hands in a dexterous manner.

The Influence of Visual Attention on The Performance of A Novel Tactile P300 Brain-Computer Interface with Cheeks-Stim Paradigm

Tactile P300 brain-computer interface (BCI) generally has a worse accuracy and information transfer rate (ITR) than the visual-based BCI. It may be due to the fact that human beings have a relatively poor tactile perception. This study investigated the influence of visual attention on the performance of a tactile P300 BCI. We designed our paradigms based on a novel cheeks-stim paradigm which attached the stimulators on the subject's cheeks. Two paradigms were designed as follows: a paradigm with no visual attention and another paradigm with visual attention to the target position. Eleven subjects were invited to perform the two paradigms. We also recorded and analyzed the eyeball movement data during the paradigm with visual attention to explore whether the eyeball movement would have an effect on the BCI classification. The average online accuracy was 89.09% for the paradigm with visual attention, which was significantly higher than that of the paradigm with no visual attention (70.45%). Significant difference in ITR was also found between the two paradigms ([Formula: see text]). The results demonstrated that visual attention was an effective method to improve the performance of tactile P300 BCI. Our findings suggested that it may be feasible to complete an efficient tactile BCI system by adding visual attention.

A New dispersion entropy and fuzzy logic system methodology for automated classification of dementia stages using electroencephalograms

A new EEG-based methodology is presented for differential diagnosis of the Alzheimer's disease (AD), Mild Cognitive Impairment (MCI), and healthy subjects employing the discrete wavelet transform (DWT), dispersion entropy index (DEI), a recently-proposed nonlinear measurement, and a fuzzy logic-based classification algorithm. The effectiveness and usefulness of the proposed methodology are evaluated by employing a database of measured EEG data acquired from 135 subjects, 45 MCI, 45 AD and 45 healthy subjects. The proposed methodology differentiates MCI and AD patients from HC subjects with an accuracy of 82.6-86.9%, sensitivity of 91 %, and specificity of 87 %.

Channel Selection for Optimal EEG Measurement in Motor Imagery-Based Brain-Computer Interfaces

A method for selecting electroencephalographic (EEG) signals in motor imagery-based brain-computer interfaces (MI-BCI) is proposed for enhancing the online interoperability and portability of BCI systems, as well as user comfort. The attempt is also to reduce variability and noise of MI-BCI, which could be affected by a large number of EEG channels. The relation between selected channels and MI-BCI performance is therefore analyzed. The proposed method is able to select acquisition channels common to all subjects, while achieving a performance compatible with the use of all the channels. Results are reported with reference to a standard benchmark dataset, the BCI competition IV dataset 2a. They prove that a performance compatible with the best state-of-the-art approaches can be achieved, while adopting a significantly smaller number of channels, both in two and in four tasks classification. In particular, classification accuracy is about 77-83% in binary classification with down to 6 EEG channels, and above 60% for the four-classes case when 10 channels are employed. This gives a contribution in optimizing the EEG measurement while developing non-invasive and wearable MI-based brain-computer interfaces.

Network-based brain computer interfaces: principles and applications

Brain-computer interfaces (BCIs) make possible to interact with the external environment by decoding the mental intention of individuals. BCIs can therefore be used to address basic neuroscience questions but also to unlock a variety of applications from exoskeleton control to neurofeedback (NFB) rehabilitation. In general, BCI usability critically depends on the ability to comprehensively characterize brain functioning and correctly identify the user's mental state. To this end, much of the efforts have focused on improving the classification algorithms taking into account localized brain activities as input features. Despite considerable improvement BCI performance is still unstable and, as a matter of fact, current features represent oversimplified descriptors of brain functioning. In the last decade, growing evidence has shown that the brain works as a networked system composed of multiple specialized and spatially distributed areas that dynamically integrate information. While more complex, looking at how remote brain regions functionally interact represents a grounded alternative to better describe brain functioning. Thanks to recent advances in network science, i.e. a modern field that draws on graph theory, statistical mechanics, data mining and inferential modelling, scientists have now powerful means to characterize complex brain networks derived from neuroimaging data. Notably, summary features can be extracted from these networks to quantitatively measure specific organizational properties across a variety of topological scales. In this topical review, we aim to provide the state-of-the-art supporting the development of a network theoretic approach as a promising tool for understanding BCIs and improve usability.

An empirical survey of electroencephalography-based brain-computer interfaces

Objectives

The Electroencephalogram (EEG) signal is modified using the Motor Imagery (MI) and it is utilized for patients with high motor impairments. Hence, the direct relationship between the computer and brain is termed as an EEG-based brain-computer interface (BCI). The objective of this survey is to presents an analysis of the existing distinct BCIs based on EEG.

Methods

This survey provides a detailed review of more than 60 research papers presenting the BCI-based EEG, like motor imagery-based techniques, spatial filtering-based techniques, Steady-State Visual Evoked Potential (SSVEP)-based techniques, machine learning-based techniques, Event-Related Potential (ERP)-based techniques, and online EEG-based techniques. Subsequently, the research gaps and issues of several EEG-based BCI systems are adopted to help the researchers for better future scope.

Results

An elaborative analyses as well as discussion have been provided by concerning the parameters, like evaluation metrics, year of publication, accuracy, implementation tool, and utilized datasets obtained by various techniques.

Conclusions

This survey paper exposes research topics on BCI-based EEG, which helps the researchers and scholars, who are interested in this domain.

Inter- and Intra-individual Variability in Brain Oscillations During Sports Motor Imagery

The aim of this work was to re-evaluate electrophysiological data from a previous study on motor imagery (MI) with a special focus on observed inter- and intra-individual differences. More concretely, we investigated event-related desynchronization/synchronization patterns during sports MI (playing tennis) compared with simple MI (squeezing a ball) and discovered high variability across participants. Thirty healthy volunteers were divided in two groups; the experimental group (EG) performed a physical exercise between two imagery sessions, and the control group (CG) watched a landscape movie without physical activity. We computed inter-individual differences by assessing the dissimilarities among subjects for each group, condition, time period, and frequency band. In the alpha band, we observe some clustering in the ranking of the subjects, therefore showing smaller distances than others. Moreover, in our statistical evaluation, we observed a consistency in ranking across time periods both for the EG and for the CG. For the latter, we also observed similar rankings across conditions. On the contrary, in the beta band, the ranking of the subjects was more similar for the EG across conditions and time periods than for the subjects of the CG. With this study, we would like to draw attention to variability measures instead of primarily focusing on the identification of common patterns across participants, which often do not reflect the whole neurophysiological reality.

Olfactory EEG Signal Classification Using a Trapezoid Difference-Based Electrode Sequence Hashing Approach

Olfactory-induced electroencephalogram (EEG) signal classification is of great significance in a variety of fields, such as disorder treatment, neuroscience research, multimedia applications and brain-computer interface. In this paper, a trapezoid difference-based electrode sequence hashing method is proposed for olfactory EEG signal classification. First, an N-layer trapezoid feature set whose size ratio of the top, bottom and height is 1:2:1 is constructed for each frequency band of each EEG sample. This construction is based on N optimized power-spectral-density features extracted from N real electrodes and N nonreal electrode's features. Subsequently, the N real electrodes' sequence (ES) codes of each layer of the constructed trapezoid feature set are obtained by arranging the feature values in ascending order. Finally, the nearest neighbor classification is used to find a class whose ES codes are the most similar to those of the testing sample. Thirteen-class olfactory EEG signals collected from 11 subjects are used to compare the classification performance of the proposed method with six traditional classification methods. The comparison shows that the proposed method gives average accuracy of 94.3%, Cohen's kappa value of 0.94, precision of 95.0%, and F1-measure of 94.6%, which are higher than those of the existing methods.

Improved Activity Recognition Combining Inertial Motion Sensors and Electroencephalogram Signals

Human activity recognition and neural activity analysis are the basis for human computational neureoethology research dealing with the simultaneous analysis of behavioral ethogram descriptions and neural activity measurements. Wireless electroencephalography (EEG) and wireless inertial measurement units (IMU) allow the realization of experimental data recording with improved ecological validity where the subjects can be carrying out natural activities while data recording is minimally invasive. Specifically, we aim to show that EEG and IMU data fusion allows improved human activity recognition in a natural setting. We have defined an experimental protocol composed of natural sitting, standing and walking activities, and we have recruited subjects in two sites: in-house ([Formula: see text]) and out-house ([Formula: see text]) populations with different demographics. Experimental protocol data capture was carried out with validated commercial systems. Classifier model training and validation were carried out with scikit-learn open source machine learning python package. EEG features consist of the amplitude of the standard EEG frequency bands. Inertial features were the instantaneous position of the body tracked points after a moving average smoothing to remove noise. We carry out three validation processes: a 10-fold cross-validation process per experimental protocol repetition, (b) the inference of the ethograms, and (c) the transfer learning from each experimental protocol repetition to the remaining repetitions. The in-house accuracy results were lower and much more variable than the out-house sessions results. In general, random forest was the best performing classifier model. Best cross-validation results, ethogram accuracy, and transfer learning were achieved from the fusion of EEG and IMUs data. Transfer learning behaved poorly compared to classification on the same protocol repetition, but it has accuracy still greater than 0.75 on average for the out-house data sessions. Transfer leaning accuracy among repetitions of the same subject was above 0.88 on average. Ethogram prediction accuracy was above 0.96 on average. Therefore, we conclude that wireless EEG and IMUs allow for the definition of natural experimental designs with high ecological validity toward human computational neuroethology research. The fusion of both EEG and IMUs signals improves activity and ethogram recognition.

Machine learning (ML) for the diagnosis of autism spectrum disorder (ASD) using brain imaging

Autism spectrum disorder (ASD) is a neurodevelopmental incurable disorder with a long diagnostic period encountered in the early years of life. If diagnosed early, the negative effects of this disease can be reduced by starting special education early. Machine learning (ML), an increasingly ubiquitous technology, can be applied for the early diagnosis of ASD. The aim of this study is to examine and provide a comprehensive state-of-the-art review of ML research for the diagnosis of ASD based on (a) structural magnetic resonance image (MRI), (b) functional MRI and (c) hybrid imaging techniques over the past decade. The accuracy of the studies with a large number of participants is in general lower than those with fewer participants leading to the conclusion that further large-scale studies are needed. An examination of the age of the participants shows that the accuracy of the automated diagnosis of ASD is higher at a younger age range. ML technology is expected to contribute significantly to the early and rapid diagnosis of ASD in the coming years and become available to clinicians in the near future. This review is aimed to facilitate that.

Learning in brain-computer interface control evidenced by joint decomposition of brain and behavior

OBJECTIVE

Motor imagery-based brain-computer interfaces (BCIs) use an individual's ability to volitionally modulate localized brain activity, often as a therapy for motor dysfunction or to probe causal relations between brain activity and behavior. However, many individuals cannot learn to successfully modulate their brain activity, greatly limiting the efficacy of BCI for therapy and for basic scientific inquiry. Formal experiments designed to probe the nature of BCI learning have offered initial evidence that coherent activity across spatially distributed and functionally diverse cognitive systems is a hallmark of individuals who can successfully learn to control the BCI. However, little is known about how these distributed networks interact through time to support learning.

APPROACH

Here, we address this gap in knowledge by constructing and applying a multimodal network approach to decipher brain-behavior relations in motor imagery-based brain-computer interface learning using magnetoencephalography. Specifically, we employ a minimally constrained matrix decomposition method - non-negative matrix factorization - to simultaneously identify regularized, covarying subgraphs of functional connectivity, to assess their similarity to task performance, and to detect their time-varying expression.

MAIN RESULTS

We find that learning is marked by diffuse brain-behavior relations: good learners displayed many subgraphs whose temporal expression tracked performance. Individuals also displayed marked variation in the spatial properties of subgraphs such as the connectivity between the frontal lobe and the rest of the brain, and in the temporal properties of subgraphs such as the stage of learning at which they reached maximum expression. From these observations, we posit a conceptual model in which certain subgraphs support learning by modulating brain activity in sensors near regions important for sustaining attention. To test this model, we use tools that stipulate regional dynamics on a networked system (network control theory), and find that good learners display a single subgraph whose temporal expression tracked performance and whose architecture supports easy modulation of sensors located near brain regions important for attention.

SIGNIFICANCE

The nature of our contribution to the neuroscience of BCI learning is therefore both computational and theoretical; we first use a minimally-constrained, individual specific method of identifying mesoscale structure in dynamic brain activity to show how global connectivity and interactions between distributed networks supports BCI learning, and then we use a formal network model of control to lend theoretical support to the hypothesis that these identified subgraphs are well suited to modulate attention.

EEG Connectivity Analysis Using Denoising Autoencoders for the Detection of Dyslexia

The Temporal Sampling Framework (TSF) theorizes that the characteristic phonological difficulties of dyslexia are caused by an atypical oscillatory sampling at one or more temporal rates. The LEEDUCA study conducted a series of Electroencephalography (EEG) experiments on children listening to amplitude modulated (AM) noise with slow-rythmic prosodic (0.5-1[Formula: see text]Hz), syllabic (4-8[Formula: see text]Hz) or the phoneme (12-40[Formula: see text]Hz) rates, aimed at detecting differences in perception of oscillatory sampling that could be associated with dyslexia. The purpose of this work is to check whether these differences exist and how they are related to children's performance in different language and cognitive tasks commonly used to detect dyslexia. To this purpose, temporal and spectral inter-channel EEG connectivity was estimated, and a denoising autoencoder (DAE) was trained to learn a low-dimensional representation of the connectivity matrices. This representation was studied via correlation and classification analysis, which revealed ability in detecting dyslexic subjects with an accuracy higher than 0.8, and balanced accuracy around 0.7. Some features of the DAE representation were significantly correlated ([Formula: see text]) with children's performance in language and cognitive tasks of the phonological hypothesis category such as phonological awareness and rapid symbolic naming, as well as reading efficiency and reading comprehension. Finally, a deeper analysis of the adjacency matrix revealed a reduced bilateral connection between electrodes of the temporal lobe (roughly the primary auditory cortex) in DD subjects, as well as an increased connectivity of the F7 electrode, placed roughly on Broca's area. These results pave the way for a complementary assessment of dyslexia using more objective methodologies such as EEG.

Hand Movement Activity-Based Character Input System on a Virtual Keyboard

Nowadays, gesture-based technology is revolutionizing the world and lifestyles, and the users are comfortable and care about their needs, for example, in communication, information security, the convenience of day-to-day operations and so forth. In this case, hand movement information provides an alternative way for users to interact with people, machines or robots. Therefore, this paper presents a character input system using a virtual keyboard based on the analysis of hand movements. We analyzed the signals of the accelerometer, gyroscope, and electromyography (EMG) for movement activity. We explored potential features of removing noise from input signals through the wavelet denoising technique. The envelope spectrum is used for the analysis of the accelerometer and gyroscope and cepstrum for the EMG signal. Furthermore, the support vector machine (SVM) is used to train and detect the signal to perform character input. In order to validate the proposed model, signal information is obtained from predefined gestures, that is, “double-tap”, “hold-fist”, “wave-left”, “wave-right” and “spread-finger” of different respondents for different input actions such as “input a character”, “change character”, “delete a character”, “line break”, “space character”. The experimental results show the superiority of hand gesture recognition and accuracy of character input compared to state-of-the-art systems.

Calibration time reduction through local activities estimation in motor imagery-based brain-computer interfaces

OBJECTIVE

One of the main limitations for the practical use of brain-computer interfaces (BCI) is the calibration phase. Several methods have been suggested for the truncating of this undesirable time, including various variants of the popular CSP method. In this study, we cope with the problem, using local activities estimation (LAE).

APPROACH

LAE is a spatial filtering technique that uses the EEG data of all electrodes along with their position information to emphasize the local activities. After spatial filtering by LAE, a few electrodes are selected based on physiological information. Then the features are extracted from the signal using FFT and classified by the support vector machine. In this study, the LAE is compared with CSP, RCSP, FBCSP and FBRCSP in two different electrode configurations of 118 and 64-channel.

MAIN RESULTS

The LAE outperforms CSP-based methods in all experiments using the different number of training samples. The LAE method also obtains an average classification accuracy of 84% even with a calibration time of fewer than 2 min Significance: Unlike CSP-based methods, the LAE does not use the covariance matrix, and also uses a priori physiological information. Therefore, LAE can significantly reduce the calibration time while maintaining proper accuracy. It works well even with a few training samples.

Controllability of Networks of Multiple Coupled Neural Populations: An Analytical Method for Neuromodulation's Feasibility

Neuromodulation plays a vital role in the prevention and treatment of neurological and psychiatric disorders. Neuromodulation's feasibility is a long-standing issue because it provides the necessity for neuromodulation to realize the desired purpose. A controllability analysis of neural dynamics is necessary to ensure neuromodulation's feasibility. Here, we present such a theoretical method by using the concept of controllability from the control theory that neuromodulation's feasibility can be studied smoothly. Firstly, networks of multiple coupled neural populations with different topologies are established to mathematically model complicated neural dynamics. Secondly, an analytical method composed of a linearization method, the Kalman controllable rank condition and a controllability index is applied to analyze the controllability of the established network models. Finally, the relationship between network dynamics or topological characteristic parameters and controllability is studied by using the analytical method. The proposed method provides a new idea for the study of neuromodulation's feasibility, and the results are expected to guide us to better modulate neurodynamics by optimizing network dynamics and network topology.

EEG Feature Extraction Based on a Bilevel Network: Minimum Spanning Tree and Regional Network

Feature extraction is essential for classifying different motor imagery (MI) tasks in a brain–computer interface (BCI). Although the methods of brain network analysis have been widely studied in the BCI field, these methods are limited by differences in network size, density, and standardization. To address this issue and improve classification accuracy, we propose a novel method, in which the hybrid features of the brain function based on the bilevel network are extracted. Minimum spanning tree (MST) based on electroencephalogram (EEG) signal nodes in different MIs is constructed as the first network layer to solve the global network connectivity problem. In addition, the regional network in different movement patterns is constructed as the second network layer to determine the network characteristics, which is consistent with the correspondence between limb movement patterns and cerebral cortex in neurophysiology. We attempt to apply MST to the classification of the MI EEG signals, and the bilevel network has better interpretability. Thereafter, a vector is formed by combining the MST fundamental features with the directional features of the regional network. Our method is validated using the BCI Competition IV Dataset I. Experimental results verify the feasibility of the bilevel network framework. Furthermore, the average classification performance of the proposed method reaches 89.50%, which is higher than that of other competing methods, thereby indicating that the bilevel network is effective for MI classification.

Tangent Space Features-Based Transfer Learning Classification Model for Two-Class Motor Imagery Brain–Computer Interface

The performance of a brain–computer interface (BCI) will generally improve by increasing the volume of training data on which it is trained. However, a classifier’s generalization ability is often negatively affected when highly non-stationary data are collected across both sessions and subjects. The aim of this work is to reduce the long calibration time in BCI systems by proposing a transfer learning model which can be used for evaluating unseen single trials for a subject without the need for training session data. A method is proposed which combines a generalization of the previously proposed subject-specific “multivariate empirical-mode decomposition” preprocessing technique by taking a fixed band of 8–30[Formula: see text]Hz for all four motor imagery tasks and a novel classification model which exploits the structure of tangent space features drawn from the Riemannian geometry framework, that is shared among the training data of multiple sessions and subjects. Results demonstrate comparable performance improvement across multiple subjects without subject-specific calibration, when compared with other state-of-the-art techniques.

On the Feasibility of Using an Ear-EEG to Develop an Endogenous Brain-Computer Interface

Brain-computer interface (BCI) studies based on electroencephalography (EEG) measured around the ears (ear-EEGs) have mostly used exogenous paradigms involving brain activity evoked by external stimuli. The objective of this study is to investigate the feasibility of ear-EEGs for development of an endogenous BCI system that uses self-modulated brain activity. We performed preliminary and main experiments where EEGs were measured on the scalp and behind the ears to check the reliability of ear-EEGs as compared to scalp-EEGs. In the preliminary and main experiments, subjects performed eyes-open and eyes-closed tasks, and they performed mental arithmetic (MA) and light cognitive (LC) tasks, respectively. For data analysis, the brain area was divided into four regions of interest (ROIs) (i.e., frontal, central, occipital, and ear area). The preliminary experiment showed that the degree of alpha activity increase of the ear area with eyes closed is comparable to those of other ROIs (occipital > ear > central > frontal). In the main experiment, similar event-related (de)synchronization (ERD/ERS) patterns were observed between the four ROIs during MA and LC, and all ROIs showed the mean classification accuracies above 70% required for effective binary communication (MA vs. LC) (occipital = ear = central = frontal). From the results, we demonstrated that ear-EEG can be used to develop an endogenous BCI system based on cognitive tasks without external stimuli, which allows the usability of ear-EEGs to be extended.