Motor imagery classification by means of source analysis for brain-computer interface applications

Neural Correlates of Motor/Tactile Imagery and Tactile Sensation in a BCI paradigm: A High-Density EEG Source Imaging Study

Complementary to brain-computer interface (BCI) based on motor imagery (MI) task, sensory imagery (SI) task provides a way for BCI construction using brain activity from somatosensory cortex. The underlying neurophysiological correlation between SI and MI was unclear and difficult to measure through behavior recording. In this study, we investigated the underlying neurodynamic of motor/tactile imagery and tactile sensation tasks through a high-density electroencephalogram (EEG) recording, and EEG source imaging was used to systematically explore the cortical activation differences and correlations between the tasks. In the experiment, participants were instructed to perform the left and right hand tasks in MI paradigm, sensory stimulation (SS) paradigm and SI paradigm. The statistical results demonstrated that the imagined MI and SI tasks differed from each other within ipsilateral sensorimotor scouts, frontal and right temporal areas in α bands, whereas real SS and imagined SI showed a similar activation pattern. The similarity between SS and SI may provide a way to train the BCI system, while the difference between MI and SI may provide a way to integrate the discriminative information between them to enhance BCI performance. The combination of the tasks and its underlying neurodynamic would provide a new approach for BCI designation for a wider application. BCI studies concentrate on the hybrid decoding method combining MI or SI with SS, but the underlining neurophysiological correlates between them were unclear. MI and SI differed from each other within the ipsilateral sensorimotor cortex in alpha bands. This is a first study to investigate the neurophysiological relationship between MI and SI through an EEG source imaging approach from high-density EEG recording.

Neural signatures of imaginary motivational states: desire for music, movement and social play

The literature has demonstrated the potential for detecting accurate electrical signals that correspond to the will or intention to move, as well as decoding the thoughts of individuals who imagine houses, faces or objects. This investigation examines the presence of precise neural markers of imagined motivational states through the combining of electrophysiological and neuroimaging methods. 20 participants were instructed to vividly imagine the desire to move, listen to music or engage in social activities. Their EEG was recorded from 128 scalp sites and analysed using individual standardized Low-Resolution Brain Electromagnetic Tomographies (LORETAs) in the N400 time window (400-600 ms). The activation of 1056 voxels was examined in relation to the 3 motivational states. The most active dipoles were grouped in eight regions of interest (ROI), including Occipital, Temporal, Fusiform, Premotor, Frontal, OBF/IF, Parietal, and Limbic areas. The statistical analysis revealed that all motivational imaginary states engaged the right hemisphere more than the left hemisphere. Distinct markers were identified for the three motivational states. Specifically, the right temporal area was more relevant for "Social Play", the orbitofrontal/inferior frontal cortex for listening to music, and the left premotor cortex for the "Movement" desire. This outcome is encouraging in terms of the potential use of neural indicators in the realm of brain-computer interface, for interpreting the thoughts and desires of individuals with locked-in syndrome.

Continuous tracking using deep learning-based decoding for noninvasive brain-computer interface

Brain-computer interfaces (BCI) using electroencephalography provide a noninvasive method for users to interact with external devices without the need for muscle activation. While noninvasive BCIs have the potential to improve the quality of lives of healthy and motor-impaired individuals, they currently have limited applications due to inconsistent performance and low degrees of freedom. In this study, we use deep learning (DL)-based decoders for online continuous pursuit (CP), a complex BCI task requiring the user to track an object in 2D space. We developed a labeling system to use CP data for supervised learning, trained DL-based decoders based on two architectures, including a newly proposed adaptation of the PointNet architecture, and evaluated the performance over several online sessions. We rigorously evaluated the DL-based decoders in a total of 28 human participants, and found that the DL-based models improved throughout the sessions as more training data became available and significantly outperformed a traditional BCI decoder by the last session. We also performed additional experiments to test an implementation of transfer learning by pretraining models on data from other subjects, and midsession training to reduce intersession variability. The results from these experiments showed that pretraining did not significantly improve performance, but updating the models' midsession may have some benefit. Overall, these findings support the use of DL-based decoders for improving BCI performance in complex tasks like CP, which can expand the potential applications of BCI devices and help to improve the quality of lives of healthy and motor-impaired individuals.

Generative Perturbation Network for Universal Adversarial Attacks on Brain-Computer Interfaces

Deep neural networks (DNNs) have successfully classified EEG-based brain-computer interface (BCI) systems. However, recent studies have found that well-designed input samples, known as adversarial examples, can easily fool well-performed deep neural networks model with minor perturbations undetectable by a human. This paper proposes an efficient generative model named generative perturbation network (GPN), which can generate universal adversarial examples with the same architecture for non-targeted and targeted attacks. Furthermore, the proposed model can be efficiently extended to conditionally or simultaneously generate perturbations for various targets and victim models. Our experimental evaluation demonstrates that perturbations generated by the proposed model outperform previous approaches for crafting signal-agnostic perturbations. We demonstrate that the extended network for signal-specific methods also significantly reduces generation time while performing similarly. The transferability across classification networks of the proposed method is superior to the other methods, which shows our perturbations' high level of generality.

Recognition of motor intentions from EEGs of the same upper limb by signal traceability and Riemannian geometry features

Introduction

The electroencephalographic (EEG) based on the motor imagery task is derived from the physiological electrical signal caused by the autonomous activity of the brain. Its weak potential difference changes make it easy to be overwhelmed by noise, and the EEG acquisition method has a natural limitation of low spatial resolution. These have brought significant obstacles to high-precision recognition, especially the recognition of the motion intention of the same upper limb.

Methods

This research proposes a method that combines signal traceability and Riemannian geometric features to identify six motor intentions of the same upper limb, including grasping/holding of the palm, flexion/extension of the elbow, and abduction/adduction of the shoulder. First, the EEG data of electrodes irrelevant to the task were screened out by low-resolution brain electromagnetic tomography. Subsequently, tangential spatial features are extracted by the Riemannian geometry framework in the covariance matrix estimated from the reconstructed EEG signals. The learned Riemannian geometric features are used for pattern recognition by a support vector machine with a linear kernel function.

Results

The average accuracy of the six classifications on the data set of 15 participants is 22.47%, the accuracy is 19.34% without signal traceability, the accuracy is 18.07% when the features are the filter bank common spatial pattern (FBCSP), and the accuracy is 16.7% without signal traceability and characterized by FBCSP.

Discussion

The results show that the proposed method can significantly improve the accuracy of intent recognition. In addressing the issue of temporal variability in EEG data for active Brain-Machine Interfaces, our method achieved an average standard deviation of 2.98 through model transfer on different days' data.

A novel framework for classification of two-class motor imagery EEG signals using logistic regression classification algorithm

Robotics and artificial intelligence have played a significant role in developing assistive technologies for people with motor disabilities. Brain-Computer Interface (BCI) is a communication system that allows humans to communicate with their environment by detecting and quantifying control signals produced from different modalities and translating them into voluntary commands for actuating an external device. For that purpose, classification the brain signals with a very high accuracy and minimization of the errors is of profound importance to the researchers. So in this study, a novel framework has been proposed to classify the binary-class electroencephalogram (EEG) data. The proposed framework is tested on BCI Competition IV dataset 1 and BCI Competition III dataset 4a. Artifact removal from EEG data is done through preprocessing, followed by feature extraction for recognizing discriminative information in the recorded brain signals. Signal preprocessing involves the application of independent component analysis (ICA) on raw EEG data, accompanied by the employment of common spatial pattern (CSP) and log-variance for extracting useful features. Six different classification algorithms, namely support vector machine, linear discriminant analysis, k-nearest neighbor, naïve Bayes, decision trees, and logistic regression, have been compared to classify the EEG data accurately. The proposed framework achieved the best classification accuracies with logistic regression classifier for both datasets. Average classification accuracy of 90.42% has been attained on BCI Competition IV dataset 1 for seven different subjects, while for BCI Competition III dataset 4a, an average accuracy of 95.42% has been attained on five subjects. This indicates that the model can be used in real time BCI systems and provide extra-ordinary results for 2-class Motor Imagery (MI) signals classification applications and with some modifications this framework can also be made compatible for multi-class classification in the future.

Stability of mental motor-imagery classification in EEG depends on the choice of classifier model and experiment design, but not on signal preprocessing

Introduction

Modern consciousness research has developed diagnostic tests to improve the diagnostic accuracy of different states of consciousness via electroencephalography (EEG)-based mental motor imagery (MI), which is still challenging and lacks a consensus on how to best analyse MI EEG-data. An optimally designed and analyzed paradigm must detect command-following in all healthy individuals, before it can be applied in patients, e.g., for the diagnosis of disorders of consciousness (DOC).

Methods

We investigated the effects of two important steps in the raw signal preprocessing on predicting participant performance (F1) and machine-learning classifier performance (area-under-curve, AUC) in eight healthy individuals, that are based solely on MI using high-density EEG (HD-EEG): artifact correction (manual correction with vs. without Independent Component Analysis [ICA]), region of interest (ROI; motor area vs. whole brain), and machine-learning algorithm (support-vector machine [SVM] vs. k-nearest neighbor [KNN]).

Results

Results revealed no significant effects of artifact correction and ROI on predicting participant performance (F1) and classifier performance (AUC) scores (all ps > 0.05) in the SVM classification model. In the KNN model, ROI had a significant influence on the classifier performance [F(1,8.939) = 7.585, p = 0.023]. There was no evidence for artifact correction and ROI selection changing the prediction of participants performance and classifier performance in EEG-based mental MI if using SVM-based classification (71-100% correct classifications across different signal preprocessing methods). The variance in the prediction of participant performance was significantly higher when the experiment started with a resting-state compared to a mental MI task block [X2(1) = 5.849, p = 0.016].

Discussion

Overall, we could show that classification is stable across different modes of EEG signal preprocessing when using SVM models. Exploratory analysis gave a hint toward potential effects of the sequence of task execution on the prediction of participant performance, which should be taken into account in future studies.

Transfer Learning in Trajectory Decoding: Sensor or Source Space?

In this study, across-participant and across-session transfer learning was investigated to minimize the calibration time of the brain-computer interface (BCI) system in the context of continuous hand trajectory decoding. We reanalyzed data from a study with 10 able-bodied participants across three sessions. A leave-one-participant-out (LOPO) model was utilized as a starting model. Recursive exponentially weighted partial least squares regression (REW-PLS) was employed to overcome the memory limitation due to the large pool of training data. We considered four scenarios: generalized with no update (Gen), generalized with cumulative update (GenC), and individual models with cumulative (IndC) and non-cumulative (Ind) updates, with each one trained with sensor-space features or source-space features. The decoding performance in generalized models (Gen and GenC) was lower than the chance level. In individual models, the cumulative update (IndC) showed no significant improvement over the non-cumulative model (Ind). The performance showed the decoder's incapability to generalize across participants and sessions in this task. The results suggested that the best correlation could be achieved with the sensor-space individual model, despite additional anatomical information in the source-space features. The decoding pattern showed a more localized pattern around the precuneus over three sessions in Ind models.

Applying Dimensionality Reduction Techniques in Source-Space Electroencephalography via Template and Magnetic Resonance Imaging-Derived Head Models to Continuously Decode Hand Trajectories

Several studies showed evidence supporting the possibility of hand trajectory decoding from low-frequency electroencephalography (EEG). However, the decoding in the source space via source localization is scarcely investigated. In this study, we tried to tackle the problem of collinearity due to the higher number of signals in the source space by two folds: first, we selected signals in predefined regions of interest (ROIs); second, we applied dimensionality reduction techniques to each ROI. The dimensionality reduction techniques were computing the mean (Mean), principal component analysis (PCA), and locality preserving projections (LPP). We also investigated the effect of decoding between utilizing a template head model and a subject-specific head model during the source localization. The results indicated that applying source-space decoding with PCA yielded slightly higher correlations and signal-to-noise (SNR) ratios than the sensor-space approach. We also observed slightly higher correlations and SNRs when applying the subject-specific head model than the template head model. However, the statistical tests revealed no significant differences between the source-space and sensor-space approaches and no significant differences between subject-specific and template head models. The decoder with Mean and PCA utilizes information mainly from precuneus and cuneus to decode the velocity kinematics similarly in the subject-specific and template head models.

A Novel Two-Stage Refine Filtering Method for EEG-Based Motor Imagery Classification

Cerebral stroke is a common disease across the world, and it is a promising method to recognize the intention of stroke patients with the help of brain-computer interface (BCI). In the field of motor imagery (MI) classification, appropriate filtering is vital for feature extracting of electroencephalogram (EEG) signals and consequently influences the accuracy of MI classification. In this case, a novel two-stage refine filtering method was proposed, inspired by Gradient-weighted Class Activation Mapping (Grad-CAM), which uses the gradients of any target concept flowing into the final convolutional layer to highlight the important part of training data for predicting the concept. In the first stage, MI classification was carried out and then the frequency band to be filtered was calculated according to the Grad-CAM of the MI classification results. In the second stage, EEG was filtered and classified for a higher classification accuracy. To evaluate the filtering effect, this method was applied to the multi-branch neural network proposed in our previous work. Experiment results revealed that the proposed method reached state-of-the-art classification kappa value levels and acquired at least 3% higher kappa values than other methods This study also proposed some promising application scenarios with this filtering method.

A novel approach of CNN for human motor imagery recognition using the virtual electrode pairs

In order to develop an efficient brain-computer interface system, the brain activity measured by electroencephalography needs to be accurately decoded. In this paper, a motor imagery classification approach is proposed, combining virtual electrodes on the cortex layer with a convolutional neural network; this can effectively improve the decoding performance of the brain-computer interface system. A three layer (cortex, skull, and scalp) head volume conduction model was established by using the symmetric boundary element method to map the scalp signal to the cortex area. Nine pairs of virtual electrodes were created on the cortex layer, and the features of the time and frequency sequence from the virtual electrodes were extracted by performing time-frequency analysis. Finally, the convolutional neural network was used to classify motor imagery tasks. The results show that the proposed approach is convergent in both the training model and the test model. Based on the Physionet motor imagery database, the averaged accuracy can reach 98.32% for a single subject, while the averaged values of accuracy, Kappa, precision, recall, and F1-score on the group-wise are 96.23%, 94.83%, 96.21%, 96.13%, and 96.14%, respectively. Based on the High Gamma database, the averaged accuracy has achieved 96.37% and 91.21% at the subject and group levels, respectively. Moreover, this approach is superior to those of other studies on the same database, which suggests robustness and adaptability to individual variability.

Progress in Brain Computer Interface: Challenges and Opportunities

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

Chaotic behaviour of EEG responses with an identical grasp posture

Individuals with severe neuromuscular ailments can benefit from restoring their grasp activities with a brain-controlled upper-limb neuroprosthesis. EEG signals can be utilized as the driving source, and to implement natural human-like grasping abilities. Although good accuracy has already been achieved in classifying the various grasp patterns for specific sets of objects, unseen objects are still a hurdle in real-life implementation. Generalization of grasp patterns should be explored without any prior knowledge of the objects. In this regard, the similarity of motor imagery for different objects requiring similar grasp pattern can be utilized. It is also necessary to identify the brain regions that exhibit prominent distinguishability during different grasp patterns. In this study, we propose a chaos-based method to decode the motor imagery of two quite similar Power grasp patterns-cylindrical and spherical-for holding various objects. Three distinct suitable objects were chosen for each of the two patterns, and a 29-channel EEG was taken of 18 healthy participants to explore motor imagery for grasping the objects. Nonlinear correlation dimension was employed on the EEG data, at sub-band levels α, upper β, and γ, to analyse the distinguishability, as well as the similarity of grasp patterns for the objects. ANOVA was subsequently performed on the obtained CD parameters to identify the contribution of each electrode channel. Furthermore, using an SVM classifier, more than 80% accuracy was obtained in classifying the grasping patterns at the upper β sub-band. The outcome may lead to identification of optimum feature sets of motor imagery from specific brain regions for random objects grasps.

A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN

OBJECTIVE

To develop and implement a novel approach which combines the technique of scout EEG source imaging (ESI) with convolutional neural network (CNN) for the classification of motor imagery (MI) tasks.

APPROACH

The technique of ESI uses a boundary element method (BEM) and weighted minimum norm estimation (WMNE) to solve the EEG forward and inverse problems, respectively. Ten scouts are then created within the motor cortex to select the region of interest (ROI). We extract features from the time series of scouts using a Morlet wavelet approach. Lastly, CNN is employed for classifying MI tasks.

MAIN RESULTS

The overall mean accuracy on the Physionet database reaches 94.5% and the individual accuracy of each task reaches 95.3%, 93.3%, 93.6%, 96% for the left fist, right fist, both fists and both feet, correspondingly, validated using ten-fold cross validation. We report an increase of up to 14.4% for overall classification compared with the competitive results from the state-of-the-art MI classification methods. Then, we add four new subjects to verify the validity of the method and the overall mean accuracy is 92.5%. Furthermore, the global classifier was adapted to single subjects improving the overall mean accuracy to 94.54%.

SIGNIFICANCE

The combination of scout ESI and CNN enhances BCI performance of decoding EEG four-class MI tasks.

Hand, foot and lip representations in primary sensorimotor cortex: a high-density electroencephalography study

The primary sensorimotor cortex plays a major role in the execution of movements of the contralateral side of the body. The topographic representation of different body parts within this brain region is commonly investigated through functional magnetic resonance imaging (fMRI). However, fMRI does not provide direct information about neuronal activity. In this study, we used high-density electroencephalography (hdEEG) to map the representations of hand, foot, and lip movements in the primary sensorimotor cortex, and to study their neural signatures. Specifically, we assessed the event-related desynchronization (ERD) in the cortical space. We found that the performance of hand, foot, and lip movements elicited an ERD in beta and gamma frequency bands. The primary regions showing significant beta- and gamma-band ERD for hand and foot movements, respectively, were consistent with previously reported using fMRI. We observed relatively weaker ERD for lip movements, which may be explained by the fact that less fine movement control was required. Overall, our study demonstrated that ERD based on hdEEG data can support the study of motor-related neural processes, with relatively high spatial resolution. An interesting avenue may be the use of hdEEG for deeper investigations into the pathophysiology of neuromotor disorders.

Dynamic Activation Patterns of the Motor Brain Revealed by Diffuse Optical Tomography. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019;2019:6028-6031. [PMID: 31947220 DOI: 10.1109/embc.2019.8857370]

Diffuse optical tomography (DOT), a subset of functional near-infrared spectroscopy (fNIRS), is a noninvasive functional imaging modality for studying the human brain in normal and diseased conditions. It measures changes in concentrations of oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (Hb) in the blood vasculature of the brain. In contrast to functional magnetic resonance imaging (fMRI), the gold standard in human brain imaging, DOT offers the advantage of higher temporal resolution, portability, lower cost, multiple contrasts and usability for persons who cannot otherwise utilize MRI-based imaging modalities, including bedridden patients and infants, etc. The goal of the present study was to evaluate performance of a DOT method in studying dynamic patterns of brain activations involving motor control. CW-fNIRS data were acquired in four sessions from a healthy male participant when he performed a motor task in a block-design experiment. Results from experimental data showed pronounced activity in the primary motor cortex (M1), contralateral to the clenching hand. It was further observed that the M1 activity was consistent over four sessions. Furthermore, temporal dynamics of motor activity at each session further revealed well-sequenced activation patterns among M1, premotor cortex (PMC), and supplementary motor area (SMA). Timed ipsilateral motor activity suppression was also observed several seconds after the onset of contralateral M1 activity. More importantly, these temporal dynamics were similarly observed in all four sessions. These preliminary results suggest that the DOT method has the sensitivity, reliability, and spatio-temporal resolutions to study activities originated from the motor cortices. A full-scope evaluation and validation in more participants on the motor system can establish it as a promising neuroimaging tool to study, such as, infants at the risk of cerebral palsy or elders with Parkinson's due to its portability and usability in clinical environments.

Noninvasive neuroimaging enhances continuous neural tracking for robotic device control

Brain-computer interfaces (BCIs) utilizing signals acquired with intracortical implants have achieved successful high-dimensional robotic device control useful for completing daily tasks. However, the substantial amount of medical and surgical expertise required to correctly implant and operate these systems significantly limits their use beyond a few clinical cases. A noninvasive counterpart requiring less intervention that can provide high-quality control would profoundly impact the integration of BCIs into the clinical and home setting. Here, we present and validate a noninvasive framework utilizing electroencephalography (EEG) to achieve the neural control of a robotic device for continuous random target tracking. This framework addresses and improves upon both the "brain" and "computer" components by respectively increasing user engagement through a continuous pursuit task and associated training paradigm, and the spatial resolution of noninvasive neural data through EEG source imaging. In all, our unique framework enhanced BCI learning by nearly 60% for traditional center-out tasks and by over 500% in the more realistic continuous pursuit task. We further demonstrated an additional enhancement in BCI control of almost 10% by using online noninvasive neuroimaging. Finally, this framework was deployed in a physical task, demonstrating a near seamless transition from the control of an unconstrained virtual cursor to the real-time control of a robotic arm. Such combined advances in the quality of neural decoding and the practical utility of noninvasive robotic arm control will have major implications on the eventual development and implementation of neurorobotics by means of noninvasive BCI.

Exploring spatial-frequency-sequential relationships for motor imagery classification with recurrent neural network

BACKGROUND

Conventional methods of motor imagery brain computer interfaces (MI-BCIs) suffer from the limited number of samples and simplified features, so as to produce poor performances with spatial-frequency features and shallow classifiers.

METHODS

Alternatively, this paper applies a deep recurrent neural network (RNN) with a sliding window cropping strategy (SWCS) to signal classification of MI-BCIs. The spatial-frequency features are first extracted by the filter bank common spatial pattern (FB-CSP) algorithm, and such features are cropped by the SWCS into time slices. By extracting spatial-frequency-sequential relationships, the cropped time slices are then fed into RNN for classification. In order to overcome the memory distractions, the commonly used gated recurrent unit (GRU) and long-short term memory (LSTM) unit are applied to the RNN architecture, and experimental results are used to determine which unit is more suitable for processing EEG signals.

RESULTS

Experimental results on common BCI benchmark datasets show that the spatial-frequency-sequential relationships outperform all other competing spatial-frequency methods. In particular, the proposed GRU-RNN architecture achieves the lowest misclassification rates on all BCI benchmark datasets.

CONCLUSION

By introducing spatial-frequency-sequential relationships with cropping time slice samples, the proposed method gives a novel way to construct and model high accuracy and robustness MI-BCIs based on limited trials of EEG signals.

Empirical Comparison of Distributed Source Localization Methods for Single-Trial Detection of Movement Preparation

The development of technologies for the treatment of movement disorders, like stroke, is still of particular interest in brain-computer interface (BCI) research. In this context, source localization methods (SLMs), that reconstruct the cerebral origin of brain activity measured outside the head, e.g., via electroencephalography (EEG), can add a valuable insight into the current state and progress of the treatment. However, in BCIs SLMs were often solely considered as advanced signal processing methods that are compared against other methods based on the classification performance alone. Though, this approach does not guarantee physiological meaningful results. We present an empirical comparison of three established distributed SLMs with the aim to use one for single-trial movement prediction. The SLMs wMNE, sLORETA, and dSPM were applied on data acquired from eight subjects performing voluntary arm movements. Besides the classification performance as quality measure, a distance metric was used to asses the physiological plausibility of the methods. For the distance metric, which is usually measured to the source position of maximum activity, we further propose a variant based on clusters that is better suited for the single-trial case in which several sources are likely and the actual maximum is unknown. The two metrics showed different results. The classification performance revealed no significant differences across subjects, indicating that all three methods are equally well-suited for single-trial movement prediction. On the other hand, we obtained significant differences in the distance measure, favoring wMNE even after correcting the distance with the number of reconstructed clusters. Further, distance results were inconsistent with the traditional method using the maximum, indicating that for wMNE the point of maximum source activity often did not coincide with the nearest activation cluster. In summary, the presented comparison might help users to select an appropriate SLM and to understand the implications of the selection. The proposed methodology pays attention to the particular properties of distributed SLMs and can serve as a framework for further comparisons.

Creamino: A Cost-Effective, Open-Source EEG-Based BCI System

This paper presents an open source framework called Creamino. It consists of an Arduino-based cost-effective quick-setup EEG platform built with off-the-shelf components and a set of software modules that easily allow users to connect this system to Simulink or BCI-oriented tools (such as BCI2000 or OpenViBE) and set up a wide number of neuroscientific experiments. Creamino is capable of processing multiple EEG channels in real-time and operates under Windows, Linux, and Mac OS X in real-time on a standard PC. Its objective is to provide a system that can be readily fabricated and used for neurophysiological experiments and, at the same time, can serve as the basis for development of novel BCI platforms by accessing and modifying its open source hardware and software libraries. Schematics, gerber files, bill of materials, source code, software modules, demonstration videos, and instructions on how to use these modules are available free of charge for research and educational purposes online at https://github.com/ArcesUnibo/creamino. Application cases show how the system can be used for neuroscientific or BCI experiments. Thanks to its low production cost and its compatibility with open-source BCI tools, the system presented is particularly suitable for use in BCI research and educational applications.

Electrophysiological Source Imaging: A Noninvasive Window to Brain Dynamics

Brain activity and connectivity are distributed in the three-dimensional space and evolve in time. It is important to image brain dynamics with high spatial and temporal resolution. Electroencephalography (EEG) and magnetoencephalography (MEG) are noninvasive measurements associated with complex neural activations and interactions that encode brain functions. Electrophysiological source imaging estimates the underlying brain electrical sources from EEG and MEG measurements. It offers increasingly improved spatial resolution and intrinsically high temporal resolution for imaging large-scale brain activity and connectivity on a wide range of timescales. Integration of electrophysiological source imaging and functional magnetic resonance imaging could further enhance spatiotemporal resolution and specificity to an extent that is not attainable with either technique alone. We review methodological developments in electrophysiological source imaging over the past three decades and envision its future advancement into a powerful functional neuroimaging technology for basic and clinical neuroscience applications.

Functional Brain Connectivity during Multiple Motor Imagery Tasks in Spinal Cord Injury

Reciprocal communication of the central and peripheral nervous systems is compromised during spinal cord injury due to neurotrauma of ascending and descending pathways. Changes in brain organization after spinal cord injury have been associated with differences in prognosis. Changes in functional connectivity may also serve as injury biomarkers. Most studies on functional connectivity have focused on chronic complete injury or resting-state condition. In our study, ten right-handed patients with incomplete spinal cord injury and ten age- and gender-matched healthy controls performed multiple visual motor imagery tasks of upper extremities and walking under high-resolution electroencephalography recording. Directed transfer function was used to study connectivity at the cortical source space between sensorimotor nodes. Chronic disruption of reciprocal communication in incomplete injury could result in permanent significant decrease of connectivity in a subset of the sensorimotor network, regardless of positive or negative neurological outcome. Cingulate motor areas consistently contributed the larger outflow (right) and received the higher inflow (left) among all nodes, across all motor imagery categories, in both groups. Injured subjects had higher outflow from left cingulate than healthy subjects and higher inflow in right cingulate than healthy subjects. Alpha networks were less dense, showing less integration and more segregation than beta networks. Spinal cord injury patients showed signs of increased local processing as adaptive mechanism. This trial is registered with NCT02443558.

Real-Time Clustered Multiple Signal Classification (RTC-MUSIC)

Magnetoencephalography (MEG) and electroencephalography provide a high temporal resolution, which allows estimation of the detailed time courses of neuronal activity. However, in real-time analysis of these data two major challenges must be handled: the low signal-to-noise ratio (SNR) and the limited time available for computations. In this work, we present real-time clustered multiple signal classification (RTC-MUSIC) a real-time source localization algorithm, which can handle low SNRs and can reduce the computational effort. It provides correlation information together with sparse source estimation results, which can, e.g., be used to identify evoked responses with high sensitivity. RTC-MUSIC clusters the forward solution based on an anatomical brain atlas and optimizes the scanning process inherent to MUSIC approaches. We evaluated RTC-MUSIC by analyzing MEG auditory and somatosensory data. The results demonstrate that the proposed method localizes sources reliably. For the auditory experiment the most dominant correlated source pair was located bilaterally in the superior temporal gyri. The highest activation in the somatosensory experiment was found in the contra-lateral primary somatosensory cortex.

Anodal Transcranial Direct Current Stimulation Increases Bilateral Directed Brain Connectivity during Motor-Imagery Based Brain-Computer Interface Control

Transcranial direct current stimulation (tDCS) has been shown to affect motor and cognitive task performance and learning when applied to brain areas involved in the task. Targeted stimulation has also been found to alter connectivity within the stimulated hemisphere during rest. However, the connectivity effect of the interaction of endogenous task specific activity and targeted stimulation is unclear. This study examined the aftereffects of concurrent anodal high-definition tDCS over the left sensorimotor cortex with motor network connectivity during a one-dimensional EEG based sensorimotor rhythm brain-computer interface (SMR-BCI) task. Directed connectivity following anodal tDCS illustrates altered connections bilaterally between frontal and parietal regions, and these alterations occur in a task specific manner; connections between similar cortical regions are altered differentially during left and right imagination trials. During right-hand imagination following anodal tDCS, there was an increase in outflow from the left premotor cortex (PMC) to multiple regions bilaterally in the motor network and increased inflow to the stimulated sensorimotor cortex from the ipsilateral PMC and contralateral sensorimotor cortex. During left-hand imagination following anodal tDCS, there was increased outflow from the stimulated sensorimotor cortex to regions across the motor network. Significant correlations between connectivity and the behavioral measures of total correct trials and time-to-hit (TTH) correct trials were also found, specifically that the input to the left PMC correlated with decreased right hand imagination performance and that flow from the ipsilateral posterior parietal cortex (PPC) to midline sensorimotor cortex correlated with improved performance for both right and left hand imagination. These results indicate that tDCS interacts with task-specific endogenous activity to alter directed connectivity during SMR-BCI. In order to predict and maximize the targeted effect of tDCS, the interaction of stimulation with the dynamics of endogenous activity needs to be examined comprehensively and understood.

Adaptive Spatial Filter Based on Similarity Indices to Preserve the Neural Information on EEG Signals during On-Line Processing

This work presents a new on-line adaptive filter, which is based on a similarity analysis between standard electrode locations, in order to reduce artifacts and common interferences throughout electroencephalography (EEG) signals, but preserving the useful information. Standard deviation and Concordance Correlation Coefficient (CCC) between target electrodes and its correspondent neighbor electrodes are analyzed on sliding windows to select those neighbors that are highly correlated. Afterwards, a model based on CCC is applied to provide higher values of weight to those correlated electrodes with lower similarity to the target electrode. The approach was applied to brain computer-interfaces (BCIs) based on Canonical Correlation Analysis (CCA) to recognize 40 targets of steady-state visual evoked potential (SSVEP), providing an accuracy (ACC) of 86.44 ± 2.81%. In addition, also using this approach, features of low frequency were selected in the pre-processing stage of another BCI to recognize gait planning. In this case, the recognition was significantly (p<0.01) improved for most of the subjects (ACC≥74.79%), when compared with other BCIs based on Common Spatial Pattern, Filter Bank-Common Spatial Pattern, and Riemannian Geometry.

A brain-computer interface driven by imagining different force loads on a single hand: an online feasibility study

Background

Motor imagery (MI) induced EEG patterns are widely used as control signals for brain-computer interfaces (BCIs). Kinetic and kinematic factors have been proved to be able to change EEG patterns during motor execution and motor imagery. However, to our knowledge, there is still no literature reporting an effective online MI-BCI using kinetic factor regulated EEG oscillations. This study proposed a novel MI-BCI paradigm in which users can online output multiple commands by imagining clenching their right hand with different force loads.

Methods

Eleven subjects participated in this study. During the experiment, they were asked to imagine clenching their right hands with two different force loads (30% maximum voluntary contraction (MVC) and 10% MVC). Multi-Common spatial patterns (Multi-CSPs) and support vector machines (SVMs) were used to build the classifier for recognizing three commands corresponding to high load MI, low load MI and relaxed status respectively. EMG were monitored to avoid voluntary muscle activities during the BCI operation. The event-related spectral perturbation (ERSP) method was used to analyse EEG variation during multiple load MI tasks.

Results

All subjects were able to drive BCI systems using motor imagery of different force loads in online experiments. We achieved an average online accuracy of 70.9%, with the highest accuracy of 83.3%, which was much higher than the chance level (33%). The event-related desynchronization (ERD) phenomenon during high load tasks was significantly higher than it was during low load tasks both in terms of intensity at electrode positions C3 (p < 0.05) and spatial distribution.

Conclusions

This paper demonstrated the feasibility of the proposed MI-BCI paradigm based on multi-force loads on the same limb through online studies. This paradigm could not only enlarge the command set of MI-BCI, but also provide a promising approach to rehabilitate patients with motor disabilities.

EEG-Based Brain-Computer Interface for Decoding Motor Imagery Tasks within the Same Hand Using Choi-Williams Time-Frequency Distribution

This paper presents an EEG-based brain-computer interface system for classifying eleven motor imagery (MI) tasks within the same hand. The proposed system utilizes the Choi-Williams time-frequency distribution (CWD) to construct a time-frequency representation (TFR) of the EEG signals. The constructed TFR is used to extract five categories of time-frequency features (TFFs). The TFFs are processed using a hierarchical classification model to identify the MI task encapsulated within the EEG signals. To evaluate the performance of the proposed approach, EEG data were recorded for eighteen intact subjects and four amputated subjects while imagining to perform each of the eleven hand MI tasks. Two performance evaluation analyses, namely channel- and TFF-based analyses, are conducted to identify the best subset of EEG channels and the TFFs category, respectively, that enable the highest classification accuracy between the MI tasks. In each evaluation analysis, the hierarchical classification model is trained using two training procedures, namely subject-dependent and subject-independent procedures. These two training procedures quantify the capability of the proposed approach to capture both intra- and inter-personal variations in the EEG signals for different MI tasks within the same hand. The results demonstrate the efficacy of the approach for classifying the MI tasks within the same hand. In particular, the classification accuracies obtained for the intact and amputated subjects are as high as 88.8% and 90.2%, respectively, for the subject-dependent training procedure, and 80.8% and 87.8%, respectively, for the subject-independent training procedure. These results suggest the feasibility of applying the proposed approach to control dexterous prosthetic hands, which can be of great benefit for individuals suffering from hand amputations.

As above, so below? Towards understanding inverse models in BCI

OBJECTIVE

In brain-computer interfaces (BCI), measurements of the user's brain activity are classified into commands for the computer. With EEG-based BCIs, the origins of the classified phenomena are often considered to be spatially localized in the cortical volume and mixed in the EEG. We investigate if more accurate BCIs can be obtained by reconstructing the source activities in the volume.

APPROACH

We contrast the physiology-driven source reconstruction with data-driven representations obtained by statistical machine learning. We explain these approaches in a common linear dictionary framework and review the different ways to obtain the dictionary parameters. We consider the effect of source reconstruction on some major difficulties in BCI classification, namely information loss, feature selection and nonstationarity of the EEG.

MAIN RESULTS

Our analysis suggests that the approaches differ mainly in their parameter estimation. Physiological source reconstruction may thus be expected to improve BCI accuracy if machine learning is not used or where it produces less optimal parameters. We argue that the considered difficulties of surface EEG classification can remain in the reconstructed volume and that data-driven techniques are still necessary. Finally, we provide some suggestions for comparing approaches.

SIGNIFICANCE

The present work illustrates the relationships between source reconstruction and machine learning-based approaches for EEG data representation. The provided analysis and discussion should help in understanding, applying, comparing and improving such techniques in the future.

A New Generation of Brain-Computer Interfaces Driven by Discovery of Latent EEG-fMRI Linkages Using Tensor Decomposition

A Brain-Computer Interface (BCI) is a setup permitting the control of external devices by decoding brain activity. Electroencephalography (EEG) has been extensively used for decoding brain activity since it is non-invasive, cheap, portable, and has high temporal resolution to allow real-time operation. Due to its poor spatial specificity, BCIs based on EEG can require extensive training and multiple trials to decode brain activity (consequently slowing down the operation of the BCI). On the other hand, BCIs based on functional magnetic resonance imaging (fMRI) are more accurate owing to its superior spatial resolution and sensitivity to underlying neuronal processes which are functionally localized. However, due to its relatively low temporal resolution, high cost, and lack of portability, fMRI is unlikely to be used for routine BCI. We propose a new approach for transferring the capabilities of fMRI to EEG, which includes simultaneous EEG/fMRI sessions for finding a mapping from EEG to fMRI, followed by a BCI run from only EEG data, but driven by fMRI-like features obtained from the mapping identified previously. Our novel data-driven method is likely to discover latent linkages between electrical and hemodynamic signatures of neural activity hitherto unexplored using model-driven methods, and is likely to serve as a template for a novel multi-modal strategy wherein cross-modal EEG-fMRI interactions are exploited for the operation of a unimodal EEG system, leading to a new generation of EEG-based BCIs.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Remembered or Forgotten?-An EEG-Based Computational Prediction Approach

Prediction of memory performance (remembered or forgotten) has various potential applications not only for knowledge learning but also for disease diagnosis. Recently, subsequent memory effects (SMEs)—the statistical differences in electroencephalography (EEG) signals before or during learning between subsequently remembered and forgotten events—have been found. This finding indicates that EEG signals convey the information relevant to memory performance. In this paper, based on SMEs we propose a computational approach to predict memory performance of an event from EEG signals. We devise a convolutional neural network for EEG, called ConvEEGNN, to predict subsequently remembered and forgotten events from EEG recorded during memory process. With the ConvEEGNN, prediction of memory performance can be achieved by integrating two main stages: feature extraction and classification. To verify the proposed approach, we employ an auditory memory task to collect EEG signals from scalp electrodes. For ConvEEGNN, the average prediction accuracy was 72.07% by using EEG data from pre-stimulus and during-stimulus periods, outperforming other approaches. It was observed that signals from pre-stimulus period and those from during-stimulus period had comparable contributions to memory performance. Furthermore, the connection weights of ConvEEGNN network can reveal prominent channels, which are consistent with the distribution of SME studied previously.

Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks

Brain-computer interface (BCI) technologies aim to provide a bridge between the human brain and external devices. Prior research using non-invasive BCI to control virtual objects, such as computer cursors and virtual helicopters, and real-world objects, such as wheelchairs and quadcopters, has demonstrated the promise of BCI technologies. However, controlling a robotic arm to complete reach-and-grasp tasks efficiently using non-invasive BCI has yet to be shown. In this study, we found that a group of 13 human subjects could willingly modulate brain activity to control a robotic arm with high accuracy for performing tasks requiring multiple degrees of freedom by combination of two sequential low dimensional controls. Subjects were able to effectively control reaching of the robotic arm through modulation of their brain rhythms within the span of only a few training sessions and maintained the ability to control the robotic arm over multiple months. Our results demonstrate the viability of human operation of prosthetic limbs using non-invasive BCI technology.

Sensorimotor Rhythm BCI with Simultaneous High Definition-Transcranial Direct Current Stimulation Alters Task Performance

BACKGROUND

Transcranial direct current stimulation (tDCS) has been used to alter the excitability of neurons within the cerebral cortex. Improvements in motor learning have been found in multiple studies when tDCS was applied to the motor cortex before or during task learning. The motor cortex is also active during the performance of motor imagination, a cognitive task during which a person imagines, but does not execute, a movement. Motor imagery can be used with noninvasive brain computer interfaces (BCIs) to control virtual objects in up to three dimensions, but to master control of such devices requires long training times.

OBJECTIVE

To evaluate the effect of high-definition tDCS on the performance and underlying electrophysiology of motor imagery based BCI.

METHODS

We utilize high-definition tDCS to investigate the effect of stimulation on motor imagery-based BCI performance across and within sessions over multiple training days.

RESULTS

We report a decreased time-to-hit with anodal stimulation both within and across sessions. We also found differing electrophysiological changes of the stimulated sensorimotor cortex during online BCI task performance for left vs. right trials. Cathodal stimulation led to a decrease in alpha and beta band power during task performance compared to sham stimulation for right hand imagination trials.

CONCLUSION

These results suggest that unilateral tDCS over the sensorimotor motor cortex differentially affects cortical areas based on task specific neural activation.

Machine-learning-based diagnosis of schizophrenia using combined sensor-level and source-level EEG features

Recently, an increasing number of researchers have endeavored to develop practical tools for diagnosing patients with schizophrenia using machine learning techniques applied to EEG biomarkers. Although a number of studies showed that source-level EEG features can potentially be applied to the differential diagnosis of schizophrenia, most studies have used only sensor-level EEG features such as ERP peak amplitude and power spectrum for machine learning-based diagnosis of schizophrenia. In this study, we used both sensor-level and source-level features extracted from EEG signals recorded during an auditory oddball task for the classification of patients with schizophrenia and healthy controls. EEG signals were recorded from 34 patients with schizophrenia and 34 healthy controls while each subject was asked to attend to oddball tones. Our results demonstrated higher classification accuracy when source-level features were used together with sensor-level features, compared to when only sensor-level features were used. In addition, the selected sensor-level features were mostly found in the frontal area, and the selected source-level features were mostly extracted from the temporal area, which coincide well with the well-known pathological region of cognitive processing in patients with schizophrenia. Our results suggest that our approach would be a promising tool for the computer-aided diagnosis of schizophrenia.

A Fully Automated Trial Selection Method for Optimization of Motor Imagery Based Brain-Computer Interface

Independent component analysis (ICA) as a promising spatial filtering method can separate motor-related independent components (MRICs) from the multichannel electroencephalogram (EEG) signals. However, the unpredictable burst interferences may significantly degrade the performance of ICA-based brain-computer interface (BCI) system. In this study, we proposed a new algorithm frame to address this issue by combining the single-trial-based ICA filter with zero-training classifier. We developed a two-round data selection method to identify automatically the badly corrupted EEG trials in the training set. The "high quality" training trials were utilized to optimize the ICA filter. In addition, we proposed an accuracy-matrix method to locate the artifact data segments within a single trial and investigated which types of artifacts can influence the performance of the ICA-based MIBCIs. Twenty-six EEG datasets of three-class motor imagery were used to validate the proposed methods, and the classification accuracies were compared with that obtained by frequently used common spatial pattern (CSP) spatial filtering algorithm. The experimental results demonstrated that the proposed optimizing strategy could effectively improve the stability, practicality and classification performance of ICA-based MIBCI. The study revealed that rational use of ICA method may be crucial in building a practical ICA-based MIBCI system.

Electroencephalographic Motor Imagery Brain Connectivity Analysis for BCI: A Review

Recent research has reached a consensus on the feasibility of motor imagery brain-computer interface (MI-BCI) for different applications, especially in stroke rehabilitation. Most MI-BCI systems rely on temporal, spectral, and spatial features of single channels to distinguish different MI patterns. However, no successful communication has been established for a completely locked-in subject. To provide more useful and informative features, it has been recommended to take into account the relationships among electroencephalographic (EEG) sensor/source signals in the form of brain connectivity as an efficient tool of neuroscience. In this review, we briefly report the challenges and limitations of conventional MI-BCIs. Brain connectivity analysis, particularly functional and effective, has been described as one of the most promising approaches for improving MI-BCI performance. An extensive literature on EEG-based MI brain connectivity analysis of healthy subjects is reviewed. We subsequently discuss the brain connectomes during left and right hand, feet, and tongue MI movements. Moreover, key components involved in brain connectivity analysis that considerably affect the results are explained. Finally, possible technical shortcomings that may have influenced the results in previous research are addressed and suggestions are provided.

EEG Source Imaging Enhances the Decoding of Complex Right-Hand Motor Imagery Tasks

GOAL

Sensorimotor-based brain-computer interfaces (BCIs) have achieved successful control of real and virtual devices in up to three dimensions; however, the traditional sensor-based paradigm limits the intuitive use of these systems. Many control signals for state-of-the-art BCIs involve imagining the movement of body parts that have little to do with the output command, revealing a cognitive disconnection between the user's intent and the action of the end effector. Therefore, there is a need to develop techniques that can identify with high spatial resolution the self-modulated neural activity reflective of the actions of a helpful output device.

METHODS

We extend previous EEG source imaging (ESI) work to decoding natural hand/wrist manipulations by applying a novel technique to classifying four complex motor imaginations of the right hand: flexion, extension, supination, and pronation.

RESULTS

We report an increase of up to 18.6% for individual task classification and 12.7% for overall classification using the proposed ESI approach over the traditional sensor-based method.

CONCLUSION

ESI is able to enhance BCI performance of decoding complex right-hand motor imagery tasks.

SIGNIFICANCE

This study may lead to the development of BCI systems with naturalistic and intuitive motor imaginations, thus facilitating broad use of noninvasive BCIs.

Systems Neuroengineering: Understanding and Interacting with the Brain

In this paper, we review the current state-of-the-art techniques used for understanding the inner workings of the brain at a systems level. The neural activity that governs our everyday lives involves an intricate coordination of many processes that can be attributed to a variety of brain regions. On the surface, many of these functions can appear to be controlled by specific anatomical structures; however, in reality, numerous dynamic networks within the brain contribute to its function through an interconnected web of neuronal and synaptic pathways. The brain, in its healthy or pathological state, can therefore be best understood by taking a systems-level approach. While numerous neuroengineering technologies exist, we focus here on three major thrusts in the field of systems neuroengineering: neuroimaging, neural interfacing, and neuromodulation. Neuroimaging enables us to delineate the structural and functional organization of the brain, which is key in understanding how the neural system functions in both normal and disease states. Based on such knowledge, devices can be used either to communicate with the neural system, as in neural interface systems, or to modulate brain activity, as in neuromodulation systems. The consideration of these three fields is key to the development and application of neuro-devices. Feedback-based neuro-devices require the ability to sense neural activity (via a neuroimaging modality) through a neural interface (invasive or noninvasive) and ultimately to select a set of stimulation parameters in order to alter neural function via a neuromodulation modality. Systems neuroengineering refers to the use of engineering tools and technologies to image, decode, and modulate the brain in order to comprehend its functions and to repair its dysfunction. Interactions between these fields will help to shape the future of systems neuroengineering-to develop neurotechniques for enhancing the understanding of whole-brain function and dysfunction, and the management of neurological and mental disorders.

Leveraging anatomical information to improve transfer learning in brain-computer interfaces

OBJECTIVE

Brain-computer interfaces (BCIs) represent a technology with the potential to rehabilitate a range of traumatic and degenerative nervous system conditions but require a time-consuming training process to calibrate. An area of BCI research known as transfer learning is aimed at accelerating training by recycling previously recorded training data across sessions or subjects. Training data, however, is typically transferred from one electrode configuration to another without taking individual head anatomy or electrode positioning into account, which may underutilize the recycled data.

APPROACH

We explore transfer learning with the use of source imaging, which estimates neural activity in the cortex. Transferring estimates of cortical activity, in contrast to scalp recordings, provides a way to compensate for variability in electrode positioning and head morphologies across subjects and sessions.

MAIN RESULTS

Based on simulated and measured electroencephalography activity, we trained a classifier using data transferred exclusively from other subjects and achieved accuracies that were comparable to or surpassed a benchmark classifier (representative of a real-world BCI). Our results indicate that classification improvements depend on the number of trials transferred and the cortical region of interest.

SIGNIFICANCE

These findings suggest that cortical source-based transfer learning is a principled method to transfer data that improves BCI classification performance and provides a path to reduce BCI calibration time.

Noninvasive Brain-Computer Interfaces Based on Sensorimotor Rhythms

Brain-computer interfaces (BCIs) have been explored in the field of neuroengineering to investigate how the brain can use these systems to control external devices. We review the principles and approaches we have taken to develop a sensorimotor rhythm EEG based brain-computer interface (BCI). The methods include developing BCI systems incorporating the control of physical devices to increase user engagement, improving BCI systems by inversely mapping scalp-recorded EEG signals to the cortical source domain, integrating BCI with noninvasive neuromodulation strategies to improve learning, and incorporating mind-body awareness training to enhance BCI learning and performance. The challenges and merits of these strategies are discussed, together with recent findings. Our work indicates that the sensorimotor-rhythm-based noninvasive BCI has the potential to provide communication and control capabilities as an alternative to physiological motor pathways.

STRAPS: A Fully Data-Driven Spatio-Temporally Regularized Algorithm for M/EEG Patch Source Imaging

For M/EEG-based distributed source imaging, it has been established that the L2-norm-based methods are effective in imaging spatially extended sources, whereas the L1-norm-based methods are more suited for estimating focal and sparse sources. However, when the spatial extents of the sources are unknown a priori, the rationale for using either type of methods is not adequately supported. Bayesian inference by exploiting the spatio-temporal information of the patch sources holds great promise as a tool for adaptive source imaging, but both computational and methodological limitations remain to be overcome. In this paper, based on state-space modeling of the M/EEG data, we propose a fully data-driven and scalable algorithm, termed STRAPS, for M/EEG patch source imaging on high-resolution cortices. Unlike the existing algorithms, the recursive penalized least squares (RPLS) procedure is employed to efficiently estimate the source activities as opposed to the computationally demanding Kalman filtering/smoothing. Furthermore, the coefficients of the multivariate autoregressive (MVAR) model characterizing the spatial-temporal dynamics of the patch sources are estimated in a principled manner via empirical Bayes. Extensive numerical experiments demonstrate STRAPS's excellent performance in the estimation of locations, spatial extents and amplitudes of the patch sources with varying spatial extents.

Electroencephalography-based real-time cortical monitoring system that uses hierarchical Bayesian estimations for the brain-machine interface

In this study, a real-time cortical activity monitoring system was constructed, which could estimate cortical activities every 125 milliseconds over 2,240 vertexes from 64 channel electroencephalography signals through the Hierarchical Bayesian estimation that uses functional magnetic resonance imaging data as its prior information. Recently, functional magnetic resonance imaging has mostly been used in the neurofeedback field because it allows for high spatial resolution. However, in functional magnetic resonance imaging, the time for the neurofeedback information to reach the patient is delayed several seconds because of its poor temporal resolution. Therefore, a number of problems need to be solved to effectively implement feedback training paradigms in patients. To address this issue, this study used a new cortical activity monitoring system that improved both spatial and temporal resolution by using both functional magnetic resonance imaging data and electroencephalography signals in conjunction with one another. This system is advantageous as it can improve applications in the fields of real-time diagnosis, neurofeedback, and the brain-machine interface.

Brain-computer interfaces using sensorimotor rhythms: current state and future perspectives

Many studies over the past two decades have shown that people can use brain signals to convey their intent to a computer using brain-computer interfaces (BCIs). BCI systems extract specific features of brain activity and translate them into control signals that drive an output. Recently, a category of BCIs that are built on the rhythmic activity recorded over the sensorimotor cortex, i.e., the sensorimotor rhythm (SMR), has attracted considerable attention among the BCIs that use noninvasive neural recordings, e.g., electroencephalography (EEG), and have demonstrated the capability of multidimensional prosthesis control. This paper reviews the current state and future perspectives of SMR-based BCI and its clinical applications, in particular focusing on the EEG SMR. The characteristic features of SMR from the human brain are described and their underlying neural sources are discussed. The functional components of SMR-based BCI, together with its current clinical applications, are reviewed. Finally, limitations of SMR-BCIs and future outlooks are also discussed.

An adaptive brain actuated system for augmenting rehabilitation

For people living with paralysis, restoration of hand function remains the top priority because it leads to independence and improvement in quality of life. In approaches to restore hand and arm function, a goal is to better engage voluntary control and counteract maladaptive brain reorganization that results from non-use. Standard rehabilitation augmented with developments from the study of brain-computer interfaces could provide a combined therapy approach for motor cortex rehabilitation and to alleviate motor impairments. In this paper, an adaptive brain-computer interface system intended for application to control a functional electrical stimulation (FES) device is developed as an experimental test bed for augmenting rehabilitation with a brain-computer interface. The system's performance is improved throughout rehabilitation by passive user feedback and reinforcement learning. By continuously adapting to the user's brain activity, similar adaptive systems could be used to support clinical brain-computer interface neurorehabilitation over multiple days.

Sitting and standing intention can be decoded from scalp EEG recorded prior to movement execution

Low frequency signals recorded from non-invasive electroencephalography (EEG), in particular movement-related cortical potentials (MRPs), are associated with preparation and execution of movement and thus present a target for use in brain-machine interfaces. We investigated the ability to decode movement intent from delta-band (0.1-4 Hz) EEG recorded immediately before movement execution in healthy volunteers. We used data from epochs starting 1.5 s before movement onset to classify future movements into one of three classes: stand-up, sit-down, or quiet. We assessed classification accuracy in both externally triggered and self-paced paradigms. Movement onset was determined from electromyography (EMG) recordings synchronized with EEG signals. We employed an artifact subspace reconstruction (ASR) algorithm to eliminate high amplitude noise before building our time-embedded EEG features. We applied local Fisher's discriminant analysis to reduce the dimensionality of our spatio-temporal features and subsequently used a Gaussian mixture model classifier for our three class problem. Our results demonstrate significantly better than chance classification accuracy (chance level = 33.3%) for the self-initiated (78.0 ± 2.6%) and triggered (74.7 ± 5.7%) paradigms. Surprisingly, we found no significant difference in classification accuracy between the self-paced and cued paradigms when using the full set of non-peripheral electrodes. However, accuracy was significantly increased for self-paced movements when only electrodes over the primary motor area were used. Overall, this study demonstrates that delta-band EEG recorded immediately before movement carries discriminative information regarding movement type. Our results suggest that EEG-based classifiers could improve lower-limb neuroprostheses and neurorehabilitation techniques by providing earlier detection of movement intent, which could be used in robot-assisted strategies for motor training and recovery of function.