Using a noninvasive decoding method to classify rhythmic movement imaginations of the arm in two planes

The Effect of Caffeine on Movement-Related Cortical Potential Morphology and Detection

Movement-related cortical potential (MRCP) is observed in EEG recordings prior to a voluntary movement. It has been used for e.g., quantifying motor learning and for brain-computer interfacing (BCIs). The MRCP amplitude is affected by various factors, but the effect of caffeine is underexplored. The aim of this study was to investigate if a cup of coffee with 85 mg caffeine modulated the MRCP amplitude and the classification of MRCPs versus idle activity, which estimates BCI performance. Twenty-six healthy participants performed 2 × 100 ankle dorsiflexion separated by a 10-min break before a cup of coffee was consumed, followed by another 100 movements. EEG was recorded during the movements and divided into epochs, which were averaged to extract three average MRCPs that were compared. Also, idle activity epochs were extracted. Features were extracted from the epochs and classified using random forest analysis. The MRCP amplitude did not change after consuming caffeine. There was a slight increase of two percentage points in the classification accuracy after consuming caffeine. In conclusion, a cup of coffee with 85 mg caffeine does not affect the MRCP amplitude, and improves MRCP-based BCI performance slightly. The findings suggest that drinking coffee is only a minor confounder in MRCP-related studies.

Towards unlocking motor control in spinal cord injured by applying an online EEG-based framework to decode motor intention, trajectory and error processing

Brain-computer interfaces (BCIs) can translate brain signals directly into commands for external devices. Electroencephalography (EEG)-based BCIs mostly rely on the classification of discrete mental states, leading to unintuitive control. The ERC-funded project "Feel Your Reach" aimed to establish a novel framework based on continuous decoding of hand/arm movement intention, for a more natural and intuitive control. Over the years, we investigated various aspects of natural control, however, the individual components had not yet been integrated. Here, we present a first implementation of the framework in a comprehensive online study, combining (i) goal-directed movement intention, (ii) trajectory decoding, and (iii) error processing in a unique closed-loop control paradigm. Testing involved twelve able-bodied volunteers, performing attempted movements, and one spinal cord injured (SCI) participant. Similar movement-related cortical potentials and error potentials to previous studies were revealed, and the attempted movement trajectories were overall reconstructed. Source analysis confirmed the involvement of sensorimotor and posterior parietal areas for goal-directed movement intention and trajectory decoding. The increased experiment complexity and duration led to a decreased performance than each single BCI. Nevertheless, the study contributes to understanding natural motor control, providing insights for more intuitive strategies for individuals with motor impairments.

Decoding movement frequencies and limbs based on steady-state movement-related rhythms from noninvasive EEG

Objective.Decoding different types of movements noninvasively from electroencephalography (EEG) is an essential topic in neural engineering, especially in brain-computer interface. Although the widely used sensorimotor rhythm (SMR) is efficient in limb decoding, it lacks efficacy in decoding movement frequencies. Accumulating evidence supports the notion that the movement frequency is encoded in the steady-state movement-related rhythm (SSMRR). Our study has two primary objectives: firstly, to investigate the spatial-spectral representation of SSMRR in EEG during voluntary movements; secondly, to assess whether movement frequencies and limbs can be effectively decoded based on SSMRR.Approach.To comprehensively examine the representation of SSMRR, we investigated the frequency characteristics and spatial patterns associated with various rhythmic finger movements. Coherence analysis was performed between the sensor or source domain EEG and finger movements recorded by data gloves. A fusion model based on spectral SNR features and filter-bank common spatial pattern features was utilized to decode movement frequencies and limbs.Main results.At the group-level, sensor domain, and source domain coherence maps demonstrated that the accurate movement frequency (f0) and its first harmonic (f1) were encoded in the contralateral motor cortex. For the four-class classification, including two movement frequencies for both hands, the decoding accuracies for externally paced and internally paced movements were 73.14 ± 15.86% and 66.30 ± 17.26% (averaged across ten subjects, chance levels at 31.05% and 30.96%). Notably, the average results of five subjects with the highest decoding accuracies reached 87.21 ± 7.44% and 80.44 ± 7.99%.Significance.Our results verified the EEG representation of SSMRR and proved that the movement frequency and limb could be effectively decoded based on spatial-spectral features extracted from SSMRR. We suggest that SSMRR can serve as a complement to SMR to expand the range of decodable movement types and the approaches of limb decoding.

Decoding movement kinematics from EEG using an interpretable convolutional neural network

Continuous decoding of hand kinematics has been recently explored for the intuitive control of electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs). Deep neural networks (DNNs) are emerging as powerful decoders, for their ability to automatically learn features from lightly pre-processed signals. However, DNNs for kinematics decoding lack in the interpretability of the learned features and are only used to realize within-subject decoders without testing other training approaches potentially beneficial for reducing calibration time, such as transfer learning. Here, we aim to overcome these limitations by using an interpretable convolutional neural network (ICNN) to decode 2-D hand kinematics (position and velocity) from EEG in a pursuit tracking task performed by 13 participants. The ICNN is trained using both within-subject and cross-subject strategies, and also testing the feasibility of transferring the knowledge learned on other subjects on a new one. Moreover, the network eases the interpretation of learned spectral and spatial EEG features. Our ICNN outperformed most of the other state-of-the-art decoders, showing the best trade-off between performance, size, and training time. Furthermore, transfer learning improved kinematics prediction in the low data regime. The network attributed the highest relevance for decoding to the delta-band across all subjects, and to higher frequencies (alpha, beta, low-gamma) for a cluster of them; contralateral central and parieto-occipital sites were the most relevant, reflecting the involvement of sensorimotor, visual and visuo-motor processing. The approach improved the quality of kinematics prediction from the EEG, at the same time allowing interpretation of the most relevant spectral and spatial features.

A comprehensive review on motion trajectory reconstruction for EEG-based brain-computer interface

The advance in neuroscience and computer technology over the past decades have made brain-computer interface (BCI) a most promising area of neurorehabilitation and neurophysiology research. Limb motion decoding has gradually become a hot topic in the field of BCI. Decoding neural activity related to limb movement trajectory is considered to be of great help to the development of assistive and rehabilitation strategies for motor-impaired users. Although a variety of decoding methods have been proposed for limb trajectory reconstruction, there does not yet exist a review that covers the performance evaluation of these decoding methods. To alleviate this vacancy, in this paper, we evaluate EEG-based limb trajectory decoding methods regarding their advantages and disadvantages from a variety of perspectives. Specifically, we first introduce the differences in motor execution and motor imagery in limb trajectory reconstruction with different spaces (2D and 3D). Then, we discuss the limb motion trajectory reconstruction methods including experiment paradigm, EEG pre-processing, feature extraction and selection, decoding methods, and result evaluation. Finally, we expound on the open problem and future outlooks.

Feel Your Reach: An EEG-Based Framework to Continuously Detect Goal-Directed Movements and Error Processing to Gate Kinesthetic Feedback Informed Artificial Arm Control

Establishing the basic knowledge, methodology, and technology for a framework for the continuous decoding of hand/arm movement intention was the aim of the ERC-funded project "Feel Your Reach". In this work, we review the studies and methods we performed and implemented in the last 6 years, which build the basis for enabling severely paralyzed people to non-invasively control a robotic arm in real-time from electroencephalogram (EEG). In detail, we investigated goal-directed movement detection, decoding of executed and attempted movement trajectories, grasping correlates, error processing, and kinesthetic feedback. Although we have tested some of our approaches already with the target populations, we still need to transfer the "Feel Your Reach" framework to people with cervical spinal cord injury and evaluate the decoders' performance while participants attempt to perform upper-limb movements. While on the one hand, we made major progress towards this ambitious goal, we also critically discuss current limitations.

A Hybrid-Domain Deep Learning-Based BCI For Discriminating Hand Motion Planning From EEG Sources

In this paper, a hybrid-domain deep learning (DL)-based neural system is proposed to decode hand movement preparation phases from electroencephalographic (EEG) recordings. The system exploits information extracted from the temporal-domain and time-frequency-domain, as part of a hybrid strategy, to discriminate the temporal windows (i.e. EEG epochs) preceding hand sub-movements (open/close) and the resting state. To this end, for each EEG epoch, the associated cortical source signals in the motor cortex and the corresponding time-frequency (TF) maps are estimated via beamforming and Continuous Wavelet Transform (CWT), respectively. Two Convolutional Neural Networks (CNNs) are designed: specifically, the first CNN is trained over a dataset of temporal (T) data (i.e. EEG sources), and is referred to as T-CNN; the second CNN is trained over a dataset of TF data (i.e. TF-maps of EEG sources), and is referred to as TF-CNN. Two sets of features denoted as T-features and TF-features, extracted from T-CNN and TF-CNN, respectively, are concatenated in a single features vector (denoted as TTF-features vector) which is used as input to a standard multi-layer perceptron for classification purposes. Experimental results show a significant performance improvement of our proposed hybrid-domain DL approach as compared to temporal-only and time-frequency-only-based benchmark approaches, achieving an average accuracy of [Formula: see text]%.

Neural decoding of continuous upper limb movements: a meta-analysis

OBJECTIVE

EEG-based motion trajectory decoding makes a promising approach for neurotechnology which can be used for neural control of motion reconstruction and neurorehabilitation tools. However, the feasibility and validity of continuous motion decoding by non-invasive brain activity are not clear. The main aim of this study was to perform a meta-analysis across studies that examined the ability of EEG-based continuous motion decoding of upper limb movements.

APPROACH

Pearson's correlation coefficient (CC) was used to evaluate the model performance of the studies and considered as an effect size. To estimate the overall effect size of neural decoding of motion trajectory across studies, characteristics of included studies were addressed and the random effect model was applied to the heterogeneous studies which estimated overall effect size distribution. Furthermore, the significant difference between the two subgroups of imagined and executed movements was analysed.

MAIN RESULTS

The mean of the overall effect size was computed 0.46 across the nonhomogeneous studies. The results showed no significant difference between imagined and executed movements (Chi²=0.28, df = 1, p = 0.60).

SIGNIFICANCE

Meta-analysis results confirm that imagination like execution movements can be used for neural decoding of motion trajectory in neural motor control systems. Also, nonlinear compare with linear model statistically confirmed to be more beneficial for complex movements. Furthermore, a new approach of synergy-based motion decoding can be significantly effective to increase model performance and more research needs to evaluate this method for different levels of complexity of movements.IMPLICATIONS FOR REHABILITATIONNeural decoding methods base on EEG as a non-invasive brain activity, are more user friendly for neurorehabilitation than invasive methods that developing of it makes it more applicable for reconstructing activities of daily living.Neurotechnology for neural control of motion reconstruction, makes the rehabilitation tools to be more synchrony with human intentional movement that can be used to improve the brain neuroplastisity in stroke or other paralysed people.The feasibility and validity of imagined movements equal with executed movements show that amputee people also can benefit EEG-based motion decoding for controling rehabilitation tools just by imagination of their intentional movements.For neurorehabilitation tools, comparing the study outcomes illucidate that the approach of synergy-based motor control in brain activities concluded significantly high performance that highlighted the need it to more investigated in future research.

Decoding of voluntary and involuntary upper-limb motor imagery based on graph fourier transform and cross-frequency coupling coefficients

OBJECTIVE

Brain-computer interface (BCI) technology based on motor imagery (MI) control has become a research hotspot but continues to encounter numerous challenges. BCI can assist in the recovery of stroke patients and serve as a key technology in robot control. Current research on MI almost exclusively focuses on the hands, feet, and tongue. Therefore, the purpose of this paper is to establish a four-class MI BCI system, in which the four types are the four articulations within the right upper limbs, involving the shoulder, elbow, wrist, and hand.

APPROACH

Ten subjects were chosen to perform nine upper-limb analytic movements, after which the differences were compared in P300, movement-related potentials(MRPS), and event-related desynchronization/event-related synchronization under voluntary MI (V-MI) and involuntary MI (INV-MI). Next, the cross-frequency coupling (CFC) coefficient based on mutual information was extracted from the electrodes and frequency bands with interest. Combined with the image Fourier transform and twin bounded support vector machine classifier, four kinds of electroencephalography data were classified, and the classifier's parameters were optimized using a genetic algorithm.

MAIN RESULTS

The results were shown to be encouraging, with an average accuracy of 93.2% and 92.2% for V-MI and INV-MI, respectively, and over 95% for any three classes and any two classes. In most cases, the accuracy of feature extraction using the proximal articulations as the basis was found to be relatively high and had better performance.

SIGNIFICANCE

This paper discussed four types of MI according to three aspects under two modes and classed them by combining graph Fourier transform and CFC. Accordingly, the theoretical discussion and classification methods may provide a fundamental theoretical basis for BCI interface applications.

Distance- and speed-informed kinematics decoding improves M/EEG based upper-limb movement decoder accuracy

OBJECTIVE

One of the main goals in brain-computer interface (BCI) research is the replacement or restoration of lost function in individuals with paralysis. One line of research investigates the inference of movement kinematics from brain activity during different volitional states. A growing number of electroencephalography (EEG) and magnetoencephalography (MEG) studies suggest that information about directional (e.g. velocity) and nondirectional (e.g. speed) movement kinematics is accessible noninvasively. We sought to assess if the neural information associated with both types of kinematics can be combined to improve the decoding accuracy.

APPROACH

In an offline analysis, we reanalyzed the data of two previous experiments containing the recordings of 34 healthy participants (15 EEG, 19 MEG). We decoded 2D movement trajectories from low-frequency M/EEG signals in executed and observed tracking movements, and compared the accuracy of an unscented Kalman filter (UKF) that explicitly modeled the nonlinear relation between directional and nondirectional kinematics to the accuracies of linear Kalman (KF) and Wiener filters which did not combine both types of kinematics.

MAIN RESULTS

At the group level, posterior-parietal and parieto-occipital (executed and observed movements) and sensorimotor areas (executed movements) encoded kinematic information. Correlations between the recorded position and velocity trajectories and the UKF decoded ones were on average 0.49 during executed and 0.36 during observed movements. Compared to the other filters, the UKF could achieve the best trade-off between maximizing the signal to noise ratio and minimizing the amplitude mismatch between the recorded and decoded trajectories.

SIGNIFICANCE

We present direct evidence that directional and nondirectional kinematic information is simultaneously detectable in low-frequency M/EEG signals. Moreover, combining directional and nondirectional kinematic information significantly improves the decoding accuracy upon a linear KF.

A Usability Study of Low-cost Wireless Brain-Computer Interface for Cursor Control Using Online Linear Model

Computer cursor control using electroencephalogram (EEG) signals is a common and well-studied brain-computer interface (BCI). The emphasis of the literature has been primarily on evaluation of the objective measures of assistive BCIs such as accuracy of the neural decoder whereas the subjective measures such as user's satisfaction play an essential role for the overall success of a BCI. As far as we know, the BCI literature lacks a comprehensive evaluation of the usability of the mind-controlled computer cursor in terms of decoder efficiency (accuracy), user experience, and relevant confounding variables concerning the platform for the public use. To fill this gap, we conducted a two-dimensional EEG-based cursor control experiment among 28 healthy participants. The computer cursor velocity was controlled by the imagery of hand movement using a paradigm presented in the literature named imagined body kinematics (IBK) with a low-cost wireless EEG headset. We evaluated the usability of the platform for different objective and subjective measures while we investigated the extent to which the training phase may influence the ultimate BCI outcome. We conducted pre- and post- BCI experiment interview questionnaires to evaluate the usability. Analyzing the questionnaires and the testing phase outcome shows a positive correlation between the individuals' ability of visualization and their level of mental controllability of the cursor. Despite individual differences, analyzing training data shows the significance of electrooculogram (EOG) on the predictability of the linear model. The results of this work may provide useful insights towards designing a personalized user-centered assistive BCI.

Non-linear online low-frequency EEG decoding of arm movements during a pursuit tracking task

Decoding upper-limb movements in invasive recordings has become a reality, but neural tuning in non-invasive low-frequency recordings is still under discussion. Recent studies managed to decode movement positions and velocities using linear decoders, even developing an online system. The decoded signals, however, exhibited smaller amplitudes than actual movements, affecting feedback and user experience. Recently, we showed that a non-linear offline decoder can combine directional (e.g., velocity) and non-directional (e.g., speed) information. In this study, it is assessed if the non-linear decoder can be used online to provide real-time feedback. Five healthy subjects were asked to track a moving target by controlling a robotic arm. Initially, the robot was controlled by their right hand; then, the control was gradually switched until it was entirely controlled by the electroencephalogram (EEG). Correlations between actual and decoded movements were generally above chance level. Results suggest that information about speed was also encoded in the EEG, demonstrating that the proposed non-linear decoder is suitable for decoding real-time arm movements.

Trajectory Decoding of Arm Reaching Movement Imageries for Brain-Controlled Robot Arm System

Development of noninvasive brain-machine interface (BMI) systems based on electroencephalography (EEG), driven by spontaneous movement intentions, is a useful tool for controlling external devices or supporting a neuro- rehabilitation. In this study, we present the possibility of brain-controlled robot arm system using arm trajectory decoding. To do that, we first constructed the experimental system that can acquire the EEG data for not only movement execution (ME) task but also movement imagery (MI) tasks. Five subjects participated in our experiments and performed four directional reaching tasks (Left, right, forward, and backward) in the 3D plane. For robust arm trajectory decoding, we propose a subject-dependent deep neural network (DNN) architecture. The decoding model applies the principle of bi-directional long short-term memory (LSTM) network. As a result, we confirmed the decoding performance (r-value: >0.8) for all X-, Y-, and Z-axis across all subjects in the MI as well as ME tasks. These results show the feasibility of the EEG-based intuitive robot arm control system for high-level tasks (e.g., drink water or moving some objects). Also, we confirm that the proposed method has no much decoding performance variations between ME and MI tasks for the offline analysis. Hence, we will demonstrate that the decoding model is capable of robust trajectory decoding even in a real-time environment.

Reconstructing Degree of Forearm Rotation from Imagined movements for BCI-based Robot Hand Control

Brain-computer interface (BCI) is an important tool for rehabilitation and control of an external device (e.g., robot arm or home appliances). Fully reconstruction of upper limb movement from brain signals is one of the critical issues for intuitive BCI. However, decoding of forearm rotation from imagined movements using electroencephalography (EEG) is difficult to decode degree of rotation accurately. In this paper, we reconstructed imagined forearm rotation from low- frequency (0.3-3 Hz) of EEG signals. We selected 20 EEG channel on motor cortex for analysis. Ten healthy subjects participated in our experiment. The subjects performed actual and imagined forearm rotation to reach different targets. We trained a reconstruction decoder which used the EEG signals measured from actual movements and the kinematic information only. Additionally, we applied a long short-term memory (LSTM) network to enhance decoding performances. As a result, we achieved the high correlation performance (Average: 0.67) to decode imagined forearm rotation angle. This result has demonstrated that the reconstruction decoder which is trained by the EEG data from actual movement has effective to decode robustly for the imagined forearm rotation angle.

Electroencephalography

The electroencephalogram (EEG) was invented almost 100 years ago and is still a method of choice for many research questions, even applications-from functional brain imaging in neuroscientific investigations during movement to real-time applications like brain-computer interfacing. This chapter gives some background information on the establishment and properties of the EEG. This chapter starts with a closer look at the sources of EEG at a micro or neuronal level, followed by recording techniques, types of electrodes, and common EEG artifacts. Then an overview on EEG phenomena, namely, spontaneous EEG and event-related potentials build the middle part of this chapter. The last part discusses brain signals, which are used in current BCI research, including short descriptions and examples of applications.

Decoding Imagined 3D Arm Movement Trajectories From EEG to Control Two Virtual Arms-A Pilot Study

Background: Realization of online control of an artificial or virtual arm using information decoded from EEG normally occurs by classifying different activation states or voluntary modulation of the sensorimotor activity linked to different overt actions of the subject. However, using a more natural control scheme, such as decoding the trajectory of imagined 3D arm movements to move a prosthetic, robotic, or virtual arm has been reported in a limited amount of studies, all using offline feed-forward control schemes.

Objective: In this study, we report the first attempt to realize online control of two virtual arms generating movements toward three targets/arm in 3D space. The 3D trajectory of imagined arm movements was decoded from power spectral density of mu, low beta, high beta, and low gamma EEG oscillations using multiple linear regression. The analysis was performed on a dataset recorded from three subjects in seven sessions wherein each session comprised three experimental blocks: an offline calibration block and two online feedback blocks. Target classification accuracy using predicted trajectories of the virtual arms was computed and compared with results of a filter-bank common spatial patterns (FBCSP) based multi-class classification method involving mutual information (MI) selection and linear discriminant analysis (LDA) modules.

Main Results: Target classification accuracy from predicted trajectory of imagined 3D arm movements in the offline runs for two subjects (mean 45%, std 5%) was significantly higher (p < 0.05) than chance level (33.3%). Nevertheless, the accuracy during real-time control of the virtual arms using the trajectory decoded directly from EEG was in the range of chance level (33.3%). However, the results of two subjects show that false-positive feedback may increase the accuracy in closed-loop. The FBCSP based multi-class classification method distinguished imagined movements of left and right arm with reasonable accuracy for two of the three subjects (mean 70%, std 5% compared to 50% chance level). However, classification of the imagined arm movement toward three targets was not successful with the FBCSP classifier as the achieved accuracy (mean 33%, std 5%) was similar to the chance level (33.3%). Sub-optimal components of the multi-session experimental paradigm were identified, and an improved paradigm proposed.

Attempted Arm and Hand Movements can be Decoded from Low-Frequency EEG from Persons with Spinal Cord Injury

We show that persons with spinal cord injury (SCI) retain decodable neural correlates of attempted arm and hand movements. We investigated hand open, palmar grasp, lateral grasp, pronation, and supination in 10 persons with cervical SCI. Discriminative movement information was provided by the time-domain of low-frequency electroencephalography (EEG) signals. Based on these signals, we obtained a maximum average classification accuracy of 45% (chance level was 20%) with respect to the five investigated classes. Pattern analysis indicates central motor areas as the origin of the discriminative signals. Furthermore, we introduce a proof-of-concept to classify movement attempts online in a closed loop, and tested it on a person with cervical SCI. We achieved here a modest classification performance of 68.4% with respect to palmar grasp vs hand open (chance level 50%).

Optimizing Prediction Model for a Noninvasive Brain-Computer Interface Platform Using Channel Selection, Classification, and Regression

A brain-computer interface (BCI) platform can be utilized by a user to control an external device without making any overt movements. An EEG-based computer cursor control task is commonly used as a testbed for BCI applications. While traditional computer cursor control schemes are based on sensorimotor rhythm, a new scheme has recently been developed using imagined body kinematics (IBK) to achieve natural cursor movement in a shorter time of training. This article attempts to explore optimal decoding algorithms for an IBK paradigm using EEG signals with application to neural cursor control. The study is based on an offline analysis of 32 healthy subjects' training data. Various machine learning techniques were implemented to predict the kinematics of the computer cursor using EEG signals during the training tasks. Our results showed that a linear regression least squares model yielded the highest goodness-of-fit scores in the cursor kinematics model (70% in horizontal prediction and 40% in vertical prediction using a Theil-Sen regressor). Additionally, the contribution of each EEG channel on the predictability of cursor kinematics was examined for horizontal and vertical directions, separately. A directional classifier was also proposed to classify horizontal versus vertical cursor kinematics using EEG signals. By incorporating features extracted from specific frequency bands, we achieved 80% classification accuracy in differentiating horizontal and vertical cursor movements. The findings of the current study could facilitate a pathway to designing an optimized online neural cursor control.

Tuning characteristics of low-frequency EEG to positions and velocities in visuomotor and oculomotor tracking tasks

Movement decoders exploit the tuning of neural activity to various movement parameters with the ultimate goal of controlling end-effector action. Invasive approaches, typically relying on spiking activity, have demonstrated feasibility. Results of recent functional neuroimaging studies suggest that information about movement parameters is even accessible non-invasively in the form of low-frequency brain signals. However, their spatiotemporal tuning characteristics to single movement parameters are still unclear. Here, we extend the current understanding of low-frequency electroencephalography (EEG) tuning to position and velocity signals. We recorded EEG from 15 healthy participants while they performed visuomotor and oculomotor pursuit tracking tasks. Linear decoders, fitted to EEG signals in the frequency range of the tracking movements, predicted positions and velocities with moderate correlations (0.2–0.4; above chance level) in both tasks. Predictive activity in terms of decoder patterns was significant in superior parietal and parieto-occipital areas in both tasks. By contrasting the two tracking tasks, we found that predictive activity in contralateral primary sensorimotor and premotor areas exhibited significantly larger tuning to end-effector velocity when the visuomotor tracking task was performed.

Improving EEG-Based Motor Imagery Classification for Real-Time Applications Using the QSA Method

We present an improvement to the quaternion-based signal analysis (QSA) technique to extract electroencephalography (EEG) signal features with a view to developing real-time applications, particularly in motor imagery (IM) cognitive processes. The proposed methodology (iQSA, improved QSA) extracts features such as the average, variance, homogeneity, and contrast of EEG signals related to motor imagery in a more efficient manner (i.e., by reducing the number of samples needed to classify the signal and improving the classification percentage) compared to the original QSA technique. Specifically, we can sample the signal in variable time periods (from 0.5 s to 3 s, in half-a-second intervals) to determine the relationship between the number of samples and their effectiveness in classifying signals. In addition, to strengthen the classification process a number of boosting-technique-based decision trees were implemented. The results show an 82.30% accuracy rate for 0.5 s samples and 73.16% for 3 s samples. This is a significant improvement compared to the original QSA technique that offered results from 33.31% to 40.82% without sampling window and from 33.44% to 41.07% with sampling window, respectively. We can thus conclude that iQSA is better suited to develop real-time applications.

EEG source space analysis of the supervised factor analytic approach for the classification of multi-directional arm movement

OBJECTIVE

In electroencephalography (EEG)-based brain-computer interface (BCI) systems for motor control tasks the conventional practice is to decode motor intentions by using scalp EEG. However, scalp EEG only reveals certain limited information about the complex tasks of movement with a higher degree of freedom. Therefore, our objective is to investigate the effectiveness of source-space EEG in extracting relevant features that discriminate arm movement in multiple directions.

APPROACH

We have proposed a novel feature extraction algorithm based on supervised factor analysis that models the data from source-space EEG. To this end, we computed the features from the source dipoles confined to Brodmann areas of interest (BA4a, BA4p and BA6). Further, we embedded class-wise labels of multi-direction (multi-class) source-space EEG to an unsupervised factor analysis to make it into a supervised learning method.

MAIN RESULTS

Our approach provided an average decoding accuracy of 71% for the classification of hand movement in four orthogonal directions, that is significantly higher (>10%) than the classification accuracy obtained using state-of-the-art spatial pattern features in sensor space. Also, the group analysis on the spectral characteristics of source-space EEG indicates that the slow cortical potentials from a set of cortical source dipoles reveal discriminative information regarding the movement parameter, direction.

SIGNIFICANCE

This study presents evidence that low-frequency components in the source space play an important role in movement kinematics, and thus it may lead to new strategies for BCI-based neurorehabilitation.

Dynamics of directional tuning and reference frames in humans: A high-density EEG study

Recent developments in EEG recording and signal processing have made it possible to record in an unconstrained, natural movement task, therefore EEG provides a promising approach to understanding the neural mechanisms of upper-limb reaching control. This study specifically addressed how EEG dynamics in the time domain encoded finger movement directions (directional tuning) and posture dependence (movement reference frames) by applying representational similarity analysis. High-density EEG covering the entire scalp was recorded while participants performed eight-directional, center-out reaching movements, thereby allowing us to explore directional selectivity of EEG sources over the brain beyond somatosensory areas. A majority of the source processes exhibited statistically significant directional tuning during peri-movement periods. In addition, directional tuning curves shifted systematically when the shoulder angle was rotated to perform the task within a more laterally positioned workspace, the degree of tuning curve rotation falling between that predicted by models assuming extrinsic and shoulder-based reference frames. We conclude that temporal dynamics of neural mechanisms for motor control can be studied noninvasively in humans using high-density EEG and that directional sensitivity of motor and non-motor processing is not limited within the sensorimotor areas but extends to the whole brain areas.

Decoding Imagined 3D Hand Movement Trajectories From EEG: Evidence to Support the Use of Mu, Beta, and Low Gamma Oscillations

Objective: To date, motion trajectory prediction (MTP) of a limb from non-invasive electroencephalography (EEG) has relied, primarily, on band-pass filtered samples of EEG potentials i.e., the potential time-series model. Most MTP studies involve decoding 2D and 3D arm movements i.e., executed arm movements. Decoding of observed or imagined 3D movements has been demonstrated with limited success and only reported in a few studies. MTP studies normally use EEG potentials filtered in the low delta (~1 Hz) band for reconstructing the trajectory of an executed or an imagined/observed movement. In contrast to MTP, multiclass classification based sensorimotor rhythm brain-computer interfaces aim to classify movements using the power spectral density of mu (8–12 Hz) and beta (12–28 Hz) bands.

Approach: We investigated if replacing the standard potentials time-series input with a power spectral density based bandpower time-series improves trajectory decoding accuracy of kinesthetically imagined 3D hand movement tasks (i.e., imagined 3D trajectory of the hand joint) and whether imagined 3D hand movements kinematics are encoded also in mu and beta bands. Twelve naïve subjects were asked to generate or imagine generating pointing movements with their right dominant arm to four targets distributed in 3D space in synchrony with an auditory cue (beep).

Main results: Using the bandpower time-series based model, the highest decoding accuracy for motor execution was observed in mu and beta bands whilst for imagined movements the low gamma (28–40 Hz) band was also observed to improve decoding accuracy for some subjects. Moreover, for both (executed and imagined) movements, the bandpower time-series model with mu, beta, and low gamma bands produced significantly higher reconstruction accuracy than the commonly used potential time-series model and delta oscillations.

Significance: Contrary to many studies that investigated only executed hand movements and recommend using delta oscillations for decoding directional information of a single limb joint, our findings suggest that motor kinematics for imagined movements are reflected mostly in power spectral density of mu, beta and low gamma bands, and that these bands may be most informative for decoding 3D trajectories of imagined limb movements.

Effects of Soft Drinks on Resting State EEG and Brain-Computer Interface Performance

Motor imagery-based (MI based) brain-computer interface (BCI) using electroencephalography (EEG) allows users to directly control a computer or external device by modulating and decoding the brain waves. A variety of factors could potentially affect the performance of BCI such as the health status of subjects or the environment. In this study, we investigated the effects of soft drinks and regular coffee on EEG signals under resting state and on the performance of MI based BCI. Twenty-six healthy human subjects participated in three or four BCI sessions with a resting period in each session. During each session, the subjects drank an unlabeled soft drink with either sugar (Caffeine Free Coca-Cola), caffeine (Diet Coke), neither ingredient (Caffeine Free Diet Coke), or a regular coffee if there was a fourth session. The resting state spectral power in each condition was compared; the analysis showed that power in alpha and beta band after caffeine consumption were decreased substantially compared to control and sugar condition. Although the attenuation of powers in the frequency range used for the online BCI control signal was shown, group averaged BCI online performance after consuming caffeine was similar to those of other conditions. This work, for the first time, shows the effect of caffeine, sugar intake on the online BCI performance and resting state brain signal.

Upper limb movements can be decoded from the time-domain of low-frequency EEG

How neural correlates of movements are represented in the human brain is of ongoing interest and has been researched with invasive and non-invasive methods. In this study, we analyzed the encoding of single upper limb movements in the time-domain of low-frequency electroencephalography (EEG) signals. Fifteen healthy subjects executed and imagined six different sustained upper limb movements. We classified these six movements and a rest class and obtained significant average classification accuracies of 55% (movement vs movement) and 87% (movement vs rest) for executed movements, and 27% and 73%, respectively, for imagined movements. Furthermore, we analyzed the classifier patterns in the source space and located the brain areas conveying discriminative movement information. The classifier patterns indicate that mainly premotor areas, primary motor cortex, somatosensory cortex and posterior parietal cortex convey discriminative movement information. The decoding of single upper limb movements is specially interesting in the context of a more natural non-invasive control of e.g., a motor neuroprosthesis or a robotic arm in highly motor disabled persons.

From classic motor imagery to complex movement intention decoding: The noninvasive Graz-BCI approach

In this chapter, we give an overview of the Graz-BCI research, from the classic motor imagery detection to complex movement intentions decoding. We start by describing the classic motor imagery approach, its application in tetraplegic end users, and the significant improvements achieved using coadaptive brain-computer interfaces (BCIs). These strategies have the drawback of not mirroring the way one plans a movement. To achieve a more natural control-and to reduce the training time-the movements decoded by the BCI need to be closely related to the user's intention. Within this natural control, we focus on the kinematic level, where movement direction and hand position or velocity can be decoded from noninvasive recordings. First, we review movement execution decoding studies, where we describe the decoding algorithms, their performance, and associated features. Second, we describe the major findings in movement imagination decoding, where we emphasize the importance of estimating the sources of the discriminative features. Third, we introduce movement target decoding, which could allow the determination of the target without knowing the exact movement-by-movement details. Aside from the kinematic level, we also address the goal level, which contains relevant information on the upcoming action. Focusing on hand-object interaction and action context dependency, we discuss the possible impact of some recent neurophysiological findings in the future of BCI control. Ideally, the goal and the kinematic decoding would allow an appropriate matching of the BCI to the end users' needs, overcoming the limitations of the classic motor imagery approach.

Workshops of the Sixth International Brain-Computer Interface Meeting: brain-computer interfaces past, present, and future

The Sixth International Brain-Computer Interface (BCI) Meeting was held 30 May-3 June 2016 at the Asilomar Conference Grounds, Pacific Grove, California, USA. The conference included 28 workshops covering topics in BCI and brain-machine interface research. Topics included BCI for specific populations or applications, advancing BCI research through use of specific signals or technological advances, and translational and commercial issues to bring both implanted and non-invasive BCIs to market. BCI research is growing and expanding in the breadth of its applications, the depth of knowledge it can produce, and the practical benefit it can provide both for those with physical impairments and the general public. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and highlighting important issues and calls for action to support future research and development.

EEG neural correlates of goal-directed movement intention

Using low-frequency time-domain electroencephalographic (EEG) signals we show, for the same type of upper limb movement, that goal-directed movements have different neural correlates than movements without a particular goal. In a reach-and-touch task, we explored the differences in the movement-related cortical potentials (MRCPs) between goal-directed and non-goal-directed movements. We evaluated if the detection of movement intention was influenced by the goal-directedness of the movement. In a single-trial classification procedure we found that classification accuracies are enhanced if there is a goal-directed movement in mind. Furthermore, by using the classifier patterns and estimating the corresponding brain sources, we show the importance of motor areas and the additional involvement of the posterior parietal lobule in the discrimination between goal-directed movements and non-goal-directed movements. We discuss next the potential contribution of our results on goal-directed movements to a more reliable brain-computer interface (BCI) control that facilitates recovery in spinal-cord injured or stroke end-users.

Bioelectric Medicine and Devices for the Treatment of Spinal Cord Injury

Recovery of motor control is paramount for patients living with paralysis following spinal cord injury (SCI). While a cure or regenerative intervention remains on the horizon for the treatment of SCI, a number of neuroprosthetic devices have been employed to treat and mitigate the symptoms of paralysis associated with injuries to the spinal column and associated comorbidities. The recent success of epidural stimulation to restore voluntary motor function in the lower limbs of a small cohort of patients has breathed new life into the promise of electric-based medicine. Recently, a number of new organic and inorganic electronic devices have been developed for brain-computer interfaces to bypass the injury, for neurorehabilitation, bladder and bowel control, and the restoration of motor or sensory control. Herein, we discuss the recent advances in neuroprosthetic devices for treating SCI and highlight future design needs for closed-loop device systems.

3D hand motion trajectory prediction from EEG mu and beta bandpower

A motion trajectory prediction (MTP) - based brain-computer interface (BCI) aims to reconstruct the three-dimensional (3D) trajectory of upper limb movement using electroencephalography (EEG). The most common MTP BCI employs a time series of bandpass-filtered EEG potentials (referred to here as the potential time-series, PTS, model) for reconstructing the trajectory of a 3D limb movement using multiple linear regression. These studies report the best accuracy when a 0.5-2Hz bandpass filter is applied to the EEG. In the present study, we show that spatiotemporal power distribution of theta (4-8Hz), mu (8-12Hz), and beta (12-28Hz) bands are more robust for movement trajectory decoding when the standard PTS approach is replaced with time-varying bandpower values of a specified EEG band, ie, with a bandpower time-series (BTS) model. A comprehensive analysis comprising of three subjects performing pointing movements with the dominant right arm toward six targets is presented. Our results show that the BTS model produces significantly higher MTP accuracy (R~0.45) compared to the standard PTS model (R~0.2). In the case of the BTS model, the highest accuracy was achieved across the three subjects typically in the mu (8-12Hz) and low-beta (12-18Hz) bands. Additionally, we highlight a limitation of the commonly used PTS model and illustrate how this model may be suboptimal for decoding motion trajectory relevant information. Although our results, showing that the mu and beta bands are prominent for MTP, are not in line with other MTP studies, they are consistent with the extensive literature on classical multiclass sensorimotor rhythm-based BCI studies (classification of limbs as opposed to motion trajectory prediction), which report the best accuracy of imagined limb movement classification using power values of mu and beta frequency bands. The methods proposed here provide a positive step toward noninvasive decoding of imagined 3D hand movements for movement-free BCIs.

Block design enhances classification of 3D reach targets from electroencephalographic signals

To date, decoding accuracy of actual or imagined pointing movements to targets in 3D space from electroencephalographic (EEG) signals has remained modest. The reason may pertain to the fact that these movements activate essentially the same neural networks. In this study, we aimed at testing whether repetitive pointing movements to each of the targets promotes the development of segregated neural patterns, resulting in enhanced decoding accuracy. Six human subjects generated slow or fast repetitive pointing movements with their right dominant arm to one of five targets distributed in 3D space, followed by repetitive imagery of movements to the same target or to a different target. Nine naive subjects generated both repetitive and non-repetitive slow actual movements to each of the five targets to test the effect of block design on decoding accuracy. In order to assure that base line drift and low frequency motion artifacts do not contaminate the data, the data were high-pass filtered in 4-30Hz, leaving out the delta and gamma band. For the repetitive trials, the model decoded target location with 81% accuracy, which is significantly higher than chance level. The average decoding rate of target location was only 30% for the non-repetitive trials, which is not significantly different than chance level. A subset of electrodes, mainly over the contralateral sensorimotor areas, was found to provide most of the discriminative features for all tested conditions. Time proximity between trained and tested blocks was found to enhance decoding accuracy of target location both by target non-specific and specific mechanisms. Our findings suggest that movement repetition promotes the development of distinct neural patterns, presumably by the formation of target-specific kinesthetic memory.

Quaternion-Based Signal Analysis for Motor Imagery Classification from Electroencephalographic Signals

Quaternions can be used as an alternative to model the fundamental patterns of electroencephalographic (EEG) signals in the time domain. Thus, this article presents a new quaternion-based technique known as quaternion-based signal analysis (QSA) to represent EEG signals obtained using a brain-computer interface (BCI) device to detect and interpret cognitive activity. This quaternion-based signal analysis technique can extract features to represent brain activity related to motor imagery accurately in various mental states. Experimental tests in which users where shown visual graphical cues related to left and right movements were used to collect BCI-recorded signals. These signals were then classified using decision trees (DT), support vector machine (SVM) and k-nearest neighbor (KNN) techniques. The quantitative analysis of the classifiers demonstrates that this technique can be used as an alternative in the EEG-signal modeling phase to identify mental states.

Movement target decoding from EEG and the corresponding discriminative sources: A preliminary study

Brain-computer interfaces (BCIs) can detect movement imaginations (MI) which can act as a control signal for a neuroprosthesis of a paralyzed person. However, today's non-invasive BCIs can only detect simply qualities of MI, like what body part is subjected to MI. More advanced future non-invasive BCIs should be able to detect many qualities of MI to allow a natural control of a neuroprosthesis. In this preliminary study, we decoded movement targets during a self-paced center-out reaching task, and calculated corresponding spatial patterns in the source space. We were able to decode the movement targets with significant classification accuracy from one out of three subjects during the movement planning phase. This subject showed a distinct spatial pattern over the central motor area.

Goal-directed or aimless? EEG differences during the preparation of a reach-and-touch task

The natural control of neuroprostheses is currently a challenge in both rehabilitation engineering and brain-computer interfaces (BCIs) research. One of the recurrent problems is to know exactly when to activate such devices. For the execution of the most common activities of daily living, these devices only need to be active when in the presence of a goal. Therefore, we believe that the distinction between the planning of goal-directed and aimless movements, using non-invasive recordings, can be useful for the implementation of a simple and effective activation method for these devices. We investigated whether those differences are detectable during a reach-and-touch task, using electroencephalography (EEG). Event-related potentials and oscillatory activity changes were studied. Our results show that there are statistically significant differences between both types of movement. Combining this information with movement decoding would allow a natural control strategy for BCIs, exclusively relying on the cognitive processes behind movement preparation and execution.