Decoding finger movement in humans using synergy of EEG cortical current signals

Modeling the cortical response elicited by wrist manipulation via a nonlinear delay differential embedding

Regarding motor processes, modeling healthy people's brains is essential to understand the brain activity in people with motor impairments. However, little research has been undertaken when external forces disturb limbs, having limited information on physiological pathways. Therefore, in this paper, a nonlinear delay differential embedding model is used to estimate the brain response elicited by externally controlled wrist movement in healthy individuals. The aim is to improve the understanding of the relationship between a controlled wrist movement and the generated cortical activity of healthy people, helping to disclose the underlying mechanisms and physiological relationships involved in the motor event. To evaluate the model, a public database from the Delft University of Technology is used, which contains electroencephalographic recordings of ten healthy subjects while wrist movement was externally provoked by a robotic system. In this work, the cortical response related to movement is identified via Independent Component Analysis and estimated based on a nonlinear delay differential embedding model. After a cross-validation analysis, the model performance reaches 90.21% ± 4.46% Variance Accounted For, and Correlation 95.14% ± 2.31%. The proposed methodology allows to select the model degree, to estimate a general predominant operation mode of the cortical response elicited by wrist movement. The obtained results revealed two facts that had not previously been reported: the movement's acceleration affects the cortical response, and a common delayed activity is shared among subjects. Going forward, identifying biomarkers related to motor tasks could aid in the evaluation of rehabilitation treatments for patients with upper limbs motor impairments.

Decoding reach-to-grasp from EEG using classifiers trained with data from the contralateral limb

Fundamental to human movement is the ability to interact with objects in our environment. How one reaches an object depends on the object's shape and intended interaction afforded by the object, e.g., grasp and transport. Extensive research has revealed that the motor intention of reach-to-grasp can be decoded from cortical activities using EEG signals. The goal of the present study is to determine the extent to which information encoded in the EEG signals is shared between two limbs to enable cross-hand decoding. We performed an experiment in which human subjects (n = 10) were tasked to interact with a novel object with multiple affordances using either right or left hands. The object had two vertical handles attached to a horizontal base. A visual cue instructs what action (lift or touch) and whether the left or right handle should be used for each trial. EEG was recorded and processed from bilateral frontal-central-parietal regions (30 channels). We trained LDA classifiers using data from trials performed by one limb and tested the classification accuracy using data from trials performed by the contralateral limb. We found that the type of hand-object interaction can be decoded with approximately 59 and 69% peak accuracy in the planning and execution stages, respectively. Interestingly, the decoding accuracy of the reaching directions was dependent on how EEG channels in the testing dataset were spatially mirrored, and whether directions were labeled in the extrinsic (object-centered) or intrinsic (body-centered) coordinates.

The simultaneous changes in motor performance and EEG patterns in beta band during learning dart throwing skill in dominant and non-dominant hand

Background: Although changes in performance during the learning of various sports skills have been studied, however, how these changes at the brain level is still unknown. The aim of this study was to investigate simultaneous changes in motor performance and EEG patterns in beta band during learning dart throwing skill in dominant and non-dominant hand. Methodology: The samples consisted of 14 non-athlete students with an average age of 23 ± 2.5, which were divided into two group dominant hand (7) and non-dominant hand (7). Repeated measures ANOVA were used to measure data at the execution level and changes in EEG activity. Results: The results of this study at the performance level showed a significant reduction in the absolute error of dart throwing and at the same time at the brain level increased EEG activity in frontal and parietal-posterior regions along with decreased central area activity in acquisition and retention stages in both groups (P<.05). Also, there was a significant difference between the activity of EEG pattern in the dominant and non-dominant hand groups except for two channels AF3 and PO4 (P<.05). Conclusion: In general, the results of this study showed that along with relatively constant changes in performance during dart skill learning, relatively constant changes in EEG activity pattern occur, so that the concept of motor learning is also visible at the brain level. Also, the results of this study supported the existence of the different motor program for dominant and non-dominant hand control in the conditions of bilateral transfer control.

Clustered event related spectral perturbation (ERSP) feature in right hand motor imagery classification

A technology that allows humans to interact with machines more directly and efficiently would be desirable. Research on brain-computer interfaces (BCIs) provides the possibility for computers to understand human thoughts in a straightforward manner thereby facilitating communication. As a branch of BCI research, motor imagery (MI) techniques analyze the brain signals and help people in many aspects such as rehabilitation, clinical applications, entertainment, and system controlling. In this study, an imagery experiment consisting of four kinds of right-hand movements (gripping, opening, pronation, and supination) was designed. Then a novel feature, namely, clustered feature was proposed based on the event-related spectral perturbation (ERSP) calculated from the EEG signal. Based on the selected features, two classical classifiers (support vector machine and linear discriminant classifier) were trained, achieving an acceptable accurate result (80%, on average).

Reconstructing Synergy-Based Hand Grasp Kinematics from Electroencephalographic Signals

Brain-machine interfaces (BMIs) have become increasingly popular in restoring the lost motor function in individuals with disabilities. Several research studies suggest that the CNS may employ synergies or movement primitives to reduce the complexity of control rather than controlling each DoF independently, and the synergies can be used as an optimal control mechanism by the CNS in simplifying and achieving complex movements. Our group has previously demonstrated neural decoding of synergy-based hand movements and used synergies effectively in driving hand exoskeletons. In this study, ten healthy right-handed participants were asked to perform six types of hand grasps representative of the activities of daily living while their neural activities were recorded using electroencephalography (EEG). From half of the participants, hand kinematic synergies were derived, and a neural decoder was developed, based on the correlation between hand synergies and corresponding cortical activity, using multivariate linear regression. Using the synergies and the neural decoder derived from the first half of the participants and only cortical activities from the remaining half of the participants, their hand kinematics were reconstructed with an average accuracy above 70%. Potential applications of synergy-based BMIs for controlling assistive devices in individuals with upper limb motor deficits, implications of the results in individuals with stroke and the limitations of the study were discussed.

Brain-inspired spiking neural networks for decoding and understanding muscle activity and kinematics from electroencephalography signals during hand movements

Compared to the abilities of the animal brain, many Artificial Intelligence systems have limitations which emphasise the need for a Brain-Inspired Artificial Intelligence paradigm. This paper proposes a novel Brain-Inspired Spiking Neural Network (BI-SNN) model for incremental learning of spike sequences. BI-SNN maps spiking activity from input channels into a high dimensional source-space which enhances the evolution of polychronising spiking neural populations. We applied the BI-SNN to predict muscle activity and kinematics from electroencephalography signals during upper limb functional movements. The BI-SNN extends our previously proposed eSPANNet computational model by integrating it with the ‘NeuCube’ brain-inspired SNN architecture. We show that BI-SNN can successfully predict continuous muscle activity and kinematics of upper-limb. The experimental results confirmed that the BI-SNN resulted in strongly correlated population activity and demonstrated the feasibility for real-time prediction. In contrast to the majority of Brain–Computer Interfaces (BCIs) that constitute a ‘black box’, BI-SNN provide quantitative and visual feedback about the related brain activity. This study is one of the first attempts to examine the feasibility of finding neural correlates of muscle activity and kinematics from electroencephalography using a brain-inspired computational paradigm. The findings suggest that BI-SNN is a better neural decoder for non-invasive BCI.

A new framework for classification of multi-category hand grasps using EMG signals

Electromyogram (EMG) signals have had a great impact on many applications, including prosthetic or rehabilitation devices, human-machine interactions, clinical and biomedical areas. In recent years, EMG signals have been used as a popular tool to generate device control commands for rehabilitation equipment, such as robotic prostheses. This intention of this study was to design an EMG signal-based expert model for hand-grasp classification that could enhance prosthetic hand movements for people with disabilities. The study, thus, aimed to introduce an innovative framework for recognising hand movements using EMG signals. The proposed framework consists of logarithmic spectrogram-based graph signal (LSGS), AdaBoost k-means (AB-k-means) and an ensemble of feature selection (FS) techniques. First, the LSGS model is applied to analyse and extract the desirable features from EMG signals. Then, to assist in selecting the most influential features, an ensemble FS is added to the design. Finally, in the classification phase, a novel classification model, named AB-k-means, is developed to classify the selected EMG features into different hand grasps. The proposed hybrid model, LSGS-based scheme is evaluated with a publicly available EMG hand movement dataset from the UCI repository. Using the same dataset, the LSGS-AB-k-means design model is also benchmarked with several classifications including the state-of-the-art algorithms. The results demonstrate that the proposed model achieves a high classification rate and demonstrates superior results compared to several previous research works. This study, therefore, establishes that the proposed model can accurately classify EMG hand grasps and can be implemented as a control unit with low cost and a high classification rate.

The Effect of ICA and Non-negative Matrix Factorization Analysis for EMG Signals Recorded From Multi-Channel EMG Sensors

Surface electromyography (EMG) measurements are affected by various noises such as power source and movement artifacts and adjacent muscle activities. Hardware solutions have been found that use multi-channel EMG signal to attenuate noise signals related to sensor positions. However, studies addressing the overcoming of crosstalk from EMG and the division of overlaid superficial and deep muscles are scarce. In this study, two signal decompositions-independent component analysis and non-negative matrix factorization-were used to create a low-dimensional input signal that divides noise, surface muscles, and deep muscles and utilizes them for movement classification based on direction. In the case of index finger movement, it was confirmed that the proposed decomposition method improved the classification performance with the least input dimensions. These results suggest a new method to analyze more dexterous movements of the hand by separating superficial and deep muscles in the future using multi-channel EMG signals.

Neural coordination of bilateral power and precision finger movements

The dexterity of hands and fingers is related to the strength of control by cortico-motoneuronal connections which exclusively exist in primates. The cortical command is associated with a task-specific, rapid proprioceptive adaptation of forces applied by hands and fingers to an object. This neural control differs between "power grip" movements (e.g., reach and grasp of a cup) where hand and fingers act as a unity and "precision grip" movements (e.g., picking up a raspberry) where fingers move independently from the hand. In motor tasks requiring hands and fingers of both sides a "neural coupling" (reflected in bilateral reflex responses to unilateral stimulations) coordinates power grip movements (e.g., opening a bottle). In contrast, during bilateral precision movements, such as playing piano, the fingers of both hands move independently, due to a direct cortico-motoneuronal control, while the hands are coupled (e.g., to maintain the rhythm between the two sides). While most studies on prehension concern unilateral hand movements, many activities of daily life are tackled by bilateral power grips where a neural coupling serves for an automatic movement performance. In primates this mode of motor control is supplemented by a system that enables the uni- or bilateral performance of skilled individual finger movements.

Investigation of Delayed Response during Real-Time Cursor Control Using Electroencephalography

Error-related brain activation has been investigated for advanced brain-machine interfaces (BMI). However, how a delayed response of cursor control in BMI systems should be handled is not clear. Therefore, the purpose of this study was to investigate how participants responded to delayed cursor control. Six subjects participated in the experiment and performed a wrist-bending task. For three distinct delay intervals (an interval where participants could not perceive the delay, an interval where participants could not be sure whether there was a delay or not, and an interval where participants could perceive the delay), we assessed two types of binary classifications (“Yes + No” vs. “I don't know” and “Yes” vs. “No”) based on participants' responses and applied delay times (thus, four types of classification, overall). For most participants, the “Yes vs. No” classification had higher accuracy than “Yes + No” vs. “I don't know” classification. For the “Yes + No” vs. “I don't know” classification, most participants displayed higher accuracy based on response classification than delay classification. Our results demonstrate that a class only for “I don't know” largely contributed to these differences. Many independent components (ICs) that exhibited high accuracy in “Yes + No” vs. “I don't know” response classification were associated with activation of areas from the frontal to parietal lobes, while many ICs that showed high accuracy in the “Yes vs. No” classification were associated with activation of an area ranging from the parietal to the occipital lobes and were more broadly localized in cortical regions than was seen for the “Yes + No” vs. “I don't know” classification. Our results suggest that small and large delays in real-time cursor control differ not only in the magnitude of the delay but should be handled as distinct information in different ways and might involve differential processing in the brain.

MEG Source Imaging and Group Analysis Using VBMEG

Variational Bayesian Multimodal EncephaloGraphy (VBMEG) is a MATLAB toolbox that estimates distributed source currents from magnetoencephalography (MEG)/electroencephalography (EEG) data by integrating functional MRI (fMRI) (https://vbmeg.atr.jp/). VBMEG also estimates whole-brain connectome dynamics using anatomical connectivity derived from a diffusion MRI (dMRI). In this paper, we introduce the VBMEG toolbox and demonstrate its usefulness. By collaborating with VBMEG's tutorial page (https://vbmeg.atr.jp/docs/v2/static/vbmeg2_tutorial_neuromag.html), we show its full pipeline using an open dataset recorded by Wakeman and Henson (2015). We import the MEG data and preprocess them to estimate the source currents. From the estimated source currents, we perform a group analysis and examine the differences of current amplitudes between conditions by controlling the false discovery rate (FDR), which yields results consistent with previous studies. We highlight VBMEG's characteristics by comparing these results with those obtained by other source imaging methods: weighted minimum norm estimate (wMNE), dynamic statistical parametric mapping (dSPM), and linearly constrained minimum variance (LCMV) beamformer. We also estimate source currents from the EEG data and the whole-brain connectome dynamics from the MEG data and dMRI. The observed results indicate the reliability, characteristics, and usefulness of VBMEG.

Continuous Wavelet Transform for Decoding Finger Movements From Single-Channel EEG

Human body movements can be reflected in brain signals and collected noninvasively with electroencephalography (EEG). Motor-related signals include sensory motor rhythms (also known as the Mu wave) in the upper-alpha band of 8-13 Hz and slow cortical potentials (SCPs) in the low frequency range of 0.1-5 Hz. This study compares the two signals for decoding finger movements. Human subjects were asked to individually lift each of the five digits of their right hand, at the rate of one every 10 s. EEG was recorded using a bipolar montage between ipsilateral and contralateral motor cortices. Electromyograms were obtained for identifying movement onsets. Linear discriminant analysis (LDA) generated significant performance with SCPs but not with Mu. Meanwhile, continuous wavelet transform (CWT) was applied to SCPs or Mu to create a spectrogram for each finger, showing distinctive amplitude and phase patterns. A dprime-based weighting algorithm was used to extract time-frequency features. With a template-matching paradigm, both SCP and Mu spectrograms yielded significant classification accuracies for multiple subjects, with the highest being >50% (chance = 20%). Interestingly, the index finger was better distinguished with Mu for most of the subjects, whereas the ring finger was better distinguished with SCPs. The CWT algorithm outperformed LDA by better decoding the thumb. This study suggests that the time-frequency characteristics of a single-channel EEG, when phase is preserved, contain critical information on finger movements. SCPs and Mu seem to work in an independent but complementary manner.

Dynamic Information Flow Based on EEG and Diffusion MRI in Stroke: A Proof-of-Principle Study

In hemiparetic stroke, functional recovery of paretic limb may occur with the reorganization of neural networks in the brain. Neuroimaging techniques, such as magnetic resonance imaging (MRI), have a high spatial resolution which can be used to reveal anatomical changes in the brain following a stroke. However, low temporal resolution of MRI provides less insight of dynamic changes of brain activity. In contrast, electro-neurophysiological techniques, such as electroencephalography (EEG), have an excellent temporal resolution to measure such transient events, however are hindered by its low spatial resolution. This proof-of-principle study assessed a novel multimodal brain imaging technique namely Variational Bayesian Multimodal Encephalography (VBMEG), which aims to improve the spatial resolution of EEG for tracking the information flow inside the brain and its changes following a stroke. The limitations of EEG are complemented by constraints derived from anatomical MRI and diffusion weighted imaging (DWI). EEG data were acquired from individuals suffering from a stroke as well as able-bodied participants while electrical stimuli were delivered sequentially at their index finger in the left and right hand, respectively. The locations of active sources related to this stimulus were precisely identified, resulting in high Variance Accounted For (VAF above 80%). An accurate estimation of dynamic information flow between sources was achieved in this study, showing a high VAF (above 90%) in the cross-validation test. The estimated dynamic information flow was compared between chronic hemiparetic stroke and able-bodied individuals. The results demonstrate the feasibility of VBMEG method in revealing the changes of information flow in the brain after stroke. This study verified the VBMEG method as an advanced computational approach to track the dynamic information flow in the brain following a stroke. This may lead to the development of a quantitative tool for monitoring functional changes of the cortical neural networks after a unilateral brain injury and therefore facilitate the research into, and the practice of stroke rehabilitation.

Perspective-Taking in Sentence Comprehension: Time and Empathy

This study examines the neural substrate of perspective-taking by analyzing the electroencephalographic (EEG) activity elicited by the auditory comprehension of sentences for which the comprehender had to adopt the perspective of the person described in them. Recent studies suggest that the ability of perspective-taking can be an integrative function of temporal and spatial information processing. We thus examined the independence and possible interaction of human perspective shifts and temporal perspective-taking by utilizing Japanese subsidiary verbs for giving, namely -ageru and -kureru. We manipulated human perspective shifts and temporal perspective-taking independently in experimental sentences by syntactically changing the subject and the object between the speaker and a third person, while we manipulated the tense to be past or non-past tense via sentence-final particles ru/ta (non-past/past). The EEG analyses via electrodes indicated the suppression of the β band for human perspective shifts in sentences in non-past tense and the absence of such suppression in sentences in past tense. The analyses for the clusters of independent components indicated β suppression for past tense against non-past tense in sentences without a human perspective shift. This response pattern suggests a close relationship between human perspective shifting and temporal perspective-taking. The β suppression for the human perspective shift in our experiment can be understood as a replication of the previous EEG findings observed for perspective-taking in the presentation of visual images. The preceding findings and our result suggest that the ability or the function of perspective-taking is not specific to the modality. Furthermore, the generator of the β suppression for past tense against non-past tense without human perspective shifting was localized in the precuneus, which is consistent with recent findings indicating that the precuneus is deeply involved in time perception.