Human EEG reveals distinct neural correlates of power and precision grasping types

Decoding of unimanual and bimanual reach-and-grasp actions from EMG and IMU signals in persons with cervical spinal cord injury

Objective. Chronic motor impairments of arms and hands as the consequence of a cervical spinal cord injury (SCI) have a tremendous impact on activities of daily life. A considerable number of people however retain minimal voluntary motor control in the paralyzed parts of the upper limbs that are measurable by electromyography (EMG) and inertial measurement units (IMUs). An integration into human-machine interfaces (HMIs) holds promise for reliable grasp intent detection and intuitive assistive device control.Approach. We used a multimodal HMI incorporating EMG and IMU data to decode reach-and-grasp movements of groups of persons with cervical SCI (n = 4) and without (control, n = 13). A post-hoc evaluation of control group data aimed to identify optimal parameters for online, co-adaptive closed-loop HMI sessions with persons with cervical SCI. We compared the performance of real-time, Random Forest-based movement versus rest (2 classes) and grasp type predictors (3 classes) with respect to their co-adaptation and evaluated the underlying feature importance maps.Main results. Our multimodal approach enabled grasp decoding significantly better than EMG or IMU data alone (p<0.05). We found the 0.25 s directly prior to the first touch of an object to hold the most discriminative information. Our HMIs correctly predicted 79.3 ± STD 7.4 (102.7 ± STD 2.3 control group) out of 105 trials with grand average movement vs. rest prediction accuracies above 99.64% (100% sensitivity) and grasp prediction accuracies of 75.39 ± STD 13.77% (97.66 ± STD 5.48% control group). Co-adaption led to higher prediction accuracies with time, and we could identify adaptions in feature importances unique to each participant with cervical SCI.Significance. Our findings foster the development of multimodal and adaptive HMIs to allow persons with cervical SCI the intuitive control of assistive devices to improve personal independence.

Coherent activity within and between hemispheres: cortico-cortical connectivity revealed by rTMS of the right posterior parietal cortex

Introduction

Low frequency (1 Hz) repetitive transcranial stimulation (rTMS) applied over right posterior parietal cortex (rPPC) has been shown to reduce cortical excitability both of the stimulated area and of the interconnected contralateral homologous areas. In the present study, we investigated the whole pattern of intra- and inter-hemispheric cortico-cortical connectivity changes induced by rTMS over rPPC.

Methods

To do so, 14 healthy participants underwent resting state EEG recording before and after 30 min of rTMS at 1 Hz or sham stimulation over the rPPC (electrode position P6). Real stimulation was applied at 90% of motor threshold. Coherence values were computed on the electrodes nearby the stimulated site (i.e., P4, P8, and CP6) considering all possible inter- and intra-hemispheric combinations for the following frequency bands: delta (0.5-4 Hz), theta (4-8 Hz), alpha (8-12Hz), low beta (12-20 Hz), high beta (20-30 Hz), and gamma (30-50 Hz).

Results and discussion

Results revealed a significant increase in coherence in delta, theta, alpha and beta frequency bands between rPPC and the contralateral homologous sites. Moreover, an increase in coherence in theta, alpha, beta and gamma frequency bands was found between rPPC and right frontal sites, reflecting the activation of the fronto-parietal network within the right hemisphere. Summarizing, subthreshold rTMS over rPPC revealed cortico-cortical inter- and intra-hemispheric connectivity as measured by the increase in coherence among these areas. Moreover, the present results further confirm previous evidence indicating that the increase of coherence values is related to intra- and inter-hemispheric inhibitory effects of rTMS. These results can have implications for devising evidence-based rehabilitation protocols after stroke.

Changes in EEG alpha-band power during prehension indicates neural motor drive inhibition

Changes in alpha band activity (8-12 Hz) indicate the downregulation of brain regions during cognitive tasks, reflecting real-time cognitive load. Despite this, its feasibility to be used in a more dynamic environment with ongoing motor corrections has not been studied. This research used electroencephalography (EEG) to explore how different brain regions are engaged during a simple grasp and lift task where unexpected changes to the object's weight or surface friction are introduced. The results suggest that alpha activity changes related to motor error correction occur only in motor-related areas (i.e. central areas) but not in error processing areas (i.e., frontoparietal network) during unexpected weight changes. This suggests that oscillations over motor areas reflect the reduction of motor drive related to motor error correction, thus, being a potential cortical electrophysiological biomarker for the process and not solely as a proxy for cognitive demands. This observation is particularly relevant in scenarios where these signals are used to evaluate high cognitive demands co-occurring with high levels of motor errors and corrections, such as prosthesis use. The establishment of electrophysiological biomarkers of mental resource allocation during movement and cognition can help identify indicators of mental workload and motor drive, which may be useful for improving brain-machine interfaces.NEW & NOTEWORTHY We demonstrated that alpha suppression, an EEG phenomenon with high temporal resolution, occurs over the primary sensorimotor area during error correction during lift movements. Interpretations of alpha activity are often attributed to high cognitive demands, thus recognizing that it is also influenced by motor processes is important in situations where cognitive demands are paired with movement errors. This could further have application as a biomarker for error correction in human-machine interfaces, such as neuroprostheses.

Decoding reach-to-grasp from EEG using classifiers trained with data from the contralateral limb

Fundamental to human movement is the ability to interact with objects in our environment. How one reaches an object depends on the object's shape and intended interaction afforded by the object, e.g., grasp and transport. Extensive research has revealed that the motor intention of reach-to-grasp can be decoded from cortical activities using EEG signals. The goal of the present study is to determine the extent to which information encoded in the EEG signals is shared between two limbs to enable cross-hand decoding. We performed an experiment in which human subjects (n = 10) were tasked to interact with a novel object with multiple affordances using either right or left hands. The object had two vertical handles attached to a horizontal base. A visual cue instructs what action (lift or touch) and whether the left or right handle should be used for each trial. EEG was recorded and processed from bilateral frontal-central-parietal regions (30 channels). We trained LDA classifiers using data from trials performed by one limb and tested the classification accuracy using data from trials performed by the contralateral limb. We found that the type of hand-object interaction can be decoded with approximately 59 and 69% peak accuracy in the planning and execution stages, respectively. Interestingly, the decoding accuracy of the reaching directions was dependent on how EEG channels in the testing dataset were spatially mirrored, and whether directions were labeled in the extrinsic (object-centered) or intrinsic (body-centered) coordinates.

Cortical Contributions to Imagined Power Grip Task: An EEG-Triggered TMS Study

Previous studies have demonstrated that motor imagery leads to desynchronization in the alpha rhythm within the contralateral primary motor cortex. However, the underlying electrophysiological mechanisms responsible for this desynchronization during motor imagery remain unclear. To examine this question, we conducted an investigation using EEG in combination with noninvasive transcranial magnetic stimulation (TMS) during index finger abduction (ABD) and power grip imaginations. The TMS was administered employing diverse coil orientations to selectively stimulate corticospinal axons, aiming to target both early and late synaptic inputs to corticospinal neurons. TMS was triggered based on the alpha power levels, categorized in 20th percentile bins, derived from the individual alpha power distribution during the imagined tasks of ABD and power grip. Our analysis revealed negative correlations between alpha power and motor evoked potential (MEP) amplitude, as well as positive correlations with MEP latency across all coil orientations for each imagined task. Furthermore, we conducted functional network analysis in the alpha band to explore network connectivity during imagined index finger abduction and power grip tasks. Our findings indicate that network connections were denser in the fronto-parietal area during imagined ABD compared to power grip conditions. Moreover, the functional network properties demonstrated potential for effectively classifying between these two imagined tasks. These results provide functional evidence supporting the hypothesis that alpha oscillations may play a role in suppressing MEP amplitude and latency during imagined power grip. We propose that imagined ABD and power grip tasks may activate different populations and densities of axons at the cortical level.

Cortico-cortical paired associative stimulation conditioning superficial ventral premotor cortex-primary motor cortex connectivity influences motor cortical activity during precision grip

The ventral premotor cortex (PMv) and primary motor cortex (M1) represent critical nodes of a parietofrontal network involved in grasping actions, such as power and precision grip. Here, we investigated how the functional PMv-M1 connectivity drives the dissociation between these two actions. We applied a PMv-M1 cortico-cortical paired associative stimulation (cc-PAS) protocol, stimulating M1 in both postero-anterior (PA) and antero-posterior (AP) directions, in order to induce long-term changes in the activity of different neuronal populations within M1. We evaluated the motor-evoked potential (MEP) amplitude, MEP latency and cortical silent period, in both PA and AP, during the isometric execution of precision and power grip, before and after the PMv-M1 cc-PAS. The repeated activation of the PMv-M1 cortico-cortical network with PA orientation over M1 did not change MEP amplitude or cortical silent period duration during both actions. In contrast, the PMv-M1 cc-PAS stimulation of M1 with an AP direction led to a specific modulation of precision grip motor drive. In particular, MEPs tested with AP stimulation showed a selective increase of corticospinal excitability during precision grip. These findings suggest that the more superficial M1 neuronal populations recruited by the PMv input are involved preferentially in the execution of precision grip actions. KEY POINTS: Ventral premotor cortex (PMv)-primary motor cortex (M1) cortico-cortical paired associative stimulation (cc-PAS) with different coil orientation targets dissociable neural populations. PMv-M1 cc-PAS with M1 antero-posterior coil orientation specifically modulates corticospinal excitability during precision grip. Superficial M1 populations are involved preferentially in the execution of precision grip. A plasticity induction protocol targeting the specific PMv-M1 subpopulation might have important translational value for the rehabilitation of hand function.

Beta-band differences in primary motor cortex between media and non-media professionals when watching motor actions in movies

To watch a person doing an activity has an impact on the viewer. In fact, the film industry hinges on viewers looking at characters doing all sorts of narrative activities. From previous works, we know that media and non-media professionals perceive differently audiovisuals with cuts. Media professionals present a lower eye-blink rate, a lower activity in frontal and central cortical areas, and a more organized functional brain connectivity when watching audiovisual cuts. Here, we aimed to determine how audiovisuals with no formal interruptions such as cuts were perceived by media and non-media professionals. Moreover, we wondered how motor actions of characters in films would have an impact on the brain activities of the two groups of observers. We presented a narrative with 24 motor actions in a one-shot movie in wide shot with no cuts to 40 participants. We recorded the electroencephalographic (EEG) activity of the participants and analyzed it for the periods corresponding to the 24 motor actions (24 actions × 40 participants = 960 potential trials). In accordance with collected results, we observed differences in the EEG activity of the left primary motor cortex. A spectral analysis of recorded EEG traces indicated the presence of significant differences in the beta band between the two groups after the onset of the motor activities, while no such differences were found in the alpha band. We concluded that media expertise is related with the beta band identified in the EEG activity of the left primary motor cortex and the observation of motor actions in videos.

Exploring EEG spectral and temporal dynamics underlying a hand grasp movement

For brain-computer interfaces, resolving the differences between pre-movement and movement requires decoding neural ensemble activity in the motor cortex’s functional regions and behavioural patterns. Here, we explored the underlying neural activity and mechanisms concerning a grasped motor task by recording electroencephalography (EEG) signals during the execution of hand movements in healthy subjects. The grasped movement included different tasks; reaching the target, grasping the target, lifting the object upwards, and moving the object in the left or right directions. 163 trials of EEG data were acquired from 30 healthy participants who performed the grasped movement tasks. Rhythmic EEG activity was analysed during the premovement (alert task) condition and compared against grasped movement tasks while the arm was moved towards the left or right directions. The short positive to negative deflection that initiated around -0.5ms as a wave before the onset of movement cue can be used as a potential biomarker to differentiate movement initiation and movement. A rebound increment of 14% of beta oscillations and 26% gamma oscillations in the central regions was observed and could be used to distinguish pre-movement and grasped movement tasks. Comparing movement initiation to grasp showed a decrease of 10% in beta oscillations and 13% in gamma oscillations, and there was a rebound increment 4% beta and 3% gamma from grasp to grasped movement. We also investigated the combination MRCPs and spectral estimates of α, β, and γ oscillations as features for machine learning classifiers that could categorize movement conditions. Support vector machines with 3^rd order polynomial kernel yielded 70% accuracy. Pruning the ranked features to 5 leaf nodes reduced the error rate by 16%. For decoding grasped movement and in the context of BCI applications, this study identifies potential biomarkers, including the spatio-temporal characteristics of MRCPs, spectral information, and choice of classifiers for optimally distinguishing initiation and grasped movement.

The Inhibition Effect of Affordances in Action Picture Naming: An ERP Study

How quickly are different kinds of conceptual knowledge activated in action picture naming? Using a masked priming paradigm, we manipulated the prime category type (artificial vs. natural), prime action type (precision, power, vs. neutral grip), and target action type (precision vs. power grip) in action picture naming, while electrophysiological signals were measured concurrently. Naming latencies showed an inhibition effect in the congruent action type condition compared with the neutral condition. ERP results showed that artificial and natural category primes induced smaller waveforms in precision or power action primes than neutral primes in the time window of 100-200 msec. Time-frequency results consistently presented a power desynchronization of the mu rhythm in the time window of 0-210 msec with precision action type artificial objects compared with neutral primes, which localized at the supplementary motor, precentral and postcentral areas in the left hemisphere. These findings suggest an inhibitory effect of affordances arising at conceptual preparation in action picture naming and provide evidence for embodied cognition.

On the Extraction of Purely Motor EEG Neural Correlates during an Upper Limb Visuomotor Task

Deciphering and analyzing the neural correlates of different movements from the same limb using electroencephalography (EEG) would represent a notable breakthrough in the field of sensorimotor neurophysiology. Functional movements involve concurrent posture co-ordination and head and eye movements, which create electrical activity that affects EEG recordings. In this paper, we revisit the identification of brain signatures of different reaching movements using EEG and present, test, and validate a protocol to separate the effect of head and eye movements from a reaching task-related visuomotor brain activity. Ten healthy participants performed reaching movements under two different conditions: avoiding head and eye movements and moving with no constrains. Reaching movements can be identified from EEG with unconstrained eye and head movement, whereas the discriminability of the signals drops to chance level otherwise. These results show that neural patterns associated with different arm movements could only be extracted from EEG if the eye and head movements occurred concurrently with the task, polluting the recordings. Although these findings do not imply that brain correlates of reaching directions cannot be identified from EEG, they show the consequences that ignoring these events can have in any EEG study that includes a visuomotor task.

Decoding Different Reach-and-Grasp Movements Using Noninvasive Electroencephalogram

Grasping is one of the most indispensable functions of humans. Decoding reach-and-grasp actions from electroencephalograms (EEGs) is of great significance for the realization of intuitive and natural neuroprosthesis control, and the recovery or reconstruction of hand functions of patients with motor disorders. In this paper, we investigated decoding five different reach-and-grasp movements closely related to daily life using movement-related cortical potentials (MRCPs). In the experiment, nine healthy subjects were asked to naturally execute five different reach-and-grasp movements on the designed experimental platform, namely palmar, pinch, push, twist, and plug grasp. A total of 480 trials per subject (80 trials per condition) were recorded. The MRCPs amplitude from low-frequency (0.3-3 Hz) EEG signals were used as decoding features for further offline analysis. Average binary classification accuracy for grasping vs. the no-movement condition peaked at 75.06 ± 6.8%. Peak average accuracy for grasping vs. grasping conditions of 64.95 ± 7.4% could be reached. Grand average peak accuracy of multiclassification for five grasping conditions reached 36.7 ± 6.8% at 1.45 s after the movement onset. The analysis of MRCPs indicated that all the grasping conditions are more pronounced than the no-movement condition, and there are also significant differences between the grasping conditions. These findings clearly proved the feasibility of decoding multiple reach-and-grasp actions from noninvasive EEG signals. This work is significant for the natural and intuitive BCI application, particularly for neuroprosthesis control or developing an active human-machine interaction system, such as rehabilitation robot.

Stimulus evoked causality estimation in stereo-EEG

Objective.Stereo-electroencephalography (SEEG) has recently gained importance in analyzing brain functions. Its high temporal resolution and spatial specificity make it a powerful tool to investigate the strength, direction, and spectral content of brain networks interactions, especially when these connections are stimulus-evoked. However, choosing the best approach to evaluate the flow of information is not trivial, due to the lack of validated methods explicitly designed for SEEG.Approach.We propose a novel non-parametric statistical test for event-related causality (ERC) assessment on SEEG recordings. Here, we refer to the ERC as the causality evoked by a particular part of the stimulus (a response window (RW)). We also present a data surrogation method to evaluate the performance of a causality estimation algorithm. We finally validated our pipeline using surrogate SEEG data derived from an experimentally collected dataset, and compared the most used and successful measures to estimate effective connectivity, belonging to the Geweke-Granger causality framework.Main results.Here we show that our workflow correctly identified all the directed connections in the RW of the surrogate data and prove the robustness of the procedure against synthetic noise with amplitude exceeding physiological-plausible values. Among the causality measures tested, partial directed coherence performed best.Significance.This is the first non-parametric statistical test for ERC estimation explicitly designed for SEEG datasets. The pipeline, in principle, can also be applied to the analysis of any type of time-varying estimator, if there exists a clearly defined RW.

Deep learning multimodal fNIRS and EEG signals for bimanual grip force decoding

Objective.Non-invasive brain-machine interfaces (BMIs) offer an alternative, safe and accessible way to interact with the environment. To enable meaningful and stable physical interactions, BMIs need to decode forces. Although previously addressed in the unimanual case, controlling forces from both hands would enable BMI-users to perform a greater range of interactions. We here investigate the decoding of hand-specific forces.Approach.We maximise cortical information by using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) and developing a deep-learning architecture with attention and residual layers (cnnatt) to improve their fusion. Our task required participants to generate hand-specific force profiles on which we trained and tested our deep-learning and linear decoders.Main results.The use of EEG and fNIRS improved the decoding of bimanual force and the deep-learning models outperformed the linear model. In both cases, the greatest gain in performance was due to the detection of force generation. In particular, the detection of forces was hand-specific and better for the right dominant hand andcnnattwas better at fusing EEG and fNIRS. Consequently, the study ofcnnattrevealed that forces from each hand were differently encoded at the cortical level.Cnnattalso revealed traces of the cortical activity being modulated by the level of force which was not previously found using linear models.Significance.Our results can be applied to avoid hand-cross talk during hand force decoding to improve the robustness of BMI robotic devices. In particular, we improve the fusion of EEG and fNIRS signals and offer hand-specific interpretability of the encoded forces which are valuable during motor rehabilitation assessment.

Decoding Hand Movement Types and Kinematic Information From Electroencephalogram

Brain-computer interfaces (BCIs) have achieved successful control of assistive devices, e.g. neuroprosthesis or robotic arm. Previous research based on hand movements Electroencephalogram (EEG) has shown limited success in precise and natural control. In this study, we explored the possibilities of decoding movement types and kinematic information for three reach-and-execute actions using movement-related cortical potentials (MRCPs). EEG signals were acquired from 12 healthy subjects during the execution of pinch, palmar and precision disk rotation actions that involved two levels of speeds and forces. In the case of discrimination between hand movement types under each of four different kinematics conditions, we obtained the average peak accuracies of 83.44% and 73.83% for the binary and 3-class classification, respectively. In the case of discrimination between different movement kinematics for each of three actions, the average peak accuracies of 82.9% and 58.2% could be achieved for the two and 4-class scenario. In both cases, peak decoding performance was significantly higher than the subject-specific chance level. We found that hand movement types all could be classified when these actions were executed at four different kinematic parameters. Meanwhile, for each of three hand movements, we decoded movement parameters successfully. Furthermore, the feasibility of decoding hand movements during hand retraction process was also demonstrated. These findings are of great importance for controlling neuroprosthesis or other rehabilitation devices in a fine and natural way, which would drastically increase the acceptance of motor impaired users.

Online detection of movement during natural and self-initiated reach-and-grasp actions from EEG signals

Movement intention detection using electroencephalography (EEG) is a challenging but essential component of brain-computer interfaces (BCIs) for people with motor disabilities.Objective.The goal of this study is to develop a new experimental paradigm to perform asynchronous online detection of movement based on low-frequency time-domain EEG features, concretely on movement-related cortical potentials. The paradigm must be easily transferable to people without any residual upper-limb movement function and the BCI must be independent of upper-limb movement onset measurements and external cues.Approach. In a study with non-disabled participants, we evaluated a novel BCI paradigm to detect self-initiated reach-and-grasp movements. Two experimental conditions were involved. In one condition, participants performed reach-and-grasp movements to a target and simultaneously shifted their gaze towards it. In a control condition, participants solely shifted their gaze towards the target (oculomotor task). The participants freely decided when to initiate the tasks. After eye artefact correction, the EEG signals were time-locked to the saccade onset and the resulting amplitude features were exploited on a hierarchical classification approach to detect movement asynchronously.Main results. With regards to BCI performance, 54.1% (14.4% SD) of the movements were correctly identified, and all participants achieved a performance above chance-level (around 12%). An average of 21.5% (14.1% SD) of the oculomotor tasks were falsely detected as upper-limb movement. In an additional rest condition, 1.7 (1.6 SD) false positives per minute were measured. Through source imaging, movement information was mapped to sensorimotor, posterior parietal and occipital areas.Significance. We present a novel approach for movement detection using EEG signals which does not rely on upper-limb movement onset measurements or on the presentation of external cues. The participants' behaviour closely matches the natural behaviour during goal-directed reach-and-grasp movements, which also constitutes an advantage with respect to current BCI protocols.

Cortical signatures of precision grip force control in children, adolescents, and adults

Human dexterous motor control improves from childhood to adulthood, but little is known about the changes in cortico-cortical communication that support such ontogenetic refinement of motor skills. To investigate age-related differences in connectivity between cortical regions involved in dexterous control, we analyzed electroencephalographic data from 88 individuals (range 8-30 years) performing a visually guided precision grip task using dynamic causal modelling and parametric empirical Bayes. Our results demonstrate that bidirectional coupling in a canonical 'grasping network' is associated with precision grip performance across age groups. We further demonstrate greater backward coupling from higher-order to lower-order sensorimotor regions from late adolescence in addition to differential associations between connectivity strength in a premotor-prefrontal network and motor performance for different age groups. We interpret these findings as reflecting greater use of top-down and executive control processes with development. These results expand our understanding of the cortical mechanisms that support dexterous abilities through development.

Scalp electroencephalograms over ipsilateral sensorimotor cortex reflect contraction patterns of unilateral finger muscles

A variety of neural substrates are implicated in the initiation, coordination, and stabilization of voluntary movements underpinned by adaptive contraction and relaxation of agonist and antagonist muscles. To achieve such flexible and purposeful control of the human body, brain systems exhibit extensive modulation during the transition from resting state to motor execution and to maintain proper joint impedance. However, the neural structures contributing to such sensorimotor control under unconstrained and naturalistic conditions are not fully characterized. To elucidate which brain regions are implicated in generating and coordinating voluntary movements, we employed a physiologically inspired, two-stage method to decode relaxation and three patterns of contraction in unilateral finger muscles (i.e., extension, flexion, and co-contraction) from high-density scalp electroencephalograms (EEG). The decoder consisted of two parts employed in series. The first discriminated between relaxation and contraction. If the EEG data were discriminated as contraction, the second stage then discriminated among the three contraction patterns. Despite the difficulty in dissociating detailed contraction patterns of muscles within a limb from scalp EEG signals, the decoder performance was higher than chance-level by 2-fold in the four-class classification. Moreover, weighted features in the trained decoders revealed EEG features differentially contributing to decoding performance. During the first stage, consistent with previous reports, weighted features were localized around sensorimotor cortex (SM1) contralateral to the activated fingers, while those during the second stage were localized around ipsilateral SM1. The loci of these weighted features suggested that the coordination of unilateral finger muscles induced different signaling patterns in ipsilateral SM1 contributing to motor control. Weighted EEG features enabled a deeper understanding of human sensorimotor processing as well as of a more naturalistic control of brain-computer interfaces.

Analyzing and Decoding Natural Reach-and-Grasp Actions Using Gel, Water and Dry EEG Systems

Reaching and grasping is an essential part of everybody's life, it allows meaningful interaction with the environment and is key to independent lifestyle. Recent electroencephalogram (EEG)-based studies have already shown that neural correlates of natural reach-and-grasp actions can be identified in the EEG. However, it is still in question whether these results obtained in a laboratory environment can make the transition to mobile applicable EEG systems for home use. In the current study, we investigated whether EEG-based correlates of natural reach-and-grasp actions can be successfully identified and decoded using mobile EEG systems, namely the water-based EEG-Versatile ^TM system and the dry-electrodes EEG-Hero ^TM headset. In addition, we also analyzed gel-based recordings obtained in a laboratory environment (g.USBamp/g.Ladybird, gold standard), which followed the same experimental parameters. For each recording system, 15 study participants performed 80 self-initiated reach-and-grasp actions toward a glass (palmar grasp) and a spoon (lateral grasp). Our results confirmed that EEG-based correlates of reach-and-grasp actions can be successfully identified using these mobile systems. In a single-trial multiclass-based decoding approach, which incorporated both movement conditions and rest, we could show that the low frequency time domain (LFTD) correlates were also decodable. Grand average peak accuracy calculated on unseen test data yielded for the water-based electrode system 62.3% (9.2% STD), whereas for the dry-electrodes headset reached 56.4% (8% STD). For the gel-based electrode system 61.3% (8.6% STD) could be achieved. To foster and promote further investigations in the field of EEG-based movement decoding, as well as to allow the interested community to make their own conclusions, we provide all datasets publicly available in the BNCI Horizon 2020 database (http://bnci-horizon-2020.eu/database/data-sets).

Neural coordination of bilateral power and precision finger movements

The dexterity of hands and fingers is related to the strength of control by cortico-motoneuronal connections which exclusively exist in primates. The cortical command is associated with a task-specific, rapid proprioceptive adaptation of forces applied by hands and fingers to an object. This neural control differs between "power grip" movements (e.g., reach and grasp of a cup) where hand and fingers act as a unity and "precision grip" movements (e.g., picking up a raspberry) where fingers move independently from the hand. In motor tasks requiring hands and fingers of both sides a "neural coupling" (reflected in bilateral reflex responses to unilateral stimulations) coordinates power grip movements (e.g., opening a bottle). In contrast, during bilateral precision movements, such as playing piano, the fingers of both hands move independently, due to a direct cortico-motoneuronal control, while the hands are coupled (e.g., to maintain the rhythm between the two sides). While most studies on prehension concern unilateral hand movements, many activities of daily life are tackled by bilateral power grips where a neural coupling serves for an automatic movement performance. In primates this mode of motor control is supplemented by a system that enables the uni- or bilateral performance of skilled individual finger movements.

Decoding hand movements from human EEG to control a robotic arm in a simulation environment

OBJECTIVE

Daily life tasks can become a significant challenge for motor impaired persons. Depending on the severity of their impairment, they require more complex solutions to retain an independent life. Brain-computer interfaces (BCIs) are targeted to provide an intuitive form of control for advanced assistive devices such as robotic arms or neuroprostheses. In our current study we aim to decode three different executed hand movements in an online BCI scenario from electroencephalographic (EEG) data.

APPROACH

Immersed in a desktop-based simulation environment, 15 non-disabled participants interacted with virtual objects from daily life by an avatar's robotic arm. In a short calibration phase, participants performed executed palmar and lateral grasps and wrist supinations. Using this data, we trained a classification model on features extracted from the low frequency time domain. In the subsequent evaluation phase, participants controlled the avatar's robotic arm and interacted with the virtual objects in case of a correct classification.

MAIN RESULTS

On average, participants scored online 48% of all movement trials correctly (3-condition scenario, adjusted chance level 40%, alpha = 0.05). The underlying movement-related cortical potentials (MRCPs) of the acquired calibration data show significant differences between conditions over contralateral central sensorimotor areas, which are retained in the data acquired from the online BCI use.

SIGNIFICANCE

We could show the successful online decoding of two grasps and one wrist supination movement using low frequency time domain features of the human EEG. These findings can potentially contribute to the development of a more natural and intuitive BCI-based control modality for upper limb motor neuroprostheses or robotic arms for people with motor impairments.

General principles of machine learning for brain-computer interfacing

Brain-computer interfaces (BCIs) are systems that translate brain activity patterns into commands that can be executed by an artificial device. This enables the possibility of controlling devices such as a prosthetic arm or exoskeleton, a wheelchair, typewriting applications, or games directly by modulating our brain activity. For this purpose, BCI systems rely on signal processing and machine learning algorithms to decode the brain activity. This chapter provides an overview of the main steps required to do such a process, including signal preprocessing, feature extraction and selection, and decoding. Given the large amount of possible methods that can be used for these processes, a comprehensive review of them is beyond the scope of this chapter, and it is focused instead on the general principles that should be taken into account, as well as discussing good practices on how these methods should be applied and evaluated for proper design of reliable BCI systems.

Multivariate Analysis of Electrophysiological Signals Reveals the Temporal Properties of Visuomotor Computations for Precision Grips

The frontoparietal networks underlying grasping movements have been extensively studied, especially using fMRI. Accordingly, whereas much is known about their cortical locus much less is known about the temporal dynamics of visuomotor transformations. Here, we show that multivariate EEG analysis allows for detailed insights into the time course of visual and visuomotor computations of precision grasps. Male and female human participants first previewed one of several objects and, upon its reappearance, reached to grasp it with the thumb and index finger along one of its two symmetry axes. Object shape classifiers reached transient accuracies of 70% at ∼105 ms, especially based on scalp sites over visual cortex, dropping to lower levels thereafter. Grasp orientation classifiers relied on a system of occipital-to-frontal electrodes. Their accuracy rose concurrently with shape classification but ramped up more gradually, and the slope of the classification curve predicted individual reaction times. Further, cross-temporal generalization revealed that dynamic shape representation involved early and late neural generators that reactivated one another. In contrast, grasp computations involved a chain of generators attaining a sustained state about 100 ms before movement onset. Our results reveal the progression of visual and visuomotor representations over the course of planning and executing grasp movements.SIGNIFICANCE STATEMENT Grasping an object requires the brain to perform visual-to-motor transformations of the object's properties. Although much of the neuroanatomic basis of visuomotor transformations has been uncovered, little is known about its time course. Here, we orthogonally manipulated object visual characteristics and grasp orientation, and used multivariate EEG analysis to reveal that visual and visuomotor computations follow similar time courses but display different properties and dynamics.

Selection of Features and Classifiers for EMG-EEG-Based Upper Limb Assistive Devices-A Review

Bio-signals are distinctive factors in the design of human-machine interface, essentially useful for prosthesis, orthosis, and exoskeletons. Despite the progress in the analysis of pattern recognition based devices; the acceptance of these devices is still questionable. One reason is the lack of information to identify the possible combinations of features and classifiers. Besides; there is also a need for optimal selection of various sensors for sensations such as touch, force, texture, along with EMGs/EEGs. This article reviews the two bio-signal techniques, named as electromyography and electroencephalography. The details of the features and the classifiers used in the data processing for upper limb assist devices are summarised here. Various features and their sets are surveyed and different classifiers for feature sets are discussed on the basis of the classification rate. The review was carried out on the basis of the last 10-12 years of published research in this area. This article also outlines the influence of modality of EMGs and EEGs with other sensors on classifications. Also, other bio-signals used in upper limb devices and future aspects are considered.

Utilizing High-Density Electroencephalography and Motion Capture Technology to Characterize Sensorimotor Integration While Performing Complex Actions

Studies of sensorimotor integration often use sensory stimuli that require a simple motor response, such as a reach or a grasp. Recent advances in neural recording techniques, motion capture technologies, and time-synchronization methods enable studying sensorimotor integration using more complex sensory stimuli and performed actions. Here, we demonstrate that prehensile actions that require using complex sensory instructions for manipulating different objects can be characterized using high-density electroencephalography and motion capture systems. In 20 participants, we presented stimuli in different sensory modalities (visual, auditory) containing different contextual information about the object with which to interact. Neural signals recorded near motor cortex and posterior parietal cortex discharged based on both the instruction delivered and object manipulated. Additionally, kinematics of the wrist movements could be discriminated between participants. These findings demonstrate a proof-of-concept behavioral paradigm for studying sensorimotor integration of multidimensional sensory stimuli to perform complex movements. The designed framework will prove vital for studying neural control of movements in clinical populations in which sensorimotor integration is impaired due to information no longer being communicated correctly between brain regions (e.g. stroke). Such a framework is the first step towards developing a neural rehabilitative system for restoring function more effectively.

Unimanual and Bimanual Reach-and-Grasp Actions Can Be Decoded From Human EEG

While most tasks of daily life can be handled through a small number of different grasps, many tasks require the action of both hands. In these bimanual tasks, the second hand has either a supporting role (e.g. for fixating a jar) or a more active role (e.g. grasping a pot on both handles). In this study we attempt to discriminate the neural correlates of unimanual (performed with left and right hand) from bimanual reach-and-grasp actions using the low-frequency time-domain electroencephalogram (EEG). In a self-initiated movement task, 15 healthy participants were asked to perform unimanual (palmar and lateral grasps with left and right hand) and bimanual (double lateral, mixed palmar/lateral) reach-and-grasps on objects of daily life. Using EEG time-domain features in the frequency range of 0.3-3 Hz, we achieved multiclass-classification accuracies of 38.6 ± 6.6% (7 classes, 17.1% chance level) for a combination of 6 movements and 1 rest condition. The grand average confusion matrix shows highest true positive rates (TPR) for the rest (63%) condition while TPR for the movement classes varied between 33 to 41%. The underlying movement-related cortical potentials (MRCPs) show significant differences between unimanual (e.g left hand vs. right hand grasps) as well unimanual vs. bimanual conditions which both can be attributed to lateralization effects. We believe that these findings can be exploited and further used for attempts in providing persons with spinal cord injury a form of natural control for bimanual neuroprostheses.

Direct comparison of supervised and semi-supervised retraining approaches for co-adaptive BCIs

For Brain-Computer interfaces (BCIs), system calibration is a lengthy but necessary process for successful operation. Co-adaptive BCIs aim to shorten training and imply positive motivation to users by presenting feedback already at early stages: After just 5 min of gathering calibration data, the systems are able to provide feedback and engage users in a mutual learning process. In this work, we investigate whether the retraining stage of co-adaptive BCIs can be adapted to a semi-supervised concept, where only a small amount of labeled data is available and all additional data needs to be labeled by the BCI itself. The aim of the current work was to evaluate whether a semi-supervised co-adaptive BCI could successfully compete with a supervised co-adaptive BCI model. In a supporting two-class (190 trials per condition) BCI study based on motor imagery tasks, we evaluated both approaches in two separate groups of 10 participants online, while we simulated the other approach in each group offline. Our results indicate that despite the lack of true labeled data, the semi-supervised driven BCI did not perform significantly worse (p > 0.05) than the supervised counterpart. We believe that these findings contribute to developing BCIs for long-term use, where continuous adaptation becomes imperative for maintaining meaningful BCI performance.

Graphical abstract

Combining frequency and time-domain EEG features for classification of self-paced reach-and-grasp actions

Brain-computer interfaces (BCIs) might provide an intuitive way for severely motor impaired persons to operate assistive devices to perform daily life activities. Recent studies have shown that complex hand movements, such as reach-and-grasp tasks, can be decoded from the low frequency of the electroencephalogram (EEG). In this work we investigated whether additional features extracted from the frequency-domain of alpha and beta bands could improve classification performance of rest vs. palmar vs. lateral grasp. We analysed two multi-class classification approaches, the first using features from the low frequency time-domain, and the second in which we combined the time-domain with frequency-domain features from alpha and beta bands. We measured EEG of ten participants without motor disability which performed self-paced reach-and-grasp actions on objects of daily life. For the time-domain classification approach, participants reached an average peak accuracy of 65%. For the combined approach, an average peak accuracy of 75% was reached. In both approaches and for all subjects, performance was significantly higher than chance level (38.1%, 3-class scenario). By computing the confusion matrices as well as feature rankings through the Fisher score, we show that movement vs. rest classification performance increased considerably in the combined approach and was the main responsible for the multi-class higher performance. These findings could help the development of BCIs in real-life scenarios, where decreasing false movement detections could drastically increase the end-user acceptance and usability of BCIs.

Attempted Arm and Hand Movements can be Decoded from Low-Frequency EEG from Persons with Spinal Cord Injury

We show that persons with spinal cord injury (SCI) retain decodable neural correlates of attempted arm and hand movements. We investigated hand open, palmar grasp, lateral grasp, pronation, and supination in 10 persons with cervical SCI. Discriminative movement information was provided by the time-domain of low-frequency electroencephalography (EEG) signals. Based on these signals, we obtained a maximum average classification accuracy of 45% (chance level was 20%) with respect to the five investigated classes. Pattern analysis indicates central motor areas as the origin of the discriminative signals. Furthermore, we introduce a proof-of-concept to classify movement attempts online in a closed loop, and tested it on a person with cervical SCI. We achieved here a modest classification performance of 68.4% with respect to palmar grasp vs hand open (chance level 50%).

On the design of EEG-based movement decoders for completely paralyzed stroke patients

Background

Brain machine interface (BMI) technology has demonstrated its efficacy for rehabilitation of paralyzed chronic stroke patients. The critical component in BMI-training consists of the associative connection (contingency) between the intention and the feedback provided. However, the relationship between the BMI design and its performance in stroke patients is still an open question.

Methods

In this study we compare different methodologies to design a BMI for rehabilitation and evaluate their effects on movement intention decoding performance. We analyze the data of 37 chronic stroke patients who underwent 4 weeks of BMI intervention with different types of association between their brain activity and the proprioceptive feedback. We simulate the pseudo-online performance that a BMI would have under different conditions, varying: (1) the cortical source of activity (i.e., ipsilesional, contralesional, bihemispheric), (2) the type of spatial filter applied, (3) the EEG frequency band, (4) the type of classifier; and also evaluated the use of residual EMG activity to decode the movement intentions.

Results

We observed a significant influence of the different BMI designs on the obtained performances. Our results revealed that using bihemispheric beta activity with a common average reference and an adaptive support vector machine led to the best classification results. Furthermore, the decoding results based on brain activity were significantly higher than those based on muscle activity.

Conclusions

This paper underscores the relevance of the different parameters used to decode movement, using EEG in severely paralyzed stroke patients. We demonstrated significant differences in performance for the different designs, which supports further research that should elucidate if those approaches leading to higher accuracies also induce higher motor recovery in paralyzed stroke patients.