HEAR to remove pops and drifts: the high-variance electrode artifact removal (HEAR) algorithm

Restoration of Genuine Sensation and Proprioception of Individual Fingers Following Transradial Amputation with Targeted Sensory Reinnervation as a Mechanoneural Interface

Background/Objectives: Tactile gnosis derives from the interplay between the hand's tactile input and the memory systems of the brain. It is the prerequisite for complex hand functions. Impaired sensation leads to profound disability. Various invasive and non-invasive sensory substitution strategies for providing feedback from prostheses have been unsuccessful when translated to clinical practice, since they fail to match the feeling to genuine sensation of the somatosensory cortex. Methods: Herein, we describe a novel surgical technique for upper-limb-targeted sensory reinnervation (ulTSR) and report how single digital nerves selectively reinnervate the forearm skin and restore the spatial sensory capacity of single digits of the amputated hand in a case series of seven patients. We explore the interplay of the redirected residual digital nerves and the interpretation of sensory perception after reinnervation of the forearm skin in the somatosensory cortex by evaluating sensory nerve action potentials (SNAPs), somatosensory evoked potentials (SEPs), and amputation-associated pain qualities. Results: Digital nerves were rerouted and reliably reinnervated the forearm skin after hand amputation, leading to somatotopy and limb maps of the thumb and four individual fingers. SNAPs were obtained from the donor digital nerves after stimulating the recipient sensory nerves of the forearm. Matching SEPs were obtained after electrocutaneous stimulation of the reinnervated skin areas of the forearm where the thumb, index, and little fingers are perceived. Pain incidence was significantly reduced or even fully resolved. Conclusions: We propose that ulTSR can lead to higher acceptance of prosthetic hands and substantially reduce the incidence of phantom limb and neuroma pain. In addition, the spatial restoration of lost-hand sensing and the somatotopic reinnervation of the forearm skin may serve as a machine interface, allowing for genuine sensation and embodiment of the prosthetic hand without the need for complex neural coding adjustments.

Detection of Movement and Lead-Popping Artifacts in Polysomnography EEG Data

Polysomnography (PSG) measures brain activity during sleep via electroencephalography (EEG) using six leads. Artifacts caused by movement or loose leads distort EEG measurements. We developed a method to automatically identify such artifacts in a PSG EEG trace. After preprocessing, we extracted power levels at frequencies of 0.5-32.5 Hz with multitaper spectral analysis using 4 s windows with 3 s overlap. For each resulting 1 s segment, we computed segment-specific correlations between power levels for all pairs of leads. We then averaged all pairwise correlation coefficients involving each lead, creating a time series of segment-specific average correlations for each lead. Our algorithm scans each averaged time series separately for "bad" segments using a local moving window. In a second pass, any segment whose averaged correlation is less than a global threshold among all remaining good segments is declared an outlier. We mark all segments between two outlier segments fewer than 300 s apart as artifact regions. This process is repeated, removing a channel with excessive outliers in each iteration. We compared artifact regions discovered by our algorithm to expert-assessed ground truth, achieving sensitivity and specificity of 80% and 91%, respectively. Our algorithm is an open-source tool, either as a Python package or a Docker.

Improving non-invasive trajectory decoding via neural correlates of continuous erroneous feedback processing

Objective. Over the last decades, error-related potentials (ErrPs) have repeatedly proven especially useful as corrective mechanisms in invasive and non-invasive brain-computer interfaces (BCIs). However, research in this context exclusively investigated the distinction of discrete events intocorrectorerroneousto the present day. Due to this predominant formulation as a binary classification problem, classical ErrP-based BCIs fail to monitor tasks demanding quantitative information on error severity rather than mere qualitative decisions on error occurrence. As a result, fine-tuned and natural feedback control based on continuously perceived deviations from an intended target remains beyond the capabilities of previously used BCI setups.Approach.To address this issue for future BCI designs, we investigated the feasibility of regressing rather than classifying error-related activity non-invasively from the brain.Main results.Using pre-recorded data from ten able-bodied participants in three sessions each and a multi-output convolutional neural network, we demonstrated the above-chance regression of ongoing target-feedback discrepancies from brain signals in a pseudo-online fashion. In a second step, we used this inferred information about the target deviation to correct the initially displayed feedback accordingly, reporting significant improvements in correlations between corrected feedback and target trajectories across feedback conditions.Significance.Our results indicate that continuous information on target-feedback discrepancies can be successfully regressed from cortical activity, paving the way to increasingly naturalistic, fine-tuned correction mechanisms for future BCI applications.

A generic error-related potential classifier based on simulated subjects

Error-related potentials (ErrPs) are brain signals known to be generated as a reaction to erroneous events. Several works have shown that not only self-made errors but also mistakes generated by external agents can elicit such event-related potentials. The possibility of reliably measuring ErrPs through non-invasive techniques has increased the interest in the brain-computer interface (BCI) community in using such signals to improve performance, for example, by performing error correction. Extensive calibration sessions are typically necessary to gather sufficient trials for training subject-specific ErrP classifiers. This procedure is not only time-consuming but also boresome for participants. In this paper, we explore the effectiveness of ErrPs in closed-loop systems, emphasizing their dependency on precise single-trial classification. To guarantee the presence of an ErrPs signal in the data we employ and to ensure that the parameters defining ErrPs are systematically varied, we utilize the open-source toolbox SEREEGA for data simulation. We generated training instances and evaluated the performance of the generic classifier on both simulated and real-world datasets, proposing a promising alternative to conventional calibration techniques. Results show that a generic support vector machine classifier reaches balanced accuracies of 72.9%, 62.7%, 71.0%, and 70.8% on each validation dataset. While performing similarly to a leave-one-subject-out approach for error class detection, the proposed classifier shows promising generalization across different datasets and subjects without further adaptation. Moreover, by utilizing SEREEGA, we can systematically adjust parameters to accommodate the variability in the ErrP, facilitating the systematic validation of closed-loop setups. Furthermore, our objective is to develop a universal ErrP classifier that captures the signal's variability, enabling it to determine the presence or absence of an ErrP in real EEG data.

Transfer Learning in Trajectory Decoding: Sensor or Source Space?

In this study, across-participant and across-session transfer learning was investigated to minimize the calibration time of the brain-computer interface (BCI) system in the context of continuous hand trajectory decoding. We reanalyzed data from a study with 10 able-bodied participants across three sessions. A leave-one-participant-out (LOPO) model was utilized as a starting model. Recursive exponentially weighted partial least squares regression (REW-PLS) was employed to overcome the memory limitation due to the large pool of training data. We considered four scenarios: generalized with no update (Gen), generalized with cumulative update (GenC), and individual models with cumulative (IndC) and non-cumulative (Ind) updates, with each one trained with sensor-space features or source-space features. The decoding performance in generalized models (Gen and GenC) was lower than the chance level. In individual models, the cumulative update (IndC) showed no significant improvement over the non-cumulative model (Ind). The performance showed the decoder's incapability to generalize across participants and sessions in this task. The results suggested that the best correlation could be achieved with the sensor-space individual model, despite additional anatomical information in the source-space features. The decoding pattern showed a more localized pattern around the precuneus over three sessions in Ind models.

FiNN: A toolbox for neurophysiological network analysis

Recently, neuroscience has seen a shift from localist approaches to network-wide investigations of brain function. Neurophysiological signals across different spatial and temporal scales provide insight into neural communication. However, additional methodological considerations arise when investigating network-wide brain dynamics rather than local effects. Specifically, larger amounts of data, investigated across a higher dimensional space, are necessary. Here, we present FiNN (Find Neurophysiological Networks), a novel toolbox for the analysis of neurophysiological data with a focus on functional and effective connectivity. FiNN provides a wide range of data processing methods and statistical and visualization tools to facilitate inspection of connectivity estimates and the resulting metrics of brain dynamics. The Python toolbox and its documentation are freely available as Supporting Information. We evaluated FiNN against a number of established frameworks on both a conceptual and an implementation level. We found FiNN to require much less processing time and memory than other toolboxes. In addition, FiNN adheres to a design philosophy of easy access and modifiability, while providing efficient data processing implementations. Since the investigation of network-level neural dynamics is experiencing increasing interest, we place FiNN at the disposal of the neuroscientific community as open-source software.

Toward passive BCI: asynchronous decoding of neural responses to direction- and angle-specific perturbations during a simulated cockpit scenario

Neuroimaging studies have provided proof that loss of balance evokes specific neural transient wave complexes in electroencephalography (EEG), called perturbation evoked potentials (PEPs). Online decoding of balance perturbations from ongoing EEG signals can establish the possibility of implementing passive brain-computer interfaces (pBCIs) as a part of aviation/driving assistant systems. In this study, we investigated the feasibility of identifying the existence and expression of perturbations in four different conditions by using EEG signals. Fifteen healthy participants experienced four various postural changes while they sat in a glider cockpit. Sudden perturbations were exposed by a robot connected to a glider and moved to the right and left directions with tilting angles of 5 and 10 degrees. Perturbations occurred in an oddball paradigm in which participants were not aware of the time and expression of the perturbations. We employed a hierarchical approach to separate the perturbation and rest, and then discriminate the expression of perturbations. The performance of the BCI system was evaluated by using classification accuracy and F1 score. Asynchronously, we achieved average accuracies of 89.83 and 73.64% and average F1 scores of 0.93 and 0.60 for binary and multiclass classification, respectively. These results manifest the practicality of pBCI for the detection of balance disturbances in a realistic situation.

Continuous 2D trajectory decoding from attempted movement: across-session performance in able-bodied and feasibility in a spinal cord injured participant

OBJECTIVE

In people with a cervical spinal cord injury (SCI) or degenerative diseases leading to limited motor function, restoration of upper limb movement has been a goal of the brain-computer interface (BCI) field for decades. Recently, research from our group investigated non-invasive and real-time decoding of continuous movement in able-bodied participants from low-frequency brain signals during a target-tracking task. To advance our setup towards motor-impaired end users, we consequently chose a new paradigm based on attempted movement.

APPROACH

Here, we present the results of two studies. During the first study, data of ten able-bodied participants completing a target-tracking/shape-tracing task on-screen were investigated in terms of improvements in decoding performance due to user training. In a second study, a spinal cord injured participant underwent the same tasks. To investigate the merit of employing attempted movement in end users with SCI, data of the spinal cord injured participant were recorded twice; once within an observation only condition, and once while simultaneously attempting movement.

MAIN RESULTS

We observed mean correlation well above chance level for continuous motor decoding based on attempted movement in able-bodied participants. No global improvement over three sessions, both in sensor and source space, could be observed across all participants and movement parameters. In the participant with SCI, decoding performance well above chance was found.

SIGNIFICANCE

No presence of a learning effect in continuous attempted movement decoding in able-bodied participants could be observed. In contrast, non-significantly varying decoding patterns may promote the use of source space decoding in terms of generalized decoders utilizing transfer learning. Furthermore, above-chance correlations for attempted movement decoding ranging between those of observation only and executed movement were seen in one spinal cord injured participant, suggesting attempted movement decoding as a possible link between feasibility studies in able-bodied and actual applications in motor impaired end users.

Applying Dimensionality Reduction Techniques in Source-Space Electroencephalography via Template and Magnetic Resonance Imaging-Derived Head Models to Continuously Decode Hand Trajectories

Several studies showed evidence supporting the possibility of hand trajectory decoding from low-frequency electroencephalography (EEG). However, the decoding in the source space via source localization is scarcely investigated. In this study, we tried to tackle the problem of collinearity due to the higher number of signals in the source space by two folds: first, we selected signals in predefined regions of interest (ROIs); second, we applied dimensionality reduction techniques to each ROI. The dimensionality reduction techniques were computing the mean (Mean), principal component analysis (PCA), and locality preserving projections (LPP). We also investigated the effect of decoding between utilizing a template head model and a subject-specific head model during the source localization. The results indicated that applying source-space decoding with PCA yielded slightly higher correlations and signal-to-noise (SNR) ratios than the sensor-space approach. We also observed slightly higher correlations and SNRs when applying the subject-specific head model than the template head model. However, the statistical tests revealed no significant differences between the source-space and sensor-space approaches and no significant differences between subject-specific and template head models. The decoder with Mean and PCA utilizes information mainly from precuneus and cuneus to decode the velocity kinematics similarly in the subject-specific and template head models.

Feel Your Reach: An EEG-Based Framework to Continuously Detect Goal-Directed Movements and Error Processing to Gate Kinesthetic Feedback Informed Artificial Arm Control

Establishing the basic knowledge, methodology, and technology for a framework for the continuous decoding of hand/arm movement intention was the aim of the ERC-funded project "Feel Your Reach". In this work, we review the studies and methods we performed and implemented in the last 6 years, which build the basis for enabling severely paralyzed people to non-invasively control a robotic arm in real-time from electroencephalogram (EEG). In detail, we investigated goal-directed movement detection, decoding of executed and attempted movement trajectories, grasping correlates, error processing, and kinesthetic feedback. Although we have tested some of our approaches already with the target populations, we still need to transfer the "Feel Your Reach" framework to people with cervical spinal cord injury and evaluate the decoders' performance while participants attempt to perform upper-limb movements. While on the one hand, we made major progress towards this ambitious goal, we also critically discuss current limitations.

Online detection of movement during natural and self-initiated reach-and-grasp actions from EEG signals

Movement intention detection using electroencephalography (EEG) is a challenging but essential component of brain-computer interfaces (BCIs) for people with motor disabilities.Objective.The goal of this study is to develop a new experimental paradigm to perform asynchronous online detection of movement based on low-frequency time-domain EEG features, concretely on movement-related cortical potentials. The paradigm must be easily transferable to people without any residual upper-limb movement function and the BCI must be independent of upper-limb movement onset measurements and external cues.Approach. In a study with non-disabled participants, we evaluated a novel BCI paradigm to detect self-initiated reach-and-grasp movements. Two experimental conditions were involved. In one condition, participants performed reach-and-grasp movements to a target and simultaneously shifted their gaze towards it. In a control condition, participants solely shifted their gaze towards the target (oculomotor task). The participants freely decided when to initiate the tasks. After eye artefact correction, the EEG signals were time-locked to the saccade onset and the resulting amplitude features were exploited on a hierarchical classification approach to detect movement asynchronously.Main results. With regards to BCI performance, 54.1% (14.4% SD) of the movements were correctly identified, and all participants achieved a performance above chance-level (around 12%). An average of 21.5% (14.1% SD) of the oculomotor tasks were falsely detected as upper-limb movement. In an additional rest condition, 1.7 (1.6 SD) false positives per minute were measured. Through source imaging, movement information was mapped to sensorimotor, posterior parietal and occipital areas.Significance. We present a novel approach for movement detection using EEG signals which does not rely on upper-limb movement onset measurements or on the presentation of external cues. The participants' behaviour closely matches the natural behaviour during goal-directed reach-and-grasp movements, which also constitutes an advantage with respect to current BCI protocols.

Long-Term Mutual Training for the CYBATHLON BCI Race With a Tetraplegic Pilot: A Case Study on Inter-Session Transfer and Intra-Session Adaptation

CYBATHLON is an international championship where people with severe physical disabilities compete with the aid of state-of-the-art assistive technology. In one of the disciplines, the BCI Race, tetraplegic pilots compete in a computer game race by controlling an avatar with a brain-computer interface (BCI). This competition offers a perfect opportunity for BCI researchers to study long-term training effects in potential end-users, and to evaluate BCI performance in a realistic environment. In this work, we describe the BCI system designed by the team Mirage91 for participation in the CYBATHLON BCI Series 2019, as well as in the CYBATHLON 2020 Global Edition. Furthermore, we present the BCI’s interface with the game and the main methodological strategies, along with a detailed evaluation of its performance over the course of the training period, which lasted 14 months. The developed system was a 4-class BCI relying on task-specific modulations of brain rhythms. We implemented inter-session transfer learning to reduce calibration time, and to reinforce the stability of the brain patterns. Additionally, in order to compensate for potential intra-session shifts in the features’ distribution, normalization parameters were continuously adapted in an unsupervised fashion. Across the aforementioned 14 months, we recorded 26 game-based training sessions. Between the first eight sessions, and the final eight sessions leading up to the CYBATHLON 2020 Global Edition, the runtimes significantly improved from 255 ± 23 s (mean ± std) to 225 ± 22 s, respectively. Moreover, we observed a significant increase in the classifier’s accuracy from 46 to 53%, driven by more distinguishable brain patterns. Compared to conventional single session, non-adaptive BCIs, the inter-session transfer learning and unsupervised intra-session adaptation techniques significantly improved the performance. This long-term study demonstrates that regular training helped the pilot to significantly increase the distance between task-specific patterns, which resulted in an improvement of performance, both with respect to class separability in the calibration data, and with respect to the game. Furthermore, it shows that our methodological approaches were beneficial in transferring the performance across sessions, and most importantly to the CYBATHLON competitions.

Distance- and speed-informed kinematics decoding improves M/EEG based upper-limb movement decoder accuracy

OBJECTIVE

One of the main goals in brain-computer interface (BCI) research is the replacement or restoration of lost function in individuals with paralysis. One line of research investigates the inference of movement kinematics from brain activity during different volitional states. A growing number of electroencephalography (EEG) and magnetoencephalography (MEG) studies suggest that information about directional (e.g. velocity) and nondirectional (e.g. speed) movement kinematics is accessible noninvasively. We sought to assess if the neural information associated with both types of kinematics can be combined to improve the decoding accuracy.

APPROACH

In an offline analysis, we reanalyzed the data of two previous experiments containing the recordings of 34 healthy participants (15 EEG, 19 MEG). We decoded 2D movement trajectories from low-frequency M/EEG signals in executed and observed tracking movements, and compared the accuracy of an unscented Kalman filter (UKF) that explicitly modeled the nonlinear relation between directional and nondirectional kinematics to the accuracies of linear Kalman (KF) and Wiener filters which did not combine both types of kinematics.

MAIN RESULTS

At the group level, posterior-parietal and parieto-occipital (executed and observed movements) and sensorimotor areas (executed movements) encoded kinematic information. Correlations between the recorded position and velocity trajectories and the UKF decoded ones were on average 0.49 during executed and 0.36 during observed movements. Compared to the other filters, the UKF could achieve the best trade-off between maximizing the signal to noise ratio and minimizing the amplitude mismatch between the recorded and decoded trajectories.

SIGNIFICANCE

We present direct evidence that directional and nondirectional kinematic information is simultaneously detectable in low-frequency M/EEG signals. Moreover, combining directional and nondirectional kinematic information significantly improves the decoding accuracy upon a linear KF.

Non-linear online low-frequency EEG decoding of arm movements during a pursuit tracking task

Decoding upper-limb movements in invasive recordings has become a reality, but neural tuning in non-invasive low-frequency recordings is still under discussion. Recent studies managed to decode movement positions and velocities using linear decoders, even developing an online system. The decoded signals, however, exhibited smaller amplitudes than actual movements, affecting feedback and user experience. Recently, we showed that a non-linear offline decoder can combine directional (e.g., velocity) and non-directional (e.g., speed) information. In this study, it is assessed if the non-linear decoder can be used online to provide real-time feedback. Five healthy subjects were asked to track a moving target by controlling a robotic arm. Initially, the robot was controlled by their right hand; then, the control was gradually switched until it was entirely controlled by the electroencephalogram (EEG). Correlations between actual and decoded movements were generally above chance level. Results suggest that information about speed was also encoded in the EEG, demonstrating that the proposed non-linear decoder is suitable for decoding real-time arm movements.

Distinct cortical networks for hand movement initiation and directional processing: An EEG study

Movement preparation and initiation have been shown to involve large scale brain networks. Recent findings suggest that movement preparation and initiation are represented in functionally distinct cortical networks. In electroencephalographic (EEG) recordings, movement initiation is reflected as a strong negative potential at medial central channels that is phase-locked to the movement onset - the movement-related cortical potential (MRCP). Movement preparation describes the process of transforming high level movement goals to low level commands. An integral part of this transformation process is directional processing (i.e., where to move). The processing of movement direction during visuomotor and oculomotor tasks is associated with medial parieto-occipital cortex (PO) activity, phase-locked to the presentation of potential movement goals. We surmised that the network generating the MRCP (movement initiation) would encode less information about movement direction than the parieto-occipital network processing movement direction. Here, we studied delta band EEG activity during center-out reaching movements (2D; 4 directions) in visuomotor and oculomotor tasks. In 15 healthy participants, we found a consistent representation of movement direction in PO 300–400 ​ms after the direction cue irrespective of the task. Despite generating the MRCP, sensorimotor areas (SM) encoded less information about the movement direction than PO. Moreover, the encoded directional information in SM was less consistent across participants and specific to the visuomotor task. In a classification approach, we could infer the four movement directions from the delta band EEG activity with moderate accuracies up to 55.9%. The accuracies for cue-aligned data were significantly higher than for movement onset-aligned data in either task, which also suggests a stronger representation of movement direction during movement preparation. Therefore, we present direct evidence that EEG delta band amplitude modulations carry information about both arm movement initiation and movement direction, and that they are represented in two distinct cortical networks.

•

Delta band EEG carries information about arm movement initiation and direction.

•

Movement initiation and direction are represented in two distinct cortical networks.

•

Information about movement direction is primarily encoded in parieto-occipital areas.

•

The activity in parieto-occipital areas is phase-locked to the direction cue.