Unsupervised neural decoding for concurrent and continuous multi-finger force prediction

Reliable prediction of multi-finger forces is crucial for neural-machine interfaces. Various neural decoding methods have progressed substantially for accurate motor output predictions. However, most neural decoding methods are performed in a supervised manner, i.e., the finger forces are needed for model training, which may not be suitable in certain contexts, especially in scenarios involving individuals with an arm amputation. To address this issue, we developed an unsupervised neural decoding approach to predict multi-finger forces using spinal motoneuron firing information. We acquired high-density surface electromyogram (sEMG) signals of the finger extensor muscle when subjects performed single-finger and multi-finger tasks of isometric extensions. We first extracted motor units (MUs) from sEMG signals of the single-finger tasks. Because of inevitable finger muscle co-activation, MUs controlling the non-targeted fingers can also be recruited. To ensure an accurate finger force prediction, these MUs need to be teased out. To this end, we clustered the decomposed MUs based on inter-MU distances measured by the dynamic time warping technique, and we then labeled the MUs using the mean firing rate or the firing rate phase amplitude. We merged the clustered MUs related to the same target finger and assigned weights based on the consistency of the MUs being retained. As a result, compared with the supervised neural decoding approach and the conventional sEMG amplitude approach, our new approach can achieve a higher R2 (0.77 ± 0.036 vs. 0.71 ± 0.11 vs. 0.61 ± 0.09) and a lower root mean square error (5.16 ± 0.58 %MVC vs. 5.88 ± 1.34 %MVC vs. 7.56 ± 1.60 %MVC). Our findings can pave the way for the development of accurate and robust neural-machine interfaces, which can significantly enhance the experience during human-robotic hand interactions in diverse contexts.

Correlation between the ERD in grasp/open tasks of BCIs and hand function of stroke patients: a cross-sectional study

BACKGROUND AND AIMS

Brain-computer interfaces (BCIs) are emerging as a promising tool for upper limb recovery after stroke, and motor tasks are an essential part of BCIs for patient training and control of rehabilitative/assistive BCIs. However, the correlation between brain activation with different levels of motor impairment and motor tasks in BCIs is still not so clear. Thus, we aim to compare the brain activation of different levels of motor impairment in performing the hand grasping and opening tasks in BCIs.

METHODS

We instructed stroke patients to perform motor attempts (MA) to grasp and open the affected hand for 30 trials, respectively. During this period, they underwent EEG acquisition and BCIs accuracy recordings. They also received detailed history records and behavioral scale assessments (the Fugl-Meyer assessment of upper limb, FMA-UE).

RESULTS

The FMA-UE was negatively correlated with the event-related desynchronization (ERD) of the affected hemisphere during open MA (R = - 0.423, P = 0.009) but not with grasp MA (R = - 0.058, P = 0.733). Then we divided the stroke patients into group 1 (Brunnstrom recovery stages between I to II, n = 19) and group 2 (Brunnstrom recovery stages between III to VI, n = 23). No difference during the grasping task (t = 0.091, P = 0.928), but a significant difference during the open task (t = 2.156, P = 0.037) was found between the two groups on the affected hemisphere. No significant difference was found in the unaffected hemisphere.

CONCLUSIONS

The study indicated that brain activation is positively correlated with the hand function of stroke in open-hand tasks. In the grasping task, the patients in the different groups have a similar brain response, while in the open task, mildly injured patients have more brain activation in open the hand than the poor hand function patients.

Corticomuscular and Intermuscular Coupling in Simple Hand Movements to Enable a Hybrid Brain-Computer Interface

Hybrid Brain-Computer Interfaces (BCIs) for upper limb rehabilitation after stroke should enable the reinforcement of "more normal" brain and muscular activity. Here, we propose the combination of corticomuscular coherence (CMC) and intermuscular coherence (IMC) as control features for a novel hybrid BCI for rehabilitation purposes. Multiple electroencephalographic (EEG) signals and surface electromyography (EMG) from 5 muscles per side were collected in 20 healthy participants performing finger extension (Ext) and grasping (Grasp) with both dominant and non-dominant hand. Grand average of CMC and IMC patterns showed a bilateral sensorimotor area as well as multiple muscles involvement. CMC and IMC values were used as features to classify each task versus rest and Ext versus Grasp. We demonstrated that a combination of CMC and IMC features allows for classification of both movements versus rest with better performance (Area Under the receiver operating characteristic Curve, AUC) for the Ext movement (0.97) with respect to Grasp (0.88). Classification of Ext versus Grasp also showed high performances (0.99). All in all, these preliminary findings indicate that the combination of CMC and IMC could provide for a comprehensive framework for simple hand movements to eventually be employed in a hybrid BCI system for post-stroke rehabilitation.

Data-Driven Approach to the Analysis of Real-Time FMRI Neurofeedback Data: Disorder-Specific Brain Synchrony in PTSD

Brain-computer interfaces (BCIs) can be used in real-time fMRI neurofeedback (rtfMRI NF) investigations to provide feedback on brain activity to enable voluntary regulation of the blood-oxygen-level dependent (BOLD) signal from localized brain regions. However, the temporal pattern of successful self-regulation is dynamic and complex. In particular, the general linear model (GLM) assumes fixed temporal model functions and misses other dynamics. We propose a novel data-driven analyses approach for rtfMRI NF using intersubject covariance (ISC) analysis. The potential of ISC was examined in a reanalysis of data from 21 healthy individuals and nine patients with post-traumatic stress-disorder (PTSD) performing up-regulation of the anterior cingulate cortex (ACC). ISC in the PTSD group differed from healthy controls in a network including the right inferior frontal gyrus (IFG). In both cohorts, ISC decreased throughout the experiment indicating the development of individual regulation strategies. ISC analyses are a promising approach to reveal novel information on the mechanisms involved in voluntary self-regulation of brain signals and thus extend the results from GLM-based methods. ISC enables a novel set of research questions that can guide future neurofeedback and neuroimaging investigations.

Inter trial coherence of low-frequency oscillations in the course of stroke recovery

OBJECTIVE

The aim was to find a sensitive method to highlight the remodeling of the brain's bioelectric activity in post-stroke repair.

METHODS

Fifteen mild upper limb paretic stroke patients and age-matched healthy controls were included. Repeated trials of finger tapping around the 10th and 100th days after stroke onset were recorded with a 128-channel EEG. Power spectra and Inter Trial Coherence (ITC) calculations were synchronized to tappings. ITC was correlated with motor performance.

RESULTS

ITC, in low frequency bands, designates the motor related bioelectric activity in channel space in both healthy subjects and patients. Ten days after stroke onset, delta-theta ITC was severely reduced compared to baseline, while three months later ITC reorganized partially over the ipsilesional central-parietal areas reflecting the improvement of motor networks. Decreased ITC in the central-parietal area remained significant compared to controls. Delta band ITC over the dorsolateral-prefrontal cortex correlates with the performance on Nine Hole Peg Test. At post-recovery, non-paretic hand tappings show significantly decreased delta-theta ITC over the supplementary motor area, which reflects network remodeling.

CONCLUSIONS

Inter Trial Coherence is a useful measure of brain reorganization during stroke recovery.

SIGNIFICANCE

Delta- theta ITC is a sensitive indicator of impaired motor execution.

Feature Selection Combining Filter and Wrapper Methods for Motor-Imagery Based Brain-Computer Interfaces

Motor imagery (MI) based brain-computer interfaces help patients with movement disorders to regain the ability to control external devices. Common spatial pattern (CSP) is a popular algorithm for feature extraction in decoding MI tasks. However, due to noise and nonstationarity in electroencephalography (EEG), it is not optimal to combine the corresponding features obtained from the traditional CSP algorithm. In this paper, we designed a novel CSP feature selection framework that combines the filter method and the wrapper method. We first evaluated the importance of every CSP feature by the infinite latent feature selection method. Meanwhile, we calculated Wasserstein distance between feature distributions of the same feature under different tasks. Then, we redefined the importance of every CSP feature based on two indicators mentioned above, which eliminates half of CSP features to create a new CSP feature subspace according to the new importance indicator. At last, we designed the improved binary gravitational search algorithm (IBGSA) by rebuilding its transfer function and applied IBGSA on the new CSP feature subspace to find the optimal feature set. To validate the proposed method, we conducted experiments on three public BCI datasets and performed a numerical analysis of the proposed algorithm for MI classification. The accuracies were comparable to those reported in related studies and the presented model outperformed other methods in literature on the same underlying data.

Optimization of Model Training Based on Iterative Minimum Covariance Determinant In Motor-Imagery BCI

The common spatial patterns (CSP) algorithm is one of the most frequently used and effective spatial filtering methods for extracting relevant features for use in motor imagery brain-computer interfaces (MI-BCIs). However, the inherent defect of the traditional CSP algorithm is that it is highly sensitive to potential outliers, which adversely affects its performance in practical applications. In this work, we propose a novel feature optimization and outlier detection method for the CSP algorithm. Specifically, we use the minimum covariance determinant (MCD) to detect and remove outliers in the dataset, then we use the Fisher score to evaluate and select features. In addition, in order to prevent the emergence of new outliers, we propose an iterative minimum covariance determinant (IMCD) algorithm. We evaluate our proposed algorithm in terms of iteration times, classification accuracy and feature distribution using two BCI competition datasets. The experimental results show that the average classification performance of our proposed method is 12% and 22.9% higher than that of the traditional CSP method in two datasets ([Formula: see text]), and our proposed method obtains better performance in comparison with other competing methods. The results show that our method improves the performance of MI-BCI systems.

Decoding Saccade Intention From Primate Prefrontal Cortical Local Field Potentials Using Spectral, Spatial, and Temporal Dimensionality Reduction

Most invasive Brain Computer Interfaces (iBCIs) use spike and Local Field Potentials (LFPs) from the motor or parietal cortices to decode movement intentions. It has been debated whether harvesting signals from other brain areas that encode global cognitive variables, such as the allocation of attention and eye movement goals in a variety of spatial reference frames, may improve the outcome of iBCIs. Here, we explore the ability of LFP signals, sampled from the lateral prefrontal cortex (LPFC) of macaque monkeys, to encode eye-movement intention during the pre-movement fixation period of a delayed saccade task. We use spectral dimensionality reduction to examine the spatiotemporal properties of the extracted non-rhythmic broadband activity and explore its usefulness in decoding saccade goals. The dynamics of the broadband signal in low spatial dimensions across the pre-movement fixation period uncovered saccade target separation; its discriminative potential was confirmed using support vector machine classifications. These findings reveal that broadband LFP from the LPFC can be used to decode intended saccade target location during pre-movement periods. We further provide a general workflow that can be implemented in iBCIs and it is relatively robust to the loss of spikes in individual electrodes.

Concurrent Prediction of Finger Forces Based on Source Separation and Classification of Neuron Discharge Information

A reliable neural-machine interface is essential for humans to intuitively interact with advanced robotic hands in an unconstrained environment. Existing neural decoding approaches utilize either discrete hand gesture-based pattern recognition or continuous force decoding with one finger at a time. We developed a neural decoding technique that allowed continuous and concurrent prediction of forces of different fingers based on spinal motoneuron firing information. High-density skin-surface electromyogram (HD-EMG) signals of finger extensor muscle were recorded, while human participants produced isometric flexion forces in a dexterous manner (i.e. produced varying forces using either a single finger or multiple fingers concurrently). Motoneuron firing information was extracted from the EMG signals using a blind source separation technique, and each identified neuron was further classified to be associated with a given finger. The forces of individual fingers were then predicted concurrently by utilizing the corresponding motoneuron pool firing frequency of individual fingers. Compared with conventional approaches, our technique led to better prediction performances, i.e. a higher correlation ([Formula: see text] versus [Formula: see text]), a lower prediction error ([Formula: see text]% MVC versus [Formula: see text]% MVC), and a higher accuracy in finger state (rest/active) prediction ([Formula: see text]% versus [Formula: see text]%). Our decoding method demonstrated the possibility of classifying motoneurons for different fingers, which significantly alleviated the cross-talk issue of EMG recordings from neighboring hand muscles, and allowed the decoding of finger forces individually and concurrently. The outcomes offered a robust neural-machine interface that could allow users to intuitively control robotic hands in a dexterous manner.

The Influence of Visual Attention on The Performance of A Novel Tactile P300 Brain-Computer Interface with Cheeks-Stim Paradigm

Tactile P300 brain-computer interface (BCI) generally has a worse accuracy and information transfer rate (ITR) than the visual-based BCI. It may be due to the fact that human beings have a relatively poor tactile perception. This study investigated the influence of visual attention on the performance of a tactile P300 BCI. We designed our paradigms based on a novel cheeks-stim paradigm which attached the stimulators on the subject's cheeks. Two paradigms were designed as follows: a paradigm with no visual attention and another paradigm with visual attention to the target position. Eleven subjects were invited to perform the two paradigms. We also recorded and analyzed the eyeball movement data during the paradigm with visual attention to explore whether the eyeball movement would have an effect on the BCI classification. The average online accuracy was 89.09% for the paradigm with visual attention, which was significantly higher than that of the paradigm with no visual attention (70.45%). Significant difference in ITR was also found between the two paradigms ([Formula: see text]). The results demonstrated that visual attention was an effective method to improve the performance of tactile P300 BCI. Our findings suggested that it may be feasible to complete an efficient tactile BCI system by adding visual attention.

Channel Selection for Optimal EEG Measurement in Motor Imagery-Based Brain-Computer Interfaces

A method for selecting electroencephalographic (EEG) signals in motor imagery-based brain-computer interfaces (MI-BCI) is proposed for enhancing the online interoperability and portability of BCI systems, as well as user comfort. The attempt is also to reduce variability and noise of MI-BCI, which could be affected by a large number of EEG channels. The relation between selected channels and MI-BCI performance is therefore analyzed. The proposed method is able to select acquisition channels common to all subjects, while achieving a performance compatible with the use of all the channels. Results are reported with reference to a standard benchmark dataset, the BCI competition IV dataset 2a. They prove that a performance compatible with the best state-of-the-art approaches can be achieved, while adopting a significantly smaller number of channels, both in two and in four tasks classification. In particular, classification accuracy is about 77-83% in binary classification with down to 6 EEG channels, and above 60% for the four-classes case when 10 channels are employed. This gives a contribution in optimizing the EEG measurement while developing non-invasive and wearable MI-based brain-computer interfaces.

Dynamic Modeling of Common Brain Neural Activity in Motor Imagery Tasks

Evaluation of brain dynamics elicited by motor imagery (MI) tasks can contribute to clinical and learning applications. The multi-subject analysis is to make inferences on the group/population level about the properties of MI brain activity. However, intrinsic neurophysiological variability of neural dynamics poses a challenge for devising efficient MI systems. Here, we develop a time-frequency model for estimating the spatial relevance of common neural activity across subjects employing an introduced statistical thresholding rule. In deriving multi-subject spatial maps, we present a comparative analysis of three feature extraction methods: Common Spatial Patterns, Functional Connectivity, and Event-Related De/Synchronization. In terms of interpretability, we evaluate the effectiveness in gathering MI data from collective populations by introducing two assumptions: (i) Non-linear assessment of the similarity between multi-subject data originating the subject-level dynamics; (ii) Assessment of time-varying brain network responses according to the ranking of individual accuracy performed in distinguishing distinct motor imagery tasks (left-hand vs. right-hand). The obtained validation results indicate that the estimated collective dynamics differently reflect the flow of sensorimotor cortex activation, providing new insights into the evolution of MI responses.

Behavioral and physiological correlates of kinetically tracking a chaotic target

Humans can innately track a moving target by anticipating its future position from a brief history of observations. While ballistic trajectories can be readily extrapolated, many natural and artificial systems are governed by more general nonlinear dynamics and, therefore, can produce highly irregular motion. Yet, relatively little is known regarding the behavioral and physiological underpinnings of prediction and tracking in the presence of chaos. Here, we investigated in lab settings whether participants could manually follow the orbit of a paradigmatic chaotic system, the Rössler equations, on the (x,y) plane under different settings of a control parameter, which determined the prominence of transients in the target position. Tracking accuracy was negatively related to the level of unpredictability and folding. Nevertheless, while participants initially reacted to the transients, they gradually learned to anticipate it. This was accompanied by a decrease in muscular co-contraction, alongside enhanced activity in the theta and beta EEG bands for the highest levels of chaoticity. Furthermore, greater phase synchronization of breathing was observed. Taken together, these findings point to the possible ability of the nervous system to implicitly learn topological regularities even in the context of highly irregular motion, reflecting in multiple observables at the physiological level.

Intrinsic Synchronization Analysis of Brain Activity in Obsessive-compulsive Disorders

Obsessive-compulsive disorder (OCD) is one of the neuropsychiatric disorders qualified by intrusive and iterative annoying thoughts and mental attitudes that are activated by these thoughts. In recent studies, advanced signal processing techniques have been favored to diagnose OCD. This research suggests four different measurements; intrinsic phase-locked value, intrinsic coherence, intrinsic synchronization likelihood, and intrinsic visibility graph similarity that quantifies the synchronization level and complexity in electroencephalography (EEG) signals. This intrinsic synchronization is achieved by utilizing Multivariate Empirical Mode Decomposition (MEMD), a data-driven method that resolves nonlinear and nonstationary data into their intrinsic mode functions. Our intrinsic technique in this study demonstrates that MEMD-based synchronization analysis gives us much more detailed knowledge rather than utilizing the synchronization method alone. Furthermore, the nonlinear synchronization method presents more consistent results considering OCD heterogeneity. Statistical evaluation using sample [Formula: see text]-test and [Formula: see text]-test has shown the significance of such new methodology.

User activity recognition system to improve the performance of environmental control interfaces: a pilot study with patients

BACKGROUND

Assistive technologies aim to increase quality of life, reduce dependence on care giver and on the long term care system. Several studies have demonstrated the effectiveness in the use of assistive technology for environment control and communication systems. The progress of brain-computer interfaces (BCI) research together with exoskeleton enable a person with motor impairment to interact with new elements in the environment. This paper aims to evaluate the environment control interface (ECI) developed under the AIDE project conditions, a multimodal interface able to analyze and extract relevant information from the environments as well as from the identification of residual abilities, behaviors, and intentions of the user.

METHODS

This study evaluated the ECI in a simulated scenario using a two screen layout: one with the ECI and the other with a simulated home environment, developed for this purpose. The sensorimotor rhythms and the horizontal oculoversion, acquired through BCI2000, a multipurpose standard BCI platform, were used to online control the ECI after the user training and system calibration. Eight subjects with different neurological diseases and spinal cord injury participated in this study. The subjects performed simulated activities of daily living (ADLs), i.e. actions in the simulated environment as drink, switch on a lamp or raise the bed head, during ten minutes in two different modes, AIDE mode, using a prediction model, to recognize the user intention facilitating the scan, and Manual mode, without a prediction model.

RESULTS

The results show that the mean task time spent in the AIDE mode was less than in the Manual, i.e the users were able to perform more tasks in the AIDE mode during the same time. The results showed a statistically significant differences with p<0.001. Regarding the steps, i.e the number of abstraction levels crossed in the ECI to perform an ADL, the users performed one step in the 90% of the tasks using the AIDE mode and three steps, at least, were necessary in the Manual mode. The user's intention prediction was performed through conditional random fields (CRF), with a global accuracy about 87%.

CONCLUSIONS

The environment analysis and the identification of the user's behaviors can be used to predict the user intention opening a new paradigm in the design of the ECIs. Although the developed ECI was tested only in a simulated home environment, it can be easily adapted to a real environment increasing the user independence at home.