An empirical comparison of neural networks and machine learning algorithms for EEG gait decoding

Decoding lower-limb kinematic parameters during pedaling tasks using deep learning approaches and EEG

Stroke is a neurological condition that usually results in the loss of voluntary control of body movements, making it difficult for individuals to perform activities of daily living (ADLs). Brain-computer interfaces (BCIs) integrated into robotic systems, such as motorized mini exercise bikes (MMEBs), have been demonstrated to be suitable for restoring gait-related functions. However, kinematic estimation of continuous motion in BCI systems based on electroencephalography (EEG) remains a challenge for the scientific community. This study proposes a comparative analysis to evaluate two artificial neural network (ANN)-based decoders to estimate three lower-limb kinematic parameters: x- and y-axis position of the ankle and knee joint angle during pedaling tasks. Long short-term memory (LSTM) was used as a recurrent neural network (RNN), which reached Pearson correlation coefficient (PCC) scores close to 0.58 by reconstructing kinematic parameters from the EEG features on the delta band using a time window of 250 ms. These estimates were evaluated through kinematic variance analysis, where our proposed algorithm showed promising results for identifying pedaling and rest periods, which could increase the usability of classification tasks. Additionally, negative linear correlations were found between pedaling speed and decoder performance, thereby indicating that kinematic parameters between slower speeds may be easier to estimate. The results allow concluding that the use of deep learning (DL)-based methods is feasible for the estimation of lower-limb kinematic parameters during pedaling tasks using EEG signals. This study opens new possibilities for implementing controllers most robust for MMEBs and BCIs based on continuous decoding, which may allow for maximizing the degrees of freedom and personalized rehabilitation.

A Method to Extract Task-Related EEG Feature Based on Lightweight Convolutional Neural Network

Unlocking task-related EEG spectra is crucial for neuroscience. Traditional convolutional neural networks (CNNs) effectively extract these features but face limitations like overfitting due to small datasets. To address this issue, we propose a lightweight CNN and assess its interpretability through the fully connected layer (FCL). Initially tested with two tasks (Task 1: open vs closed eyes, Task 2: interictal vs ictal stage), the CNN demonstrated enhanced spectral features in the alpha band for Task 1 and the theta band for Task 2, aligning with established neurophysiological characteristics. Subsequent experiments on two brain-computer interface tasks revealed a correlation between delta activity (around 1.55 Hz) and hand movement, with consistent results across pericentral electroencephalogram (EEG) channels. Compared to recent research, our method stands out by delivering task-related spectral features through FCL, resulting in significantly fewer trainable parameters while maintaining comparable interpretability. This indicates its potential suitability for a wider array of EEG decoding scenarios.

Brain-machine interface based on deep learning to control asynchronously a lower-limb robotic exoskeleton: a case-of-study

BACKGROUND

This research focused on the development of a motor imagery (MI) based brain-machine interface (BMI) using deep learning algorithms to control a lower-limb robotic exoskeleton. The study aimed to overcome the limitations of traditional BMI approaches by leveraging the advantages of deep learning, such as automated feature extraction and transfer learning. The experimental protocol to evaluate the BMI was designed as asynchronous, allowing subjects to perform mental tasks at their own will.

METHODS

A total of five healthy able-bodied subjects were enrolled in this study to participate in a series of experimental sessions. The brain signals from two of these sessions were used to develop a generic deep learning model through transfer learning. Subsequently, this model was fine-tuned during the remaining sessions and subjected to evaluation. Three distinct deep learning approaches were compared: one that did not undergo fine-tuning, another that fine-tuned all layers of the model, and a third one that fine-tuned only the last three layers. The evaluation phase involved the exclusive closed-loop control of the exoskeleton device by the participants' neural activity using the second deep learning approach for the decoding.

RESULTS

The three deep learning approaches were assessed in comparison to an approach based on spatial features that was trained for each subject and experimental session, demonstrating their superior performance. Interestingly, the deep learning approach without fine-tuning achieved comparable performance to the features-based approach, indicating that a generic model trained on data from different individuals and previous sessions can yield similar efficacy. Among the three deep learning approaches compared, fine-tuning all layer weights demonstrated the highest performance.

CONCLUSION

This research represents an initial stride toward future calibration-free methods. Despite the efforts to diminish calibration time by leveraging data from other subjects, complete elimination proved unattainable. The study's discoveries hold notable significance for advancing calibration-free approaches, offering the promise of minimizing the need for training trials. Furthermore, the experimental evaluation protocol employed in this study aimed to replicate real-life scenarios, granting participants a higher degree of autonomy in decision-making regarding actions such as walking or stopping gait.

End-to-end deep learning approach to mouse behavior classification from cortex-wide calcium imaging

Deep learning is a powerful tool for neural decoding, broadly applied to systems neuroscience and clinical studies. Interpretable and transparent models that can explain neural decoding for intended behaviors are crucial to identifying essential features of deep learning decoders in brain activity. In this study, we examine the performance of deep learning to classify mouse behavioral states from mesoscopic cortex-wide calcium imaging data. Our convolutional neural network (CNN)-based end-to-end decoder combined with recurrent neural network (RNN) classifies the behavioral states with high accuracy and robustness to individual differences on temporal scales of sub-seconds. Using the CNN-RNN decoder, we identify that the forelimb and hindlimb areas in the somatosensory cortex significantly contribute to behavioral classification. Our findings imply that the end-to-end approach has the potential to be an interpretable deep learning method with unbiased visualization of critical brain regions.

User- and Speed-Independent Slope Estimation for Lower-Extremity Wearable Robots

Wearable robots can help users traverse unstructured slopes by providing mode-specific hip, knee, and ankle joint assistance. However, generalizing the same assistance pattern across different slopes is not optimal. Control strategies that scale assistance based on slope are expected to improve the feel of the device and improve outcome measures such as decreasing metabolic cost. Prior numerical methods for slope estimation struggled to estimate slopes at variable walking speeds or were limited to a single estimation per gait cycle. This study overcomes these limitations by developing machine-learning methods that yield continuous, user- and speed-independent slope estimators for a variety of wearable robot applications using an able-bodied wearable sensor dataset. In a leave-one-subject-out cross-validation (N = 9), four-phase XGBoost regression models were trained on static-slope (fixed-slope) data and evaluated on a novel subject's static-slope and dynamic-slope (variable-slope) data. Using all available sensors, we achieved an average error of 0.88° and 1.73° mean absolute error (MAE) on static and dynamic slopes, respectively. Ankle prosthesis, knee-ankle prosthesis, and hip exoskeleton sensor suites yielded average errors under 2° MAE on static and dynamic slopes, except for the ankle prosthesis and hip exoskeleton cases on dynamic slopes which yielded an average error of 2.2° and 3.2° MAE, respectively. We found that the thigh inertial measurement unit contributed the most to a reduction in average error. Our findings suggest that reliable slope estimators can be trained using only static-slope data regardless of the type of lower-extremity wearable robot.

Exploring Convolutional Neural Network Architectures for EEG Feature Extraction

The main purpose of this paper is to provide information on how to create a convolutional neural network (CNN) for extracting features from EEG signals. Our task was to understand the primary aspects of creating and fine-tuning CNNs for various application scenarios. We considered the characteristics of EEG signals, coupled with an exploration of various signal processing and data preparation techniques. These techniques include noise reduction, filtering, encoding, decoding, and dimension reduction, among others. In addition, we conduct an in-depth analysis of well-known CNN architectures, categorizing them into four distinct groups: standard implementation, recurrent convolutional, decoder architecture, and combined architecture. This paper further offers a comprehensive evaluation of these architectures, covering accuracy metrics, hyperparameters, and an appendix that contains a table outlining the parameters of commonly used CNN architectures for feature extraction from EEG signals.

A review of combined functional neuroimaging and motion capture for motor rehabilitation

BACKGROUND

Technological advancements in functional neuroimaging and motion capture have led to the development of novel methods that facilitate the diagnosis and rehabilitation of motor deficits. These advancements allow for the synchronous acquisition and analysis of complex signal streams of neurophysiological data (e.g., EEG, fNIRS) and behavioral data (e.g., motion capture). The fusion of those data streams has the potential to provide new insights into cortical mechanisms during movement, guide the development of rehabilitation practices, and become a tool for assessment and therapy in neurorehabilitation.

RESEARCH OBJECTIVE

This paper aims to review the existing literature on the combined use of motion capture and functional neuroimaging in motor rehabilitation. The objective is to understand the diversity and maturity of technological solutions employed and explore the clinical advantages of this multimodal approach.

METHODS

This paper reviews literature related to the combined use of functional neuroimaging and motion capture for motor rehabilitation following the PRISMA guidelines. Besides study and participant characteristics, technological aspects of the used systems, signal processing methods, and the nature of multimodal feature synchronization and fusion were extracted.

RESULTS

Out of 908 publications, 19 were included in the final review. Basic or translation studies were mainly represented and based predominantly on healthy participants or stroke patients. EEG and mechanical motion capture technologies were most used for biomechanical data acquisition, and their subsequent processing is based mainly on traditional methods. The system synchronization techniques at large were underreported. The fusion of multimodal features mainly supported the identification of movement-related cortical activity, and statistical methods were occasionally employed to examine cortico-kinematic relationships.

CONCLUSION

The fusion of motion capture and functional neuroimaging might offer advantages for motor rehabilitation in the future. Besides facilitating the assessment of cognitive processes in real-world settings, it could also improve rehabilitative devices' usability in clinical environments. Further, by better understanding cortico-peripheral coupling, new neuro-rehabilitation methods can be developed, such as personalized proprioceptive training. However, further research is needed to advance our knowledge of cortical-peripheral coupling, evaluate the validity and reliability of multimodal parameters, and enhance user-friendly technologies for clinical adaptation.

Decoding movement kinematics from EEG using an interpretable convolutional neural network

Continuous decoding of hand kinematics has been recently explored for the intuitive control of electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs). Deep neural networks (DNNs) are emerging as powerful decoders, for their ability to automatically learn features from lightly pre-processed signals. However, DNNs for kinematics decoding lack in the interpretability of the learned features and are only used to realize within-subject decoders without testing other training approaches potentially beneficial for reducing calibration time, such as transfer learning. Here, we aim to overcome these limitations by using an interpretable convolutional neural network (ICNN) to decode 2-D hand kinematics (position and velocity) from EEG in a pursuit tracking task performed by 13 participants. The ICNN is trained using both within-subject and cross-subject strategies, and also testing the feasibility of transferring the knowledge learned on other subjects on a new one. Moreover, the network eases the interpretation of learned spectral and spatial EEG features. Our ICNN outperformed most of the other state-of-the-art decoders, showing the best trade-off between performance, size, and training time. Furthermore, transfer learning improved kinematics prediction in the low data regime. The network attributed the highest relevance for decoding to the delta-band across all subjects, and to higher frequencies (alpha, beta, low-gamma) for a cluster of them; contralateral central and parieto-occipital sites were the most relevant, reflecting the involvement of sensorimotor, visual and visuo-motor processing. The approach improved the quality of kinematics prediction from the EEG, at the same time allowing interpretation of the most relevant spectral and spatial features.

Survey on the research direction of EEG-based signal processing

Electroencephalography (EEG) is increasingly important in Brain-Computer Interface (BCI) systems due to its portability and simplicity. In this paper, we provide a comprehensive review of research on EEG signal processing techniques since 2021, with a focus on preprocessing, feature extraction, and classification methods. We analyzed 61 research articles retrieved from academic search engines, including CNKI, PubMed, Nature, IEEE Xplore, and Science Direct. For preprocessing, we focus on innovatively proposed preprocessing methods, channel selection, and data augmentation. Data augmentation is classified into conventional methods (sliding windows, segmentation and recombination, and noise injection) and deep learning methods [Generative Adversarial Networks (GAN) and Variation AutoEncoder (VAE)]. We also pay attention to the application of deep learning, and multi-method fusion approaches, including both conventional algorithm fusion and fusion between conventional algorithms and deep learning. Our analysis identifies 35 (57.4%), 18 (29.5%), and 37 (60.7%) studies in the directions of preprocessing, feature extraction, and classification, respectively. We find that preprocessing methods have become widely used in EEG classification (96.7% of reviewed papers) and comparative experiments have been conducted in some studies to validate preprocessing. We also discussed the adoption of channel selection and data augmentation and concluded several mentionable matters about data augmentation. Furthermore, deep learning methods have shown great promise in EEG classification, with Convolutional Neural Networks (CNNs) being the main structure of deep neural networks (92.3% of deep learning papers). We summarize and analyze several innovative neural networks, including CNNs and multi-structure fusion. However, we also identified several problems and limitations of current deep learning techniques in EEG classification, including inappropriate input, low cross-subject accuracy, unbalanced between parameters and time costs, and a lack of interpretability. Finally, we highlight the emerging trend of multi-method fusion approaches (49.2% of reviewed papers) and analyze the data and some examples. We also provide insights into some challenges of multi-method fusion. Our review lays a foundation for future studies to improve EEG classification performance.

Gait Pattern Recognition Based on Supervised Contrastive Learning Between EEG and EMG

Electroencephalography (EEG) and lower-limb electromyography (EMG) signals are widely used in lower-limb kinematic classification and regression tasks. Since it directly measures muscle responses, EMG usually works better. However, due to the susceptibility of EMG signals to muscle fatigue, insufficient residual myoelectric activity, and the difficulty of precise localization, it is difficult to acquire EMG signals in practice. In contrast, EEG signals are stable and easy to sample. Therefore, in this work, we propose a multimodal training strategy based on supervised contrastive learning. With this training strategy, we can effectively use the guiding role of EMG in the training phase to help the model fit the gait with EEG signal while using only EEG signal in the testing phase to obtain better results than using any single modal signal to train and test the model. Finally, we compared the models trained with the strategy proposed in this paper with other models trained with EEG signals. The obtained Pearson's Correlation Coefficient value exceeds those of all baseline models.

Quasi-Distributed Fiber Sensor-Based Approach for Pipeline Health Monitoring: Generating and Analyzing Physics-Based Simulation Datasets for Classification

This study presents a framework for detecting mechanical damage in pipelines, focusing on generating simulated data and sampling to emulate distributed acoustic sensing (DAS) system responses. The workflow transforms simulated ultrasonic guided wave (UGW) responses into DAS or quasi-DAS system responses to create a physically robust dataset for pipeline event classification, including welds, clips, and corrosion defects. This investigation examines the effects of sensing systems and noise on classification performance, emphasizing the importance of selecting the appropriate sensing system for a specific application. The framework shows the robustness of different sensor number deployments to experimentally relevant noise levels, demonstrating its applicability in real-world scenarios where noise is present. Overall, this study contributes to the development of a more reliable and effective method for detecting mechanical damage to pipelines by emphasizing the generation and utilization of simulated DAS system responses for pipeline classification efforts. The results on the effects of sensing systems and noise on classification performance further enhance the robustness and reliability of the framework.

DeePay: deep learning decodes EEG to predict consumer's willingness to pay for neuromarketing

There is an increasing demand within consumer-neuroscience (or neuromarketing) for objective neural measures to quantify consumers' subjective valuations and predict responses to marketing campaigns. However, the properties of EEG raise difficulties for these aims: small datasets, high dimensionality, elaborate manual feature extraction, intrinsic noise, and between-subject variations. We aimed to overcome these limitations by combining unique techniques of Deep Learning Networks (DLNs), while providing interpretable results for neuroscientific and decision-making insight. In this study, we developed a DLN to predict subjects' willingness to pay (WTP) based on their EEG data. In each trial, 213 subjects observed a product's image, from 72 possible products, and then reported their WTP for the product. The DLN employed EEG recordings from product observation to predict the corresponding reported WTP values. Our results showed 0.276 test root-mean-square-error and 75.09% test accuracy in predicting high vs. low WTP, surpassing other models and a manual feature extraction approach. Network visualizations provided the predictive frequencies of neural activity, their scalp distributions, and critical timepoints, shedding light on the neural mechanisms involved with evaluation. In conclusion, we show that DLNs may be the superior method to perform EEG-based predictions, to the benefit of decision-making researchers and marketing practitioners alike.

Lower-limb kinematic reconstruction during pedaling tasks from EEG signals using Unscented Kalman filter

Kinematic reconstruction of lower-limb movements using electroencephalography (EEG) has been used in several rehabilitation systems. However, the nonlinear relationship between neural activity and limb movement may challenge decoders in real-time Brain-Computer Interface (BCI) applications. This paper proposes a nonlinear neural decoder using an Unscented Kalman Filter (UKF) to infer lower-limb kinematics from EEG signals during pedaling. The results demonstrated maximum decoding accuracy using slow cortical potentials in the delta band (0.1-4 Hz) of 0.33 for Pearson's r-value and 8 for the signal-to-noise ratio (SNR). This leaves an open door to the development of closed-loop EEG-based BCI systems for kinematic monitoring during pedaling rehabilitation tasks.

The Utility of Machine Learning Models for Predicting Chemical Contaminants in Drinking Water: Promise, Challenges, and Opportunities

PURPOSE OF REVIEW

This review aims to better understand the utility of machine learning algorithms for predicting spatial patterns of contaminants in the United States (U.S.) drinking water.

RECENT FINDINGS

We found 27 U.S. drinking water studies in the past ten years that used machine learning algorithms to predict water quality. Most studies (42%) developed random forest classification models for groundwater. Continuous models show low predictive power, suggesting that larger datasets and additional predictors are needed. Categorical/classification models for arsenic and nitrate that predict exceedances of pollution thresholds are most common in the literature because of good national scale data coverage and priority as environmental health concerns. Most groundwater data used to develop models were obtained from the United States Geological Survey (USGS) National Water Information System (NWIS). Predictors were similar across contaminants but challenges are posed by the lack of a standard methodology for imputation, pre-processing, and differing availability of data across regions. We reviewed 27 articles that focused on seven drinking water contaminants. Good performance metrics were reported for binary models that classified chemical concentrations above a threshold value by finding significant predictors. Classification models are especially useful for assisting in the design of sampling efforts by identifying high-risk areas. Only a few studies have developed continuous models and obtaining good predictive performance for such models is still challenging. Improving continuous models is important for potential future use in epidemiological studies to supplement data gaps in exposure assessments for drinking water contaminants. While significant progress has been made over the past decade, methodological advances are still needed for selecting appropriate model performance metrics and accounting for spatial autocorrelations in data. Finally, improved infrastructure for code and data sharing would spearhead more rapid advances in machine-learning models for drinking water quality.

Overcoming the Domain Gap in Neural Action Representations

Relating behavior to brain activity in animals is a fundamental goal in neuroscience, with practical applications in building robust brain-machine interfaces. However, the domain gap between individuals is a major issue that prevents the training of general models that work on unlabeled subjects. Since 3D pose data can now be reliably extracted from multi-view video sequences without manual intervention, we propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations exploiting the properties of microscopy imaging. To test our method, we collect a large dataset that features flies and their neural activity. To reduce the domain gap, during training, we mix features of neural and behavioral data across flies that seem to be performing similar actions. To show our method can generalize further neural modalities and other downstream tasks, we test our method on a human neural Electrocorticography dataset, and another RGB video data of human activities from different viewpoints. We believe our work will enable more robust neural decoding algorithms to be used in future brain-machine interfaces.

Combined Feature Extraction and Random Forest for Laser Self-Mixing Vibration Measurement without Determining Feedback Intensity

To measure the vibration of a target by laser self-mixing interference (SMI), we propose a method that combines feature extraction and random forest (RF) without determining the feedback strength (C). First, the temporal, spectral, and statistical features of the SMI signal are extracted to characterize the original SMI signal. Secondly, these interpretable features are fed into the pretrained RF model to directly predict the amplitude and frequency (A and f) of the vibrating target, recovering the periodic vibration of the target. The results show that the combination of RF and feature extraction yields a fit of more than 0.94 for simple and quick measurement of A and f of unsmooth planar vibrations, regardless of the feedback intensity and the misalignment of the retromirror. Without a complex optical stage, this method can quickly recover arbitrary periodic vibrations from SMI signals without C, which provides a novel method for quickly implementing vibration measurements.

A data-driven machine learning approach for brain-computer interfaces targeting lower limb neuroprosthetics

Prosthetic devices that replace a lost limb have become increasingly performant in recent years. Recent advances in both software and hardware allow for the decoding of electroencephalogram (EEG) signals to improve the control of active prostheses with brain-computer interfaces (BCI). Most BCI research is focused on the upper body. Although BCI research for the lower extremities has increased in recent years, there are still gaps in our knowledge of the neural patterns associated with lower limb movement. Therefore, the main objective of this study is to show the feasibility of decoding lower limb movements from EEG data recordings. The second aim is to investigate whether well-known neuroplastic adaptations in individuals with an amputation have an influence on decoding performance. To address this, we collected data from multiple individuals with lower limb amputation and a matched able-bodied control group. Using these data, we trained and evaluated common BCI methods that have already been proven effective for upper limb BCI. With an average test decoding accuracy of 84% for both groups, our results show that it is possible to discriminate different lower extremity movements using EEG data with good accuracy. There are no significant differences (p = 0.99) in the decoding performance of these movements between healthy subjects and subjects with lower extremity amputation. These results show the feasibility of using BCI for lower limb prosthesis control and indicate that decoding performance is not influenced by neuroplasticity-induced differences between the two groups.

A Motor Imagery Signals Classification Method via the Difference of EEG Signals Between Left and Right Hemispheric Electrodes

Brain-computer interface (BCI) based on motor imagery (MI) can help patients with limb movement disorders in their normal life. In order to develop an efficient BCI system, it is necessary to decode high-accuracy motion intention by electroencephalogram (EEG) with low signal-to-noise ratio. In this article, a MI classification approach is proposed, combining the difference in EEG signals between the left and right hemispheric electrodes with a dual convolutional neural network (dual-CNN), which effectively improved the decoding performance of BCI. The positive and inverse problems of EEG were solved by the boundary element method (BEM) and weighted minimum norm estimation (WMNE), and then the scalp signals were mapped to the cortex layer. We created nine pairs of new electrodes on the cortex as the region of interest. The time series of the nine electrodes on the left and right hemispheric are respectively used as the input of the dual-CNN model to classify four MI tasks. The results show that this method has good results in both group-level subjects and individual subjects. On the Physionet database, the averaged accuracy on group-level can reach 96.36%, while the accuracies of four MI tasks reach 98.54, 95.02, 93.66, and 96.19%, respectively. As for the individual subject, the highest accuracy is 98.88%, and its four MI accuracies are 99.62, 99.68, 98.47, and 97.73%, respectively.

Decoding Neural Activity Preceding Balance Loss During Standing with a Lower-limb Exoskeleton using an Interpretable Deep Learning Model

Falls are a leading cause of death in adults 65 and older. Recent efforts to restore lower-limb function in these populations have seen an increase in the use of wearable robotic systems; however, fall prevention measures in these systems require early detection of balance loss to be effective. Prior studies have investigated whether kinematic variables contain information about an impending fall, but few have examined the potential of using electroencephalography (EEG) as a fall-predicting signal and how the brain responds to avoid a fall. To address this, we decoded neural activity in a balance perturbation task while wearing an exoskeleton. We acquired EEG, electromyography (EMG), and center of pressure (COP) data from 7 healthy participants during mechanical perturbations while standing. The timing of the perturbations was randomized in all trials. We found perturbation evoked potentials (PEP) components as early as 75-134 ms after the onset of the external perturbation, which preceded both the peak in EMG (∼ 180 ms) and the COP (∼ 350 ms). A convolutional neural network trained to predict balance perturbations from single-trial EEG had a mean F-score of 75.0 ± 4.3 %. Clustering GradCAM-based model explanations demonstrated that the model utilized components in the PEP and was not driven by artifacts. Additionally, dynamic functional connectivity results agreed with model explanations; the nodal connectivity measured using phase difference derivative was higher in the occipital-parietal region in the early stage of perturbations, before shifting to the parietal, motor, and back to the frontal-parietal channels. Continuous-time decoding of COP trajectories from EEG, using a gated recurrent unit model, achieved a mean Pearson's correlation coefficient of 0.7 ± 0.06. Overall, our findings suggest that EEG signals contain short-latency neural information related to an impending fall, which may be useful for developing brain-machine interface systems for fall prevention in robotic exoskeletons.

Past, Present, and Future of EEG-Based BCI Applications

An electroencephalography (EEG)-based brain-computer interface (BCI) is a system that provides a pathway between the brain and external devices by interpreting EEG. EEG-based BCI applications have initially been developed for medical purposes, with the aim of facilitating the return of patients to normal life. In addition to the initial aim, EEG-based BCI applications have also gained increasing significance in the non-medical domain, improving the life of healthy people, for instance, by making it more efficient, collaborative and helping develop themselves. The objective of this review is to give a systematic overview of the literature on EEG-based BCI applications from the period of 2009 until 2019. The systematic literature review has been prepared based on three databases PubMed, Web of Science and Scopus. This review was conducted following the PRISMA model. In this review, 202 publications were selected based on specific eligibility criteria. The distribution of the research between the medical and non-medical domain has been analyzed and further categorized into fields of research within the reviewed domains. In this review, the equipment used for gathering EEG data and signal processing methods have also been reviewed. Additionally, current challenges in the field and possibilities for the future have been analyzed.

Decoding ECoG signal into 3D hand translation using deep learning

Objective.Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.Approach.In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity.Main results.Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.Significance.This study shows that DL-based models could increase the accuracy of BCI systems in the case of 3D hand translation prediction in a tetraplegic subject.

A multi-step blind source separation approach for the attenuation of artifacts in mobile high-density electroencephalography data

Objective.Electroencephalography (EEG) is a widely used technique to address research questions about brain functioning, from controlled laboratorial conditions to naturalistic environments. However, EEG data are affected by biological (e.g. ocular, myogenic) and non-biological (e.g. movement-related) artifacts, which-depending on their extent-may limit the interpretability of the study results. Blind source separation (BSS) approaches have demonstrated to be particularly promising for the attenuation of artifacts in high-density EEG (hdEEG) data. Previous EEG artifact removal studies suggested that it may not be optimal to use the same BSS method for different kinds of artifacts.Approach.In this study, we developed a novel multi-step BSS approach to optimize the attenuation of ocular, movement-related and myogenic artifacts from hdEEG data. For validation purposes, we used hdEEG data collected in a group of healthy participants in standing, slow-walking and fast-walking conditions. During part of the experiment, a series of tone bursts were used to evoke auditory responses. We quantified event-related potentials (ERPs) using hdEEG signals collected during an auditory stimulation, as well as the event-related desynchronization (ERD) by contrasting hdEEG signals collected in walking and standing conditions, without auditory stimulation. We compared the results obtained in terms of auditory ERP and motor-related ERD using the proposed multi-step BSS approach, with respect to two classically used single-step BSS approaches.Main results. The use of our approach yielded the lowest residual noise in the hdEEG data, and permitted to retrieve stronger and more reliable modulations of neural activity than alternative solutions. Overall, our study confirmed that the performance of BSS-based artifact removal can be improved by using specific BSS methods and parameters for different kinds of artifacts.Significance.Our technological solution supports a wider use of hdEEG-based source imaging in movement and rehabilitation studies, and contributes to the further development of mobile brain/body imaging applications.

The significance of neural inter-frequency power correlations

It is of great interest in neuroscience to determine what frequency bands in the brain have covarying power. This would help us robustly identify the frequency signatures of neural processes. However to date, to the best of the author's knowledge, a comprehensive statistical approach to this question that accounts for intra-frequency autocorrelation, frequency-domain oversampling, and multiple testing under dependency has not been undertaken. As such, this work presents a novel statistical significance test for correlated power across frequency bands for a broad class of non-stationary time series. It is validated on synthetic data. It is then used to test all of the inter-frequency power correlations between 0.2 and 8500 Hz in continuous intracortical extracellular neural recordings in Macaque M1, using a very large, publicly available dataset. The recordings were Current Source Density referenced and were recorded with a Utah array. The results support previous results in the literature that show that neural processes in M1 have power signatures across a very broad range of frequency bands. In particular, the power in LFP frequency bands as low as 20 Hz was found to almost always be statistically significantly correlated to the power in kHz frequency ranges. It is proposed that this test can also be used to discover the superimposed frequency domain signatures of all the neural processes in a neural signal, allowing us to identify every interesting neural frequency band.

Investigation on Robustness of EEG-based Brain-Computer Interfaces

Electroencephalogram (EEG)-based brain-computer interface (BCI) systems tend to suffer from performance degradation due to the presence of noise and artifacts in EEG data. This study is aimed at systematically investigating the robustness of state-of-the-art machine learning and deep learning based EEG-BCI models for motor imagery classification against simulated channel-specific noise in EEG data, at various low values of signal-to-noise ratio (SNR). Our results illustrate higher robustness of deep learning based MI classification models compared to the traditional machine learning based model, while identifying a set of channels with large sensitivity to simulated channel-specific noise. The EEGNet is relatively more robust towards channel-specific noise than Shallow ConvNet and FBCSP. We propose a preliminary solution, based on activation function, to improve the robustness of the deep learning models. By using saturating nonlinearities, the percentage drop in classification accuracy for SNR of -18 dB had reduced from 10.99% to 6.53% for EEGNet and 14.05% to 3.57% for Shallow ConvNet. Through this study, we emphasize the need for a more precise solution for enhancing the robustness, and thereby usability of EEG-BCI systems.

Comparing Methods of Feature Extraction of Brain Activities for Octave Illusion Classification Using Machine Learning

The aim of this study was to find an efficient method to determine features that characterize octave illusion data. Specifically, this study compared the efficiency of several automatic feature selection methods for automatic feature extraction of the auditory steady-state responses (ASSR) data in brain activities to distinguish auditory octave illusion and nonillusion groups by the difference in ASSR amplitudes using machine learning. We compared univariate selection, recursive feature elimination, principal component analysis, and feature importance by testifying the results of feature selection methods by using several machine learning algorithms: linear regression, random forest, and support vector machine. The univariate selection with the SVM as the classification method showed the highest accuracy result, 75%, compared to 66.6% without using feature selection. The received results will be used for future work on the explanation of the mechanism behind the octave illusion phenomenon and creating an algorithm for automatic octave illusion classification.

Tremor-Suppression Orthoses for the Upper Limb: Current Developments and Future Challenges

Introduction: Pathological tremor is the most common motor disorder in adults and characterized by involuntary, rhythmic muscular contraction leading to shaking movements in one or more parts of the body. Functional Electrical Stimulation (FES) and biomechanical loading using wearable orthoses have emerged as effective and non-invasive methods for tremor suppression. A variety of upper-limb orthoses for tremor suppression have been introduced; however, a systematic review of the mechanical design, algorithms for tremor extraction, and the experimental design is still missing.

Methods: To address this gap, we applied a standard systematic review methodology to conduct a literature search in the PubMed and PMC databases. Inclusion criteria and full-text access eligibility were used to filter the studies from the search results. Subsequently, we extracted relevant information, such as suppression mechanism, system weights, degrees of freedom (DOF), algorithms for tremor estimation, experimental settings, and the efficacy.

Results: The results show that the majority of tremor-suppression orthoses are active with 47% prevalence. Active orthoses are also the heaviest with an average weight of 561 ± 467 g, followed by semi-active 486 ± 395 g, and passive orthoses 191 ± 137 g. Most of the orthoses only support one DOF (54.5%). Two-DOF and three-DOF orthoses account for 33 and 18%, respectively. The average efficacy of tremor suppression using wearable orthoses is 83 ± 13%. Active orthoses are the most efficient with an average efficacy of 83 ± 8%, following by the semi-active 77 ± 19%, and passive orthoses 75 ± 12%. Among different experimental setups, bench testing shows the highest efficacy at 95 ± 5%, this value dropped to 86 ± 8% when evaluating with tremor-affected subjects. The majority of the orthoses (92%) measured voluntary and/or tremorous motions using biomechanical sensors (e.g., IMU, force sensor). Only one system was found to utilize EMG for tremor extraction.

Conclusions: Our review showed an improvement in efficacy of using robotic orthoses in tremor suppression. However, significant challenges for the translations of these systems into clinical or home use remain unsolved. Future challenges include improving the wearability of the orthoses (e.g., lightweight, aesthetic, and soft structure), and user control interfaces (i.e., neural machine interface). We also suggest addressing non-technical challenges (e.g., regulatory compliance, insurance reimbursement) to make the technology more accessible.

Differential Theta-Band Signatures of the Anterior Cingulate and Motor Cortices During Seated Locomotor Perturbations

Quantifying motor and cortical responses to perturbations during seated locomotor tasks such as recumbent stepping and cycling will expand and improve the understanding of locomotor adaptation processes beyond just perturbed gait. Using a perturbed recumbent stepping protocol, we hypothesized motor errors and anterior cingulate activity would decrease with time, and perturbation timing would influence electrocortical elicitation. Young adults (n = 17) completed four 10-minute arms and legs stepping tasks, with perturbations applied at every left or right leg extension-onset or mid-extension. A random no-perturbation "catch" stride occurred in every five perturbed strides. We instructed subjects to follow a pacing cue and to step smoothly, and we quantified temporal and spatial motor errors. We used high-density electroencephalography to estimate sources of electrocortical fluctuations shared among >70% of subjects. Temporal and spatial errors did not decrease from early to late for either perturbed or catch strides. Interestingly, spatial errors post-perturbation did not return to pre-perturbation levels, suggesting use-dependent learning occurred. Theta (3-8 Hz) synchronization in the anterior cingulate cortex and left and right supplementary motor areas (SMA) emerged near the perturbation event, and extension-onset perturbations elicited greater theta-band power than mid-extension perturbations. Even though motor errors did not adapt, anterior cingulate theta synchronization decreased from early to late perturbed strides, but only during the right-side tasks. Additionally, SMA mainly demonstrated specialized, not contralateral, lateralization. Overall, seated locomotor perturbations produced differential theta-band responses in the anterior cingulate and SMAs, suggesting that tuning perturbation parameters, e.g., timing, can potentially modify electrocortical responses.

Deep learning approaches for neural decoding across architectures and recording modalities

Decoding behavior, perception or cognitive state directly from neural signals is critical for brain-computer interface research and an important tool for systems neuroscience. In the last decade, deep learning has become the state-of-the-art method in many machine learning tasks ranging from speech recognition to image segmentation. The success of deep networks in other domains has led to a new wave of applications in neuroscience. In this article, we review deep learning approaches to neural decoding. We describe the architectures used for extracting useful features from neural recording modalities ranging from spikes to functional magnetic resonance imaging. Furthermore, we explore how deep learning has been leveraged to predict common outputs including movement, speech and vision, with a focus on how pretrained deep networks can be incorporated as priors for complex decoding targets like acoustic speech or images. Deep learning has been shown to be a useful tool for improving the accuracy and flexibility of neural decoding across a wide range of tasks, and we point out areas for future scientific development.

Hybrid Human-Machine Interface for Gait Decoding Through Bayesian Fusion of EEG and EMG Classifiers

Despite the advances in the field of brain computer interfaces (BCI), the use of the sole electroencephalography (EEG) signal to control walking rehabilitation devices is currently not viable in clinical settings, due to its unreliability. Hybrid interfaces (hHMIs) represent a very recent solution to enhance the performance of single-signal approaches. These are classification approaches that combine multiple human-machine interfaces, normally including at least one BCI with other biosignals, such as the electromyography (EMG). However, their use for the decoding of gait activity is still limited. In this work, we propose and evaluate a hybrid human-machine interface (hHMI) to decode walking phases of both legs from the Bayesian fusion of EEG and EMG signals. The proposed hHMI significantly outperforms its single-signal counterparts, by providing high and stable performance even when the reliability of the muscular activity is compromised temporarily (e.g., fatigue) or permanently (e.g., weakness). Indeed, the hybrid approach shows a smooth degradation of classification performance after temporary EMG alteration, with more than 75% of accuracy at 30% of EMG amplitude, with respect to the EMG classifier whose performance decreases below 60% of accuracy. Moreover, the fusion of EEG and EMG information helps keeping a stable recognition rate of each gait phase of more than 80% independently on the permanent level of EMG degradation. From our study and findings from the literature, we suggest that the use of hybrid interfaces may be the key to enhance the usability of technologies restoring or assisting the locomotion on a wider population of patients in clinical applications and outside the laboratory environment.