Prediction of gait intention from pre-movement EEG signals: a feasibility study

Research of intent recognition in rehabilitation robots: a systematic review

PURPOSE

Rehabilitation robots with intent recognition are helping people with dysfunction to enjoy better lives. Many rehabilitation robots with intent recognition have been developed by academic institutions and commercial companies. However, there is no systematic summary about the application of intent recognition in the field of rehabilitation robots. Therefore, the purpose of this paper is to summarize the application of intent recognition in rehabilitation robots, analyze the current status of their research, and provide cutting-edge research directions for colleagues.

MATERIALS AND METHODS

Literature searches were conducted on Web of Science, IEEE Xplore, ScienceDirect, SpringerLink, and Medline. Search terms included "rehabilitation robot", "intent recognition", "exoskeleton", "prosthesis", "surface electromyography (sEMG)" and "electroencephalogram (EEG)". References listed in relevant literature were further screened according to inclusion and exclusion criteria.

RESULTS

In this field, most studies have recognized movement intent by kinematic, sEMG, and EEG signals. However, in practical studies, the development of intent recognition in rehabilitation robots is limited by the hysteresis of kinematic signals and the weak anti-interference ability of sEMG and EEG signals.

CONCLUSIONS

Intent recognition has achieved a lot in the field of rehabilitation robotics but the key factors limiting its development are still timeliness and accuracy. In the future, intent recognition strategy with multi-sensor information fusion may be a good solution.

EEG generation mechanism of lower limb active movement intention and its virtual reality induction enhancement: a preliminary study

Introduction

Active rehabilitation requires active neurological participation when users use rehabilitation equipment. A brain-computer interface (BCI) is a direct communication channel for detecting changes in the nervous system. Individuals with dyskinesia have unclear intentions to initiate movement due to physical or psychological factors, which is not conducive to detection. Virtual reality (VR) technology can be a potential tool to enhance the movement intention from pre-movement neural signals in clinical exercise therapy. However, its effect on electroencephalogram (EEG) signals is not yet known. Therefore, the objective of this paper is to construct a model of the EEG signal generation mechanism of lower limb active movement intention and then investigate whether VR induction could improve movement intention detection based on EEG.

Methods

Firstly, a neural dynamic model of lower limb active movement intention generation was established from the perspective of signal transmission and information processing. Secondly, the movement-related EEG signal was calculated based on the model, and the effect of VR induction was simulated. Movement-related cortical potential (MRCP) and event-related desynchronization (ERD) features were extracted to analyze the enhancement of movement intention. Finally, we recorded EEG signals of 12 subjects in normal and VR environments to verify the effectiveness and feasibility of the above model and VR induction enhancement of lower limb active movement intention for individuals with dyskinesia.

Results

Simulation and experimental results show that VR induction can effectively enhance the EEG features of subjects and improve the detectability of movement intention.

Discussion

The proposed model can simulate the EEG signal of lower limb active movement intention, and VR induction can enhance the early and accurate detectability of lower limb active movement intention. It lays the foundation for further robot control based on the actual needs of users.

EEG Dynamics of Locomotion and Balancing: Solution to Neuro-Rehabilitation

The past decade has witnessed tremendous growth in analyzing the cortical representation of human locomotion and balance using Electroencephalography (EEG). With the advanced developments in miniaturized electronics, wireless brain recording systems have been developed for mobile recordings, such as in locomotion. In this review, the cortical dynamics during locomotion are presented with extensive focus on motor imagery, and employing the treadmill as a tool for performing different locomotion tasks. Further, the studies that examine the cortical dynamics during balancing, focusing on two types of balancing tasks, ie, static and dynamic, with the challenges in sensory inputs and cognition (dual-task), are presented. Moreover, the current literature demonstrates the advancements in signal processing methods to detect and remove the artifacts from EEG signals. Prior studies show the electrocortical sources in the anterior cingulate, posterior parietal, and sensorimotor cortex was found to be activated during locomotion. The event-related potential has been observed to increase in the fronto-central region for a wide range of balance tasks. The advanced knowledge of cortical dynamics during mobility can benefit various application areas such as neuroprosthetics and gait/balance rehabilitation. This review will be beneficial for the development of neuroprostheses, and rehabilitation devices for patients suffering from movement or neurological disorders.

Recognition of walking directional intention employed ground reaction forces and center of pressure during gait initiation

BACKGROUND

Movement intentions are generally classified by Electroencephalogram (EEG) and have been used in gait initiation prediction. However, it is not easy to collect EEG data and practical in reality. Alternatively, ground reaction force (GRF) and the center of pressure (COP) is produced by the contact between the foot and the ground during a specific period of walking, which are the characteristics of evaluating gait performance RESEARCH QUESTION: The study aims to use a deep learning technique to recognize the data of the COP and GRF to classify straight walking and right turn. Second, the study aims to reveal gait characteristics that could replace EEG to predict walking directional intentions METHODS: Ten healthy male adults were instructed to stand on the force platform and self-selected to perform three conditions: standstill, straight walking, and right turn. The onset of gait initiation was evaluated by muscle activation of the right tibialis anterior, and EEG and the COP displacement evaluated the onset of gait intention. Subsequently, GRF and COP would be treated as features to classify the gait intention in the Long Short-Term Memory (LSTM) model.

RESULTS

The results revealed that the onset of EEG and the COP displacement initiation were statistically significant differences between straight walking and right turn. For the classification, the average accuracy of the LSTM model with GRF and COP as features reached the highest one, 94.79 %, depending on the heel- or toe-off of the swing leg. The results indicated that gait intentions could be classified based on the GRF and COP.

SIGNIFICANCE

The machine learning technique of LSTM with gait parameters can recognize the gait intention of changing walking orientation. Our model and approach would be expected to provide advanced predictions, such as exoskeleton control or pedestrian traffic flow.

Research on two-class and four-class action recognition based on EEG signals

BMI has attracted widespread attention in the past decade, which has greatly improved the living conditions of patients with motor disorders. The application of EEG signals in lower limb rehabilitation robots and human exoskeleton has also been gradually applied by researchers. Therefore, the recognition of EEG signals is of great significance. In this paper, a CNN-LSTM neural network model is designed to study the two-class and four-class motion recognition of EEG signals. In this paper, a brain-computer interface experimental scheme is designed. Combining the characteristics of EEG signals, the time-frequency characteristics of EEG signals and event-related potential phenomena are analyzed, and the ERD/ERS characteristics are obtained. Pre-process EEG signals, and propose a CNN-LSTM neural network model to classify the collected binary and four-class EEG signals. The experimental results show that the CNN-LSTM neural network model has a good effect, and its average accuracy and kappa coefficient are higher than the other two classification algorithms, which also shows that the classification algorithm selected in this paper has a good classification effect.

Low limb prostheses and complex human prosthetic interaction: A systematic literature review

A few years ago, powered prostheses triggered new technological advances in diverse areas such as mobility, comfort, and design, which have been essential to improving the quality of life of individuals with lower limb disability. The human body is a complex system involving mental and physical health, meaning a dependant relationship between its organs and lifestyle. The elements used in the design of these prostheses are critical and related to lower limb amputation level, user morphology and human-prosthetic interaction. Hence, several technologies have been employed to accomplish the end user's needs, for example, advanced materials, control systems, electronics, energy management, signal processing, and artificial intelligence. This paper presents a systematic literature review on such technologies, to identify the latest advances, challenges, and opportunities in developing lower limb prostheses with the analysis on the most significant papers. Powered prostheses for walking in different terrains were illustrated and examined, with the kind of movement the device should perform by considering the electronics, automatic control, and energy efficiency. Results show a lack of a specific and generalised structure to be followed by new developments, gaps in energy management and improved smoother patient interaction. Additionally, Human Prosthetic Interaction (HPI) is a term introduced in this paper since no other research has integrated this interaction in communication between the artificial limb and the end-user. The main goal of this paper is to provide, with the found evidence, a set of steps and components to be followed by new researchers and experts looking to improve knowledge in this field.

Coordinating human-robot collaboration by EEG-based human intention prediction and vigilance control

In human-robot collaboration scenarios with shared workspaces, a highly desired performance boost is offset by high requirements for human safety, limiting speed and torque of the robot drives to levels which cannot harm the human body. Especially for complex tasks with flexible human behavior, it becomes vital to maintain safe working distances and coordinate tasks efficiently. An established approach in this regard is reactive servo in response to the current human pose. However, such an approach does not exploit expectations of the human's behavior and can therefore fail to react to fast human motions in time. To adapt the robot's behavior as soon as possible, predicting human intention early becomes a factor which is vital but hard to achieve. Here, we employ a recently developed type of brain-computer interface (BCI) which can detect the focus of the human's overt attention as a predictor for impending action. In contrast to other types of BCI, direct projection of stimuli onto the workspace facilitates a seamless integration in workflows. Moreover, we demonstrate how the signal-to-noise ratio of the brain response can be used to adjust the velocity of the robot movements to the vigilance or alertness level of the human. Analyzing this adaptive system with respect to performance and safety margins in a physical robot experiment, we found the proposed method could improve both collaboration efficiency and safety distance.

NeuralTree: A 256-Channel 0.227-μJ/Class Versatile Neural Activity Classification and Closed-Loop Neuromodulation SoC

Closed-loop neural interfaces with on-chip machine learning can detect and suppress disease symptoms in neurological disorders or restore lost functions in paralyzed patients. While high-density neural recording can provide rich neural activity information for accurate disease-state detection, existing systems have low channel counts and poor scalability, which could limit their therapeutic efficacy. This work presents a highly scalable and versatile closed-loop neural interface SoC that can overcome these limitations. A 256-channel time-division multiplexed (TDM) front-end with a two-step fast-settling mixed-signal DC servo loop (DSL) is proposed to record high-spatial-resolution neural activity and perform channel-selective brain-state inference. A tree-structured neural network (NeuralTree) classification processor extracts a rich set of neural biomarkers in a patient- and disease-specific manner. Trained with an energy-aware learning algorithm, the NeuralTree classifier detects the symptoms of underlying disorders (e.g., epilepsy and movement disorders) at an optimal energy-accuracy trade-off. A 16-channel high-voltage (HV) compliant neurostimulator closes the therapeutic loop by delivering charge-balanced biphasic current pulses to the brain. The proposed SoC was fabricated in 65nm CMOS and achieved a 0.227μJ/class energy efficiency in a compact area of 0.014mm2/channel. The SoC was extensively verified on human electroencephalography (EEG) and intracranial EEG (iEEG) epilepsy datasets, obtaining 95.6%/94% sensitivity and 96.8%/96.9% specificity, respectively. In-vivo neural recordings using soft μECoG arrays and multi-domain biomarker extraction were further performed on a rat model of epilepsy. In addition, for the first time in literature, on-chip classification of rest-state tremor in Parkinson's disease (PD) from human local field potentials (LFPs) was demonstrated.

Decoding Brain Signals to Classify Gait Direction Anticipation

The use of brain-computer interface (BCI) technology has emerged as a promising rehabilitation approach for patients with motor function and motor-related disorders. BCIs provide an augmentative communication platform for controlling advanced assistive robots such as a lower-limb exoskeleton. Brain recordings collected by an electroencephalography (EEG) system have been employed in the BCI platform to command the exoskeleton. To date, the literature on this topic is limited to the prediction of gait intention and gait variations from EEG signals. This study, however, aims to predict the anticipated gait direction using a stream of EEG signals collected from the brain cortex. Three healthy participants (age range: 29-31, 2 female) were recruited. While wearing the EEG device, the participants were instructed to initiate gait movement toward the direction of the arrow triggers (pointing forward, backward, left, or right) being shown on a screen with a blank white background. Collected EEG data was then epoched around the trigger timepoints. These epochs were then converted to the time-frequency domain using event- related synchronization (ERS) and event-related desynchronization (ERD) methods. Finally, the classification pipeline was constructed using logistic regression (LR), support vector machine (SVM), and convolutional neural network (CNN). A ten-fold cross-validation scheme was used to evaluate the classification performance. The results revealed that the CNN classifier outperforms the other two classifiers with an accuracy of 0.75. Clinical Relevance - The outcome of this study has the potential to be ultimately used for interactive navigation of the lower-limb exoskeletons during robotic rehabilitation therapy and enhance neurodegeneration and neuroplasticity in a wide range of individuals with lower-limb motor function disabilities.

Development of an Adaptive Artifact Subspace Reconstruction Based on Hebbian/Anti-Hebbian Learning Networks for Enhancing BCI Performance

Brain-computer interface (BCI) actively translates the brain signals into executable actions by establishing direct communication between the human brain and external devices. Recording brain activity through electroencephalography (EEG) is generally contaminated with both physiological and nonphysiological artifacts, which significantly hinders the BCI performance. Artifact subspace reconstruction (ASR) is a well-known statistical technique that automatically removes artifact components by determining the rejection threshold based on the initial reference EEG segment in multichannel EEG recordings. In real-world applications, the fixed threshold may limit the efficacy of the artifact correction, especially when the quality of the reference data is poor. This study proposes an adaptive online ASR technique by integrating the Hebbian/anti-Hebbian neural networks into the ASR algorithm, namely, principle subspace projection ASR (PSP-ASR) and principal subspace whitening ASR (PSW-ASR) that segmentwise self-organize the artifact subspace by updating the synaptic weights according to the Hebbian and anti-Hebbian learning rules. The effectiveness of the proposed algorithm is compared to the conventional ASR approaches on benchmark EEG dataset and three BCI frameworks, including steady-state visual evoked potential (SSVEP), rapid serial visual presentation (RSVP), and motor imagery (MI) by evaluating the root-mean-square error (RMSE), the signal-to-noise ratio (SNR), the Pearson correlation, and classification accuracy. The results demonstrated that the PSW-ASR algorithm effectively removed the EEG artifacts and retained the activity-specific brain signals compared to the PSP-ASR, standard ASR (Init-ASR), and moving-window ASR (MW-ASR) methods, thereby enhancing the SSVEP, RSVP, and MI BCI performances. Finally, our empirical results from the PSW-ASR algorithm suggested the choice of an aggressive cutoff range of c = 1-10 for activity-specific BCI applications and a moderate range of for the benchmark dataset and general BCI applications.

Integration of Inertial Sensors in a Lower Limb Robotic Exoskeleton

Motion assistance exoskeletons are designed to support the joint movement of people who perform repetitive tasks that cause damage to their health. To guarantee motion accompaniment, the integration between sensors and actuators should ensure a near-zero delay between the signal acquisition and the actuator response. This study presents the integration of a platform based on Imocap-GIS inertial sensors, with a motion assistance exoskeleton that generates joint movement by means of Maxon motors and Harmonic drive reducers, where a near zero-lag is required for the gait accompaniment to be correct. The Imocap-GIS sensors acquire positional data from the user's lower limbs and send the information through the UDP protocol to the CompactRio system, which constitutes a high-performance controller. These data are processed by the card and subsequently a control signal is sent to the motors that move the exoskeleton joints. Simulations of the proposed controller performance were conducted. The experimental results show that the motion accompaniment exhibits a delay of between 20 and 30 ms, and consequently, it may be stated that the integration between the exoskeleton and the sensors achieves a high efficiency. In this work, the integration between inertial sensors and an exoskeleton prototype has been proposed, where it is evident that the integration met the initial objective. In addition, the integration between the exoskeleton and IMOCAP is among the highest efficiency ranges of similar systems that are currently being developed, and the response lag that was obtained could be improved by means of the incorporation of complementary systems.

Detecting the Speed Change Intention from EEG Signals: From the Offline and Pseudo-Online Analysis to an Online Closed-Loop Validation

Control of assistive devices by voluntary user intention is an underdeveloped topic in the Brain–Machine Interfaces (BMI) literature. In this work, a preliminary real-time BMI for the speed control of an exoskeleton is presented. First, an offline analysis for the selection of the intention patterns based on the optimum features and electrodes is proposed. This is carried out comparing three different classification models: monotonous walk vs. increasing and decreasing change speed intentions, monotonous walk vs. only increasing intention, and monotonous walk vs. only decreasing intention. The results indicate that, among the features tested, the most suitable parameter to represent these models are the Hjorth statistics in alpha and beta frequency bands. The average offline classification accuracy for the offline cross-validation of the three models obtained is 68 ± 11%. This selection is also tested following a pseudo-online analysis, simulating a real-time detection of the subject’s intentions to change speed. The average results indices of the three models during this pseudoanalysis are of a 42% true positive ratio and a false positive rate per minute of 9. Finally, in order to check the viability of the approach with an exoskeleton, a case of study is presented. During the experimental session, the pros and cons of the implementation of a closed-loop control of speed change for the H3 exoskeleton through EEG analysis are commented.

Brain Activity Reflects Subjective Response to Delayed Input When Using an Electromyography-Controlled Robot

In various experimental settings, electromyography (EMG) signals have been used to control robots. EMG-based robot control requires intrinsic parameters for control, which makes it difficult for users to understand the input protocol. When a proper input is not provided, the response time of the system varies; as such, the user’s subjective delay should be investigated regardless of the actual delay. In this study, we investigated the influence of the subjective perception of delay on brain activation. Brain recordings were taken while subjects used EMG signals to control a robot hand, which requires a basic processing delay. We used muscle synergy for the grip command of the robot hand. After controlling the robot by grasping their hand, one of four additional delay durations (0 ms, 50 ms, 125 ms, and 250 ms) was applied in every trial, and subjects were instructed to answer whether the delay was natural, additional, or whether they were not sure. We compared brain activity based on responses (“sure” and “not sure”). Our results revealed a significant power difference in the theta band of the parietal lobe, and this time range included the interval in which the subjects could not feel the delay. Our study provides important insights that should be considered when constructing an adaptive system and evaluating its usability.

Emotion Recognition Based on EEG Using Generative Adversarial Nets and Convolutional Neural Network

Emotion recognition plays an important role in the field of human-computer interaction (HCI). Automatic emotion recognition based on EEG is an important topic in brain-computer interface (BCI) applications. Currently, deep learning has been widely used in the field of EEG emotion recognition and has achieved remarkable results. However, due to the cost of data collection, most EEG datasets have only a small amount of EEG data, and the sample categories are unbalanced in these datasets. These problems will make it difficult for the deep learning model to predict the emotional state. In this paper, we propose a new sample generation method using generative adversarial networks to solve the problem of EEG sample shortage and sample category imbalance. In experiments, we explore the performance of emotion recognition with the frequency band correlation and frequency band separation computational models before and after data augmentation on standard EEG-based emotion datasets. Our experimental results show that the method of generative adversarial networks for data augmentation can effectively improve the performance of emotion recognition based on the deep learning model. And we find that the frequency band correlation deep learning model is more conducive to emotion recognition.

Prediction of gait trajectories based on the Long Short Term Memory neural networks

The forecasting of lower limb trajectories can improve the operation of assistive devices and minimise the risk of tripping and balance loss. The aim of this work was to examine four Long Short Term Memory (LSTM) neural network architectures (Vanilla, Stacked, Bidirectional and Autoencoders) in predicting the future trajectories of lower limb kinematics, i.e. Angular Velocity (AV) and Linear Acceleration (LA). Kinematics data of foot, shank and thigh (LA and AV) were collected from 13 male and 3 female participants (28 ± 4 years old, 1.72 ± 0.07 m in height, 66 ± 10 kg in mass) who walked for 10 minutes at preferred walking speed (4.34 ± 0.43 km.h-1) and at an imposed speed (5km.h-1, 15.4% ± 7.6% faster) on a 0% gradient treadmill. The sliding window technique was adopted for training and testing the LSTM models with total kinematics time-series data of 10,500 strides. Results based on leave-one-out cross validation, suggested that the LSTM autoencoders is the top predictor of the lower limb kinematics trajectories (i.e. up to 0.1s). The normalised mean squared error was evaluated on trajectory predictions at each time-step and it obtained 2.82-5.31% for the LSTM autoencoders. The ability to predict future lower limb motions may have a wide range of applications including the design and control of bionics allowing improved human-machine interface and mitigating the risk of falls and balance loss.

A multi-modal modified feedback self-paced BCI to control the gait of an avatar

Brain-computer interfaces (BCI) have been used to control the gait of a virtual self-avatar with a proposed application in the field of gait rehabilitation.

OBJECTIVE

to develop a high performance multi-modal BCI to control single steps and forward walking of an immersive virtual reality avatar. This system will overcome the limitation of existing systems.

APPROACH

This system used MI of these actions, in cue-paced and self-paced modes. Twenty healthy participants participated in this 4 sessions study across 4 different days. They were cued to imagine a single step forward with their right or left foot, or to imagine walking forward. They were instructed to reach a target by using the MI of multiple steps (self-paced switch-control mode) or by maintaining MI of forward walking (continuous-control mode). The movement of the avatar was controlled by two calibrated RLDA classifiers that used the µ power spectral density (PSD) over the foot area of the motor cortex as a feature. The classifiers were retrained after every session. For a subset of the trials, positive modified feedback was presented to half of the participants.

MAIN RESULTS

All participants were able to operate the BCI. Their average offline performance, after retraining the classifiers was 86.0 ± 6.1%, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9 ± 8.4% showing that modified feedback enhanced BCI performance (p =0.001). The average performance was 83% at self-paced switch control and 92% at continuous control mode.

SIGNIFICANCE

This study reports on the first novel integration of different design approaches, different control modes and different performance enhancement techniques, all in parallel in one single high performance and multi-modal BCI system, to control single steps and forward walking of an immersive virtual reality avatar.

Neural Correlates of Freezing of Gait in Parkinson's Disease: An Electrophysiology Mini-Review

Freezing of gait (FoG) is a disabling symptom characterized as a brief inability to step or by short steps, which occurs when initiating gait or while turning, affecting over half the population with advanced Parkinson's disease (PD). Several non-competing hypotheses have been proposed to explain the pathophysiology and mechanism behind FoG. Yet, due to the complexity of FoG and the lack of a complete understanding of its mechanism, no clear consensus has been reached on the best treatment options. Moreover, most studies that aim to explore neural biomarkers of FoG have been limited to semi-static or imagined paradigms. One of the biggest unmet needs in the field is the identification of reliable biomarkers that can be construed from real walking scenarios to guide better treatments and validate medical and therapeutic interventions. Advances in neural electrophysiology exploration, including EEG and DBS, will allow for pathophysiology research on more real-to-life scenarios for better FoG biomarker identification and validation. The major aim of this review is to highlight the most up-to-date studies that explain the mechanisms underlying FoG through electrophysiology explorations. The latest methodological approaches used in the neurophysiological study of FoG are summarized, and potential future research directions are discussed.

Asynchronous Prediction of Human Gait Intention in a Pseudo Online Paradigm Using Wavelet Transform

Prediction of human voluntary gait intention is a very significant task to ensure direct cortical control of real-life assistive technologies for locomotion rehabilitation. Neurophysiological studies provide that human voluntary gait intention is represented by slow DC potentials and power shifts in specific frequency ranges of brain wave, which can be detected 1.5- 2 seconds before the actual onset. The goal of this study was to determine whether it is possible to reliably detect the intention of voluntary gait 'starting' and 'stopping' intention before it takes place. A computational algorithm was designed to implement asynchronous prediction of gait intention in an offline and pseudo-online environment using support vector machine. Six healthy subjects participated in the study and performed self- paced voluntary gait cycles. A combination of advanced wavelet transform algorithms resulted in 88.23± 1.59% accuracy, 85.42± 4.03% sensitivity and 90.24± 2.78% specificity for intention of start detection and 87.04± 1.72% accuracy, 82.69± 4.13% sensitivity and 89.59± 3.04% specificity for intention to stop walking in offline testing. Additionally, the wavelet transform methods accompanied with threshold regulation and majority voting algorithm resulted in a True Positive Rate of 85.5± 5.0% and 81.2± 3.3% for 'start' and 'stop' prediction with 6.8± 0.7 and 9.4± 1.0 False Positives per Minute respectively in pseudo online testing. The average detection latencies were -1002 ± 603 ms and -943 ± 603 ms, respectively, for 'start' and 'stop' prediction. The study provides promising outcomes in terms of TPR, FP/min, and detection latency, which suggests that human voluntary gait intention can be predicted before the onset of movement.