Brain-machine interfaces for controlling lower-limb powered robotic systems

State of the Art of Brain Function Detection Technologies in Robot-Assisted Lower Limb Rehabilitation

Background: With an aging population, the prevalence of neurological disorders is increasing, leading to a rise in lower limb movement disorders and, in turn, a growing need for rehabilitation training. Previous neuroimaging studies have shown a growing scientific interest in the study of brain mechanisms in robot-assisted lower limb rehabilitation (RALLR). Objective: This review aimed to determine differences in neural activity patterns during different RALLR tasks and the impact on neurofunctional plasticity. Methods: Sixty-five articles in the field of RALLR were selected and tested using three brain function detection technologies. Results: Most studies have focused on changes in activity in various regions of the cerebral cortex during different lower limb rehabilitation tasks but have also increasingly focused on functional changes in other cortical and deep subcortical structures. Our analysis also revealed a neglect of certain task types. Conclusion: We identify and discuss future research directions that may contribute to a clear understanding of neural functional plasticity under different RALLR tasks. Impact Statement The evaluation of robot-assisted lower limb rehabilitation based on brain function detection technology can assess the neurological changes of patients in the rehabilitation process by monitoring brain activities and can also provide more accurate guidance for robot-assisted lower limb rehabilitation. By monitoring the patient's brain activity, the robot can adjust according to the real-time status of the patient to achieve more effective rehabilitation training. This has potential impact on improving the rehabilitation effect and speeding up the rehabilitation process of patients.

EEG classification based on visual stimuli via adversarial learning

In this work, we propose a dual path deep learning architecture for the application of visual brain decoding. The inputs to the proposed network are the electroencephalogram (EEG) signals which are evoked due to the external stimuli, specifically, images in this case. The objective is to classify the EEG signals based on the image categories under which they were evoked. Our approach involves the combinations of convolution neural networks (CNN) on the time axis and the channel axis. Importantly, for the purpose of learning subject-invariant features, we also make use of the gradient reversal layer (GRL). This addition to our network boosts the performance of our system. In addition, we also propose to use guided back-propagation for the selection of more informative EEG channels, and finally, with the reduced number of channels, we estimate the performance of the proposed network which is almost similar to the version when considering all EEG channels.

Attention-based convolutional neural network with multi-modal temporal information fusion for motor imagery EEG decoding

Convolutional neural network (CNN) has been widely applied in motor imagery (MI)-based brain computer interface (BCI) to decode electroencephalography (EEG) signals. However, due to the limited perceptual field of convolutional kernel, CNN only extracts features from local region without considering long-term dependencies for EEG decoding. Apart from long-term dependencies, multi-modal temporal information is equally important for EEG decoding because it can offer a more comprehensive understanding of the temporal dynamics of neural processes. In this paper, we propose a novel deep learning network that combines CNN with self-attention mechanism to encapsulate multi-modal temporal information and global dependencies. The network first extracts multi-modal temporal information from two distinct perspectives: average and variance. A shared self-attention module is then designed to capture global dependencies along these two feature dimensions. We further design a convolutional encoder to explore the relationship between average-pooled and variance-pooled features and fuse them into more discriminative features. Moreover, a data augmentation method called signal segmentation and recombination is proposed to improve the generalization capability of the proposed network. The experimental results on the BCI Competition IV-2a (BCIC-IV-2a) and BCI Competition IV-2b (BCIC-IV-2b) datasets show that our proposed method outperforms the state-of-the-art methods and achieves 4-class average accuracy of 85.03% on the BCIC-IV-2a dataset. The proposed method implies the effectiveness of multi-modal temporal information fusion in attention-based deep learning networks and provides a new perspective for MI-EEG decoding. The code is available at https://github.com/Ma-Xinzhi/EEG-TransNet.

Cortical signals analysis to recognize intralimb mobility using modified RNN and various EEG quantities

Electroencephalogram (EEG) signals are critical in interpreting sensorimotor activities for predicting body movements. However, their efficacy in identifying intralimb movements, such as the dorsiflexion and plantar flexion of the foot, remains suboptimal. This study aims to explore whether various EEG signal quantities can effectively recognize intralimb movements to facilitate the development of Brain-Computer Interface (BCI) devices for foot rehabilitation. This research involved twenty-two healthy, right-handed participants. EEG data were collected using 21 electrodes positioned over the motor cortex, while two electromyography (EMG) electrodes recorded the onset of ankle joint movements. The study focused on analyzing slow cortical potential (SCP) and sensorimotor rhythms (SMR) in alpha and beta bands from the EEG. Five key features-fourth-order Autoregressive feature, variance, waveform length, standard deviation, and permutation entropy-were extracted. A modified Recurrent Neural Network (RNN) including Long Short-term Memory (LSTM) and Gated Recurrent Unit (GRU) algorithms was developed for movement recognition. These were compared against conventional machine learning algorithms, including nonlinear Support Vector Machine (SVM) and k Nearest Neighbourhood (kNN) classifiers. The performance of the proposed models was assessed using two data schemes: within-subject and across-subjects. The findings demonstrated that the GRU and LSTM models significantly outperformed traditional machine learning algorithms in recognizing different EEG signal quantities for intralimb movement. The study indicates that deep learning models, particularly GRU and LSTM, hold superior potential over standard machine learning techniques in identifying intralimb movements using EEG signals. Where the accuracies of LSTM for within and across subjects were 98.87 ± 1.80 % and 87.38 ± 0.86 % respectively. Whereas the accuracy of GRU within and across subjects were 99.18 ± 1.28 % and 86.44 ± 0.69 % respectively. This advancement could significantly benefit the development of BCI devices aimed at foot rehabilitation, suggesting a new avenue for enhancing physical therapy outcomes.

Brain-machine interface based on deep learning to control asynchronously a lower-limb robotic exoskeleton: a case-of-study

BACKGROUND

This research focused on the development of a motor imagery (MI) based brain-machine interface (BMI) using deep learning algorithms to control a lower-limb robotic exoskeleton. The study aimed to overcome the limitations of traditional BMI approaches by leveraging the advantages of deep learning, such as automated feature extraction and transfer learning. The experimental protocol to evaluate the BMI was designed as asynchronous, allowing subjects to perform mental tasks at their own will.

METHODS

A total of five healthy able-bodied subjects were enrolled in this study to participate in a series of experimental sessions. The brain signals from two of these sessions were used to develop a generic deep learning model through transfer learning. Subsequently, this model was fine-tuned during the remaining sessions and subjected to evaluation. Three distinct deep learning approaches were compared: one that did not undergo fine-tuning, another that fine-tuned all layers of the model, and a third one that fine-tuned only the last three layers. The evaluation phase involved the exclusive closed-loop control of the exoskeleton device by the participants' neural activity using the second deep learning approach for the decoding.

RESULTS

The three deep learning approaches were assessed in comparison to an approach based on spatial features that was trained for each subject and experimental session, demonstrating their superior performance. Interestingly, the deep learning approach without fine-tuning achieved comparable performance to the features-based approach, indicating that a generic model trained on data from different individuals and previous sessions can yield similar efficacy. Among the three deep learning approaches compared, fine-tuning all layer weights demonstrated the highest performance.

CONCLUSION

This research represents an initial stride toward future calibration-free methods. Despite the efforts to diminish calibration time by leveraging data from other subjects, complete elimination proved unattainable. The study's discoveries hold notable significance for advancing calibration-free approaches, offering the promise of minimizing the need for training trials. Furthermore, the experimental evaluation protocol employed in this study aimed to replicate real-life scenarios, granting participants a higher degree of autonomy in decision-making regarding actions such as walking or stopping gait.

Boosting lower-limb motor imagery performance through an ensemble method for gait rehabilitation

Lower-limb exoskeletons have been used extensively in many rehabilitation applications to assist disabled people with their therapies. Brain-machine interfaces (BMIs) further provide effective and natural control schemes. However, the limited performance of brain signal decoding from lower-limb kinematics restricts the broad growth of both BMI and rehabilitation industry. To address these challenges, we propose an ensemble method for lower-limb motor imagery (MI) classification. The proposed model employs multiple techniques to boost performance, including deep and shallow parts. Traditional wavelet transformation followed by filter-bank common spatial pattern (CSP) employs neurophysiologically reasonable patterns, while multi-head self-attention (MSA) followed by temporal convolutional network (TCN) extracts deeper encoded generalized patterns. Experimental results in a customized lower-limb exoskeleton on 8 subjects in 3 consecutive sessions showed that the proposed method achieved 60.27% and 64.20% for three (MI of left leg, MI of right leg, and rest) and two classes (lower-limb MI vs. rest), respectively. Besides, the proposed model achieves improvements of up to 4% and 2% accuracy for the subject-specific and subject-independent modes compared to the current state-of-the-art (SOTA) techniques, respectively. Finally, feature analysis was conducted to show discriminative brain patterns in each MI task and sessions with different feedback modalities. The proposed models integrated in the brain-actuated lower-limb exoskeleton established a potential BMI for gait training and neuroprosthesis.

Flexible Dry Electrode Based on a Wrinkled Surface That Uses Carbon Nanotube/Polymer Composites for Recording Electroencephalograms

Electroencephalography (EEG) captures minute electrical signals emanating from the brain. These signals are vulnerable to interference from external noise and dynamic artifacts; hence, accurately recording such signals is challenging. Although dry electrodes are convenient, their signals are of limited quality; consequently, wet electrodes are predominantly used in EEG. Therefore, developing dry electrodes for accurately and stably recording EEG signals is crucial. In this study, we developed flexible dry electrodes using polydimethylsiloxane (PDMS)/carbon-nanotube (CNT) composites with isotropically wrinkled surfaces that effectively combine the advantages of wet and dry electrodes. Adjusting the PDMS crosslinker ratio led to good adhesion, resulting in a highly adhesive CNT/PDMS composite with a low Young's modulus that exhibited excellent electrical and mechanical properties owing to its ability to conformally contact skin. The isotropically wrinkled surface also effectively controls dynamic artifacts during EEG signal detection and ensures accurate signal analysis. The results of this study demonstrate that dry electrodes based on flexible CNT/PDMS composites and corrugated structures can outperform wet electrodes. The introduction of such electrodes is expected to enable the accurate analysis and monitoring of EEG signals in various scenarios, including clinical trials.

Antioxidant Dimethyl Fumarate Temporarily but Not Chronically Improves Intracortical Microelectrode Performance

Intracortical microelectrode arrays (MEAs) can be used in a range of applications, from basic neuroscience research to providing an intimate interface with the brain as part of a brain-computer interface (BCI) system aimed at restoring function for people living with neurological disorders or injuries. Unfortunately, MEAs tend to fail prematurely, leading to a loss in functionality for many applications. An important contributing factor in MEA failure is oxidative stress resulting from chronically inflammatory-activated microglia and macrophages releasing reactive oxygen species (ROS) around the implant site. Antioxidants offer a means for mitigating oxidative stress and improving tissue health and MEA performance. Here, we investigate using the clinically available antioxidant dimethyl fumarate (DMF) to reduce the neuroinflammatory response and improve MEA performance in a rat MEA model. Daily treatment of DMF for 16 weeks resulted in a significant improvement in the recording capabilities of MEA devices during the sub-chronic (Weeks 5-11) phase (42% active electrode yield vs. 35% for control). However, these sub-chronic improvements were lost in the chronic implantation phase, as a more exacerbated neuroinflammatory response occurs in DMF-treated animals by 16 weeks post-implantation. Yet, neuroinflammation was indiscriminate between treatment and control groups during the sub-chronic phase. Although worse for chronic use, a temporary improvement (<12 weeks) in MEA performance is meaningful. Providing short-term improvement to MEA devices using DMF can allow for improved use for limited-duration studies. Further efforts should be taken to explore the mechanism behind a worsened neuroinflammatory response at the 16-week time point for DMF-treated animals and assess its usefulness for specific applications.

A New Compound-Limbs Paradigm: Integrating Upper-Limb Swing Improves Lower-Limb Stepping Intention Decoding From EEG

Brain-computer interface (BCI) systems based on spontaneous electroencephalography (EEG) hold the promise to implement human voluntary control of lower-extremity powered exoskeletons. However, current EEG-BCI paradigms do not consider the cooperation of upper and lower limbs during walking, which is inconsistent with natural human stepping patterns. To deal with this problem, this study proposed a stepping-matched human EEG-BCI paradigm that involved actions of both unilateral lower and contralateral upper limbs (also referred to as compound-limbs movement). Experiments were conducted in motor execution (ME) and motor imagery (MI) conditions to validate the feasibility. Common spatial pattern (CSP) proposed subject-specific CSP (SSCSP), and filter-bank CSP (FBCSP) methods were applied for feature extraction, respectively. The best average classification results based on SSCSP indicated that the accuracies of compound-limbs paradigms in ME and MI conditions achieved 89.02% ± 12.84% and 73.70% ± 12.47%, respectively. Although they were 2.03% and 5.68% lower than those of the single-upper-limb mode that does not match human stepping patterns, they were 24.30% and 11.02% higher than those of the single-lower-limb mode. These findings indicated that the proposed compound-limbs EEG-BCI paradigm is feasible for decoding human stepping intention and thus provides a potential way for natural human control of walking assistance devices.

Online semi-supervised learning for motor imagery EEG classification

OBJECTIVE

Time-consuming data labeling in brain-computer interfaces (BCIs) raises many problems such as mental fatigue and is one key factor that hinders the real-world adoption of motor imagery (MI)-based BCIs. An alternative approach is to integrate readily available, as well as informative, unlabeled data online, whereas this approach is less investigated.

APPROACH

We proposed an online semi-supervised learning scheme to improve the classification performance of MI-based BCI. This scheme uses regularized weighted online sequential extreme learning machine (RWOS-ELM) as the base classifier and updates its model parameters with incoming balanced data chunk-by-chunk. In the initial stage, we designed a technique that combines the synthetic minority oversampling with the edited nearest neighbor rule for data augmentation to construct more discriminative initial classifiers. When used online, the incoming chunk of data is first pseudo-labeled by RWOS-ELM as well as an auxiliary classifier, and then balanced again by the above-mentioned technique. Initial classifiers are further updated based on these class-balanced data.

MAIN RESULTS

Offline experimental results on two publicly available MI datasets demonstrate the superiority of the proposed scheme over its counterparts. Further online experiments on six subjects show that their BCI performance gradually improved by learning from incoming unlabeled data.

SIGNIFICANCE

Our proposed online semi-supervised learning scheme has higher computation and memory usage efficiency, which is promising for online MI-based BCIs, especially in the case of insufficient labeled training data.

Metabolic cost of walking with electromechanical ankle exoskeletons under proportional myoelectric control on a treadmill and outdoors

Lower limb robotic exoskeletons are often studied in the context of steady state treadmill walking in a laboratory environment. However, the end goal for exoskeletons is to be used in real world, complex environments. To reach the point that exoskeletons are openly adopted into our everyday lives, we need to understand how the human and robot interact outside of a laboratory. Metabolic cost is often viewed as a gold standard metric for measuring exoskeleton performance but is rarely used to evaluate performance at non steady state walking outside of a laboratory. In this study, we tested the effects of robotic ankle exoskeletons under proportional myoelectric control on the cost of transport of walking both inside on a treadmill and outside overground. We hypothesized that walking with the exoskeletons would lead to a lower cost of transport compared to walking without them both on a treadmill and outside. We saw no significant increases or decreases in cost of transport or exoskeleton mechanics when walking with the exoskeletons compared to walking without them both on a treadmill and outside. We saw a strong negative correlation between walking speed and cost of transport when walking with and without the exoskeletons. In the future, research should consider how performing more difficult tasks, such as incline and loaded walking, affects the cost of transport while walking with and without robotic ankle exoskeletons. The value of this study to the literature is that it emphasizes the importance of both hardware dynamics and controller design towards reducing metabolic cost of transport with robotic ankle exoskeletons. When comparing our results to other studies using the same hardware with different controllers or very similar controllers with different hardware, there are a wide range of outcomes as to metabolic benefit.

Data-efficient human walking speed intent identification

The ability to accurately identify human gait intent is a challenge relevant to the success of many applications in robotics, including, but not limited to, assistive devices. Most existing intent identification approaches, however, are either sensor-specific or use a pattern-recognition approach that requires large amounts of training data. This paper introduces a real-time walking speed intent identification algorithm based on the Mahalanobis distance that requires minimal training data. This data efficiency is enabled by making the simplifying assumption that each time step of walking data is independent of all other time steps. The accuracy of the algorithm was analyzed through human-subject experiments that were conducted using controlled walking speed changes on a treadmill. Experimental results confirm that the model used for intent identification converges quickly (within 5 min of training data). On average, the algorithm successfully detected the change in desired walking speed within one gait cycle and had a maximum of 87% accuracy at responding with the correct intent category of speed up, slow down, or no change. The findings also show that the accuracy of the algorithm improves with the magnitude of the speed change, while speed increases were more easily detected than speed decreases.

The Application of Deep Learning to Electroencephalograms, Magnetic Resonance Imaging, and Implants for the Detection of Epileptic Seizures: A Narrative Review

Epilepsy is a neurological disorder characterized by recurrent seizures affecting millions worldwide. Medically intractable seizures in epilepsy patients are not only detrimental to the quality of life but also pose a significant threat to their safety. Outcomes of epilepsy therapy can be improved by early detection and intervention during the interictal window period. Electroencephalography is the primary diagnostic tool for epilepsy, but accurate interpretation of seizure activity is challenging and highly time-consuming. Machine learning (ML) and deep learning (DL) algorithms enable us to analyze complex EEG data, which can not only help us diagnose but also locate epileptogenic zones and predict medical and surgical treatment outcomes. DL models such as convolutional neural networks (CNNs), inspired by visual processing, can be used to classify EEG activity. By applying preprocessing techniques, signal quality can be enhanced by denoising and artifact removal. DL can also be incorporated into the analysis of magnetic resonance imaging (MRI) data, which can help in the localization of epileptogenic zones in the brain. Proper detection of these zones can help in good neurosurgical outcomes. Recent advancements in DL have facilitated the implementation of these systems in neural implants and wearable devices, allowing for real-time seizure detection. This has the potential to transform the management of drug-refractory epilepsy. This review explores the application of ML and DL techniques to Electroencephalograms (EEGs), MRI, and wearable devices for epileptic seizure detection. This review briefly explains the fundamentals of both artificial intelligence (AI) and DL, highlighting these systems' potential advantages and undeniable limitations.

Transfer Learning with CNN Models for Brain-Machine Interfaces to command lower-limb exoskeletons: A Solution for Limited Data. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083615 DOI: 10.1109/embc40787.2023.10340008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

This study evaluates the performance of two convolutional neural networks (CNNs) in a brain-machine interface (BMI) based on motor imagery (MI) by using a small dataset collected from five participants wearing a lower-limb exoskeleton. To address the issue of limited data availability, transfer learning was employed by training models on EEG signals from other subjects and subsequently fine-tuning them to specific users. A combination of common spatial patterns (CSP) and linear discriminant analysis (LDA) was used as a benchmark for comparison. The study's primary aim is to examine the potential of CNNs and transfer learning in the development of an automatic neural classification system for a BMI based on MI to command a lower-limb exoskeleton that can be used by individuals without specialized training.Clinical Relevance- BMI can be used in rehabilitation for patients with motor impairment by using mental simulation of movement to activate robotic exoskeletons. This can promote neural plasticity and aid in recovery.

Design and Validation of a Low-Cost Mobile EEG-Based Brain-Computer Interface

Objective: We designed and validated a wireless, low-cost, easy-to-use, mobile, dry-electrode headset for scalp electroencephalography (EEG) recordings for closed-loop brain-computer (BCI) interface and internet-of-things (IoT) applications. Approach: The EEG-based BCI headset was designed from commercial off-the-shelf (COTS) components using a multi-pronged approach that balanced interoperability, cost, portability, usability, form factor, reliability, and closed-loop operation. Main Results: The adjustable headset was designed to accommodate 90% of the population. A patent-pending self-positioning dry electrode bracket allowed for vertical self-positioning while parting the user's hair to ensure contact of the electrode with the scalp. In the current prototype, five EEG electrodes were incorporated in the electrode bracket spanning the sensorimotor cortices bilaterally, and three skin sensors were included to measure eye movement and blinks. An inertial measurement unit (IMU) provides monitoring of head movements. The EEG amplifier operates with 24-bit resolution up to 500 Hz sampling frequency and can communicate with other devices using 802.11 b/g/n WiFi. It has high signal-to-noise ratio (SNR) and common-mode rejection ratio (CMRR) (121 dB and 110 dB, respectively) and low input noise. In closed-loop BCI mode, the system can operate at 40 Hz, including real-time adaptive noise cancellation and 512 MB of processor memory. It supports LabVIEW as a backend coding language and JavaScript (JS), Cascading Style Sheets (CSS), and HyperText Markup Language (HTML) as front-end coding languages and includes training and optimization of support vector machine (SVM) neural classifiers. Extensive bench testing supports the technical specifications and human-subject pilot testing of a closed-loop BCI application to support upper-limb rehabilitation and provides proof-of-concept validation for the device's use at both the clinic and at home. Significance: The usability, interoperability, portability, reliability, and programmability of the proposed wireless closed-loop BCI system provides a low-cost solution for BCI and neurorehabilitation research and IoT applications.

An EEG database for the cognitive assessment of motor imagery during walking with a lower-limb exoskeleton

One important point in the development of a brain-machine Interface (BMI) commanding an exoskeleton is the assessment of the cognitive engagement of the subject during the motor imagery tasks conducted. However, there are not many databases that provide electroencephalography (EEG) data during the use of a lower-limb exoskeleton. The current paper presents a database designed with an experimental protocol aiming to assess not only motor imagery during the control of the device, but also the attention to gait on flat and inclined surfaces. The research was conducted as an EUROBENCH subproject in the facilities sited in Hospital Los Madroños, Brunete (Madrid). The data validation reaches accuracies over 70% in the assessment of motor imagery and attention to gait, which marks the present database as a valuable resource for researches interested on developing and testing new EEG-based BMIs.

Brain-computer interface enhanced by virtual reality training for controlling a lower limb exoskeleton

This study explores the use of a brain-computer interface (BCI) based on motor imagery (MI) for the control of a lower limb exoskeleton to aid in motor recovery after a neural injury. The BCI was evaluated in ten able-bodied subjects and two patients with spinal cord injuries. Five able-bodied subjects underwent a virtual reality (VR) training session to accelerate training with the BCI. Results from this group were compared with a control group of five able-bodied subjects, and it was found that the employment of shorter training by VR did not reduce the effectiveness of the BCI and even improved it in some cases. Patients gave positive feedback about the system and were able to handle experimental sessions without reaching high levels of physical and mental exertion. These results are promising for the inclusion of BCI in rehabilitation programs, and future research should investigate the potential of the MI-based BCI system.

Impact of dataset size and long-term ECoG-based BCI usage on deep learning decoders performance

Introduction

In brain-computer interfaces (BCI) research, recording data is time-consuming and expensive, which limits access to big datasets. This may influence the BCI system performance as machine learning methods depend strongly on the training dataset size. Important questions arise: taking into account neuronal signal characteristics (e.g., non-stationarity), can we achieve higher decoding performance with more data to train decoders? What is the perspective for further improvement with time in the case of long-term BCI studies? In this study, we investigated the impact of long-term recordings on motor imagery decoding from two main perspectives: model requirements regarding dataset size and potential for patient adaptation.

Methods

We evaluated the multilinear model and two deep learning (DL) models on a long-term BCI &amp; Tetraplegia (ClinicalTrials.gov identifier: NCT02550522) clinical trial dataset containing 43 sessions of ECoG recordings performed with a tetraplegic patient. In the experiment, a participant executed 3D virtual hand translation using motor imagery patterns. We designed multiple computational experiments in which training datasets were increased or translated to investigate the relationship between models' performance and different factors influencing recordings.

Results

Our results showed that DL decoders showed similar requirements regarding the dataset size compared to the multilinear model while demonstrating higher decoding performance. Moreover, high decoding performance was obtained with relatively small datasets recorded later in the experiment, suggesting motor imagery patterns improvement and patient adaptation during the long-term experiment. Finally, we proposed UMAP embeddings and local intrinsic dimensionality as a way to visualize the data and potentially evaluate data quality.

Discussion

DL-based decoding is a prospective approach in BCI which may be efficiently applied with real-life dataset size. Patient-decoder co-adaptation is an important factor to consider in long-term clinical BCI.

Brain-machine interface based on transfer-learning for detecting the appearance of obstacles during exoskeleton-assisted walking

Introduction

Brain-machine interfaces (BMIs) attempt to establish communication between the user and the device to be controlled. BMIs have great challenges to face in order to design a robust control in the real field of application. The artifacts, high volume of training data, and non-stationarity of the signal of EEG-based interfaces are challenges that classical processing techniques do not solve, showing certain shortcomings in the real-time domain. Recent advances in deep-learning techniques open a window of opportunity to solve some of these problems. In this work, an interface able to detect the evoked potential that occurs when a person intends to stop due to the appearance of an unexpected obstacle has been developed.

Material and methods

First, the interface was tested on a treadmill with five subjects, in which the user stopped when an obstacle appeared (simulated by a laser). The analysis is based on two consecutive convolutional networks: the first one to discern the intention to stop against normal walking and the second one to correct false detections of the previous one.

Results and discussion

The results were superior when using the methodology of the two consecutive networks vs. only the first one in a cross-validation pseudo-online analysis. The false positives per min (FP/min) decreased from 31.8 to 3.9 FP/min and the number of repetitions in which there were no false positives and true positives (TP) improved from 34.9% to 60.3% NOFP/TP. This methodology was tested in a closed-loop experiment with an exoskeleton, in which the brain-machine interface (BMI) detected an obstacle and sent the command to the exoskeleton to stop. This methodology was tested with three healthy subjects, and the online results were 3.8 FP/min and 49.3% NOFP/TP. To make this model feasible for non-able bodied patients with a reduced and manageable time frame, transfer-learning techniques were applied and validated in the previous tests, and were then applied to patients. The results for two incomplete Spinal Cord Injury (iSCI) patients were 37.9% NOFP/TP and 7.7 FP/min.

A Long Short-Term Memory Network Using Resting-State Electroencephalogram to Predict Outcomes Following Moderate Traumatic Brain Injury

Although traumatic brain injury (TBI) is a global public health issue, not all injuries necessitate additional hospitalisation. Thinking, memory, attention, personality, and movement can all be negatively impacted by TBI. However, only a small proportion of nonsevere TBIs necessitate prolonged observation. Clinicians would benefit from an electroencephalography (EEG)-based computational intelligence model for outcome prediction by having access to an evidence-based analysis that would allow them to securely discharge patients who are at minimal risk of TBI-related mortality. Despite the increasing popularity of EEG-based deep learning research to create predictive models with breakthrough performance, particularly in epilepsy prediction, its use in clinical decision making for the diagnosis and prognosis of TBI has not been as widely exploited. Therefore, utilising 60s segments of unprocessed resting-state EEG data as input, we suggest a long short-term memory (LSTM) network that can distinguish between improved and unimproved outcomes in moderate TBI patients. Complex feature extraction and selection are avoided in this architecture. The experimental results show that, with a classification accuracy of 87.50 ± 0.05%, the proposed prognostic model outperforms three related works. The results suggest that the proposed methodology is an efficient and reliable strategy to assist clinicians in creating an automated tool for predicting treatment outcomes from EEG signals.

Control strategies used in lower limb exoskeletons for gait rehabilitation after brain injury: a systematic review and analysis of clinical effectiveness

BACKGROUND

In the past decade, there has been substantial progress in the development of robotic controllers that specify how lower-limb exoskeletons should interact with brain-injured patients. However, it is still an open question which exoskeleton control strategies can more effectively stimulate motor function recovery. In this review, we aim to complement previous literature surveys on the topic of exoskeleton control for gait rehabilitation by: (1) providing an updated structured framework of current control strategies, (2) analyzing the methodology of clinical validations used in the robotic interventions, and (3) reporting the potential relation between control strategies and clinical outcomes.

METHODS

Four databases were searched using database-specific search terms from January 2000 to September 2020. We identified 1648 articles, of which 159 were included and evaluated in full-text. We included studies that clinically evaluated the effectiveness of the exoskeleton on impaired participants, and which clearly explained or referenced the implemented control strategy.

RESULTS

(1) We found that assistive control (100% of exoskeletons) that followed rule-based algorithms (72%) based on ground reaction force thresholds (63%) in conjunction with trajectory-tracking control (97%) were the most implemented control strategies. Only 14% of the exoskeletons implemented adaptive control strategies. (2) Regarding the clinical validations used in the robotic interventions, we found high variability on the experimental protocols and outcome metrics selected. (3) With high grade of evidence and a moderate number of participants (N = 19), assistive control strategies that implemented a combination of trajectory-tracking and compliant control showed the highest clinical effectiveness for acute stroke. However, they also required the longest training time. With high grade of evidence and low number of participants (N = 8), assistive control strategies that followed a threshold-based algorithm with EMG as gait detection metric and control signal provided the highest improvements with the lowest training intensities for subacute stroke. Finally, with high grade of evidence and a moderate number of participants (N = 19), assistive control strategies that implemented adaptive oscillator algorithms together with trajectory-tracking control resulted in the highest improvements with reduced training intensities for individuals with chronic stroke.

CONCLUSIONS

Despite the efforts to develop novel and more effective controllers for exoskeleton-based gait neurorehabilitation, the current level of evidence on the effectiveness of the different control strategies on clinical outcomes is still low. There is a clear lack of standardization in the experimental protocols leading to high levels of heterogeneity. Standardized comparisons among control strategies analyzing the relation between control parameters and biomechanical metrics will fill this gap to better guide future technical developments. It is still an open question whether controllers that provide an on-line adaptation of the control parameters based on key biomechanical descriptors associated to the patients' specific pathology outperform current control strategies.

Input Shape Effect on Classification Performance of Raw EEG Motor Imagery Signals with Convolutional Neural Networks for Use in Brain-Computer Interfaces

EEG signals are interpreted, analyzed and classified by many researchers for use in brain-computer interfaces. Although there are many different EEG signal acquisition methods, one of the most interesting is motor imagery signals. Many different signal processing methods, machine learning and deep learning models have been developed for the classification of motor imagery signals. Among these, Convolutional Neural Network models generally achieve better results than other models. Because the size and shape of the data is important for training Convolutional Neural Network models and discovering the right relationships, researchers have designed and experimented with many different input shape structures. However, no study has been found in the literature evaluating the effect of different input shapes on model performance and accuracy. In this study, the effects of different input shapes on model performance and accuracy in the classification of EEG motor imagery signals were investigated, which had not been specifically studied before. In addition, signal preprocessing methods, which take a long time before classification, were not used; rather, two CNN models were developed for training and classification using raw data. Two different datasets, BCI Competition IV 2A and 2B, were used in classification processes. For different input shapes, 53.03-89.29% classification accuracy and 2-23 s epoch time were obtained for 2A dataset, 64.84-84.94% classification accuracy and 4-10 s epoch time were obtained for 2B dataset. This study showed that the input shape has a significant effect on the classification performance, and when the correct input shape is selected and the correct CNN architecture is developed, feature extraction and classification can be done well by the CNN architecture without any signal preprocessing.

Brain-Computer interface control of stepping from invasive electrocorticography upper-limb motor imagery in a patient with quadriplegia

Introduction: Most spinal cord injuries (SCI) result in lower extremities paralysis, thus diminishing ambulation. Using brain-computer interfaces (BCI), patients may regain leg control using neural signals that actuate assistive devices. Here, we present a case of a subject with cervical SCI with an implanted electrocorticography (ECoG) device and determined whether the system is capable of motor-imagery-initiated walking in an assistive ambulator. Methods: A 24-year-old male subject with cervical SCI (C5 ASIA A) was implanted before the study with an ECoG sensing device over the sensorimotor hand region of the brain. The subject used motor-imagery (MI) to train decoders to classify sensorimotor rhythms. Fifteen sessions of closed-loop trials followed in which the subject ambulated for one hour on a robotic-assisted weight-supported treadmill one to three times per week. We evaluated the stability of the best-performing decoder over time to initiate walking on the treadmill by decoding upper-limb (UL) MI. Results: An online bagged trees classifier performed best with an accuracy of 84.15% averaged across 9 weeks. Decoder accuracy remained stable following throughout closed-loop data collection. Discussion: These results demonstrate that decoding UL MI is a feasible control signal for use in lower-limb motor control. Invasive BCI systems designed for upper-extremity motor control can be extended for controlling systems beyond upper extremity control alone. Importantly, the decoders used were able to use the invasive signal over several weeks to accurately classify MI from the invasive signal. More work is needed to determine the long-term consequence between UL MI and the resulting lower-limb control.

Recognition of single upper limb motor imagery tasks from EEG using multi-branch fusion convolutional neural network

Motor imagery-based brain-computer interfaces (MI-BCI) have important application values in the field of neurorehabilitation and robot control. At present, MI-BCI mostly use bilateral upper limb motor tasks, but there are relatively few studies on single upper limb MI tasks. In this work, we conducted studies on the recognition of motor imagery EEG signals of the right upper limb and proposed a multi-branch fusion convolutional neural network (MF-CNN) for learning the features of the raw EEG signals as well as the two-dimensional time-frequency maps at the same time. The dataset used in this study contained three types of motor imagery tasks: extending the arm, rotating the wrist, and grasping the object, 25 subjects were included. In the binary classification experiment between the grasping object and the arm-extending tasks, MF-CNN achieved an average classification accuracy of 78.52% and kappa value of 0.57. When all three tasks were used for classification, the accuracy and kappa value were 57.06% and 0.36, respectively. The comparison results showed that the classification performance of MF-CNN is higher than that of single CNN branch algorithms in both binary-class and three-class classification. In conclusion, MF-CNN makes full use of the time-domain and frequency-domain features of EEG, can improve the decoding accuracy of single limb motor imagery tasks, and it contributes to the application of MI-BCI in motor function rehabilitation training after stroke.

Multibranch convolutional neural network with contrastive representation learning for decoding same limb motor imagery tasks

Introduction

Emerging deep learning approaches to decode motor imagery (MI) tasks have significantly boosted the performance of brain-computer interfaces. Although recent studies have produced satisfactory results in decoding MI tasks of different body parts, the classification of such tasks within the same limb remains challenging due to the activation of overlapping brain regions. A single deep learning model may be insufficient to effectively learn discriminative features among tasks.

Methods

The present study proposes a framework to enhance the decoding of multiple hand-MI tasks from the same limb using a multi-branch convolutional neural network. The CNN framework utilizes feature extractors from established deep learning models, as well as contrastive representation learning, to derive meaningful feature representations for classification.

Results

The experimental results suggest that the proposed method outperforms several state-of-the-art methods by obtaining a classification accuracy of 62.98% with six MI classes and 76.15 % with four MI classes on the Tohoku University MI-BCI and BCI Competition IV datasets IIa, respectively.

Discussion

Despite requiring heavy data augmentation and multiple optimization steps, resulting in a relatively long training time, this scheme is still suitable for online use. However, the trade-of between the number of base learners, training time, prediction time, and system performance should be carefully considered.

Brain-Computer Interface-Controlled Exoskeletons in Clinical Neurorehabilitation: Ready or Not?

The development of brain-computer interface-controlled exoskeletons promises new treatment strategies for neurorehabilitation after stroke or spinal cord injury. By converting brain/neural activity into control signals of wearable actuators, brain/neural exoskeletons (B/NEs) enable the execution of movements despite impaired motor function. Beyond the use as assistive devices, it was shown that-upon repeated use over several weeks-B/NEs can trigger motor recovery, even in chronic paralysis. Recent development of lightweight robotic actuators, comfortable and portable real-world brain recordings, as well as reliable brain/neural control strategies have paved the way for B/NEs to enter clinical care. Although B/NEs are now technically ready for broader clinical use, their promotion will critically depend on early adopters, for example, research-oriented physiotherapists or clinicians who are open for innovation. Data collected by early adopters will further elucidate the underlying mechanisms of B/NE-triggered motor recovery and play a key role in increasing efficacy of personalized treatment strategies. Moreover, early adopters will provide indispensable feedback to the manufacturers necessary to further improve robustness, applicability, and adoption of B/NEs into existing therapy plans.

BCI system with lower-limb robot improves rehabilitation in spinal cord injury patients through short-term training: a pilot study

In the recent years, the increasing applications of brain-computer interface (BCI) in rehabilitation programs have enhanced the chances of functional recovery for patients with neurological disorders. We presented and validated a BCI system with a lower-limb robot for short-term training of patients with spinal cord injury (SCI). The cores of this system included: (1) electroencephalogram (EEG) features related to motor intention reported through experiments and used to drive the robot; (2) a decision tree to determine the training mode provided for patients with different degrees of injuries. Seven SCI patients (one American Spinal Injury Association Impairment Scale (AIS) A, three AIS B, and three AIS C) participated in the short-term training with this system. All patients could learn to use the system rapidly and maintained a high intensity during the training program. The strength of the lower limb key muscles of the patients was improved. Four AIS A/B patients were elevated to AIS C. The cumulative results indicate that clinical application of the BCI system with lower-limb robot is feasible and safe, and has potentially positive effects on SCI patients.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09801-6.

Emerging Wearable Biosensor Technologies for Stress Monitoring and Their Real-World Applications

Wearable devices are being developed faster and applied more widely. Wearables have been used to monitor movement-related physiological indices, including heartbeat, movement, and other exercise metrics, for health purposes. People are also paying more attention to mental health issues, such as stress management. Wearable devices can be used to monitor emotional status and provide preliminary diagnoses and guided training functions. The nervous system responds to stress, which directly affects eye movements and sweat secretion. Therefore, the changes in brain potential, eye potential, and cortisol content in sweat could be used to interpret emotional changes, fatigue levels, and physiological and psychological stress. To better assess users, stress-sensing devices can be integrated with applications to improve cognitive function, attention, sports performance, learning ability, and stress release. These application-related wearables can be used in medical diagnosis and treatment, such as for attention-deficit hyperactivity disorder (ADHD), traumatic stress syndrome, and insomnia, thus facilitating precision medicine. However, many factors contribute to data errors and incorrect assessments, including the various wearable devices, sensor types, data reception methods, data processing accuracy and algorithms, application reliability and validity, and actual user actions. Therefore, in the future, medical platforms for wearable devices and applications should be developed, and product implementations should be evaluated clinically to confirm product accuracy and perform reliable research.

A Parallel Feature Fusion Network Combining GRU and CNN for Motor Imagery EEG Decoding

In recent years, deep-learning-based motor imagery (MI) electroencephalography (EEG) decoding methods have shown great potential in the field of the brain–computer interface (BCI). The existing literature is relatively mature in decoding methods for two classes of MI tasks. However, with the increase in MI task classes, decoding studies for four classes of MI tasks need to be further explored. In addition, it is difficult to obtain large-scale EEG datasets. When the training data are limited, deep-learning-based decoding models are prone to problems such as overfitting and poor robustness. In this study, we design a data augmentation method for MI-EEG. The original EEG is slid along the time axis and reconstructed to expand the size of the dataset. Second, we combine the gated recurrent unit (GRU) and convolutional neural network (CNN) to construct a parallel-structured feature fusion network to decode four classes of MI tasks. The parallel structure can avoid temporal, frequency and spatial features interfering with each other. Experimenting on the well-known four-class MI dataset BCI Competition IV 2a shows a global average classification accuracy of 80.7% and a kappa value of 0.74. The proposed method improves the robustness of deep learning to decode small-scale EEG datasets and alleviates the overfitting phenomenon caused by insufficient data. The method can be applied to BCI systems with a small amount of daily recorded data.

Decoding EEG rhythms offline and online during motor imagery for standing and sitting based on a brain-computer interface

Motor imagery (MI)-based brain-computer interface (BCI) systems have shown promising advances for lower limb motor rehabilitation. The purpose of this study was to develop an MI-based BCI for the actions of standing and sitting. Thirty-two healthy subjects participated in the study using 17 active EEG electrodes. We used a combination of the filter bank common spatial pattern (FBCSP) method and the regularized linear discriminant analysis (RLDA) technique for decoding EEG rhythms offline and online during motor imagery for standing and sitting. The offline analysis indicated the classification of motor imagery and idle state provided a mean accuracy of 88.51 ± 1.43% and 85.29 ± 1.83% for the sit-to-stand and stand-to-sit transitions, respectively. The mean accuracies of the sit-to-stand and stand-to-sit online experiments were 94.69 ± 1.29% and 96.56 ± 0.83%, respectively. From these results, we believe that the MI-based BCI may be useful to future brain-controlled standing systems.

A Transformer-Based Approach Combining Deep Learning Network and Spatial-Temporal Information for Raw EEG Classification

The attention mechanism of the Transformer has the advantage of extracting feature correlation in the long-sequence data and visualizing the model. As time-series data, the spatial and temporal dependencies of the EEG signals between the time points and the different channels contain important information for accurate classification. So far, Transformer-based approaches have not been widely explored in motor-imagery EEG classification and visualization, especially lacking general models based on cross-individual validation. Taking advantage of the Transformer model and the spatial-temporal characteristics of the EEG signals, we designed Transformer-based models for classifications of motor imagery EEG based on the PhysioNet dataset. With 3s EEG data, our models obtained the best classification accuracy of 83.31%, 74.44%, and 64.22% on two-, three-, and four-class motor-imagery tasks in cross-individual validation, which outperformed other state-of-the-art models by 0.88%, 2.11%, and 1.06%. The inclusion of the positional embedding modules in the Transformer could improve the EEG classification performance. Furthermore, the visualization results of attention weights provided insights into the working mechanism of the Transformer-based networks during motor imagery tasks. The topography of the attention weights revealed a pattern of event-related desynchronization (ERD) which was consistent with the results from the spectral analysis of Mu and beta rhythm over the sensorimotor areas. Together, our deep learning methods not only provide novel and powerful tools for classifying and understanding EEG data but also have broad applications for brain-computer interface (BCI) systems.

Distinct Kinematic and Neuromuscular Activation Strategies During Quiet Stance and in Response to Postural Perturbations in Healthy Individuals Fitted With and Without a Lower-Limb Exoskeleton

Many individuals with disabling conditions have difficulty with gait and balance control that may result in a fall. Exoskeletons are becoming an increasingly popular technology to aid in walking. Despite being a significant aid in increasing mobility, little attention has been paid to exoskeleton features to mitigate falls. To develop improved exoskeleton stability, quantitative information regarding how a user reacts to postural challenges while wearing the exoskeleton is needed. Assessing the unique responses of individuals to postural perturbations while wearing an exoskeleton provides critical information necessary to effectively accommodate a variety of individual response patterns. This report provides kinematic and neuromuscular data obtained from seven healthy, college-aged individuals during posterior support surface translations with and without wearing a lower limb exoskeleton. A 2-min, static baseline standing trial was also obtained. Outcome measures included a variety of 0 dimensional (OD) measures such as center of pressure (COP) RMS, peak amplitude, velocities, pathlength, and electromyographic (EMG) RMS, and peak amplitudes. These measures were obtained during epochs associated with the response to the perturbations: baseline, response, and recovery. T-tests were used to explore potential statistical differences between the exoskeleton and no exoskeleton conditions. Time series waveforms (1D) of the COP and EMG data were also analyzed. Statistical parametric mapping (SPM) was used to evaluate the 1D COP and EMG waveforms obtained during the epochs with and without wearing the exoskeleton. The results indicated that during quiet stance, COP velocity was increased while wearing the exoskeleton, but the magnitude of sway was unchanged. The OD COP measures revealed that wearing the exoskeleton significantly reduced the sway magnitude and velocity in response to the perturbations. There were no systematic effects of wearing the exoskeleton on EMG. SPM analysis revealed that there was a range of individual responses; both behaviorally (COP) and among neuromuscular activation patterns (EMG). Using both the OD and 1D measures provided a more comprehensive representation of how wearing the exoskeleton impacts the responses to posterior perturbations. This study supports a growing body of evidence that exoskeletons must be personalized to meet the specific capabilities and needs of each individual end-user.

EEG based stress analysis using rhythm specific spectral feature for video game play

BACKGROUND AND OBJECTIVE

For the emerging significance of mental stress, various research directives have been established over time to understand better the causes of stress and how to deal with it. In recent years, the rise of video gameplay has been unprecedented, further triggered by the lockdown imposed due to the COVID-19 pandemic. Several researchers and organizations have contributed to the practical analysis of the impacts of such extended periods of gameplay, which lacks coordinated studies to underline the outcomes and reflect those in future game designing and public awareness about video gameplay. Investigations have mainly focused on the "gameplay stress" based on physical syndromes. Some studies have analyzed the effects of video gameplay with Electroencephalogram (EEG), Magnetic resonance imaging (MRI), etc., without concentrating on the relaxation procedure after video gameplay.

METHODS

This paper presents an end-to-end stress analysis for video gaming stimuli using EEG. The power spectral density (PSD) of the Alpha and Beta bands is computed to calculate the Beta-to-Alpha ratio (BAR). The Alpha and Beta band power is computed, and the Beta-to-Alpha band power ratio (BAR) has been determined. In this article, BAR is used to denote mental stress. Subjects are chosen based on various factors such as gender, gameplay experience, age, and Body mass index (BMI). EEG is recorded using Scan SynAmps2 Express equipment. There are three types of video gameplay: strategic, puzzle, and combinational. Relaxation is accomplished in this study by using music of various pitches. Two types of regression analysis are done to mathematically model stress and relaxation curve. Brain topography is rendered to indicate the stressed and relaxed region of the brain.

RESULTS

In the relaxed state, the subjects have BAR 0.701, which is considered the baseline value. Non-gamer subjects have an average BAR of 2.403 for 1 h of strategic video gameplay, whereas gamers have 2.218 BAR concurrently. After 12 minutes of listening to low-pitch music, gamers achieved 0.709 BAR, which is nearly the baseline value. In comparison to Quartic regression, the 4PL symmetrical sigmoid function performs regression analysis with fewer parameters and computational power.

CONCLUSION

Non-gamers experience more stress than gamers, whereas strategic games stress the human brain more. During gameplay, the beta band in the frontal region is mostly activated. For relaxation, low pitch music is the most useful medium. Residual stress is evident in the frontal lobe when the subjects have listened to high pitch music. Quartic regression and 4PL symmetrical sigmoid function have been employed to find the model parameters of the relaxation curve. Among them, quartic regression performs better in terms of Akaike information criterion (AIC) and R² measure.

Machine-learned Adaptive Switching in Voluntary Lower-limb Exoskeleton Control: Preliminary Results

Lower-limb exoskeletons utilize fixed control strategies and are not adaptable to user's intention. To this end, the goal of this study was to investigate the potential of using temporal-difference learning and general value functions for predicting the next possible walking mode that will be selected by users wearing exoskeletons in order to reduce the effort and cognitive load while switching between different modes of walking. Experiments were performed with a user wearing the Indego exoskeleton and given the authority to switch between five walking modes that were different in terms of speed and turn direction. The user's switching preferences were learned and predicted from device-centric and room-centric measurements by considering similarities in the movements being performed. A switching list was updated to show the most probable future next modes to be selected by the user. In contrast to other approaches that either can only predict a single time-step or require intensive offline training, this work used a computationally inexpensive method for learning and has the potential of providing temporally extended sets of predictions in real-time. Comparing the number of required manual switches between the machine-learned switching list and the best possible static lists showed an average decrease of 42.44% in the required switches for the machine-learned adaptive strategy. These promising results will facilitate the path for real-time application of this technique.

Embodiment Comfort Levels During Motor Imagery Training Combined With Immersive Virtual Reality in a Spinal Cord Injury Patient

Brain-machine interfaces combining visual, auditory, and tactile feedback have been previously used to generate embodiment experiences during spinal cord injury (SCI) rehabilitation. It is not known if adding temperature to these modalities can result in discomfort with embodiment experiences. Here, comfort levels with the embodiment experiences were investigated in an intervention that required a chronic pain SCI patient to generate lower limb motor imagery commands in an immersive environment combining visual (virtual reality -VR), auditory, tactile, and thermal feedback. Assessments were made pre-/ post-, throughout the intervention (Weeks 0-5), and at 7 weeks follow up. Overall, high levels of embodiment in the adapted three-domain scale of embodiment were found throughout the sessions. No significant adverse effects of VR were reported. Although sessions induced only a modest reduction in pain levels, an overall reduction occurred in all pain scales (Faces, Intensity, and Verbal) at follow up. A high degree of comfort in the comfort scale for the thermal-tactile sleeve, in both the thermal and tactile feedback components of the sleeve was reported. This study supports the feasibility of combining multimodal stimulation involving visual (VR), auditory, tactile, and thermal feedback to generate embodiment experiences in neurorehabilitation programs.

Decoding Neural Activity Preceding Balance Loss During Standing with a Lower-limb Exoskeleton using an Interpretable Deep Learning Model

Falls are a leading cause of death in adults 65 and older. Recent efforts to restore lower-limb function in these populations have seen an increase in the use of wearable robotic systems; however, fall prevention measures in these systems require early detection of balance loss to be effective. Prior studies have investigated whether kinematic variables contain information about an impending fall, but few have examined the potential of using electroencephalography (EEG) as a fall-predicting signal and how the brain responds to avoid a fall. To address this, we decoded neural activity in a balance perturbation task while wearing an exoskeleton. We acquired EEG, electromyography (EMG), and center of pressure (COP) data from 7 healthy participants during mechanical perturbations while standing. The timing of the perturbations was randomized in all trials. We found perturbation evoked potentials (PEP) components as early as 75-134 ms after the onset of the external perturbation, which preceded both the peak in EMG (∼ 180 ms) and the COP (∼ 350 ms). A convolutional neural network trained to predict balance perturbations from single-trial EEG had a mean F-score of 75.0 ± 4.3 %. Clustering GradCAM-based model explanations demonstrated that the model utilized components in the PEP and was not driven by artifacts. Additionally, dynamic functional connectivity results agreed with model explanations; the nodal connectivity measured using phase difference derivative was higher in the occipital-parietal region in the early stage of perturbations, before shifting to the parietal, motor, and back to the frontal-parietal channels. Continuous-time decoding of COP trajectories from EEG, using a gated recurrent unit model, achieved a mean Pearson's correlation coefficient of 0.7 ± 0.06. Overall, our findings suggest that EEG signals contain short-latency neural information related to an impending fall, which may be useful for developing brain-machine interface systems for fall prevention in robotic exoskeletons.

Auxiliary controller design and performance comparative analysis in closed-loop brain-machine interface system

Brain-machine interface (BMI) can realize information interaction between the brain and external devices, and yet the control accuracy is limited by the change of electroencephalogram signals. The introduction of auxiliary controller can overcome the above problems, but the performance of different auxiliary controllers is quite different. Hence, in this paper, we comprehensively compare and analyze the performance of different auxiliary controllers to provide a theoretical basis for designing BMI system. The main work includes: (1) designing four kinds of auxiliary controllers based on simultaneous perturbation stochastic approximation-function approximator (SPSA-FA), iterative feedback tuning-PID (IFT-PID), model predictive control (MPC) and model-free control (MFC); (2) based on the model of improved single-joint information transmission, constructing the closed-loop BMI systems with the decoder-based Wiener filter; and (3) comparing their performance in the constructed closed-loop BMI systems for dynamic motion restoration. The results show that the order of tracking accuracy is MPC, IFT-PID, SPSA-FA, MFC, and the order of time consumed is opposite. A good control effectiveness is achieved at the expense of time, so a suitable auxiliary controller should be selected according to the actual requirements.

BCI-Based Control for Ankle Exoskeleton T-FLEX: Comparison of Visual and Haptic Stimuli with Stroke Survivors

Brain–computer interface (BCI) remains an emerging tool that seeks to improve the patient interaction with the therapeutic mechanisms and to generate neuroplasticity progressively through neuromotor abilities. Motor imagery (MI) analysis is the most used paradigm based on the motor cortex’s electrical activity to detect movement intention. It has been shown that motor imagery mental practice with movement-associated stimuli may offer an effective strategy to facilitate motor recovery in brain injury patients. In this sense, this study aims to present the BCI associated with visual and haptic stimuli to facilitate MI generation and control the T-FLEX ankle exoskeleton. To achieve this, five post-stroke patients (55–63 years) were subjected to three different strategies using T-FLEX: stationary therapy (ST) without motor imagination, motor imagination with visual stimulation (MIV), and motor imagination with visual-haptic inducement (MIVH). The quantitative characterization of both BCI stimuli strategies was made through the motor imagery accuracy rate, the electroencephalographic (EEG) analysis during the MI active periods, the statistical analysis, and a subjective patient’s perception. The preliminary results demonstrated the viability of the BCI-controlled ankle exoskeleton system with the beta rebound, in terms of patient’s performance during MI active periods and satisfaction outcomes. Accuracy differences employing haptic stimulus were detected with an average of 68% compared with the 50.7% over only visual stimulus. However, the power spectral density (PSD) did not present changes in prominent activation of the MI band but presented significant variations in terms of laterality. In this way, visual and haptic stimuli improved the subject’s MI accuracy but did not generate differential brain activity over the affected hemisphere. Hence, long-term sessions with a more extensive sample and a more robust algorithm should be carried out to evaluate the impact of the proposed system on neuronal and motor evolution after stroke.

Brain Symmetry Analysis during the Use of a BCI Based on Motor Imagery for the Control of a Lower-Limb Exoskeleton

Brain–Computer Interfaces (BCI) are systems that allow external devices to be controlled by means of brain activity. There are different such technologies, and electroencephalography (EEG) is an example. One of the most common EEG control methods is based on detecting changes in sensorimotor rhythms (SMRs) during motor imagery (MI). The aim of this study was to assess the laterality of cortical function when performing MI of the lower limb. Brain signals from five subjects were analyzed in two conditions, during exoskeleton-assisted gait and while static. Three different EEG electrode configurations were evaluated: covering both hemispheres, covering the non-dominant hemisphere and covering the dominant hemisphere. In addition, the evolution of performance and laterality with practice was assessed. Although sightly superior results were achieved with information from all electrodes, differences between electrode configurations were not statistically significant. Regarding the evolution during the experimental sessions, the performance of the BCI generally evolved positively the higher the experience was.

Decoding Hand Movement Types and Kinematic Information From Electroencephalogram

Brain-computer interfaces (BCIs) have achieved successful control of assistive devices, e.g. neuroprosthesis or robotic arm. Previous research based on hand movements Electroencephalogram (EEG) has shown limited success in precise and natural control. In this study, we explored the possibilities of decoding movement types and kinematic information for three reach-and-execute actions using movement-related cortical potentials (MRCPs). EEG signals were acquired from 12 healthy subjects during the execution of pinch, palmar and precision disk rotation actions that involved two levels of speeds and forces. In the case of discrimination between hand movement types under each of four different kinematics conditions, we obtained the average peak accuracies of 83.44% and 73.83% for the binary and 3-class classification, respectively. In the case of discrimination between different movement kinematics for each of three actions, the average peak accuracies of 82.9% and 58.2% could be achieved for the two and 4-class scenario. In both cases, peak decoding performance was significantly higher than the subject-specific chance level. We found that hand movement types all could be classified when these actions were executed at four different kinematic parameters. Meanwhile, for each of three hand movements, we decoded movement parameters successfully. Furthermore, the feasibility of decoding hand movements during hand retraction process was also demonstrated. These findings are of great importance for controlling neuroprosthesis or other rehabilitation devices in a fine and natural way, which would drastically increase the acceptance of motor impaired users.

Review of control strategies for lower-limb exoskeletons to assist gait

Background

Many lower-limb exoskeletons have been developed to assist gait, exhibiting a large range of control methods. The goal of this paper is to review and classify these control strategies, that determine how these devices interact with the user.

Methods

In addition to covering the recent publications on the control of lower-limb exoskeletons for gait assistance, an effort has been made to review the controllers independently of the hardware and implementation aspects. The common 3-level structure (high, middle, and low levels) is first used to separate the continuous behavior (mid-level) from the implementation of position/torque control (low-level) and the detection of the terrain or user’s intention (high-level). Within these levels, different approaches (functional units) have been identified and combined to describe each considered controller.

Results

291 references have been considered and sorted by the proposed classification. The methods identified in the high-level are manual user input, brain interfaces, or automatic mode detection based on the terrain or user’s movements. In the mid-level, the synchronization is most often based on manual triggers by the user, discrete events (followed by state machines or time-based progression), or continuous estimations using state variables. The desired action is determined based on position/torque profiles, model-based calculations, or other custom functions of the sensory signals. In the low-level, position or torque controllers are used to carry out the desired actions. In addition to a more detailed description of these methods, the variants of implementation within each one are also compared and discussed in the paper.

Conclusions

By listing and comparing the features of the reviewed controllers, this work can help in understanding the numerous techniques found in the literature. The main identified trends are the use of pre-defined trajectories for full-mobilization and event-triggered (or adaptive-frequency-oscillator-synchronized) torque profiles for partial assistance. More recently, advanced methods to adapt the position/torque profiles online and automatically detect terrains or locomotion modes have become more common, but these are largely still limited to laboratory settings. An analysis of the possible underlying reasons of the identified trends is also carried out and opportunities for further studies are discussed.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-021-00906-3.

Experimental Protocol to Assess Neuromuscular Plasticity Induced by an Exoskeleton Training Session

Exoskeleton gait rehabilitation is an emerging area of research, with potential applications in the elderly and in people with central nervous system lesions, e.g., stroke, traumatic brain/spinal cord injury. However, adaptability of such technologies to the user is still an unmet goal. Despite important technological advances, these robotic systems still lack the fine tuning necessary to adapt to the physiological modification of the user and are not yet capable of a proper human-machine interaction. Interfaces based on physiological signals, e.g., recorded by electroencephalography (EEG) and/or electromyography (EMG), could contribute to solving this technological challenge. This protocol aims to: (1) quantify neuro-muscular plasticity induced by a single training session with a robotic exoskeleton on post-stroke people and on a group of age and sex-matched controls; (2) test the feasibility of predicting lower limb motor trajectory from physiological signals for future use as control signal for the robot. An active exoskeleton that can be set in full mode (i.e., the robot fully replaces and drives the user motion), adaptive mode (i.e., assistance to the user can be tuned according to his/her needs), and free mode (i.e., the robot completely follows the user movements) will be used. Participants will undergo a preparation session, i.e., EMG sensors and EEG cap placement and inertial sensors attachment to measure, respectively, muscular and cortical activity, and motion. They will then be asked to walk in a 15 m corridor: (i) self-paced without the exoskeleton (pre-training session); (ii) wearing the exoskeleton and walking with the three modes of use; (iii) self-paced without the exoskeleton (post-training session). From this dataset, we will: (1) quantitatively estimate short-term neuroplasticity of brain connectivity in chronic stroke survivors after a single session of gait training; (2) compare muscle activation patterns during exoskeleton-gait between stroke survivors and age and sex-matched controls; and (3) perform a feasibility analysis on the use of physiological signals to decode gait intentions.

Cross-correlation based discriminant criterion for channel selection in motor imagery BCI systems

Objective.Many electroencephalogram (EEG)-based brain-computer interface (BCI) systems use a large amount of channels for higher performance, which is time-consuming to set up and inconvenient for practical applications. Finding an optimal subset of channels without compromising the performance is a necessary and challenging task.Approach.In this article, we proposed a cross-correlation based discriminant criterion (XCDC) which assesses the importance of a channel for discriminating the mental states of different motor imagery (MI) tasks. Channels are ranked and selected according to the proposed criterion. The efficacy of XCDC is evaluated on two MI EEG datasets.Main results.On the two datasets, the proposed method reduces the channel number from 71 and 15 to under 18 and 11 respectively without compromising the classification accuracy on unseen data. Under the same constraint of accuracy, the proposed method requires fewer channels than existing channel selection methods based on Pearson's correlation coefficient and common spatial pattern. Visualization of XCDC shows consistent results with neurophysiological principles.Significance.This work proposes a quantitative criterion for assessing and ranking the importance of EEG channels in MI tasks and provides a practical method for selecting the ranked channels in the calibration phase of MI BCI systems, which alleviates the computational complexity and configuration difficulty in the subsequent steps, leading to real-time and more convenient BCI systems.