Motor Imagery Hand Movement Direction Decoding Using Brain Computer Interface to Aid Stroke Recovery and Rehabilitation

Motor imagery of finger movements: Effects on cortical and muscle activities

PURPOSE

The purpose of the current study was to explore the immediate effect of motor imagery (MI) involving finger movement of a given limb on cortical response and muscle activity in healthy subjects.

METHODS

Twenty healthy right-handed adults (7 females and 13 males) with a mean + SD age of 22.05 + 6.08 years participated in the study. The beta-band event-related desynchronization (ERD) at the sensorimotor cortex and muscle activity during finger movement tasks using either the index, middle, or thumb digits on the non-dominant left hand were compared before and after an MI training session. Subjects underwent a pre-MI, MI training, and finally a post-MI session where they either performed or imagined performing a button-pushing action 50 times per session with each of the three digits.

RESULTS

The ERD power in the beta frequency band was lower in pre-MI compared to post-MI and was significantly different between the pre- and post-MI sessions for both the index and middle fingers, but not the thumb. A significant decrease was seen in the mean muscle activity during post-MI compared to pre-MI for all the digits except the thumb.

CONCLUSIONS

The results from the current study suggest that complex MI can result in motor learning and improvement in motor performance, thereby requiring less effort during motion.

Multi-scale self-attention approach for analysing motor imagery signals in brain-computer interfaces

BACKGROUND

Motor imagery-based electroencephalogram (EEG) brain-computer interface (BCI) technology has seen tremendous advancements in the past several years. Deep learning has outperformed more traditional approaches, such next-gen neuro-technologies, in terms of productivity. It is still challenging to develop and train an end-to-end network that can sufficiently extract the possible characteristics from EEG data used in motor imaging. Brain-computer interface research is largely reliant on the fundamental problem of accurately classifying EEG data. There are still many challenges in the field of MI classification even after researchers have proposed a variety of methods, such as deep learning and machine learning techniques.

METHODOLOGY

We provide a model for four-class categorization of motor imagery EEG signals using attention mechanisms: left hand, right hand, foot, and tongue/rest. The model is built on multi-scale spatiotemporal self-attention networks. To determine the most effective channels, self-attention networks are implemented spatially to assign greater weight to channels associated with motion and lesser weight to channels unrelated to motion. To eliminate noise in the temporal domain, parallel multi-scale Temporal Convolutional Network (TCN) layers are utilized to extract temporal domain features at various scales.

RESULT

On the IV-2b dataset from the BCI Competition, the suggested model achieved an accuracy of 85.09 %; on the IV-2a and IV-2b datasets from the HGD datasets, it was 96.26 %.

COMPARISON WITH EXISTING METHODS

In single-subject classification, this approach demonstrates superior accuracy when compared to existing methods.

CONCLUSION

The findings suggest that this approach exhibits commendable performance, resilience, and capacity for transfer learning.

Characterization and classification of kinesthetic motor imagery levels

Objective.Kinesthetic Motor Imagery (KMI) represents a robust brain paradigm intended for electroencephalography (EEG)-based commands in brain-computer interfaces (BCIs). However, ensuring high accuracy in multi-command execution remains challenging, with data from C3 and C4 electrodes reaching up to 92% accuracy. This paper aims to characterize and classify EEG-based KMI of multilevel muscle contraction without relying on primary motor cortex signals.Approach.A new method based on Hurst exponents is introduced to characterize EEG signals of multilevel KMI of muscle contraction from electrodes placed on the premotor, dorsolateral prefrontal, and inferior parietal cortices. EEG signals were recorded during a hand-grip task at four levels of muscle contraction (0%, 10%, 40%, and 70% of the maximal isometric voluntary contraction). The task was executed under two conditions: first, physically, to train subjects in achieving muscle contraction at each level, followed by mental imagery under the KMI paradigm for each contraction level. EMG signals were recorded in both conditions to correlate muscle contraction execution, whether correct or null accurately. Independent component analysis (ICA) maps EEG signals from the sensor to the source space for preprocessing. For characterization, three algorithms based on Hurst exponents were used: the original (HO), using partitions (HRS), and applying semivariogram (HV). Finally, seven classifiers were used: Bayes network (BN), naive Bayes (NB), support vector machine (SVM), random forest (RF), random tree (RT), multilayer perceptron (MP), and k-nearest neighbors (kNN).Main results.A combination of the three Hurst characterization algorithms produced the highest average accuracy of 96.42% from kNN, followed by MP (92.85%), SVM (92.85%), NB (91.07%), RF (91.07%), BN (91.07%), and RT (80.35%). of 96.42% for kNN.Significance.Results show the feasibility of KMI multilevel muscle contraction detection and, thus, the viability of non-binary EEG-based BCI applications without using signals from the motor cortex.

EEGGAN-Net: enhancing EEG signal classification through data augmentation

Background

Emerging brain-computer interface (BCI) technology holds promising potential to enhance the quality of life for individuals with disabilities. Nevertheless, the constrained accuracy of electroencephalography (EEG) signal classification poses numerous hurdles in real-world applications.

Methods

In response to this predicament, we introduce a novel EEG signal classification model termed EEGGAN-Net, leveraging a data augmentation framework. By incorporating Conditional Generative Adversarial Network (CGAN) data augmentation, a cropped training strategy and a Squeeze-and-Excitation (SE) attention mechanism, EEGGAN-Net adeptly assimilates crucial features from the data, consequently enhancing classification efficacy across diverse BCI tasks.

Results

The EEGGAN-Net model exhibits notable performance metrics on the BCI Competition IV-2a and IV-2b datasets. Specifically, it achieves a classification accuracy of 81.3% with a kappa value of 0.751 on the IV-2a dataset, and a classification accuracy of 90.3% with a kappa value of 0.79 on the IV-2b dataset. Remarkably, these results surpass those of four other CNN-based decoding models.

Conclusions

In conclusion, the amalgamation of data augmentation and attention mechanisms proves instrumental in acquiring generalized features from EEG signals, ultimately elevating the overall proficiency of EEG signal classification.

Three-stage transfer learning for motor imagery EEG recognition

Motor imagery (MI) paradigms have been widely used in neural rehabilitation and drowsiness state assessment. The progress in brain-computer interface (BCI) technology has emphasized the importance of accurately and efficiently detecting motor imagery intentions from electroencephalogram (EEG). Despite the recent breakthroughs made in developing EEG-based algorithms for decoding MI, the accuracy and efficiency of these models remain limited by technical challenges posed by cross-subject heterogeneity in EEG data processing and the scarcity of EEG data for training. Inspired by the optimal transport theory, this study aims to develop a novel three-stage transfer learning (TSTL) method, which uses the existing labeled data from a source domain to improve classification performance on an unlabeled target domain. Notably, the proposed method comprises three components, namely, the Riemannian tangent space mapping (RTSM), source domain transformer (SDT), and optimal subspace mapping (OSM). The RTSM maps a symmetric positive definite matrix from the Riemannian space to the tangent space to minimize the marginal probability distribution drift. The SDT transforms the source domain to a target domain by finding the optimal transport mapping matrix to reduce the joint probability distribution differences. The OSM finally maps the transformed source domain and original target domain to the same subspace to further mitigate the distribution discrepancy. The performance of the proposed method was validated on two public BCI datasets, and the average accuracy of the algorithm on two datasets was 72.24% and 69.29%. Our results demonstrated the improved performance of EEG-based MI detection in comparison with state-of-the-art algorithms.

Multi-Task Heterogeneous Ensemble Learning-Based Cross-Subject EEG Classification Under Stroke Patients

Robot-assisted motor training is applied for neurorehabilitation in stroke patients, using motor imagery (MI) as a representative paradigm of brain-computer interfaces to offer real-life assistance to individuals facing movement challenges. However, the effectiveness of training with MI may vary depending on the location of the stroke lesion, which should be considered. This paper introduces a multi-task electroencephalogram-based heterogeneous ensemble learning (MEEG-HEL) specifically designed for cross-subject training. In the proposed framework, common spatial patterns were used for feature extraction, and the features according to stroke lesions are shared and selected through sequential forward floating selection. The heterogeneous ensembles were used as classifiers. Nine patients with chronic ischemic stroke participated, engaging in MI and motor execution (ME) paradigms involving finger tapping. The classification criteria for the multi-task were established in two ways, taking into account the characteristics of stroke patients. In the cross-subject session, the first involved a direction recognition task for two-handed classification, achieving a performance of 0.7419 (±0.0811) in MI and 0.7061 (±0.1270) in ME. The second task focused on motor assessment for lesion location, resulting in a performance of 0.7457 (±0.1317) in MI and 0.6791 (±0.1253) in ME. Comparing the specific-subject session, except for ME on the motor assessment task, performance on both tasks was significantly higher than the cross-subject session. Furthermore, classification performance was similar to or statistically higher in cross-subject sessions compared to baseline models. The proposed MEEG-HEL holds promise in improving the practicality of neurorehabilitation in clinical settings and facilitating the detection of lesions.

Brain-computer interface in critical care and rehabilitation

This comprehensive review explores the broad landscape of brain-computer interface (BCI) technology and its potential use in intensive care units (ICUs), particularly for patients with motor impairments such as quadriplegia or severe brain injury. By employing brain signals from various sensing techniques, BCIs offer enhanced communication and motor rehabilitation strategies for patients. This review underscores the concept and efficacy of noninvasive, electroencephalogram-based BCIs in facilitating both communicative interactions and motor function recovery. Additionally, it highlights the current research gap in intuitive "stop" mechanisms within motor rehabilitation protocols, emphasizing the need for advancements that prioritize patient safety and individualized responsiveness. Furthermore, it advocates for more focused research that considers the unique requirements of ICU environments to address the challenges arising from patient variability, fatigue, and limited applicability of current BCI systems outside of experimental settings.

Spatio-Temporal Explanation of 3D-EEGNet for Motor Imagery EEG Classification Using Permutation and Saliency

Recently, convolutional neural network (CNN)-based classification models have shown good performance for motor imagery (MI) brain-computer interfaces (BCI) using electroencephalogram (EEG) in end-to-end learning. Although a few explainable artificial intelligence (XAI) techniques have been developed, it is still challenging to interpret the CNN models for EEG-based BCI classification effectively. In this research, we propose 3D-EEGNet as a 3D CNN model to improve both the explainability and performance of MI EEG classification. The proposed approach exhibited better performances on two MI EEG datasets than the existing EEGNet, which uses a 2D input shape. The MI classification accuracies are improved around 1.8% and 6.1% point in average on the datasets, respectively. The permutation-based XAI method is first applied for the reliable explanation of the 3D-EEGNet. Next, to find a faster XAI method for spatio-temporal explanation, we design a novel technique based on the normalized discounted cumulative gain (NDCG) for selecting the best among a few saliency-based methods due to their higher time complexity than the permutation-based method. Among the saliency-based methods, DeepLIFT was selected because the NDCG scores indicated its results are the most similar to the permutation-based results. Finally, the fast spatio-temporal explanation using DeepLIFT provides deeper understanding for the classification results of the 3D-EEGNet and the important properties in the MI EEG experiments.

Creating an AI-Enhanced Morse Code Translation System Based on Images for People with Severe Disabilities

(1) Background: Patients with severe physical impairments (spinal cord injury, cerebral palsy, amyotrophic lateral sclerosis) often have limited mobility due to physical limitations, and may even be bedridden all day long, losing the ability to take care of themselves. In more severe cases, the ability to speak may even be lost, making even basic communication very difficult. (2) Methods: This research will design a set of image-assistive communication equipment based on artificial intelligence to solve communication problems of daily needs. Using artificial intelligence for facial positioning, and facial-motion-recognition-generated Morse code, and then translating it into readable characters or commands, it allows users to control computer software by themselves and communicate through wireless networks or a Bluetooth protocol to control environment peripherals. (3) Results: In this study, 23 human-typed data sets were subjected to recognition using fuzzy algorithms. The average recognition rates for expert-generated data and data input by individuals with disabilities were 99.83% and 98.6%, respectively. (4) Conclusions: Through this system, users can express their thoughts and needs through their facial movements, thereby improving their quality of life and having an independent living space. Moreover, the system can be used without touching external switches, greatly improving convenience and safety.

A Phase-based EEG Epoch Selection Method for Decoding Bi-directional Hand Movement Imagination in Stroke Patients

Electroencephalogram (EEG) based non-invasive Brain Computer Interface (BCI) system is gaining significant attention as a promising solution for stroke rehabilitation. Accurate selection of informative EEG time segment, that accommodates the specific neural activity patterns associated with the underlying mental task can help to improve the efficacy of the BCI system. In this work, we propose a phase-based EEG epoch selection algorithm to extract the discriminative EEG time segment corresponding to bi-directional hand motor imagery. The imagined center-out hand movement in two directions is decoded using the selected epoch of the EEG, recorded from 16 stroke patients with hemiparesis and specifically hand weakness. Phase Lock Value (PLV) EEG features extracted from the selected EEG epoch is used as discriminative feature for binary classification of imagined hand movement direction using Linear Discriminant Analysis. The use of selected EEG epoch yielded an improvement of 11.5% and 11.7% in the average direction classification accuracy of calibration and feedback session data respectively, compared to the baseline method employing the whole EEG signal. In addition to improvement in decoding accuracy, the epoch selection also yielded an average Information Transfer Rate (ITR) of 39.8±24.6 bits per minute, which is 86% improvement compared to the baseline method.Clinical Relevance- The proposed Motor Imagery (MI)-BCI system may be of clinical relevance as an active rehabilitation tool for stroke-affected patients, to enhance neural plasticity and recovery of centre-out activities of affected hand and forms a strong platform for MI-BCI coupled with exoskeletons or prosthesis rehabilitation.

The effect of combining action observation in virtual reality with kinesthetic motor imagery on cortical activity

Introduction

In the past, various techniques have been used to improve motor imagery (MI), such as immersive virtual-reality (VR) and kinesthetic rehearsal. While electroencephalography (EEG) has been used to study the differences in brain activity between VR-based action observation and kinesthetic motor imagery (KMI), there has been no investigation into their combined effect. Prior research has demonstrated that VR-based action observation can enhance MI by providing both visual information and embodiment, which is the perception of oneself as part of the observed entity. Additionally, KMI has been found to produce similar brain activity to physically performing a task. Therefore, we hypothesized that utilizing VR to offer an immersive visual scenario for action observation while participants performed kinesthetic motor imagery would significantly improve cortical activity related to MI.

Methods

In this study, 15 participants (9 male, 6 female) performed kinesthetic motor imagery of three hand tasks (drinking, wrist flexion-extension, and grabbing) both with and without VR-based action observation.

Results

Our results indicate that combining VR-based action observation with KMI enhances brain rhythmic patterns and provides better task differentiation compared to KMI without action observation.

Discussion

These findings suggest that using VR-based action observation alongside kinesthetic motor imagery can improve motor imagery performance.

Enhanced motor imagery of digits within the same hand via vibrotactile stimulation

Purpose

The aim of the present study is to evaluate the effect of vibrotactile stimulation prior to repeated complex motor imagery of finger movements using the non-dominant hand on motor imagery (MI) performance.

Methods

Ten healthy right-handed adults (4 females and 6 males) participated in the study. The subjects performed motor imagery tasks with and without a brief vibrotactile sensory stimulation prior to performing motor imagery using either their left-hand index, middle, or thumb digits. Mu- and beta-band event-related desynchronization (ERD) at the sensorimotor cortex and an artificial neural network-based digit classification was evaluated.

Results

The ERD and digit discrimination results from our study showed that ERD was significantly different between the vibration conditions for the index, middle, and thumb. It was also found that digit classification accuracy with-vibration (mean ± SD = 66.31 ± 3.79%) was significantly higher than without-vibration (mean ± SD = 62.68 ± 6.58%).

Conclusion

The results showed that a brief vibration was more effective at improving MI-based brain-computer interface classification of digits within a single limb through increased ERD compared to performing MI without vibrotactile stimulation.

Evaluation of EEG Oscillatory Patterns and Classification of Compound Limb Tactile Imagery

Objective: The purpose of this study was to investigate the cortical activity and digit classification performance during tactile imagery (TI) of a vibratory stimulus at the index, middle, and thumb digits within the left hand in healthy individuals. Furthermore, the cortical activities and classification performance of the compound TI were compared with similar compound motor imagery (MI) with the same digits as TI in the same subjects. Methods: Twelve healthy right-handed adults with no history of upper limb injury, musculoskeletal condition, or neurological disorder participated in the study. The study evaluated the event-related desynchronization (ERD) response and brain-computer interface (BCI) classification performance on discriminating between the digits in the left-hand during the imagery of vibrotactile stimuli to either the index, middle, or thumb finger pads for TI and while performing a motor activity with the same digits for MI. A supervised machine learning technique was applied to discriminate between the digits within the same given limb for both imagery conditions. Results: Both TI and MI exhibited similar patterns of ERD in the alpha and beta bands at the index, middle, and thumb digits within the left hand. While TI had significantly lower ERD for all three digits in both bands, the classification performance of TI-based BCI (77.74 ± 6.98%) was found to be similar to the MI-based BCI (78.36 ± 5.38%). Conclusions: The results of this study suggest that compound tactile imagery can be a viable alternative to MI for BCI classification. The study contributes to the growing body of evidence supporting the use of TI in BCI applications, and future research can build on this work to explore the potential of TI-based BCI for motor rehabilitation and the control of external devices.

Continual Learning of a Transformer-Based Deep Learning Classifier Using an Initial Model from Action Observation EEG Data to Online Motor Imagery Classification

The motor imagery (MI)-based brain computer interface (BCI) is an intuitive interface that enables users to communicate with external environments through their minds. However, current MI-BCI systems ask naïve subjects to perform unfamiliar MI tasks with simple textual instruction or a visual/auditory cue. The unclear instruction for MI execution not only results in large inter-subject variability in the measured EEG patterns but also causes the difficulty of grouping cross-subject data for big-data training. In this study, we designed an BCI training method in a virtual reality (VR) environment. Subjects wore a head-mounted device (HMD) and executed action observation (AO) concurrently with MI (i.e., AO + MI) in VR environments. EEG signals recorded in AO + MI task were used to train an initial model, and the initial model was continually improved by the provision of EEG data in the following BCI training sessions. We recruited five healthy subjects, and each subject was requested to participate in three kinds of tasks, including an AO + MI task, an MI task, and the task of MI with visual feedback (MI-FB) three times. This study adopted a transformer- based spatial-temporal network (TSTN) to decode the user's MI intentions. In contrast to other convolutional neural network (CNN) or recurrent neural network (RNN) approaches, the TSTN extracts spatial and temporal features, and applies attention mechanisms along spatial and temporal dimensions to perceive the global dependencies. The mean detection accuracies of TSTN were 0.63, 0.68, 0.75, and 0.77 in the MI, first MI-FB, second MI-FB, and third MI-FB sessions, respectively. This study demonstrated the AO + MI gave an easier way for subjects to conform their imagery actions, and the BCI performance was improved with the continual learning of the MI-FB training process.

A Review of Online Classification Performance in Motor Imagery-Based Brain–Computer Interfaces for Stroke Neurorehabilitation

Motor imagery (MI)-based brain–computer interfaces (BCI) have shown increased potential for the rehabilitation of stroke patients; nonetheless, their implementation in clinical practice has been restricted due to their low accuracy performance. To date, although a lot of research has been carried out in benchmarking and highlighting the most valuable classification algorithms in BCI configurations, most of them use offline data and are not from real BCI performance during the closed-loop (or online) sessions. Since rehabilitation training relies on the availability of an accurate feedback system, we surveyed articles of current and past EEG-based BCI frameworks who report the online classification of the movement of two upper limbs in both healthy volunteers and stroke patients. We found that the recently developed deep-learning methods do not outperform the traditional machine-learning algorithms. In addition, patients and healthy subjects exhibit similar classification accuracy in current BCI configurations. Lastly, in terms of neurofeedback modality, functional electrical stimulation (FES) yielded the best performance compared to non-FES systems.

Time-frequency analysis of brain activity in response to directional and non-directional visual stimuli: an event related spectral perturbations (ERSP) study

Objective.A large part of the cerebral cortex is dedicated to the processing of visual stimuli and there is still much to understand about such processing modalities and hierarchies. The main aim of the present study is to investigate the differences between directional visual stimuli (DS) and non-directional visual stimuli (n-DS) processing by time-frequency analysis of brain electroencephalographic activity during a visuo-motor task. Electroencephalography (EEG) data were divided into four regions of interest (ROIs) (frontal, central, parietal, occipital).Approach.The analysis of the visual stimuli processing was based on the combination of electroencephalographic recordings and time-frequency analysis. Event related spectral perturbations (ERSPs) were computed with spectrum analysis that allow to obtain the average time course of relative changes induced by the stimulus presentation in spontaneous EEG amplitude spectrum.Main results.Visual stimuli processing enhanced the same pattern of spectral modulation in all investigated ROIs with differences in amplitudes and timing. Additionally, statistically significant differences in occipital ROI between the DS and n-DS visual stimuli processing in theta, alpha and beta bands were found.Significance.These evidences suggest that ERSPs could be a useful tool to investigate the encoding of visual information in different brain regions. Because of their simplicity and their capability in the representation of brain activity, the ERSPs might be used as biomarkers of functional recovery for example in the rehabilitation of visual dysfunction and motor impairment following a stroke, as well as diagnostic tool of anomalies in brain functions in neurological diseases tailored to personalized treatments in clinical environment.

An Effective Fusing Approach by Combining Connectivity Network Pattern and Temporal-Spatial Analysis for EEG-Based BCI Rehabilitation

Motor-modality-based brain computer interface (BCI) could promote the neural rehabilitation for stroke patients. Temporal-spatial analysis was commonly used for pattern recognition in this task. This paper introduced a novel connectivity network analysis for EEG-based feature selection. The network features of connectivity pattern not only captured the spatial activities responding to motor task, but also mined the interactive pattern among these cerebral regions. Furthermore, the effective combination between temporal-spatial analysis and network analysis was evaluated for improving the performance of BCI classification (81.7%). And the results demonstrated that it could raise the classification accuracies for most of patients (6 of 7 patients). This proposed method was meaningful for developing the effective BCI training program for stroke rehabilitation.

A Semi-Supervised Progressive Learning Algorithm for Brain-Computer Interface

Brain-computer interface (BCI) usually suffers from the problem of low recognition accuracy and large calibration time, especially when identifying motor imagery tasks for subjects with indistinct features and classifying fine grained motion control tasks by electroencephalogram (EEG)-electromyogram (EMG) fusion analysis. To fill the research gap, this paper presents an end-to-end semi-supervised learning framework for EEG classification and EEG-EMG fusion analysis. Benefiting from the proposed metric learning based label estimation strategy, sampling criterion and progressive learning scheme, the proposed framework efficiently extracts distinctive feature embedding from the unlabeled EEG samples and achieves a 5.40% improvement on BCI Competition IV Dataset IIa with 80% unlabeled samples and an average 3.35% improvement on two public BCI datasets. By employing synchronous EMG features as pseudo labels for the unlabeled EEG samples, the proposed framework further extracts deep level features of the synergistic complementarity between the EEG signals and EMG features based on the deep encoders, which improves the performance of hybrid BCI (with a 5.53% improvement for the Upper Limb Motion Dataset and an average 4.34% improvement on two hybrid datasets). Moreover, the ablation experiments show that the proposed framework can substantially improve the performance of the deep encoders (with an average 5.53% improvement). The proposed framework not only largely improves the performance of deep networks in the BCI system, but also significantly reduces the calibration time for EEG-EMG fusion analysis, which shows great potential for building an efficient and high-performance hybrid BCI for the motor rehabilitation process.

Subject-Dependent Artifact Removal for Enhancing Motor Imagery Classifier Performance under Poor Skills

The Electroencephalography (EEG)-based motor imagery (MI) paradigm is one of the most studied technologies for Brain-Computer Interface (BCI) development. Still, the low Signal-to-Noise Ratio (SNR) poses a challenge when constructing EEG-based BCI systems. Moreover, the non-stationary and nonlinear signal issues, the low-spatial data resolution, and the inter- and intra-subject variability hamper the extraction of discriminant features. Indeed, subjects with poor motor skills have difficulties in practicing MI tasks against low SNR scenarios. Here, we propose a subject-dependent preprocessing approach that includes the well-known Surface Laplacian Filtering and Independent Component Analysis algorithms to remove signal artifacts based on the MI performance. In addition, power- and phase-based functional connectivity measures are studied to extract relevant and interpretable patterns and identify subjects of inefficency. As a result, our proposal, Subject-dependent Artifact Removal (SD-AR), improves the MI classification performance in subjects with poor motor skills. Consequently, electrooculography and volume-conduction EEG artifacts are mitigated within a functional connectivity feature-extraction strategy, which favors the classification performance of a straightforward linear classifier.

EEG-EMG hybrid real-time classification of hand grasp and release movements intention in chronic stroke patients

Rehabilitation of the hand motor function is essential for stroke patients to resume activities of daily living. Recent studies have shown that wearable robot systems, like a multi degree-of-freedom soft glove, have the potential to improve hand motor impairment. The rehabilitation system, which is intuitively controlled according to the user's intention, is expected to induce active participation of the user and further promote brain plasticity. However, due to the patient-specific nature of stroke patients, extracting the intention from stroke patients is still challenging. In this study, we implemented a classifier that combines EEG and EMG to detect chronic stroke patients' four types of intention: rest, grasp, hold, and release. Three chronic stroke patients participated in the experiment and performed rest, grasp, hold, and release actions. The rest vs. grasp binary classifier and release vs. hold binary classifier showed 76.9% and 86.6% classification accuracy in real-time, respectively. In addition, patient-specific accuracy comparisons showed that the hybrid approach was robust to upper limb impairment level compared to other approaches. We believe that these results could pave the way for the development of BCI-based robotic hand rehabilitation therapy.

Decoding Brain Signals to Classify Gait Direction Anticipation

The use of brain-computer interface (BCI) technology has emerged as a promising rehabilitation approach for patients with motor function and motor-related disorders. BCIs provide an augmentative communication platform for controlling advanced assistive robots such as a lower-limb exoskeleton. Brain recordings collected by an electroencephalography (EEG) system have been employed in the BCI platform to command the exoskeleton. To date, the literature on this topic is limited to the prediction of gait intention and gait variations from EEG signals. This study, however, aims to predict the anticipated gait direction using a stream of EEG signals collected from the brain cortex. Three healthy participants (age range: 29-31, 2 female) were recruited. While wearing the EEG device, the participants were instructed to initiate gait movement toward the direction of the arrow triggers (pointing forward, backward, left, or right) being shown on a screen with a blank white background. Collected EEG data was then epoched around the trigger timepoints. These epochs were then converted to the time-frequency domain using event- related synchronization (ERS) and event-related desynchronization (ERD) methods. Finally, the classification pipeline was constructed using logistic regression (LR), support vector machine (SVM), and convolutional neural network (CNN). A ten-fold cross-validation scheme was used to evaluate the classification performance. The results revealed that the CNN classifier outperforms the other two classifiers with an accuracy of 0.75. Clinical Relevance - The outcome of this study has the potential to be ultimately used for interactive navigation of the lower-limb exoskeletons during robotic rehabilitation therapy and enhance neurodegeneration and neuroplasticity in a wide range of individuals with lower-limb motor function disabilities.

Machine Learning for Motor Imagery Wrist Dorsiflexion Prediction in Brain-Computer Interface Assisted Stroke Rehabilitation

Stroke is a life-changing event that can affect the survivors' physical, cognitive and emotional state. Stroke care focuses on helping the survivors to regain their strength; recover as much functionality as possible and return to independent living through rehabilitation therapies. Automated training protocols have been reported to improve the efficiency of the rehabilitation process. These protocols also decrease the dependency of the process on a professional trainer. Brain-Computer Interface (BCI) based systems are examples of such systems where they make use of the motor imagery (MI) based electroencephalogram (EEG) signals to drive the rehabilitation protocols. In this paper, we have proposed the use of well-known machine learning (ML) algorithms, such as, the decision tree (DT), Naive Bayesian (NB), linear discriminant analysis (LDA), support vector machine (SVM), ensemble learning classifier (ELC), and artificial neural network (ANN) for MI wrist dorsiflexion prediction in a BCI assisted stroke rehabilitation study conducted on eleven stroke survivors with either the left or right paresis. The doubling sub-band selection filter bank common spatial pattern (DSBS-FBCSP) has been proposed as feature extractor and it is observed that the ANN based classifier produces the best results.

Multimodal Neural Response and Effect Assessment During a BCI-Based Neurofeedback Training After Stroke

Stroke caused by cerebral infarction or hemorrhage can lead to motor dysfunction. The recovery of motor function is vital for patients with stroke in daily activities. Traditional rehabilitation of stroke generally depends on physical practice under passive affected limbs movement. Motor imagery-based brain computer interface (MI-BCI) combined with functional electrical stimulation (FES) is a potential active neural rehabilitation technology for patients with stroke recently, which complements traditional passive rehabilitation methods. As the predecessor of BCI technology, neurofeedback training (NFT) is a psychological process that feeds back neural activities online to users for self-regulation. In this work, BCI-based NFT were proposed to promote the active repair and reconstruction of the whole nerve conduction pathway and motor function. We designed and implemented a multimodal, training type motor NFT system (BCI-NFT-FES) by integrating the visual, auditory, and tactile multisensory pathway feedback mode and using the joint detection of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). The results indicated that after 4 weeks of training, the clinical scale score, event-related desynchronization (ERD) of EEG patterns, and cerebral oxygen response of patients with stroke were enhanced obviously. This study preliminarily verified the clinical effectiveness of the long-term NFT system and the prospect of motor function rehabilitation.

Effects of Mirror Neurons-Based Rehabilitation Techniques in Hand Injuries: A Systematic Review and Meta-Analysis

Background: Hand trauma requires specific rehabilitation protocol depending on the different structures involved. According to type of surgical intervention, and for monitoring pain and edema, post-operative rehabilitation of a hand that has experienced trauma involves different timings for immobilization. Several protocols have been used to reduce immobilization time, and various techniques and methods are adopted, depending on the structures involved. Objective: To measure the effects of mirror neurons-based rehabilitation techniques in hand injuries throughout a systematic review and meta-analysis. Methods: The protocol was accepted in PROSPERO database. A literature search was conducted in Cinahl, Scopus, Medline, PEDro, OTseeker. Two authors independently identified eligible studies, based on predefined inclusion criteria, and extracted the data. RCT quality was assessed using the JADAD scale. Results: Seventy-nine suitable studies were screened, and only eleven were included for qualitative synthesis, while four studies were selected for quantitative analysis. Four studies were case reports/series, and seven were RCTs. Nine investigate the effect of Mirror Therapy and two the effect of Motor Imagery. Quantitative analyses revealed Mirror Therapy as effective for hand function recovery (mean difference = −14.80 95% Confidence Interval (CI) = −17.22, −12.38) (p < 0.00001) in the short term, as well as in long follow-up groups (mean difference = −13.11 95% Confidence Interval (CI) = −17.53, −8.69) (p < 0.00001). Clinical, but not statistical, efficacy was found for manual dexterity (p = 0.15), while no benefit was reported for range of motion. Conclusions: Mirror neurons-based rehabilitation techniques, combined with conventional occupational and physical therapy, can be a useful approach in hand trauma. Mirror therapy seems to be effective for hand function recovery, but, for motor imagery and action observation, there is not sufficient evidence to recommend its use. Further research on the efficacy of the mirror neurons-based technique in hand injury is recommended.

Corticomuscular integrated representation of voluntary motor effort in robotic control for wrist-hand rehabilitation after stroke

OBJECTIVE

The central-to-peripheral voluntary motor effort (VME) in physical practice of the paretic limb is a dominant force for driving functional neuroplasticity on motor restoration post-stroke. However, current rehabilitation robots isolated the central and peripheral involvements in the control design, resulting in limited rehabilitation effectiveness. The purpose of this study was to design a corticomuscular coherence (CMC) and electromyography (EMG)-driven (CMC-EMG-driven) system with central-and-peripheral integrated representation of VME for wrist-hand rehabilitation after stroke.

APPROACH

The CMC-EMG-driven control was developed in a neuromuscular electrical stimulation (NMES)-robot system, i.e., CMC-EMG-driven NMES-robot system, to instruct and assist the wrist-hand extension and flexion in persons after stroke. A pilot single-group trial of 20 training sessions was conducted with the developed system to assess the feasibility for wrist-hand practice on the chronic strokes (n=16). The rehabilitation effectiveness was evaluated through clinical assessments, CMC, and EMG activation levels.

MAIN RESULTS

The trigger success rate and laterality index (LI) of CMC were significantly increased in wrist-hand extension across training sessions (p<0.05). After the training, significant improvements in the target wrist-hand joints and suppressed compensation from the proximal shoulder-elbow joints were observed through the clinical scores and EMG activation levels (p<0.05). The central-to-peripheral VME distribution across upper extremity (UE) muscles was also significantly improved, as revealed by the CMC values (p<0.05).

SIGNIFICANCE

Precise wrist-hand rehabilitation was achieved by the developed system, presenting suppressed cortical and muscular compensation from the contralesional hemisphere and the proximal UE, and improved distribution of the central-and-peripheral VME on UE muscles.

Motor Imagery Classification Using Inter-Task Transfer Learning via A Channel-Wise Variational Autoencoder-based Convolutional Neural Network

Highly sophisticated control based on a brain-computer interface (BCI) requires decoding kinematic information from brain signals. The forearm is a region of the upper limb that is often used in everyday life, but intuitive movements within the same limb have rarely been investigated in previous BCI studies. In this study, we focused on various forearm movement decoding from electroencephalography (EEG) signals using a small number of samples. Ten healthy participants took part in an experiment and performed motor execution (ME) and motor imagery (MI) of the intuitive movement tasks (Dataset I). We propose a convolutional neural network using a channel-wise variational autoencoder (CVNet) based on inter-task transfer learning. We approached that training the reconstructed ME-EEG signals together will also achieve more sufficient classification performance with only a small amount of MI-EEG signals. The proposed CVNet was validated on our own Dataset I and a public dataset, BNCI Horizon 2020 (Dataset II). The classification accuracies of various movements are confirmed to be 0.83 (±0.04) and 0.69 (±0.04) for Dataset I and II, respectively. The results show that the proposed method exhibits performance increases of approximately 0.09~0.27 and 0.08~0.24 compared with the conventional models for Dataset I and II, respectively. The outcomes suggest that the training model for decoding imagined movements can be performed using data from ME and a small number of data samples from MI. Hence, it is presented the feasibility of BCI learning strategies that can sufficiently learn deep learning with a few amount of calibration dataset and time only, with stable performance.

Decoding Finger Tapping With the Affected Hand in Chronic Stroke Patients During Motor Imagery and Execution

In stroke rehabilitation, motor imagery based on a brain-computer interface is an extremely useful method to control an external device and utilize neurofeedback. Many studies have reported on the classification performance of motor imagery to decode individual fingers in stroke patients compared with healthy controls. However, classification performance for a given limb is still low because the differences between patients owing to brain reorganization after stroke are not considered. We used electroencephalography signals from eleven healthy controls and eleven stroke patients in this study. The subjects performed a finger tapping task during motor execution, and motor imagery was performed with the dominant and affected hands in the healthy controls and stroke patients, respectively. All fingers except for the thumb were classified using the proposed framework based on a voting module. The averaged four-class accuracies during motor execution and motor imagery were 53.16 ± 8.42% and 46.94 ± 5.99% for the healthy controls and 53.17 ± 14.09% and 66.00 ± 14.96% for the stroke patients, respectively. Importantly, the classification accuracies in the stroke patients were statistically higher than those in healthy controls during motor imagery. However, there was no significant difference between the accuracies of motor execution and motor imagery. These findings show the potential for high classification performance for a given limb during motor imagery in stroke patients. These results can also provide insights into controlling an external device on the basis of a brain-computer interface.