1
|
Sengupta P, Lakshminarayanan K. Motor imagery of finger movements: Effects on cortical and muscle activities. Behav Brain Res 2024; 471:115100. [PMID: 38852744 DOI: 10.1016/j.bbr.2024.115100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2024] [Revised: 06/06/2024] [Accepted: 06/06/2024] [Indexed: 06/11/2024]
Abstract
PURPOSE The purpose of the current study was to explore the immediate effect of motor imagery (MI) involving finger movement of a given limb on cortical response and muscle activity in healthy subjects. METHODS Twenty healthy right-handed adults (7 females and 13 males) with a mean + SD age of 22.05 + 6.08 years participated in the study. The beta-band event-related desynchronization (ERD) at the sensorimotor cortex and muscle activity during finger movement tasks using either the index, middle, or thumb digits on the non-dominant left hand were compared before and after an MI training session. Subjects underwent a pre-MI, MI training, and finally a post-MI session where they either performed or imagined performing a button-pushing action 50 times per session with each of the three digits. RESULTS The ERD power in the beta frequency band was lower in pre-MI compared to post-MI and was significantly different between the pre- and post-MI sessions for both the index and middle fingers, but not the thumb. A significant decrease was seen in the mean muscle activity during post-MI compared to pre-MI for all the digits except the thumb. CONCLUSIONS The results from the current study suggest that complex MI can result in motor learning and improvement in motor performance, thereby requiring less effort during motion.
Collapse
Affiliation(s)
- Puja Sengupta
- Neuro-rehabilitation Lab, Department of Sensors and Biomedical Technology, School of Electronics Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India
| | - Kishor Lakshminarayanan
- Neuro-rehabilitation Lab, Department of Sensors and Biomedical Technology, School of Electronics Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India.
| |
Collapse
|
2
|
Bhatt MW, Sharma S. Multi-scale self-attention approach for analysing motor imagery signals in brain-computer interfaces. J Neurosci Methods 2024; 408:110182. [PMID: 38795979 DOI: 10.1016/j.jneumeth.2024.110182] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2024] [Revised: 05/01/2024] [Accepted: 05/22/2024] [Indexed: 05/28/2024]
Abstract
BACKGROUND Motor imagery-based electroencephalogram (EEG) brain-computer interface (BCI) technology has seen tremendous advancements in the past several years. Deep learning has outperformed more traditional approaches, such next-gen neuro-technologies, in terms of productivity. It is still challenging to develop and train an end-to-end network that can sufficiently extract the possible characteristics from EEG data used in motor imaging. Brain-computer interface research is largely reliant on the fundamental problem of accurately classifying EEG data. There are still many challenges in the field of MI classification even after researchers have proposed a variety of methods, such as deep learning and machine learning techniques. METHODOLOGY We provide a model for four-class categorization of motor imagery EEG signals using attention mechanisms: left hand, right hand, foot, and tongue/rest. The model is built on multi-scale spatiotemporal self-attention networks. To determine the most effective channels, self-attention networks are implemented spatially to assign greater weight to channels associated with motion and lesser weight to channels unrelated to motion. To eliminate noise in the temporal domain, parallel multi-scale Temporal Convolutional Network (TCN) layers are utilized to extract temporal domain features at various scales. RESULT On the IV-2b dataset from the BCI Competition, the suggested model achieved an accuracy of 85.09 %; on the IV-2a and IV-2b datasets from the HGD datasets, it was 96.26 %. COMPARISON WITH EXISTING METHODS In single-subject classification, this approach demonstrates superior accuracy when compared to existing methods. CONCLUSION The findings suggest that this approach exhibits commendable performance, resilience, and capacity for transfer learning.
Collapse
Affiliation(s)
- Mohammed Wasim Bhatt
- Department of Computer Science & Engineering, National Institute of Technology, Srinagar, J&K, India.
| | - Sparsh Sharma
- Department of Computer Science & Engineering, National Institute of Technology, Srinagar, J&K, India.
| |
Collapse
|
3
|
Martinez-Peon D, Garcia-Hernandez NV, Benavides-Bravo FG, Parra-Vega V. Characterization and classification of kinesthetic motor imagery levels. J Neural Eng 2024; 21:046024. [PMID: 38963179 DOI: 10.1088/1741-2552/ad5f27] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Accepted: 06/27/2024] [Indexed: 07/05/2024]
Abstract
Objective.Kinesthetic Motor Imagery (KMI) represents a robust brain paradigm intended for electroencephalography (EEG)-based commands in brain-computer interfaces (BCIs). However, ensuring high accuracy in multi-command execution remains challenging, with data from C3 and C4 electrodes reaching up to 92% accuracy. This paper aims to characterize and classify EEG-based KMI of multilevel muscle contraction without relying on primary motor cortex signals.Approach.A new method based on Hurst exponents is introduced to characterize EEG signals of multilevel KMI of muscle contraction from electrodes placed on the premotor, dorsolateral prefrontal, and inferior parietal cortices. EEG signals were recorded during a hand-grip task at four levels of muscle contraction (0%, 10%, 40%, and 70% of the maximal isometric voluntary contraction). The task was executed under two conditions: first, physically, to train subjects in achieving muscle contraction at each level, followed by mental imagery under the KMI paradigm for each contraction level. EMG signals were recorded in both conditions to correlate muscle contraction execution, whether correct or null accurately. Independent component analysis (ICA) maps EEG signals from the sensor to the source space for preprocessing. For characterization, three algorithms based on Hurst exponents were used: the original (HO), using partitions (HRS), and applying semivariogram (HV). Finally, seven classifiers were used: Bayes network (BN), naive Bayes (NB), support vector machine (SVM), random forest (RF), random tree (RT), multilayer perceptron (MP), and k-nearest neighbors (kNN).Main results.A combination of the three Hurst characterization algorithms produced the highest average accuracy of 96.42% from kNN, followed by MP (92.85%), SVM (92.85%), NB (91.07%), RF (91.07%), BN (91.07%), and RT (80.35%). of 96.42% for kNN.Significance.Results show the feasibility of KMI multilevel muscle contraction detection and, thus, the viability of non-binary EEG-based BCI applications without using signals from the motor cortex.
Collapse
Affiliation(s)
- D Martinez-Peon
- Department of Electrical and Electronic Engineering, National Technological Institute of Mexico (TecNM)- IT Nuevo Leon, Guadalupe, Mexico
| | - N V Garcia-Hernandez
- National Council on Science and Technology, Saltillo, Mexico
- Robotics and Advanced Manufacturing, Research Center for Advanced Studies (Cinvestav), Saltillo, Mexico
| | - F G Benavides-Bravo
- Department of Basic Sciences, National Technological Institute of Mexico (TecNM)- IT Nuevo Leon, Guadalupe, Mexico
| | - V Parra-Vega
- Robotics and Advanced Manufacturing, Research Center for Advanced Studies (Cinvestav), Saltillo, Mexico
| |
Collapse
|
4
|
Song J, Zhai Q, Wang C, Liu J. EEGGAN-Net: enhancing EEG signal classification through data augmentation. Front Hum Neurosci 2024; 18:1430086. [PMID: 39010893 PMCID: PMC11247432 DOI: 10.3389/fnhum.2024.1430086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2024] [Accepted: 06/10/2024] [Indexed: 07/17/2024] Open
Abstract
Background Emerging brain-computer interface (BCI) technology holds promising potential to enhance the quality of life for individuals with disabilities. Nevertheless, the constrained accuracy of electroencephalography (EEG) signal classification poses numerous hurdles in real-world applications. Methods In response to this predicament, we introduce a novel EEG signal classification model termed EEGGAN-Net, leveraging a data augmentation framework. By incorporating Conditional Generative Adversarial Network (CGAN) data augmentation, a cropped training strategy and a Squeeze-and-Excitation (SE) attention mechanism, EEGGAN-Net adeptly assimilates crucial features from the data, consequently enhancing classification efficacy across diverse BCI tasks. Results The EEGGAN-Net model exhibits notable performance metrics on the BCI Competition IV-2a and IV-2b datasets. Specifically, it achieves a classification accuracy of 81.3% with a kappa value of 0.751 on the IV-2a dataset, and a classification accuracy of 90.3% with a kappa value of 0.79 on the IV-2b dataset. Remarkably, these results surpass those of four other CNN-based decoding models. Conclusions In conclusion, the amalgamation of data augmentation and attention mechanisms proves instrumental in acquiring generalized features from EEG signals, ultimately elevating the overall proficiency of EEG signal classification.
Collapse
Affiliation(s)
- Jiuxiang Song
- School of Advanced Manufacturing, Nanchang University, Nanchang, Jiangxi, China
| | - Qiang Zhai
- School of Advanced Manufacturing, Nanchang University, Nanchang, Jiangxi, China
- Shaoxing Institute of Advanced Research, Wuhan University of Technology, Shaoxing, Zhejiang, China
| | - Chuang Wang
- Xiangyang Auto Vocational Technical College, Intelligent Manufacturing College, Xiangyang, Hubei, China
| | - Jizhong Liu
- School of Advanced Manufacturing, Nanchang University, Nanchang, Jiangxi, China
| |
Collapse
|
5
|
Li J, She Q, Meng M, Du S, Zhang Y. Three-stage transfer learning for motor imagery EEG recognition. Med Biol Eng Comput 2024; 62:1689-1701. [PMID: 38342784 DOI: 10.1007/s11517-024-03036-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Accepted: 01/17/2024] [Indexed: 02/13/2024]
Abstract
Motor imagery (MI) paradigms have been widely used in neural rehabilitation and drowsiness state assessment. The progress in brain-computer interface (BCI) technology has emphasized the importance of accurately and efficiently detecting motor imagery intentions from electroencephalogram (EEG). Despite the recent breakthroughs made in developing EEG-based algorithms for decoding MI, the accuracy and efficiency of these models remain limited by technical challenges posed by cross-subject heterogeneity in EEG data processing and the scarcity of EEG data for training. Inspired by the optimal transport theory, this study aims to develop a novel three-stage transfer learning (TSTL) method, which uses the existing labeled data from a source domain to improve classification performance on an unlabeled target domain. Notably, the proposed method comprises three components, namely, the Riemannian tangent space mapping (RTSM), source domain transformer (SDT), and optimal subspace mapping (OSM). The RTSM maps a symmetric positive definite matrix from the Riemannian space to the tangent space to minimize the marginal probability distribution drift. The SDT transforms the source domain to a target domain by finding the optimal transport mapping matrix to reduce the joint probability distribution differences. The OSM finally maps the transformed source domain and original target domain to the same subspace to further mitigate the distribution discrepancy. The performance of the proposed method was validated on two public BCI datasets, and the average accuracy of the algorithm on two datasets was 72.24% and 69.29%. Our results demonstrated the improved performance of EEG-based MI detection in comparison with state-of-the-art algorithms.
Collapse
Affiliation(s)
- Junhao Li
- School of Automation, Hangzhou Dianzi University, Hangzhou, 310018, Zhejiang, China
| | - Qingshan She
- School of Automation, Hangzhou Dianzi University, Hangzhou, 310018, Zhejiang, China.
| | - Ming Meng
- School of Automation, Hangzhou Dianzi University, Hangzhou, 310018, Zhejiang, China
| | - Shengzhi Du
- Department of Electrical Engineering, Tshwane University of Technology, Pretoria, 0001, South Africa
| | - Yingchun Zhang
- Department of Biomedical Engineering, University of Houston, Houston, TX, 77204, USA
| |
Collapse
|
6
|
Lee M, Park HY, Park W, Kim KT, Kim YH, Jeong JH. Multi-Task Heterogeneous Ensemble Learning-Based Cross-Subject EEG Classification Under Stroke Patients. IEEE Trans Neural Syst Rehabil Eng 2024; 32:1767-1778. [PMID: 38683717 DOI: 10.1109/tnsre.2024.3395133] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/02/2024]
Abstract
Robot-assisted motor training is applied for neurorehabilitation in stroke patients, using motor imagery (MI) as a representative paradigm of brain-computer interfaces to offer real-life assistance to individuals facing movement challenges. However, the effectiveness of training with MI may vary depending on the location of the stroke lesion, which should be considered. This paper introduces a multi-task electroencephalogram-based heterogeneous ensemble learning (MEEG-HEL) specifically designed for cross-subject training. In the proposed framework, common spatial patterns were used for feature extraction, and the features according to stroke lesions are shared and selected through sequential forward floating selection. The heterogeneous ensembles were used as classifiers. Nine patients with chronic ischemic stroke participated, engaging in MI and motor execution (ME) paradigms involving finger tapping. The classification criteria for the multi-task were established in two ways, taking into account the characteristics of stroke patients. In the cross-subject session, the first involved a direction recognition task for two-handed classification, achieving a performance of 0.7419 (±0.0811) in MI and 0.7061 (±0.1270) in ME. The second task focused on motor assessment for lesion location, resulting in a performance of 0.7457 (±0.1317) in MI and 0.6791 (±0.1253) in ME. Comparing the specific-subject session, except for ME on the motor assessment task, performance on both tasks was significantly higher than the cross-subject session. Furthermore, classification performance was similar to or statistically higher in cross-subject sessions compared to baseline models. The proposed MEEG-HEL holds promise in improving the practicality of neurorehabilitation in clinical settings and facilitating the detection of lesions.
Collapse
|
7
|
Oh E, Shin S, Kim SP. Brain-computer interface in critical care and rehabilitation. Acute Crit Care 2024; 39:24-33. [PMID: 38224957 PMCID: PMC11002623 DOI: 10.4266/acc.2023.01382] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 11/08/2023] [Indexed: 01/17/2024] Open
Abstract
This comprehensive review explores the broad landscape of brain-computer interface (BCI) technology and its potential use in intensive care units (ICUs), particularly for patients with motor impairments such as quadriplegia or severe brain injury. By employing brain signals from various sensing techniques, BCIs offer enhanced communication and motor rehabilitation strategies for patients. This review underscores the concept and efficacy of noninvasive, electroencephalogram-based BCIs in facilitating both communicative interactions and motor function recovery. Additionally, it highlights the current research gap in intuitive "stop" mechanisms within motor rehabilitation protocols, emphasizing the need for advancements that prioritize patient safety and individualized responsiveness. Furthermore, it advocates for more focused research that considers the unique requirements of ICU environments to address the challenges arising from patient variability, fatigue, and limited applicability of current BCI systems outside of experimental settings.
Collapse
Affiliation(s)
- Eunseo Oh
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, Korea
| | - Seyoung Shin
- Department of Mechanical Engineering, Sungkyunkwan University, Suwon, Korea
| | - Sung-Phil Kim
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, Korea
| |
Collapse
|
8
|
Park D, Park H, Kim S, Choo S, Lee S, Nam CS, Jung JY. Spatio-Temporal Explanation of 3D-EEGNet for Motor Imagery EEG Classification Using Permutation and Saliency. IEEE Trans Neural Syst Rehabil Eng 2023; 31:4504-4513. [PMID: 37934650 DOI: 10.1109/tnsre.2023.3330922] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2023]
Abstract
Recently, convolutional neural network (CNN)-based classification models have shown good performance for motor imagery (MI) brain-computer interfaces (BCI) using electroencephalogram (EEG) in end-to-end learning. Although a few explainable artificial intelligence (XAI) techniques have been developed, it is still challenging to interpret the CNN models for EEG-based BCI classification effectively. In this research, we propose 3D-EEGNet as a 3D CNN model to improve both the explainability and performance of MI EEG classification. The proposed approach exhibited better performances on two MI EEG datasets than the existing EEGNet, which uses a 2D input shape. The MI classification accuracies are improved around 1.8% and 6.1% point in average on the datasets, respectively. The permutation-based XAI method is first applied for the reliable explanation of the 3D-EEGNet. Next, to find a faster XAI method for spatio-temporal explanation, we design a novel technique based on the normalized discounted cumulative gain (NDCG) for selecting the best among a few saliency-based methods due to their higher time complexity than the permutation-based method. Among the saliency-based methods, DeepLIFT was selected because the NDCG scores indicated its results are the most similar to the permutation-based results. Finally, the fast spatio-temporal explanation using DeepLIFT provides deeper understanding for the classification results of the 3D-EEGNet and the important properties in the MI EEG experiments.
Collapse
|
9
|
Wu CM, Chen YJ, Chen SC, Zheng SF. Creating an AI-Enhanced Morse Code Translation System Based on Images for People with Severe Disabilities. Bioengineering (Basel) 2023; 10:1281. [PMID: 38002405 PMCID: PMC10669406 DOI: 10.3390/bioengineering10111281] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Revised: 10/24/2023] [Accepted: 10/30/2023] [Indexed: 11/26/2023] Open
Abstract
(1) Background: Patients with severe physical impairments (spinal cord injury, cerebral palsy, amyotrophic lateral sclerosis) often have limited mobility due to physical limitations, and may even be bedridden all day long, losing the ability to take care of themselves. In more severe cases, the ability to speak may even be lost, making even basic communication very difficult. (2) Methods: This research will design a set of image-assistive communication equipment based on artificial intelligence to solve communication problems of daily needs. Using artificial intelligence for facial positioning, and facial-motion-recognition-generated Morse code, and then translating it into readable characters or commands, it allows users to control computer software by themselves and communicate through wireless networks or a Bluetooth protocol to control environment peripherals. (3) Results: In this study, 23 human-typed data sets were subjected to recognition using fuzzy algorithms. The average recognition rates for expert-generated data and data input by individuals with disabilities were 99.83% and 98.6%, respectively. (4) Conclusions: Through this system, users can express their thoughts and needs through their facial movements, thereby improving their quality of life and having an independent living space. Moreover, the system can be used without touching external switches, greatly improving convenience and safety.
Collapse
Affiliation(s)
- Chung-Min Wu
- Department of Intelligent Automation Engineering, National Chin-Yi University of Technology, Taichung 411030, Taiwan;
| | - Yeou-Jiunn Chen
- Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan 710301, Taiwan;
| | - Shih-Chung Chen
- Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan 710301, Taiwan;
| | - Sheng-Feng Zheng
- Department of Intelligent Robotics Engineering, Kun Shan University, Tainan 710303, Taiwan;
| |
Collapse
|
10
|
K SG, Vinod AP, Subasree R. A Phase-based EEG Epoch Selection Method for Decoding Bi-directional Hand Movement Imagination in Stroke Patients. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38082644 DOI: 10.1109/embc40787.2023.10340319] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Electroencephalogram (EEG) based non-invasive Brain Computer Interface (BCI) system is gaining significant attention as a promising solution for stroke rehabilitation. Accurate selection of informative EEG time segment, that accommodates the specific neural activity patterns associated with the underlying mental task can help to improve the efficacy of the BCI system. In this work, we propose a phase-based EEG epoch selection algorithm to extract the discriminative EEG time segment corresponding to bi-directional hand motor imagery. The imagined center-out hand movement in two directions is decoded using the selected epoch of the EEG, recorded from 16 stroke patients with hemiparesis and specifically hand weakness. Phase Lock Value (PLV) EEG features extracted from the selected EEG epoch is used as discriminative feature for binary classification of imagined hand movement direction using Linear Discriminant Analysis. The use of selected EEG epoch yielded an improvement of 11.5% and 11.7% in the average direction classification accuracy of calibration and feedback session data respectively, compared to the baseline method employing the whole EEG signal. In addition to improvement in decoding accuracy, the epoch selection also yielded an average Information Transfer Rate (ITR) of 39.8±24.6 bits per minute, which is 86% improvement compared to the baseline method.Clinical Relevance- The proposed Motor Imagery (MI)-BCI system may be of clinical relevance as an active rehabilitation tool for stroke-affected patients, to enhance neural plasticity and recovery of centre-out activities of affected hand and forms a strong platform for MI-BCI coupled with exoskeletons or prosthesis rehabilitation.
Collapse
|
11
|
Lakshminarayanan K, Shah R, Daulat SR, Moodley V, Yao Y, Madathil D. The effect of combining action observation in virtual reality with kinesthetic motor imagery on cortical activity. Front Neurosci 2023; 17:1201865. [PMID: 37383098 PMCID: PMC10299830 DOI: 10.3389/fnins.2023.1201865] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 05/25/2023] [Indexed: 06/30/2023] Open
Abstract
Introduction In the past, various techniques have been used to improve motor imagery (MI), such as immersive virtual-reality (VR) and kinesthetic rehearsal. While electroencephalography (EEG) has been used to study the differences in brain activity between VR-based action observation and kinesthetic motor imagery (KMI), there has been no investigation into their combined effect. Prior research has demonstrated that VR-based action observation can enhance MI by providing both visual information and embodiment, which is the perception of oneself as part of the observed entity. Additionally, KMI has been found to produce similar brain activity to physically performing a task. Therefore, we hypothesized that utilizing VR to offer an immersive visual scenario for action observation while participants performed kinesthetic motor imagery would significantly improve cortical activity related to MI. Methods In this study, 15 participants (9 male, 6 female) performed kinesthetic motor imagery of three hand tasks (drinking, wrist flexion-extension, and grabbing) both with and without VR-based action observation. Results Our results indicate that combining VR-based action observation with KMI enhances brain rhythmic patterns and provides better task differentiation compared to KMI without action observation. Discussion These findings suggest that using VR-based action observation alongside kinesthetic motor imagery can improve motor imagery performance.
Collapse
Affiliation(s)
- Kishor Lakshminarayanan
- Neuro-Rehabilitation Lab, School of Electronics Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India
| | - Rakshit Shah
- Department of Chemical and Biomedical Engineering, Cleveland State University, Cleveland, OH, United States
| | - Sohail R. Daulat
- Department of Physiology, University of Arizona College of Medicine – Tucson, Tucson, AZ, United States
| | - Viashen Moodley
- Arizona Center for Hand to Shoulder Surgery, Phoenix, AZ, United States
| | - Yifei Yao
- Soft Tissue Biomechanics Laboratory, School of Biomedical Engineering, Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China
| | - Deepa Madathil
- Jindal Institute of Behavioural Sciences, O.P. Jindal Global University, Sonipat, Haryana, India
| |
Collapse
|
12
|
Ramu V, Lakshminarayanan K. Enhanced motor imagery of digits within the same hand via vibrotactile stimulation. Front Neurosci 2023; 17:1152563. [PMID: 37360173 PMCID: PMC10289883 DOI: 10.3389/fnins.2023.1152563] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Accepted: 05/26/2023] [Indexed: 06/28/2023] Open
Abstract
Purpose The aim of the present study is to evaluate the effect of vibrotactile stimulation prior to repeated complex motor imagery of finger movements using the non-dominant hand on motor imagery (MI) performance. Methods Ten healthy right-handed adults (4 females and 6 males) participated in the study. The subjects performed motor imagery tasks with and without a brief vibrotactile sensory stimulation prior to performing motor imagery using either their left-hand index, middle, or thumb digits. Mu- and beta-band event-related desynchronization (ERD) at the sensorimotor cortex and an artificial neural network-based digit classification was evaluated. Results The ERD and digit discrimination results from our study showed that ERD was significantly different between the vibration conditions for the index, middle, and thumb. It was also found that digit classification accuracy with-vibration (mean ± SD = 66.31 ± 3.79%) was significantly higher than without-vibration (mean ± SD = 62.68 ± 6.58%). Conclusion The results showed that a brief vibration was more effective at improving MI-based brain-computer interface classification of digits within a single limb through increased ERD compared to performing MI without vibrotactile stimulation.
Collapse
|
13
|
Lakshminarayanan K, Shah R, Daulat SR, Moodley V, Yao Y, Sengupta P, Ramu V, Madathil D. Evaluation of EEG Oscillatory Patterns and Classification of Compound Limb Tactile Imagery. Brain Sci 2023; 13:brainsci13040656. [PMID: 37190621 DOI: 10.3390/brainsci13040656] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Revised: 04/10/2023] [Accepted: 04/12/2023] [Indexed: 05/17/2023] Open
Abstract
Objective: The purpose of this study was to investigate the cortical activity and digit classification performance during tactile imagery (TI) of a vibratory stimulus at the index, middle, and thumb digits within the left hand in healthy individuals. Furthermore, the cortical activities and classification performance of the compound TI were compared with similar compound motor imagery (MI) with the same digits as TI in the same subjects. Methods: Twelve healthy right-handed adults with no history of upper limb injury, musculoskeletal condition, or neurological disorder participated in the study. The study evaluated the event-related desynchronization (ERD) response and brain-computer interface (BCI) classification performance on discriminating between the digits in the left-hand during the imagery of vibrotactile stimuli to either the index, middle, or thumb finger pads for TI and while performing a motor activity with the same digits for MI. A supervised machine learning technique was applied to discriminate between the digits within the same given limb for both imagery conditions. Results: Both TI and MI exhibited similar patterns of ERD in the alpha and beta bands at the index, middle, and thumb digits within the left hand. While TI had significantly lower ERD for all three digits in both bands, the classification performance of TI-based BCI (77.74 ± 6.98%) was found to be similar to the MI-based BCI (78.36 ± 5.38%). Conclusions: The results of this study suggest that compound tactile imagery can be a viable alternative to MI for BCI classification. The study contributes to the growing body of evidence supporting the use of TI in BCI applications, and future research can build on this work to explore the potential of TI-based BCI for motor rehabilitation and the control of external devices.
Collapse
Affiliation(s)
- Kishor Lakshminarayanan
- Neuro-Rehabilitation Lab, Department of Sensors and Biomedical Engineering, School of Electronics Engineering, Vellore Institute of Technology, Vellore 632014, Tamil Nadu, India
| | - Rakshit Shah
- Department of Chemical and Biomedical Engineering, Cleveland State University, Cleveland, OH 44115, USA
| | - Sohail R Daulat
- Department of Physiology, University of Arizona College of Medicine, Tucson, AZ 85724, USA
| | - Viashen Moodley
- Arizona Center for Hand to Shoulder Surgery, Phoenix, AZ 85004, USA
| | - Yifei Yao
- Soft Tissue Biomechanics Laboratory, Med-X Research Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China
| | - Puja Sengupta
- Neuro-Rehabilitation Lab, Department of Sensors and Biomedical Engineering, School of Electronics Engineering, Vellore Institute of Technology, Vellore 632014, Tamil Nadu, India
| | - Vadivelan Ramu
- Neuro-Rehabilitation Lab, Department of Sensors and Biomedical Engineering, School of Electronics Engineering, Vellore Institute of Technology, Vellore 632014, Tamil Nadu, India
| | - Deepa Madathil
- Jindal Institute of Behavioral Sciences, O. P. Jindal Global University, Sonipat 131001, Haryana, India
| |
Collapse
|
14
|
Thanigaivelu PS, Sridhar SS, Sulthana SF. OISVM: Optimal Incremental Support Vector Machine-based EEG Classification for Brain-computer Interface Model. Cognit Comput 2023. [DOI: 10.1007/s12559-023-10120-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/02/2023]
|
15
|
Lee PL, Chen SH, Chang TC, Lee WK, Hsu HT, Chang HH. Continual Learning of a Transformer-Based Deep Learning Classifier Using an Initial Model from Action Observation EEG Data to Online Motor Imagery Classification. Bioengineering (Basel) 2023; 10:bioengineering10020186. [PMID: 36829681 PMCID: PMC9952173 DOI: 10.3390/bioengineering10020186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2022] [Revised: 01/20/2023] [Accepted: 01/24/2023] [Indexed: 02/04/2023] Open
Abstract
The motor imagery (MI)-based brain computer interface (BCI) is an intuitive interface that enables users to communicate with external environments through their minds. However, current MI-BCI systems ask naïve subjects to perform unfamiliar MI tasks with simple textual instruction or a visual/auditory cue. The unclear instruction for MI execution not only results in large inter-subject variability in the measured EEG patterns but also causes the difficulty of grouping cross-subject data for big-data training. In this study, we designed an BCI training method in a virtual reality (VR) environment. Subjects wore a head-mounted device (HMD) and executed action observation (AO) concurrently with MI (i.e., AO + MI) in VR environments. EEG signals recorded in AO + MI task were used to train an initial model, and the initial model was continually improved by the provision of EEG data in the following BCI training sessions. We recruited five healthy subjects, and each subject was requested to participate in three kinds of tasks, including an AO + MI task, an MI task, and the task of MI with visual feedback (MI-FB) three times. This study adopted a transformer- based spatial-temporal network (TSTN) to decode the user's MI intentions. In contrast to other convolutional neural network (CNN) or recurrent neural network (RNN) approaches, the TSTN extracts spatial and temporal features, and applies attention mechanisms along spatial and temporal dimensions to perceive the global dependencies. The mean detection accuracies of TSTN were 0.63, 0.68, 0.75, and 0.77 in the MI, first MI-FB, second MI-FB, and third MI-FB sessions, respectively. This study demonstrated the AO + MI gave an easier way for subjects to conform their imagery actions, and the BCI performance was improved with the continual learning of the MI-FB training process.
Collapse
Affiliation(s)
- Po-Lei Lee
- Department of Electrical Engineering, National Central University, Taoyuan 320, Taiwan
- Pervasive Artificial Intelligence Research Labs, Hsinchu 300, Taiwan
| | - Sheng-Hao Chen
- Department of Electrical Engineering, National Central University, Taoyuan 320, Taiwan
| | - Tzu-Chien Chang
- Department of Electrical Engineering, National Central University, Taoyuan 320, Taiwan
| | - Wei-Kung Lee
- Department of Rehabilitation, Taoyuan General Hospital, Taoyuan 330, Taiwan
| | - Hao-Teng Hsu
- Department of Electrical Engineering, National Central University, Taoyuan 320, Taiwan
- Pervasive Artificial Intelligence Research Labs, Hsinchu 300, Taiwan
| | - Hsiao-Huang Chang
- Division of Cardiovascular Surgery, Department of Surgery, Taipei Veterans General Hospital, Taipei 112, Taiwan
- Department of Surgery, School of Medicine, College of Medicine, Taipei Medical University, Taipei 110, Taiwan
- Correspondence: ; Tel.: +886-937-919-107
| |
Collapse
|
16
|
He Q, He J, Yang Y, Zhao J. Brain-Computer Interfaces in Disorders of Consciousness. Neurosci Bull 2023; 39:348-352. [PMID: 35941403 PMCID: PMC9905465 DOI: 10.1007/s12264-022-00920-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Accepted: 06/03/2022] [Indexed: 11/25/2022] Open
Affiliation(s)
- Qiheng He
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100070, China
- China National Clinical Research Center for Neurological Diseases, Beijing, 100070, China
| | - Jianghong He
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100070, China.
| | - Yi Yang
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100070, China.
- Chinese Institute for Brain Research, Beijing, 100010, China.
- Beijing Institute of Brain Disorders, Beijing, 100069, China.
- China National Clinical Research Center for Neurological Diseases, Beijing, 100070, China.
| | - Jizong Zhao
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100070, China
- China National Clinical Research Center for Neurological Diseases, Beijing, 100070, China
| |
Collapse
|
17
|
A Review of Online Classification Performance in Motor Imagery-Based Brain–Computer Interfaces for Stroke Neurorehabilitation. SIGNALS 2023. [DOI: 10.3390/signals4010004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023] Open
Abstract
Motor imagery (MI)-based brain–computer interfaces (BCI) have shown increased potential for the rehabilitation of stroke patients; nonetheless, their implementation in clinical practice has been restricted due to their low accuracy performance. To date, although a lot of research has been carried out in benchmarking and highlighting the most valuable classification algorithms in BCI configurations, most of them use offline data and are not from real BCI performance during the closed-loop (or online) sessions. Since rehabilitation training relies on the availability of an accurate feedback system, we surveyed articles of current and past EEG-based BCI frameworks who report the online classification of the movement of two upper limbs in both healthy volunteers and stroke patients. We found that the recently developed deep-learning methods do not outperform the traditional machine-learning algorithms. In addition, patients and healthy subjects exhibit similar classification accuracy in current BCI configurations. Lastly, in terms of neurofeedback modality, functional electrical stimulation (FES) yielded the best performance compared to non-FES systems.
Collapse
|
18
|
Vecchio F, Nucci L, Pappalettera C, Miraglia F, Iacoviello D, Rossini PM. Time-frequency analysis of brain activity in response to directional and non-directional visual stimuli: an event related spectral perturbations (ERSP) study. J Neural Eng 2022; 19. [PMID: 36270505 DOI: 10.1088/1741-2552/ac9c96] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Accepted: 10/21/2022] [Indexed: 01/11/2023]
Abstract
Objective.A large part of the cerebral cortex is dedicated to the processing of visual stimuli and there is still much to understand about such processing modalities and hierarchies. The main aim of the present study is to investigate the differences between directional visual stimuli (DS) and non-directional visual stimuli (n-DS) processing by time-frequency analysis of brain electroencephalographic activity during a visuo-motor task. Electroencephalography (EEG) data were divided into four regions of interest (ROIs) (frontal, central, parietal, occipital).Approach.The analysis of the visual stimuli processing was based on the combination of electroencephalographic recordings and time-frequency analysis. Event related spectral perturbations (ERSPs) were computed with spectrum analysis that allow to obtain the average time course of relative changes induced by the stimulus presentation in spontaneous EEG amplitude spectrum.Main results.Visual stimuli processing enhanced the same pattern of spectral modulation in all investigated ROIs with differences in amplitudes and timing. Additionally, statistically significant differences in occipital ROI between the DS and n-DS visual stimuli processing in theta, alpha and beta bands were found.Significance.These evidences suggest that ERSPs could be a useful tool to investigate the encoding of visual information in different brain regions. Because of their simplicity and their capability in the representation of brain activity, the ERSPs might be used as biomarkers of functional recovery for example in the rehabilitation of visual dysfunction and motor impairment following a stroke, as well as diagnostic tool of anomalies in brain functions in neurological diseases tailored to personalized treatments in clinical environment.
Collapse
Affiliation(s)
- Fabrizio Vecchio
- Brain Connectivity Laboratory, Department of Neuroscience and Neurorehabilitation, IRCCS San Raffaele Roma, Rome, Italy.,Department of Theoretical and Applied Sciences, eCampus University, Novedrate, Como, Italy
| | - Lorenzo Nucci
- Brain Connectivity Laboratory, Department of Neuroscience and Neurorehabilitation, IRCCS San Raffaele Roma, Rome, Italy
| | - Chiara Pappalettera
- Brain Connectivity Laboratory, Department of Neuroscience and Neurorehabilitation, IRCCS San Raffaele Roma, Rome, Italy.,Department of Theoretical and Applied Sciences, eCampus University, Novedrate, Como, Italy
| | - Francesca Miraglia
- Brain Connectivity Laboratory, Department of Neuroscience and Neurorehabilitation, IRCCS San Raffaele Roma, Rome, Italy.,Department of Theoretical and Applied Sciences, eCampus University, Novedrate, Como, Italy
| | - Daniela Iacoviello
- Department of Computer, Control and Management Engineering Antonio Ruberti, Sapienza University of Rome, Rome, Italy
| | - Paolo Maria Rossini
- Brain Connectivity Laboratory, Department of Neuroscience and Neurorehabilitation, IRCCS San Raffaele Roma, Rome, Italy
| |
Collapse
|
19
|
Wang C, Wu Y, Wang C, Zhu Y, Wang C, Niu Y, Shao Z, Gao X, Zhao Z, Yu Y. MI-EEG classification using Shannon complex wavelet and convolutional neural networks. Appl Soft Comput 2022. [DOI: 10.1016/j.asoc.2022.109685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
20
|
Cao L, Wang W, Huang C, Xu Z, Wang H, Jia J, Chen S, Dong Y, Fan C, de Albuquerque VHC. An Effective Fusing Approach by Combining Connectivity Network Pattern and Temporal-Spatial Analysis for EEG-Based BCI Rehabilitation. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2264-2274. [PMID: 35969547 DOI: 10.1109/tnsre.2022.3198434] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Motor-modality-based brain computer interface (BCI) could promote the neural rehabilitation for stroke patients. Temporal-spatial analysis was commonly used for pattern recognition in this task. This paper introduced a novel connectivity network analysis for EEG-based feature selection. The network features of connectivity pattern not only captured the spatial activities responding to motor task, but also mined the interactive pattern among these cerebral regions. Furthermore, the effective combination between temporal-spatial analysis and network analysis was evaluated for improving the performance of BCI classification (81.7%). And the results demonstrated that it could raise the classification accuracies for most of patients (6 of 7 patients). This proposed method was meaningful for developing the effective BCI training program for stroke rehabilitation.
Collapse
|
21
|
Wei Y, Li J, Ji H, Jin L, Liu L, Bai Z, Ye C. A Semi-Supervised Progressive Learning Algorithm for Brain-Computer Interface. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2067-2076. [PMID: 35853068 DOI: 10.1109/tnsre.2022.3192448] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Brain-computer interface (BCI) usually suffers from the problem of low recognition accuracy and large calibration time, especially when identifying motor imagery tasks for subjects with indistinct features and classifying fine grained motion control tasks by electroencephalogram (EEG)-electromyogram (EMG) fusion analysis. To fill the research gap, this paper presents an end-to-end semi-supervised learning framework for EEG classification and EEG-EMG fusion analysis. Benefiting from the proposed metric learning based label estimation strategy, sampling criterion and progressive learning scheme, the proposed framework efficiently extracts distinctive feature embedding from the unlabeled EEG samples and achieves a 5.40% improvement on BCI Competition IV Dataset IIa with 80% unlabeled samples and an average 3.35% improvement on two public BCI datasets. By employing synchronous EMG features as pseudo labels for the unlabeled EEG samples, the proposed framework further extracts deep level features of the synergistic complementarity between the EEG signals and EMG features based on the deep encoders, which improves the performance of hybrid BCI (with a 5.53% improvement for the Upper Limb Motion Dataset and an average 4.34% improvement on two hybrid datasets). Moreover, the ablation experiments show that the proposed framework can substantially improve the performance of the deep encoders (with an average 5.53% improvement). The proposed framework not only largely improves the performance of deep networks in the BCI system, but also significantly reduces the calibration time for EEG-EMG fusion analysis, which shows great potential for building an efficient and high-performance hybrid BCI for the motor rehabilitation process.
Collapse
|
22
|
Subject-Dependent Artifact Removal for Enhancing Motor Imagery Classifier Performance under Poor Skills. SENSORS 2022; 22:s22155771. [PMID: 35957329 PMCID: PMC9371054 DOI: 10.3390/s22155771] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 07/26/2022] [Accepted: 07/29/2022] [Indexed: 11/28/2022]
Abstract
The Electroencephalography (EEG)-based motor imagery (MI) paradigm is one of the most studied technologies for Brain-Computer Interface (BCI) development. Still, the low Signal-to-Noise Ratio (SNR) poses a challenge when constructing EEG-based BCI systems. Moreover, the non-stationary and nonlinear signal issues, the low-spatial data resolution, and the inter- and intra-subject variability hamper the extraction of discriminant features. Indeed, subjects with poor motor skills have difficulties in practicing MI tasks against low SNR scenarios. Here, we propose a subject-dependent preprocessing approach that includes the well-known Surface Laplacian Filtering and Independent Component Analysis algorithms to remove signal artifacts based on the MI performance. In addition, power- and phase-based functional connectivity measures are studied to extract relevant and interpretable patterns and identify subjects of inefficency. As a result, our proposal, Subject-dependent Artifact Removal (SD-AR), improves the MI classification performance in subjects with poor motor skills. Consequently, electrooculography and volume-conduction EEG artifacts are mitigated within a functional connectivity feature-extraction strategy, which favors the classification performance of a straightforward linear classifier.
Collapse
|
23
|
Jo S, Jung JH, Yang MJ, Lee Y, Jang SJ, Feng J, Heo SH, Kim J, Shin JH, Jeong J, Park HS. EEG-EMG hybrid real-time classification of hand grasp and release movements intention in chronic stroke patients. IEEE Int Conf Rehabil Robot 2022; 2022:1-6. [PMID: 36176084 DOI: 10.1109/icorr55369.2022.9896592] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Rehabilitation of the hand motor function is essential for stroke patients to resume activities of daily living. Recent studies have shown that wearable robot systems, like a multi degree-of-freedom soft glove, have the potential to improve hand motor impairment. The rehabilitation system, which is intuitively controlled according to the user's intention, is expected to induce active participation of the user and further promote brain plasticity. However, due to the patient-specific nature of stroke patients, extracting the intention from stroke patients is still challenging. In this study, we implemented a classifier that combines EEG and EMG to detect chronic stroke patients' four types of intention: rest, grasp, hold, and release. Three chronic stroke patients participated in the experiment and performed rest, grasp, hold, and release actions. The rest vs. grasp binary classifier and release vs. hold binary classifier showed 76.9% and 86.6% classification accuracy in real-time, respectively. In addition, patient-specific accuracy comparisons showed that the hybrid approach was robust to upper limb impairment level compared to other approaches. We believe that these results could pave the way for the development of BCI-based robotic hand rehabilitation therapy.
Collapse
|
24
|
Vaghei Y, Park EJ, Arzanpour S. Decoding Brain Signals to Classify Gait Direction Anticipation. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:309-312. [PMID: 36086221 DOI: 10.1109/embc48229.2022.9871566] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
The use of brain-computer interface (BCI) technology has emerged as a promising rehabilitation approach for patients with motor function and motor-related disorders. BCIs provide an augmentative communication platform for controlling advanced assistive robots such as a lower-limb exoskeleton. Brain recordings collected by an electroencephalography (EEG) system have been employed in the BCI platform to command the exoskeleton. To date, the literature on this topic is limited to the prediction of gait intention and gait variations from EEG signals. This study, however, aims to predict the anticipated gait direction using a stream of EEG signals collected from the brain cortex. Three healthy participants (age range: 29-31, 2 female) were recruited. While wearing the EEG device, the participants were instructed to initiate gait movement toward the direction of the arrow triggers (pointing forward, backward, left, or right) being shown on a screen with a blank white background. Collected EEG data was then epoched around the trigger timepoints. These epochs were then converted to the time-frequency domain using event- related synchronization (ERS) and event-related desynchronization (ERD) methods. Finally, the classification pipeline was constructed using logistic regression (LR), support vector machine (SVM), and convolutional neural network (CNN). A ten-fold cross-validation scheme was used to evaluate the classification performance. The results revealed that the CNN classifier outperforms the other two classifiers with an accuracy of 0.75. Clinical Relevance - The outcome of this study has the potential to be ultimately used for interactive navigation of the lower-limb exoskeletons during robotic rehabilitation therapy and enhance neurodegeneration and neuroplasticity in a wide range of individuals with lower-limb motor function disabilities.
Collapse
|
25
|
Uyanik C, Khan MA, Brunner IC, Hansen JP, Puthusserypady S. Machine Learning for Motor Imagery Wrist Dorsiflexion Prediction in Brain-Computer Interface Assisted Stroke Rehabilitation. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:715-719. [PMID: 36086493 DOI: 10.1109/embc48229.2022.9871600] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Stroke is a life-changing event that can affect the survivors' physical, cognitive and emotional state. Stroke care focuses on helping the survivors to regain their strength; recover as much functionality as possible and return to independent living through rehabilitation therapies. Automated training protocols have been reported to improve the efficiency of the rehabilitation process. These protocols also decrease the dependency of the process on a professional trainer. Brain-Computer Interface (BCI) based systems are examples of such systems where they make use of the motor imagery (MI) based electroencephalogram (EEG) signals to drive the rehabilitation protocols. In this paper, we have proposed the use of well-known machine learning (ML) algorithms, such as, the decision tree (DT), Naive Bayesian (NB), linear discriminant analysis (LDA), support vector machine (SVM), ensemble learning classifier (ELC), and artificial neural network (ANN) for MI wrist dorsiflexion prediction in a BCI assisted stroke rehabilitation study conducted on eleven stroke survivors with either the left or right paresis. The doubling sub-band selection filter bank common spatial pattern (DSBS-FBCSP) has been proposed as feature extractor and it is observed that the ANN based classifier produces the best results.
Collapse
|
26
|
Wang Z, Cao C, Chen L, Gu B, Liu S, Xu M, He F, Ming D. Multimodal Neural Response and Effect Assessment During a BCI-Based Neurofeedback Training After Stroke. Front Neurosci 2022; 16:884420. [PMID: 35784834 PMCID: PMC9247245 DOI: 10.3389/fnins.2022.884420] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2022] [Accepted: 05/17/2022] [Indexed: 11/20/2022] Open
Abstract
Stroke caused by cerebral infarction or hemorrhage can lead to motor dysfunction. The recovery of motor function is vital for patients with stroke in daily activities. Traditional rehabilitation of stroke generally depends on physical practice under passive affected limbs movement. Motor imagery-based brain computer interface (MI-BCI) combined with functional electrical stimulation (FES) is a potential active neural rehabilitation technology for patients with stroke recently, which complements traditional passive rehabilitation methods. As the predecessor of BCI technology, neurofeedback training (NFT) is a psychological process that feeds back neural activities online to users for self-regulation. In this work, BCI-based NFT were proposed to promote the active repair and reconstruction of the whole nerve conduction pathway and motor function. We designed and implemented a multimodal, training type motor NFT system (BCI-NFT-FES) by integrating the visual, auditory, and tactile multisensory pathway feedback mode and using the joint detection of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). The results indicated that after 4 weeks of training, the clinical scale score, event-related desynchronization (ERD) of EEG patterns, and cerebral oxygen response of patients with stroke were enhanced obviously. This study preliminarily verified the clinical effectiveness of the long-term NFT system and the prospect of motor function rehabilitation.
Collapse
Affiliation(s)
- Zhongpeng Wang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
| | - Cong Cao
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Long Chen
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
- *Correspondence: Long Chen
| | - Bin Gu
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
| | - Shuang Liu
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Minpeng Xu
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
- Tianjin International Joint Research Center for Neural Engineering, Tianjin, China
| | - Feng He
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
- Tianjin International Joint Research Center for Neural Engineering, Tianjin, China
| | - Dong Ming
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
- Tianjin International Joint Research Center for Neural Engineering, Tianjin, China
- Dong Ming
| |
Collapse
|
27
|
Xu F, Dong G, Li J, Yang Q, Wang L, Zhao Y, Yan Y, Zhao J, Pang S, Guo D, Zhang Y, Leng J. Deep Convolution Generative Adversarial Network Based Electroencephalogram Data Augmentation For Post-Stroke Rehabilitation With Motor Imagery. Int J Neural Syst 2022; 32:2250039. [DOI: 10.1142/s0129065722500393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
28
|
Tofani M, Santecchia L, Conte A, Berardi A, Galeoto G, Sogos C, Petrarca M, Panuccio F, Castelli E. Effects of Mirror Neurons-Based Rehabilitation Techniques in Hand Injuries: A Systematic Review and Meta-Analysis. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:5526. [PMID: 35564920 PMCID: PMC9104298 DOI: 10.3390/ijerph19095526] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 04/26/2022] [Accepted: 04/27/2022] [Indexed: 02/04/2023]
Abstract
Background: Hand trauma requires specific rehabilitation protocol depending on the different structures involved. According to type of surgical intervention, and for monitoring pain and edema, post-operative rehabilitation of a hand that has experienced trauma involves different timings for immobilization. Several protocols have been used to reduce immobilization time, and various techniques and methods are adopted, depending on the structures involved. Objective: To measure the effects of mirror neurons-based rehabilitation techniques in hand injuries throughout a systematic review and meta-analysis. Methods: The protocol was accepted in PROSPERO database. A literature search was conducted in Cinahl, Scopus, Medline, PEDro, OTseeker. Two authors independently identified eligible studies, based on predefined inclusion criteria, and extracted the data. RCT quality was assessed using the JADAD scale. Results: Seventy-nine suitable studies were screened, and only eleven were included for qualitative synthesis, while four studies were selected for quantitative analysis. Four studies were case reports/series, and seven were RCTs. Nine investigate the effect of Mirror Therapy and two the effect of Motor Imagery. Quantitative analyses revealed Mirror Therapy as effective for hand function recovery (mean difference = −14.80 95% Confidence Interval (CI) = −17.22, −12.38) (p < 0.00001) in the short term, as well as in long follow-up groups (mean difference = −13.11 95% Confidence Interval (CI) = −17.53, −8.69) (p < 0.00001). Clinical, but not statistical, efficacy was found for manual dexterity (p = 0.15), while no benefit was reported for range of motion. Conclusions: Mirror neurons-based rehabilitation techniques, combined with conventional occupational and physical therapy, can be a useful approach in hand trauma. Mirror therapy seems to be effective for hand function recovery, but, for motor imagery and action observation, there is not sufficient evidence to recommend its use. Further research on the efficacy of the mirror neurons-based technique in hand injury is recommended.
Collapse
Affiliation(s)
- Marco Tofani
- Professional Development, Continuous Education and Research Service, Bambino Gesù Children’s Hospital, 00165 Rome, Italy
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy; (A.C.); (A.B.); (G.G.); (C.S.)
| | - Luigino Santecchia
- Orthopedic Unit, Department of Surgery, Bambino Gesù Children’s Hospital, 00100 Rome, Italy;
| | - Antonella Conte
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy; (A.C.); (A.B.); (G.G.); (C.S.)
- Neuromed IRCCS, 86077 Pozzili, Italy
| | - Anna Berardi
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy; (A.C.); (A.B.); (G.G.); (C.S.)
| | - Giovanni Galeoto
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy; (A.C.); (A.B.); (G.G.); (C.S.)
- Neuromed IRCCS, 86077 Pozzili, Italy
| | - Carla Sogos
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy; (A.C.); (A.B.); (G.G.); (C.S.)
| | - Maurizio Petrarca
- Department of Intensive Neurorehabilitation and Robotics, Bambino Gesù Children’s Hospital, 00100 Rome, Italy; (M.P.); (E.C.)
| | | | - Enrico Castelli
- Department of Intensive Neurorehabilitation and Robotics, Bambino Gesù Children’s Hospital, 00100 Rome, Italy; (M.P.); (E.C.)
| |
Collapse
|
29
|
Guo Z, Zhou S, Ji K, Zhuang Y, Song J, Nam C, Hu X, Zheng Y. Corticomuscular integrated representation of voluntary motor effort in robotic control for wrist-hand rehabilitation after stroke. J Neural Eng 2022; 19. [PMID: 35193124 DOI: 10.1088/1741-2552/ac5757] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2021] [Accepted: 02/22/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE The central-to-peripheral voluntary motor effort (VME) in physical practice of the paretic limb is a dominant force for driving functional neuroplasticity on motor restoration post-stroke. However, current rehabilitation robots isolated the central and peripheral involvements in the control design, resulting in limited rehabilitation effectiveness. The purpose of this study was to design a corticomuscular coherence (CMC) and electromyography (EMG)-driven (CMC-EMG-driven) system with central-and-peripheral integrated representation of VME for wrist-hand rehabilitation after stroke. APPROACH The CMC-EMG-driven control was developed in a neuromuscular electrical stimulation (NMES)-robot system, i.e., CMC-EMG-driven NMES-robot system, to instruct and assist the wrist-hand extension and flexion in persons after stroke. A pilot single-group trial of 20 training sessions was conducted with the developed system to assess the feasibility for wrist-hand practice on the chronic strokes (n=16). The rehabilitation effectiveness was evaluated through clinical assessments, CMC, and EMG activation levels. MAIN RESULTS The trigger success rate and laterality index (LI) of CMC were significantly increased in wrist-hand extension across training sessions (p<0.05). After the training, significant improvements in the target wrist-hand joints and suppressed compensation from the proximal shoulder-elbow joints were observed through the clinical scores and EMG activation levels (p<0.05). The central-to-peripheral VME distribution across upper extremity (UE) muscles was also significantly improved, as revealed by the CMC values (p<0.05). SIGNIFICANCE Precise wrist-hand rehabilitation was achieved by the developed system, presenting suppressed cortical and muscular compensation from the contralesional hemisphere and the proximal UE, and improved distribution of the central-and-peripheral VME on UE muscles.
Collapse
Affiliation(s)
- Ziqi Guo
- The Hong Kong Polytechnic University, Rm S107a, Dept. of BME, PolyU, Hung H, Hung Hom, Kowloon, Kowloon, Nil, HONG KONG
| | - Sa Zhou
- The Hong Kong Polytechnic University, Rm S107a, Dept. of BME, PolyU, Hung H, Hung Hom, Kowloon, Hong Kong, Kowloon, HONG KONG
| | - Kailai Ji
- The Hong Kong Polytechnic University, Dept. of BME, PolyU, Hung H, Hung Hom, Kowloon, Kowloon, Hong Kong, HONG KONG
| | - Yongqi Zhuang
- Biomedical Engineering, Hong Kong Polytechnic University, BME PolyU, Kowloon, HONG KONG
| | - Jie Song
- The Hong Kong Polytechnic University, Rm S107a, Dept. of BME, PolyU, Hung H, Hung Hom, Kowloon, Hong Kong, Kowloon, Nil, HONG KONG
| | - Chingyi Nam
- The Hong Kong Polytechnic University, Rm S107a, Dept. of BME, PolyU, Hung H, Hung Hom, Kowloon, Hong Kong, Kowloon, Nil, HONG KONG
| | - Xiaoling Hu
- Biomedical Engineering, Hong Kong Polytechnic University, Rm ST420, Dept. of BME, PolyU, Hung H, Hung Hom, Kowloon, Hong Kong, Kowloon, HONG KONG
| | - Yongping Zheng
- Biomedical Engineering, The Hong Kong Polytechnic University, BME PolyU, Hong Kong, Nil, CHINA
| |
Collapse
|
30
|
Lee DY, Jeong JH, Lee BH, Lee SW. Motor Imagery Classification Using Inter-Task Transfer Learning via A Channel-Wise Variational Autoencoder-based Convolutional Neural Network. IEEE Trans Neural Syst Rehabil Eng 2022; 30:226-237. [PMID: 35041605 DOI: 10.1109/tnsre.2022.3143836] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Highly sophisticated control based on a brain-computer interface (BCI) requires decoding kinematic information from brain signals. The forearm is a region of the upper limb that is often used in everyday life, but intuitive movements within the same limb have rarely been investigated in previous BCI studies. In this study, we focused on various forearm movement decoding from electroencephalography (EEG) signals using a small number of samples. Ten healthy participants took part in an experiment and performed motor execution (ME) and motor imagery (MI) of the intuitive movement tasks (Dataset I). We propose a convolutional neural network using a channel-wise variational autoencoder (CVNet) based on inter-task transfer learning. We approached that training the reconstructed ME-EEG signals together will also achieve more sufficient classification performance with only a small amount of MI-EEG signals. The proposed CVNet was validated on our own Dataset I and a public dataset, BNCI Horizon 2020 (Dataset II). The classification accuracies of various movements are confirmed to be 0.83 (±0.04) and 0.69 (±0.04) for Dataset I and II, respectively. The results show that the proposed method exhibits performance increases of approximately 0.09~0.27 and 0.08~0.24 compared with the conventional models for Dataset I and II, respectively. The outcomes suggest that the training model for decoding imagined movements can be performed using data from ME and a small number of data samples from MI. Hence, it is presented the feasibility of BCI learning strategies that can sufficiently learn deep learning with a few amount of calibration dataset and time only, with stable performance.
Collapse
|
31
|
Li W, Xu D. Application of intelligent rehabilitation equipment in occupational therapy for enhancing upper limb function of patients in the whole phase of stroke. MEDICINE IN NOVEL TECHNOLOGY AND DEVICES 2021. [DOI: 10.1016/j.medntd.2021.100097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022] Open
|
32
|
Lee M, Jeong JH, Kim YH, Lee SW. Decoding Finger Tapping With the Affected Hand in Chronic Stroke Patients During Motor Imagery and Execution. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1099-1109. [PMID: 34101595 DOI: 10.1109/tnsre.2021.3087506] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
In stroke rehabilitation, motor imagery based on a brain-computer interface is an extremely useful method to control an external device and utilize neurofeedback. Many studies have reported on the classification performance of motor imagery to decode individual fingers in stroke patients compared with healthy controls. However, classification performance for a given limb is still low because the differences between patients owing to brain reorganization after stroke are not considered. We used electroencephalography signals from eleven healthy controls and eleven stroke patients in this study. The subjects performed a finger tapping task during motor execution, and motor imagery was performed with the dominant and affected hands in the healthy controls and stroke patients, respectively. All fingers except for the thumb were classified using the proposed framework based on a voting module. The averaged four-class accuracies during motor execution and motor imagery were 53.16 ± 8.42% and 46.94 ± 5.99% for the healthy controls and 53.17 ± 14.09% and 66.00 ± 14.96% for the stroke patients, respectively. Importantly, the classification accuracies in the stroke patients were statistically higher than those in healthy controls during motor imagery. However, there was no significant difference between the accuracies of motor execution and motor imagery. These findings show the potential for high classification performance for a given limb during motor imagery in stroke patients. These results can also provide insights into controlling an external device on the basis of a brain-computer interface.
Collapse
|