1
|
Kim JS, Kim H, Chung CK, Kim JS. Dual model transfer learning to compensate for individual variability in brain-computer interface. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 254:108294. [PMID: 38943984 DOI: 10.1016/j.cmpb.2024.108294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 06/14/2024] [Accepted: 06/16/2024] [Indexed: 07/01/2024]
Abstract
BACKGROUND AND OBJECTIVE Recent advancements in brain-computer interface (BCI) technology have seen a significant shift towards incorporating complex decoding models such as deep neural networks (DNNs) to enhance performance. These models are particularly crucial for sophisticated tasks such as regression for decoding arbitrary movements. However, these BCI models trained and tested on individual data often face challenges with limited performance and generalizability across different subjects. This limitation is primarily due to a tremendous number of parameters of DNN models. Training complex models demands extensive datasets. Nevertheless, group data from many subjects may not produce sufficient decoding performance because of inherent variability in neural signals both across individuals and over time METHODS: To address these challenges, this study proposed a transfer learning approach that could effectively adapt to subject-specific variability in cortical regions. Our method involved training two separate movement decoding models: one on individual data and another on pooled group data. We then created a salience map for each cortical region from the individual model, which helped us identify the input's contribution variance across subjects. Based on the contribution variance, we combined individual and group models using a modified knowledge distillation framework. This approach allowed the group model to be universally applicable by assigning greater weights to input data, while the individual model was fine-tuned to focus on areas with significant individual variance RESULTS: Our combined model effectively encapsulated individual variability. We validated this approach with nine subjects performing arm-reaching tasks, with our method outperforming (mean correlation coefficient, r = 0.75) both individual (r = 0.70) and group models (r = 0.40) in decoding performance. In particular, there were notable improvements in cases where individual models showed low performances (e.g., r = 0.50 in the individual decoder to r = 0.61 in the proposed decoder) CONCLUSIONS: These results not only demonstrate the potential of our method for robust BCI, but also underscore its ability to generalize individual data for broader applicability.
Collapse
Affiliation(s)
- Jun Su Kim
- Dept. of Brain and Cognitive Sciences, Seoul National University, Seoul, Republic of Korea; Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea
| | - HongJune Kim
- Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea
| | - Chun Kee Chung
- Dept. of Brain and Cognitive Sciences, Seoul National University, Seoul, Republic of Korea; Interdisciplinary Program in Neuroscience, Seoul National University, Seoul, Republic of Korea; Dept. of Neurosurgery, Seoul National University Hospital, Seoul, Republic of Korea; Neuroscience Research Institute, Seoul National University Medical Research Center, Seoul, Republic of Korea
| | - June Sic Kim
- Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea; Research Institute of Biomedical Science & Technology, Konkuk University, Seoul, Republic of Korea.
| |
Collapse
|
2
|
Yeo TS, Kim JS, Kim HJ, Chung CK. Macroscopic brain dynamics beyond contralateral primary motor cortex for movement prediction. Neuroimage 2024; 297:120727. [PMID: 39069222 DOI: 10.1016/j.neuroimage.2024.120727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2024] [Revised: 04/12/2024] [Accepted: 07/08/2024] [Indexed: 07/30/2024] Open
Abstract
This study investigates the complex relationship between upper limb movement direction and macroscopic neural signals in the brain, which is critical for understanding brain-computer interfaces (BCI). Conventional BCI research has primarily focused on a local area, such as the contralateral primary motor cortex (M1), relying on the population-based decoding method with microelectrode arrays. In contrast, macroscopic approaches such as electroencephalography (EEG) and magnetoencephalography (MEG) utilize numerous electrodes to cover broader brain regions. This study probes the potential differences in the mechanisms of microscopic and macroscopic methods. It is important to determine which neural activities effectively predict movements. To investigate this, we analyzed MEG data from nine right-handed participants while performing arm-reaching tasks. We employed dynamic statistical parametric mapping (dSPM) to estimate source activity and built a decoding model composed of long short-term memory (LSTM) and a multilayer perceptron to predict movement trajectories. This model achieved a high correlation coefficient of 0.79 between actual and predicted trajectories. Subsequently, we identified brain regions sensitive to predicting movement direction using the integrated gradients (IG) method, which assesses the predictive contribution of each source activity. The resulting salience map demonstrated a distribution without significant differences across motor-related regions, including M1. Predictions based solely on M1 activity yielded a correlation coefficient of 0.42, nearly half as effective as predictions incorporating all source activities. This suggests that upper limb movements are influenced by various factors such as movement coordination, planning, body and target position recognition, and control, beyond simple muscle activity. All of the activities are needed in the decoding model using macroscopic signals. Our findings also revealed that contralateral and ipsilateral hemispheres contribute equally to movement prediction, implying that BCIs could potentially benefit patients with brain damage in the contralateral hemisphere by utilizing brain signals from the ipsilateral hemisphere. In conclusion, this study demonstrates that macroscopic activity from large brain regions significantly contributes to predicting upper limb movement. Non-invasive BCI systems would require a comprehensive collection of neural signals from multiple brain regions.
Collapse
Affiliation(s)
- Tae Soo Yeo
- Dept. of Brain and Cognitive Sciences, Seoul National University, Seoul, Republic of Korea; Clinical Research Institute, Konkuk University Medical Center, Seoul, Republic of Korea
| | - June Sic Kim
- Clinical Research Institute, Konkuk University Medical Center, Seoul, Republic of Korea; Research Institute of Biomedical Science and Technology, Konkuk University, Seoul, Republic of Korea.
| | - Hong June Kim
- Clinical Research Institute, Konkuk University Medical Center, Seoul, Republic of Korea
| | - Chun Kee Chung
- Neuroscience Research Institute, Seoul National University Medical Research Center, Seoul, Republic of Korea
| |
Collapse
|
3
|
Khaliq fard M, Fallah A, Maleki A. Decoding temporal muscle synergy patterns based on brain activity for upper extremity in ADL movements. Cogn Neurodyn 2024; 18:349-356. [PMID: 38699620 PMCID: PMC11061060 DOI: 10.1007/s11571-022-09885-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Revised: 02/11/2022] [Accepted: 02/17/2022] [Indexed: 11/03/2022] Open
Abstract
Muscle synergies have been hypothesized as specific predefined motor primitives that the central nervous system can reduce the complexity of motor control by using them, but how these are expressed in brain activity is ambiguous yet. The main purpose of this paper is to develop synergy-based neural decoding of motor primitives, so for the first time, brain activity and muscle synergy map of the upper extremity was investigated in the activity of daily living movements. To find the relationship between brain activities and muscle synergies, electroencephalogram (EEG) and electromyogram (EMG) signals were acquired simultaneously during activities of daily living. To extract the maximum correlation of neural commands with muscle synergies, application of a combined partial least squares and canonical correlation analysis (PLS-CCA) method was proposed. The Elman neural network was used to decode the relationship between extracted motor commands and muscle synergies. The performance of proposed method was evaluated with tenfold cross-validation and muscle synergy estimation of brain activity with R, VAF, and MSE of 84 ± 2.6%, 70 ± 4.7%, and 0.00011 ± 0.00002 were quantified respectively. Furthermore, the similarity between actual and reconstructed muscle activations was achieved more than 92% for correlation coefficient. To compare with the existing methods, our results showed significantly more accuracy of the model performance. Our results confirm that use of the expression of muscle synergies in brain activity can estimate the neural decoding performance for motor control that can be used to develop neurorehabilitation tools such as neuroprosthesis.
Collapse
Affiliation(s)
- Mahdie Khaliq fard
- Biomedical Engineering Department, Amirkabir University of Technology, Tehran, Iran
| | - Ali Fallah
- Biomedical Engineering Department, Amirkabir University of Technology, Tehran, Iran
| | | |
Collapse
|
4
|
Borra D, Mondini V, Magosso E, Müller-Putz GR. Decoding movement kinematics from EEG using an interpretable convolutional neural network. Comput Biol Med 2023; 165:107323. [PMID: 37619325 DOI: 10.1016/j.compbiomed.2023.107323] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Revised: 07/28/2023] [Accepted: 08/07/2023] [Indexed: 08/26/2023]
Abstract
Continuous decoding of hand kinematics has been recently explored for the intuitive control of electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs). Deep neural networks (DNNs) are emerging as powerful decoders, for their ability to automatically learn features from lightly pre-processed signals. However, DNNs for kinematics decoding lack in the interpretability of the learned features and are only used to realize within-subject decoders without testing other training approaches potentially beneficial for reducing calibration time, such as transfer learning. Here, we aim to overcome these limitations by using an interpretable convolutional neural network (ICNN) to decode 2-D hand kinematics (position and velocity) from EEG in a pursuit tracking task performed by 13 participants. The ICNN is trained using both within-subject and cross-subject strategies, and also testing the feasibility of transferring the knowledge learned on other subjects on a new one. Moreover, the network eases the interpretation of learned spectral and spatial EEG features. Our ICNN outperformed most of the other state-of-the-art decoders, showing the best trade-off between performance, size, and training time. Furthermore, transfer learning improved kinematics prediction in the low data regime. The network attributed the highest relevance for decoding to the delta-band across all subjects, and to higher frequencies (alpha, beta, low-gamma) for a cluster of them; contralateral central and parieto-occipital sites were the most relevant, reflecting the involvement of sensorimotor, visual and visuo-motor processing. The approach improved the quality of kinematics prediction from the EEG, at the same time allowing interpretation of the most relevant spectral and spatial features.
Collapse
Affiliation(s)
- Davide Borra
- Department of Electrical, Electronic and Information Engineering "Guglielmo Marconi" (DEI), University of Bologna, Cesena Campus, Cesena, Italy.
| | - Valeria Mondini
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Elisa Magosso
- Department of Electrical, Electronic and Information Engineering "Guglielmo Marconi" (DEI), University of Bologna, Cesena Campus, Cesena, Italy; Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Bologna, Italy; Interdepartmental Center for Industrial Research on Health Sciences & Technologies, University of Bologna, Bologna, Italy
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria; BioTechMed, Graz, Austria
| |
Collapse
|
5
|
Wang P, Cao X, Zhou Y, Gong P, Yousefnezhad M, Shao W, Zhang D. A comprehensive review on motion trajectory reconstruction for EEG-based brain-computer interface. Front Neurosci 2023; 17:1086472. [PMID: 37332859 PMCID: PMC10272365 DOI: 10.3389/fnins.2023.1086472] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Accepted: 05/03/2023] [Indexed: 06/20/2023] Open
Abstract
The advance in neuroscience and computer technology over the past decades have made brain-computer interface (BCI) a most promising area of neurorehabilitation and neurophysiology research. Limb motion decoding has gradually become a hot topic in the field of BCI. Decoding neural activity related to limb movement trajectory is considered to be of great help to the development of assistive and rehabilitation strategies for motor-impaired users. Although a variety of decoding methods have been proposed for limb trajectory reconstruction, there does not yet exist a review that covers the performance evaluation of these decoding methods. To alleviate this vacancy, in this paper, we evaluate EEG-based limb trajectory decoding methods regarding their advantages and disadvantages from a variety of perspectives. Specifically, we first introduce the differences in motor execution and motor imagery in limb trajectory reconstruction with different spaces (2D and 3D). Then, we discuss the limb motion trajectory reconstruction methods including experiment paradigm, EEG pre-processing, feature extraction and selection, decoding methods, and result evaluation. Finally, we expound on the open problem and future outlooks.
Collapse
Affiliation(s)
| | | | | | | | | | - Wei Shao
- *Correspondence: Wei Shao, ; Daoqiang Zhang,
| | | |
Collapse
|
6
|
Srisrisawang N, Müller-Putz GR. Transfer Learning in Trajectory Decoding: Sensor or Source Space? SENSORS (BASEL, SWITZERLAND) 2023; 23:3593. [PMID: 37050653 PMCID: PMC10098869 DOI: 10.3390/s23073593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/19/2023] [Revised: 03/08/2023] [Accepted: 03/28/2023] [Indexed: 06/19/2023]
Abstract
In this study, across-participant and across-session transfer learning was investigated to minimize the calibration time of the brain-computer interface (BCI) system in the context of continuous hand trajectory decoding. We reanalyzed data from a study with 10 able-bodied participants across three sessions. A leave-one-participant-out (LOPO) model was utilized as a starting model. Recursive exponentially weighted partial least squares regression (REW-PLS) was employed to overcome the memory limitation due to the large pool of training data. We considered four scenarios: generalized with no update (Gen), generalized with cumulative update (GenC), and individual models with cumulative (IndC) and non-cumulative (Ind) updates, with each one trained with sensor-space features or source-space features. The decoding performance in generalized models (Gen and GenC) was lower than the chance level. In individual models, the cumulative update (IndC) showed no significant improvement over the non-cumulative model (Ind). The performance showed the decoder's incapability to generalize across participants and sessions in this task. The results suggested that the best correlation could be achieved with the sensor-space individual model, despite additional anatomical information in the source-space features. The decoding pattern showed a more localized pattern around the precuneus over three sessions in Ind models.
Collapse
Affiliation(s)
- Nitikorn Srisrisawang
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, 8010 Graz, Austria
| | - Gernot R. Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, 8010 Graz, Austria
- BioTechMed Graz, 8010 Graz, Austria
| |
Collapse
|
7
|
Shi Y, Ma Q, Feng C, Wang M, Wang H, Li B, Fang J, Ma S, Guo X, Li T. Microstate feature fusion for distinguishing AD from MCI. Health Inf Sci Syst 2022; 10:16. [PMID: 35911952 PMCID: PMC9325930 DOI: 10.1007/s13755-022-00186-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Accepted: 07/12/2022] [Indexed: 11/25/2022] Open
Abstract
Electroencephalogram (EEG) microstates provide powerful tools for identifying EEG features due to their rich temporal information. In this study, we tested whether microstates can measure the severity of Alzheimer's disease (AD) and mild cognitive impairment (MCI) in patients and effectively distinguish AD from MCI. We defined two features using transition probabilities (TPs), and one was used to evaluate between-group differences in microstate parameters to assess the within-group consistency of TPs and MMSE scores. Another feature was used to distinguish AD from MCI in machine learning models. Tests showed that there were between-group differences in the temporal characteristics of microstates, and some kinds of TPs were significantly correlated with MMSE scores within groups. Based on our newly defined time-factor transition probabilities (TTPs) feature and partial accumulation strategy, we obtained promising scores for accuracy, sensitivity, and specificity of 0.938, 0.923, and 0.947, respectively. These results provide evidence for microstates as a neurobiological marker of AD.
Collapse
Affiliation(s)
- Yupan Shi
- Institute of Applied Mathematics, Hebei Academy of Sciences, Shijiazhuang, China
- Hebei Authentication Technology Engineering Research Center, Shijiazhuang, China
| | - Qinying Ma
- Department of Neurology, The First Hospital of Hebei Medical University, Shijiazhuang, China
- Brain Aging and Cognitive Neuroscience Key Laboratory of Hebei Province, Shijiazhuang, China
| | - Chunyu Feng
- Institute of Applied Mathematics, Hebei Academy of Sciences, Shijiazhuang, China
- Hebei Authentication Technology Engineering Research Center, Shijiazhuang, China
| | - Mingwei Wang
- Department of Neurology, The First Hospital of Hebei Medical University, Shijiazhuang, China
- Brain Aging and Cognitive Neuroscience Key Laboratory of Hebei Province, Shijiazhuang, China
| | - Hualong Wang
- Department of Neurology, The First Hospital of Hebei Medical University, Shijiazhuang, China
- Brain Aging and Cognitive Neuroscience Key Laboratory of Hebei Province, Shijiazhuang, China
| | - Bing Li
- Department of Neurology, The First Hospital of Hebei Medical University, Shijiazhuang, China
- Brain Aging and Cognitive Neuroscience Key Laboratory of Hebei Province, Shijiazhuang, China
| | - Jiyu Fang
- Department of Neurology, The First Hospital of Hebei Medical University, Shijiazhuang, China
- Brain Aging and Cognitive Neuroscience Key Laboratory of Hebei Province, Shijiazhuang, China
| | - Shaochen Ma
- Department of Neurology, The First Hospital of Hebei Medical University, Shijiazhuang, China
- Brain Aging and Cognitive Neuroscience Key Laboratory of Hebei Province, Shijiazhuang, China
| | - Xin Guo
- Department of Neurology, The First Hospital of Hebei Medical University, Shijiazhuang, China
- Brain Aging and Cognitive Neuroscience Key Laboratory of Hebei Province, Shijiazhuang, China
| | - Tongliang Li
- Institute of Applied Mathematics, Hebei Academy of Sciences, Shijiazhuang, China
- Institute of Biology, Hebei Academy of Sciences, Shijiazhuang, China
- Hebei Authentication Technology Engineering Research Center, Shijiazhuang, China
| |
Collapse
|
8
|
Cisotto G, Capuzzo M, Guglielmi AV, Zanella A. Feature stability and setup minimization for EEG-EMG-enabled monitoring systems. EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING 2022; 2022:103. [PMID: 36320592 PMCID: PMC9612609 DOI: 10.1186/s13634-022-00939-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Accepted: 10/18/2022] [Indexed: 06/16/2023]
Abstract
Delivering health care at home emerged as a key advancement to reduce healthcare costs and infection risks, as during the SARS-Cov2 pandemic. In particular, in motor training applications, wearable and portable devices can be employed for movement recognition and monitoring of the associated brain signals. This is one of the contexts where it is essential to minimize the monitoring setup and the amount of data to collect, process, and share. In this paper, we address this challenge for a monitoring system that includes high-dimensional EEG and EMG data for the classification of a specific type of hand movement. We fuse EEG and EMG into the magnitude squared coherence (MSC) signal, from which we extracted features using different algorithms (one from the authors) to solve binary classification problems. Finally, we propose a mapping-and-aggregation strategy to increase the interpretability of the machine learning results. The proposed approach provides very low mis-classification errors ( < 0.1 ), with very few and stable MSC features ( < 10 % of the initial set of available features). Furthermore, we identified a common pattern across algorithms and classification problems, i.e., the activation of the centro-parietal brain areas and arm's muscles in 8-80 Hz frequency band, in line with previous literature. Thus, this study represents a step forward to the minimization of a reliable EEG-EMG setup to enable gesture recognition.
Collapse
Affiliation(s)
- Giulia Cisotto
- Department of Information Engineering, University of Padova, Via Gradenigo, 6, 35121 Padova, Italy
- Inter-University Consortium for Telecommunications (CNIT), Padova, Italy
- Department of Informatics, Systems and Communications, University of Milano-Bicocca, Viale Sarca, 336, 20126 Milano, Italy
| | - Martina Capuzzo
- Department of Information Engineering, University of Padova, Via Gradenigo, 6, 35121 Padova, Italy
- Human Inspired Technologies Research Center, University of Padova, Via Luzzatti, 4, 35121 Padova, Italy
| | - Anna Valeria Guglielmi
- Department of Information Engineering, University of Padova, Via Gradenigo, 6, 35121 Padova, Italy
| | - Andrea Zanella
- Department of Information Engineering, University of Padova, Via Gradenigo, 6, 35121 Padova, Italy
- Inter-University Consortium for Telecommunications (CNIT), Padova, Italy
- Human Inspired Technologies Research Center, University of Padova, Via Luzzatti, 4, 35121 Padova, Italy
| |
Collapse
|
9
|
Jang SJ, Yang YJ, Ryun S, Kim JS, Chung CK, Jeong J. Decoding trajectories of imagined hand movement using electrocorticograms for brain-machine interface. J Neural Eng 2022; 19. [PMID: 35985293 DOI: 10.1088/1741-2552/ac8b37] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Accepted: 08/19/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Reaching hand movement is an important motor skill actively examined in brain-computer interface (BCI). Among various components of movement analyzed is the hand's trajectory, which describes the hand's continuous positions in three-dimensional space. While a large body of studies have investigated the decoding of real movements and the reconstruction of real hand movement trajectories from neural signals, fewer studies have attempted to decode the trajectory of imagined hand movement. To develop BCI systems for patients with hand motor dysfunctions, the systems essentially require to achieve movement-free control of external devices, which is only possible through successful decoding of purely imagined hand movement. APPROACH To achieve this goal, this study used a machine learning technique (i.e., the variational Bayesian least square) to analyze the electrocorticogram (ECoG) of eighteen epilepsy patients obtained from when they performed movement execution (ME) and kinesthetic movement imagination (KMI) of the reach-and-grasp hand action. MAIN RESULTS The variational Bayesian decoding model was able to successfully predict the imagined trajectories of hand movement significantly above chance level. The Pearson's correlation coefficient between imagined and predicted trajectories was 0.3393 and 0.4936 for the KMI (KMI trials only) and MEKMI paradigm (alternating trials of ME and KMI) respectively. SIGNIFICANCE This study demonstrated a high accuracy of prediction for trajectories of imagined hand movement, and more importantly, higher decoding accuracy of imagined trajectories in the MEKMI paradigm than in the KMI paradigm solely.
Collapse
Affiliation(s)
- Sang Jin Jang
- Bio and Brain Engineering, Korea Advanced Institute of Science and Technology, 411 E16-1(YBS Building) Korea Advanced Institute of Science and Technology, 291 Daehak-ro, Yuseong-gu, Daejeon, South Korea 34141, Daejeon, Daejeon, 34141, Korea (the Republic of)
| | - Yu Jin Yang
- Seoul National University College of Natural Sciences, 103, Daehak-ro, Jongno-gu, Seoul, Republic of Korea, Seoul, 03080, Korea (the Republic of)
| | - Seokyun Ryun
- Seoul National University College of Natural Sciences, 103, Daehak-ro, Jongno-gu, Seoul, Republic of Korea, Seoul, 03080, Korea (the Republic of)
| | - June Sic Kim
- Seoul National University College of Natural Sciences, 103, Daehak-ro, Jongno-gu, Seoul, Republic of Korea, Seoul, 03080, Korea (the Republic of)
| | - Chun Kee Chung
- Seoul National University College of Natural Sciences, 103, Daehak-ro, Jongno-gu, Seoul, Republic of Korea, Seoul, 03080, Korea (the Republic of)
| | - Jaeseung Jeong
- Bio and Brain Engineering, Korea Advanced Institute of Science and Technology, 514 E16-1(YBS Building) Korea Advanced Institute of Science and Technology, 291 Daehak-ro, Yuseong-gu, Daejeon, South Korea 34141, Daejeon, 34141, Korea (the Republic of)
| |
Collapse
|
10
|
Gangadharan K S, Vinod AP. Direction decoding of imagined hand movements using subject-specific features from parietal EEG. J Neural Eng 2022; 19. [PMID: 35901779 DOI: 10.1088/1741-2552/ac8501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Accepted: 07/28/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Research on decoding brain signals for controlling external devices is rapidly emerging, owing to its versatile potential applications including neuro-prosthetic control and neurorehabilitation. Electroencephalogram (EEG)-based non-invasive Brain Computer Interface (BCI) systems decode brain signals to establish an augmented communication and control pathway between the brain and the computer. The development of an efficient BCI system requires accurate decoding of neural activity underlying user's intentions. This study investigates the directional tuning of Electroencephalogram (EEG) characteristics from posterior parietal region, associated with bidirectional hand movement imagination (motor imagery) in left and right directions. APPROACH The imagined movement directions of the chosen hand were decoded using a combination of envelope and phase features derived from parietal EEG of both hemispheres. The proposed algorithm uses wavelet for spectral decomposition, and discriminative subject-specific subband levels are identified based on Fisher analysis of envelope and phase features. The selected features from the discriminative subband levels are used for classifying left and right motor imagery directions of the hand using Support Vector Machine Classifier. Furthermore, the performance of the proposed algorithm is evaluated by incorporating a maximum variance-based EEG time bin selection algorithm. MAIN RESULTS With the time bin selection approach using subject-specific features, the proposed algorithm yielded an average left vs right motor imagery direction decoding accuracy of 73.33% across 15 healthy subjects. In addition, decoding accuracy offered by the phase features was higher than that of the envelope features, indicating the importance of phase features in MI kinematics decoding. SIGNIFICANCE The results reveal the significance of parietal EEG in decoding imagined kinematics and open new possibilities for future BCI research.
Collapse
Affiliation(s)
- Sagila Gangadharan K
- Electrical Engineering, Indian Institute of Technology Palakkad, Indian Institute of Technology Palakkad, Pudussery West, Palakkad, Palakkad, Kerala, 678623, INDIA
| | - A P Vinod
- Singapore Institute of Technology InfoComm Technology, 10 Dover Drive, Singapore 138683, Singapore, 138683, SINGAPORE
| |
Collapse
|
11
|
Pei D, Olikkal P, Adali T, Vinjamuri R. Reconstructing Synergy-Based Hand Grasp Kinematics from Electroencephalographic Signals. SENSORS (BASEL, SWITZERLAND) 2022; 22:5349. [PMID: 35891029 PMCID: PMC9318424 DOI: 10.3390/s22145349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 07/15/2022] [Accepted: 07/16/2022] [Indexed: 06/15/2023]
Abstract
Brain-machine interfaces (BMIs) have become increasingly popular in restoring the lost motor function in individuals with disabilities. Several research studies suggest that the CNS may employ synergies or movement primitives to reduce the complexity of control rather than controlling each DoF independently, and the synergies can be used as an optimal control mechanism by the CNS in simplifying and achieving complex movements. Our group has previously demonstrated neural decoding of synergy-based hand movements and used synergies effectively in driving hand exoskeletons. In this study, ten healthy right-handed participants were asked to perform six types of hand grasps representative of the activities of daily living while their neural activities were recorded using electroencephalography (EEG). From half of the participants, hand kinematic synergies were derived, and a neural decoder was developed, based on the correlation between hand synergies and corresponding cortical activity, using multivariate linear regression. Using the synergies and the neural decoder derived from the first half of the participants and only cortical activities from the remaining half of the participants, their hand kinematics were reconstructed with an average accuracy above 70%. Potential applications of synergy-based BMIs for controlling assistive devices in individuals with upper limb motor deficits, implications of the results in individuals with stroke and the limitations of the study were discussed.
Collapse
|
12
|
Le DT, Ogawa H, Tsuyuhara M, Watanabe K, Watanabe T, Ochi R, Nishijo H, Mihara M, Fujita N, Urakawa S. Coupled versus decoupled visuomotor feedback: Differential frontoparietal activity during curved reach planning on simultaneous functional near-infrared spectroscopy and electroencephalography. Brain Behav 2022; 12:e2681. [PMID: 35701382 PMCID: PMC9304848 DOI: 10.1002/brb3.2681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/18/2022] [Revised: 04/20/2022] [Accepted: 06/01/2022] [Indexed: 11/10/2022] Open
Abstract
INTRODUCTION Interacting with the environment requires the planning and execution of reach-to-target movements along given reach trajectory paths. Human neural mechanisms for the motor planning of linear, or point-to-point, reaching movements are relatively well studied. However, the corresponding representations for curved and more complex reaching movements require further investigation. Additionally, the visual and proprioceptive feedback of hand positioning can be spatially and sequentially coupled in alignment (e.g., directly reaching for an object), termed coupled visuomotor feedback, or spatially decoupled (e.g., dragging the computer mouse forward to move the cursor upward), termed decoupled visuomotor feedback. During reach planning, visuomotor processing routes may differ across feedback types. METHODS We investigated the involvement of the frontoparietal regions, including the superior parietal lobule (SPL), dorsal premotor cortex (PMd), and dorsolateral prefrontal cortex (dlPFC), in curved reach planning under different feedback conditions. Participants engaged in two delayed-response reaching tasks with identical starting and target position sets but different reach trajectory paths (linear or curved) under two feedback conditions (coupled or decoupled). Neural responses in frontoparietal regions were analyzed using a combination of functional near-infrared spectroscopy and electroencephalography. RESULTS The results revealed that, regarding the cue period, curved reach planning had a higher hemodynamic response in the left SPL and bilateral PMd and a smaller high-beta power in the left parietal regions than linear reach planning. Regarding the delay period, higher hemodynamic responses during curved reach planning were observed in the right dlPFC for decoupled feedback than those for coupled feedback. CONCLUSION These findings suggest the crucial involvement of both SPL and PMd activities in trajectory-path processing for curved reach planning. Moreover, the dlPFC may be especially involved in the planning of curved reaching movements under decoupled feedback conditions. Thus, this study provides insight into the neural mechanisms underlying reaching function via different feedback conditions.
Collapse
Affiliation(s)
- Duc Trung Le
- Department of Musculoskeletal Functional Research and Regeneration, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, Japan
| | - Hiroki Ogawa
- Department of Musculoskeletal Functional Research and Regeneration, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, Japan
| | - Masato Tsuyuhara
- Department of Musculoskeletal Functional Research and Regeneration, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, Japan
| | - Kazuki Watanabe
- Department of Musculoskeletal Functional Research and Regeneration, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, Japan
| | - Tatsunori Watanabe
- Department of Sensorimotor Neuroscience, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, Japan
| | - Ryosuke Ochi
- Department of Musculoskeletal Functional Research and Regeneration, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, Japan
| | - Hisao Nishijo
- Department of System Emotional Science, Graduate School of Medicine and Pharmaceutical Science, University of Toyama, Toyama, Japan.,Research Center for Idling Brain Science (RCIBS), University of Toyama, Toyama, Japan
| | - Masahito Mihara
- Department of Neurology, Kawasaki Medical School, Okayama, Japan
| | - Naoto Fujita
- Department of Musculoskeletal Functional Research and Regeneration, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, Japan
| | - Susumu Urakawa
- Department of Musculoskeletal Functional Research and Regeneration, Graduate School of Biomedical and Health Sciences, Hiroshima University, Hiroshima, Japan
| |
Collapse
|
13
|
Hosseini SM, Shalchyan V. Continuous Decoding of Hand Movement From EEG Signals Using Phase-Based Connectivity Features. Front Hum Neurosci 2022; 16:901285. [PMID: 35845243 PMCID: PMC9279670 DOI: 10.3389/fnhum.2022.901285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Accepted: 06/06/2022] [Indexed: 11/13/2022] Open
Abstract
The principal goal of the brain-computer interface (BCI) is to translate brain signals into meaningful commands to control external devices or neuroprostheses to restore lost functions of patients with severe motor disabilities. The invasive recording of brain signals involves numerous health issues. Therefore, BCIs based on non-invasive recording modalities such as electroencephalography (EEG) are safer and more comfortable for the patients. The BCI requires reconstructing continuous movement parameters such as position or velocity for practical application of neuroprostheses. The BCI studies in continuous decoding have extensively relied on extracting features from the amplitude of brain signals, whereas the brain connectivity features have rarely been explored. This study aims to investigate the feasibility of using phase-based connectivity features in decoding continuous hand movements from EEG signals. To this end, the EEG data were collected from seven healthy subjects performing a 2D center-out hand movement task in four orthogonal directions. The phase-locking value (PLV) and magnitude-squared coherence (MSC) are exploited as connectivity features along with multiple linear regression (MLR) for decoding hand positions. A brute-force search approach is employed to find the best channel pairs for extracting features related to hand movements. The results reveal that the regression models based on PLV and MSC features achieve the average Pearson correlations of 0.43 ± 0.03 and 0.42 ± 0.06, respectively, between predicted and actual trajectories over all subjects. The delta and alpha band features have the most contribution in regression analysis. The results also demonstrate that both PLV and MSC decoding models lead to superior results on our data compared to two recently proposed feature extraction methods solely based on the amplitude or phase of recording signals (p < 0.05). This study verifies the ability of PLV and MSC features in the continuous decoding of hand movements with linear regression. Thus, our findings suggest that extracting features based on brain connectivity can improve the accuracy of trajectory decoder BCIs.
Collapse
|
14
|
Wang J, Bi L, Fei W. Using Non-linear Dynamics of EEG Signals to Classify Primary Hand Movement Intent Under Opposite Hand Movement. Front Neurorobot 2022; 16:845127. [PMID: 35574232 PMCID: PMC9097551 DOI: 10.3389/fnbot.2022.845127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2021] [Accepted: 03/29/2022] [Indexed: 11/13/2022] Open
Abstract
Decoding human hand movement from electroencephalograms (EEG) signals is essential for developing an active human augmentation system. Although existing studies have contributed much to decoding single-hand movement direction from EEG signals, decoding primary hand movement direction under the opposite hand movement condition remains open. In this paper, we investigated the neural signatures of the primary hand movement direction from EEG signals under the opposite hand movement and developed a novel decoding method based on non-linear dynamics parameters of movement-related cortical potentials (MRCPs). Experimental results showed significant differences in MRCPs between hand movement directions under an opposite hand movement. Furthermore, the proposed method performed well with an average binary decoding accuracy of 89.48 ± 5.92% under the condition of the opposite hand movement. This study may lay a foundation for the future development of EEG-based human augmentation systems for upper limbs impaired patients and healthy people and open a new avenue to decode other hand movement parameters (e.g., velocity and position) from EEG signals.
Collapse
|
15
|
Pulferer HS, Ásgeirsdóttir B, Mondini V, Sburlea AI, Müller-Putz GR. Continuous 2D trajectory decoding from attempted movement: across-session performance in able-bodied and feasibility in a spinal cord injured participant. J Neural Eng 2022; 19. [PMID: 35443233 DOI: 10.1088/1741-2552/ac689f] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 04/19/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE In people with a cervical spinal cord injury (SCI) or degenerative diseases leading to limited motor function, restoration of upper limb movement has been a goal of the brain-computer interface (BCI) field for decades. Recently, research from our group investigated non-invasive and real-time decoding of continuous movement in able-bodied participants from low-frequency brain signals during a target-tracking task. To advance our setup towards motor-impaired end users, we consequently chose a new paradigm based on attempted movement. APPROACH Here, we present the results of two studies. During the first study, data of ten able-bodied participants completing a target-tracking/shape-tracing task on-screen were investigated in terms of improvements in decoding performance due to user training. In a second study, a spinal cord injured participant underwent the same tasks. To investigate the merit of employing attempted movement in end users with SCI, data of the spinal cord injured participant were recorded twice; once within an observation only condition, and once while simultaneously attempting movement. MAIN RESULTS We observed mean correlation well above chance level for continuous motor decoding based on attempted movement in able-bodied participants. No global improvement over three sessions, both in sensor and source space, could be observed across all participants and movement parameters. In the participant with SCI, decoding performance well above chance was found. SIGNIFICANCE No presence of a learning effect in continuous attempted movement decoding in able-bodied participants could be observed. In contrast, non-significantly varying decoding patterns may promote the use of source space decoding in terms of generalized decoders utilizing transfer learning. Furthermore, above-chance correlations for attempted movement decoding ranging between those of observation only and executed movement were seen in one spinal cord injured participant, suggesting attempted movement decoding as a possible link between feasibility studies in able-bodied and actual applications in motor impaired end users.
Collapse
Affiliation(s)
| | | | - Valeria Mondini
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, Graz, 8010, AUSTRIA
| | - Andreea Ioana Sburlea
- Institute of Neural Engineering, Technische Universitat Graz, Stremayrgasse 16/IV, 8010 Graz, Austria, Graz, 8010, AUSTRIA
| | | |
Collapse
|
16
|
Śliwowski M, Martin M, Souloumiac A, Blanchart P, Aksenova T. Decoding ECoG signal into 3D hand translation using deep learning. J Neural Eng 2022; 19. [PMID: 35287119 DOI: 10.1088/1741-2552/ac5d69] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 03/14/2022] [Indexed: 12/29/2022]
Abstract
Objective.Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.Approach.In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity.Main results.Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.Significance.This study shows that DL-based models could increase the accuracy of BCI systems in the case of 3D hand translation prediction in a tetraplegic subject.
Collapse
Affiliation(s)
- Maciej Śliwowski
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France.,Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France
| | - Matthieu Martin
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France
| | | | | | - Tetiana Aksenova
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France
| |
Collapse
|
17
|
Srisrisawang N, Müller-Putz GR. Applying Dimensionality Reduction Techniques in Source-Space Electroencephalography via Template and Magnetic Resonance Imaging-Derived Head Models to Continuously Decode Hand Trajectories. Front Hum Neurosci 2022; 16:830221. [PMID: 35399364 PMCID: PMC8988304 DOI: 10.3389/fnhum.2022.830221] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Accepted: 02/28/2022] [Indexed: 11/13/2022] Open
Abstract
Several studies showed evidence supporting the possibility of hand trajectory decoding from low-frequency electroencephalography (EEG). However, the decoding in the source space via source localization is scarcely investigated. In this study, we tried to tackle the problem of collinearity due to the higher number of signals in the source space by two folds: first, we selected signals in predefined regions of interest (ROIs); second, we applied dimensionality reduction techniques to each ROI. The dimensionality reduction techniques were computing the mean (Mean), principal component analysis (PCA), and locality preserving projections (LPP). We also investigated the effect of decoding between utilizing a template head model and a subject-specific head model during the source localization. The results indicated that applying source-space decoding with PCA yielded slightly higher correlations and signal-to-noise (SNR) ratios than the sensor-space approach. We also observed slightly higher correlations and SNRs when applying the subject-specific head model than the template head model. However, the statistical tests revealed no significant differences between the source-space and sensor-space approaches and no significant differences between subject-specific and template head models. The decoder with Mean and PCA utilizes information mainly from precuneus and cuneus to decode the velocity kinematics similarly in the subject-specific and template head models.
Collapse
Affiliation(s)
| | - Gernot R. Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
- BioTechMed Graz, Graz, Austria
- *Correspondence: Gernot R. Müller-Putz,
| |
Collapse
|
18
|
Huggins JE, Krusienski D, Vansteensel MJ, Valeriani D, Thelen A, Stavisky S, Norton JJS, Nijholt A, Müller-Putz G, Kosmyna N, Korczowski L, Kapeller C, Herff C, Halder S, Guger C, Grosse-Wentrup M, Gaunt R, Dusang AN, Clisson P, Chavarriaga R, Anderson CW, Allison BZ, Aksenova T, Aarnoutse E. Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier. BRAIN-COMPUTER INTERFACES 2022; 9:69-101. [PMID: 36908334 PMCID: PMC9997957 DOI: 10.1080/2326263x.2021.2009654] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2021] [Accepted: 11/15/2021] [Indexed: 12/11/2022]
Abstract
The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.
Collapse
Affiliation(s)
- Jane E Huggins
- Department of Physical Medicine and Rehabilitation, Department of Biomedical Engineering, Neuroscience Graduate Program, University of Michigan, Ann Arbor, Michigan, United States 325 East Eisenhower, Room 3017; Ann Arbor, Michigan 48108-5744, 734-936-7177
| | - Dean Krusienski
- Department of Biomedical Engineering, Virginia Commonwealth University, Richmond, VA 23219
| | - Mariska J Vansteensel
- UMC Utrecht Brain Center, Dept of Neurosurgery, University Medical Center Utrecht, The Netherlands
| | | | - Antonia Thelen
- eemagine Medical Imaging Solutions GmbH, Berlin, Germany
| | | | - James J S Norton
- National Center for Adaptive Neurotechnologies, US Department of Veterans Affairs, 113 Holland Ave, Albany, NY 12208
| | - Anton Nijholt
- Faculty EEMCS, University of Twente, Enschede, The Netherlands
| | - Gernot Müller-Putz
- Institute of Neural Engineering, GrazBCI Lab, Graz University of Technology, Stremayrgasse 16/4, 8010 Graz, Austria
| | - Nataliya Kosmyna
- Massachusetts Institute of Technology (MIT), Media Lab, E14-548, Cambridge, MA 02139, Unites States
| | | | | | - Christian Herff
- School of Mental Health and Neuroscience, Maastricht University, Maastricht, The Netherlands
| | | | - Christoph Guger
- g.tec medical engineering GmbH/Guger Technologies OG, Austria, Sierningstrasse 14, 4521 Schiedlberg, Austria, +43725122240-0
| | - Moritz Grosse-Wentrup
- Research Group Neuroinformatics, Faculty of Computer Science, Vienna Cognitive Science Hub, Data Science @ Uni Vienna University of Vienna
| | - Robert Gaunt
- Rehab Neural Engineering Labs, Department of Physical Medicine and Rehabilitation, Center for the Neural Basis of Cognition, University of Pittsburgh, Pittsburgh, PA, USA, 3520 5th Ave, Suite 300, Pittsburgh, PA 15213, 412-383-1426
| | - Aliceson Nicole Dusang
- Department of Electrical and Computer Engineering, School of Engineering, Brown University, Carney Institute for Brain Science, Brown University, Providence, RI
- Department of Veterans Affairs Medical Center, Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence, RI
- Center for Neurotechnology and Neurorecovery, Neurology, Massachusetts General Hospital, Boston, MA
| | | | - Ricardo Chavarriaga
- IEEE Standards Association Industry Connections group on neurotechnologies for brain-machine interface, Center for Artificial Intelligence, School of Engineering, ZHAW-Zurich University of Applied Sciences, Switzerland, Switzerland
| | - Charles W Anderson
- Department of Computer Science, Molecular, Cellular and Integrative Neurosience Program, Colorado State University, Fort Collins, CO 80523
| | - Brendan Z Allison
- Dept. of Cognitive Science, Mail Code 0515, University of California at San Diego, La Jolla, United States, 619-534-9754
| | - Tetiana Aksenova
- University Grenoble Alpes, CEA, LETI, Clinatec, Grenoble 38000, France
| | - Erik Aarnoutse
- UMC Utrecht Brain Center, Department of Neurology & Neurosurgery, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands
| |
Collapse
|
19
|
Lee DY, Jeong JH, Lee BH, Lee SW. Motor Imagery Classification Using Inter-Task Transfer Learning via A Channel-Wise Variational Autoencoder-based Convolutional Neural Network. IEEE Trans Neural Syst Rehabil Eng 2022; 30:226-237. [PMID: 35041605 DOI: 10.1109/tnsre.2022.3143836] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Highly sophisticated control based on a brain-computer interface (BCI) requires decoding kinematic information from brain signals. The forearm is a region of the upper limb that is often used in everyday life, but intuitive movements within the same limb have rarely been investigated in previous BCI studies. In this study, we focused on various forearm movement decoding from electroencephalography (EEG) signals using a small number of samples. Ten healthy participants took part in an experiment and performed motor execution (ME) and motor imagery (MI) of the intuitive movement tasks (Dataset I). We propose a convolutional neural network using a channel-wise variational autoencoder (CVNet) based on inter-task transfer learning. We approached that training the reconstructed ME-EEG signals together will also achieve more sufficient classification performance with only a small amount of MI-EEG signals. The proposed CVNet was validated on our own Dataset I and a public dataset, BNCI Horizon 2020 (Dataset II). The classification accuracies of various movements are confirmed to be 0.83 (±0.04) and 0.69 (±0.04) for Dataset I and II, respectively. The results show that the proposed method exhibits performance increases of approximately 0.09~0.27 and 0.08~0.24 compared with the conventional models for Dataset I and II, respectively. The outcomes suggest that the training model for decoding imagined movements can be performed using data from ME and a small number of data samples from MI. Hence, it is presented the feasibility of BCI learning strategies that can sufficiently learn deep learning with a few amount of calibration dataset and time only, with stable performance.
Collapse
|
20
|
Davarinia F, Maleki A. SSVEP-gated EMG-based decoding of elbow angle during goal-directed reaching movement. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103222] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
21
|
du Bois N, Bigirimana AD, Korik A, Kéthina LG, Rutembesa E, Mutabaruka J, Mutesa L, Prasad G, Jansen S, Coyle DH. Neurofeedback with low-cost, wearable electroencephalography (EEG) reduces symptoms in chronic Post-Traumatic Stress Disorder. J Affect Disord 2021; 295:1319-1334. [PMID: 34706446 DOI: 10.1016/j.jad.2021.08.071] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Revised: 07/19/2021] [Accepted: 08/25/2021] [Indexed: 11/30/2022]
Abstract
BACKGROUND The study examines the effectiveness of both neurofeedback and motor-imagery brain-computer interface (BCI) training, which promotes self-regulation of brain activity, using low-cost electroencephalography (EEG)-based wearable neurotechnology outside a clinical setting, as a potential treatment for post-traumatic stress disorder (PTSD) in Rwanda. METHODS Participants received training/treatment sessions along with a pre- and post- intervention clinical assessment, (N = 29; control n = 9, neurofeedback (NF, 7 sessions) n = 10, and motor-imagery (MI, 6 sessions) n = 10). Feedback was presented visually via a videogame. Participants were asked to regulate (NF) or intentionally modulate (MI) brain activity to affect/control the game. RESULTS The NF group demonstrated an increase in resting-state alpha 8-12 Hz bandpower following individual training sessions, termed alpha 'rebound' (Pz channel, p = 0.025, all channels, p = 0.024), consistent with previous research findings. This alpha 'rebound', unobserved in the MI group, produced a clinically relevant reduction in symptom severity in NF group, as revealed in three of seven clinical outcome measures: PCL-5 (p = 0.005), PTSD screen (p = 0.005), and HTQ (p = 0.005). LIMITATIONS Data collection took place in environments that posed difficulties in controlling environmental factors. Nevertheless, this limitation improves ecological validity, as neurotechnology treatments must be deployable outside controlled environments, to be a feasible technological treatment. CONCLUSIONS The study produced the first evidence to support a low-cost, neurotechnological solution for neurofeedback as an effective treatment of PTSD for victims of acute trauma in conflict zones in a developing country.
Collapse
Affiliation(s)
- N du Bois
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom
| | - A D Bigirimana
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom
| | - A Korik
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom
| | - L Gaju Kéthina
- Department of Clinical Psychology, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - E Rutembesa
- Department of Clinical Psychology, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - J Mutabaruka
- Department of Clinical Psychology, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - L Mutesa
- Centre for Human Genetics, School of Medicine and Pharmacy, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - G Prasad
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom
| | - S Jansen
- Department of Clinical Psychology, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - D H Coyle
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom.
| |
Collapse
|
22
|
Wang Y, Luo J, Guo Y, Du Q, Cheng Q, Wang H. Changes in EEG Brain Connectivity Caused by Short-Term BCI Neurofeedback-Rehabilitation Training: A Case Study. Front Hum Neurosci 2021; 15:627100. [PMID: 34366808 PMCID: PMC8336868 DOI: 10.3389/fnhum.2021.627100] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2020] [Accepted: 05/31/2021] [Indexed: 12/22/2022] Open
Abstract
Background In combined with neurofeedback, Motor Imagery (MI) based Brain-Computer Interface (BCI) has been an effective long-term treatment therapy for motor dysfunction caused by neurological injury in the brain (e.g., post-stroke hemiplegia). However, individual neurological differences have led to variability in the single sessions of rehabilitation training. Research on the impact of short training sessions on brain functioning patterns can help evaluate and standardize the short duration of rehabilitation training. In this paper, we use the electroencephalogram (EEG) signals to explore the brain patterns’ changes after a short-term rehabilitation training. Materials and Methods Using an EEG-BCI system, we analyzed the changes in short-term (about 1-h) MI training data with and without visual feedback, respectively. We first examined the EEG signal’s Mu band power’s attenuation caused by Event-Related Desynchronization (ERD). Then we use the EEG’s Event-Related Potentials (ERP) features to construct brain networks and evaluate the training from multiple perspectives: small-scale based on single nodes, medium-scale based on hemispheres, and large-scale based on all-brain. Results Results showed no significant difference in the ERD power attenuation estimation in both groups. But the neurofeedback group’s ERP brain network parameters had substantial changes and trend properties compared to the group without feedback. The neurofeedback group’s Mu band power’s attenuation increased but not significantly (fitting line slope = 0.2, t-test value p > 0.05) after the short-term MI training, while the non-feedback group occurred an insignificant decrease (fitting line slope = −0.4, t-test value p > 0.05). In the ERP-based brain network analysis, the neurofeedback group’s network parameters were attenuated in all scales significantly (t-test value: p < 0.01); while the non-feedback group’s most network parameters didn’t change significantly (t-test value: p > 0.05). Conclusion The MI-BCI training’s short-term effects does not show up in the ERD analysis significantly but can be detected by ERP-based network analysis significantly. Results inspire the efficient evaluation of short-term rehabilitation training and provide a useful reference for subsequent studies.
Collapse
Affiliation(s)
- Youhao Wang
- Academy for Engineering and Technology, Fudan University (FAET), Shanghai, China
| | - Jingjing Luo
- Academy for Engineering and Technology, Fudan University (FAET), Shanghai, China.,Jihua Laboratory, Foshan, China
| | - Yuzhu Guo
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China
| | - Qiang Du
- Academy for Engineering and Technology, Fudan University (FAET), Shanghai, China
| | - Qiying Cheng
- Academy for Engineering and Technology, Fudan University (FAET), Shanghai, China
| | - Hongbo Wang
- Academy for Engineering and Technology, Fudan University (FAET), Shanghai, China
| |
Collapse
|
23
|
Sosnik R, Li Z. Reconstruction of hand, elbow and shoulder actual and imagined trajectories in 3D space using EEG current source dipoles. J Neural Eng 2021; 18. [PMID: 33752186 DOI: 10.1088/1741-2552/abf0d7] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2020] [Accepted: 03/22/2021] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Growing evidence suggests that EEG electrode (sensor) potential time series (PTS) of slow cortical potentials (SCPs) hold motor neural correlates that can be used for motion trajectory prediction (MTP), commonly by multiple linear regression (mLR). It is not yet known whether arm-joint trajectories can be reliably decoded from current sources, computed from sensor data, from which brain areas they can be decoded and using which neural features. APPROACH In this study, the PTS of 44 sensors were fed into sLORETA source localization software to compute current source activity in 30 regions of interest (ROIs) found in a recent meta-analysis to be engaged in action execution, motor imagery and motor preparation. The current sources PTS and band-power time series (BTS) in several frequency bands and time lags were used to predict actual and imagined trajectories in 3D space of the three velocity components of the hand, elbow and shoulder of nine subjects using an mLR model. MAIN RESULTS For all arm joints and movement types, current source SCPs PTS contributed most to trajectory reconstruction with time lags 150ms, 116ms and 84ms providing the highest contribution, and current source BTS in any of the tested frequency bands was not informative. Person's correlation coefficient (r) averaged across movement types, arm joints and velocity components using source data was slightly lower than using sensor data (r=0.25 and r=0.28, respectively). For each ROI, the three current source dipoles had different contribution to the reconstruction of each of the three velocity components. SIGNIFICANCE Overall, our results demonstrate the feasibility of predicting of actual and imagined 3D trajectories of all arm joints from current sources, computed from scalp EEG. These findings may be used by developers of a future BCI as a validated set of contributing ROIs.
Collapse
Affiliation(s)
- Ronen Sosnik
- Electrical, Electronics and Communication Engineering, Holon Institute of Technology, 52 Golomb St., Holon, 5810201, ISRAEL
| | - Zheng Li
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Yingdong Building, Xinjiekouwai Street 19, Beijing Haidian, Beijing, 100875, CHINA
| |
Collapse
|
24
|
Kobler RJ, Sburlea AI, Mondini V, Hirata M, Müller-Putz GR. Distance- and speed-informed kinematics decoding improves M/EEG based upper-limb movement decoder accuracy. J Neural Eng 2020; 17:056027. [PMID: 33146148 DOI: 10.1088/1741-2552/abb3b3] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE One of the main goals in brain-computer interface (BCI) research is the replacement or restoration of lost function in individuals with paralysis. One line of research investigates the inference of movement kinematics from brain activity during different volitional states. A growing number of electroencephalography (EEG) and magnetoencephalography (MEG) studies suggest that information about directional (e.g. velocity) and nondirectional (e.g. speed) movement kinematics is accessible noninvasively. We sought to assess if the neural information associated with both types of kinematics can be combined to improve the decoding accuracy. APPROACH In an offline analysis, we reanalyzed the data of two previous experiments containing the recordings of 34 healthy participants (15 EEG, 19 MEG). We decoded 2D movement trajectories from low-frequency M/EEG signals in executed and observed tracking movements, and compared the accuracy of an unscented Kalman filter (UKF) that explicitly modeled the nonlinear relation between directional and nondirectional kinematics to the accuracies of linear Kalman (KF) and Wiener filters which did not combine both types of kinematics. MAIN RESULTS At the group level, posterior-parietal and parieto-occipital (executed and observed movements) and sensorimotor areas (executed movements) encoded kinematic information. Correlations between the recorded position and velocity trajectories and the UKF decoded ones were on average 0.49 during executed and 0.36 during observed movements. Compared to the other filters, the UKF could achieve the best trade-off between maximizing the signal to noise ratio and minimizing the amplitude mismatch between the recorded and decoded trajectories. SIGNIFICANCE We present direct evidence that directional and nondirectional kinematic information is simultaneously detectable in low-frequency M/EEG signals. Moreover, combining directional and nondirectional kinematic information significantly improves the decoding accuracy upon a linear KF.
Collapse
Affiliation(s)
- Reinmar J Kobler
- Institute of Neural Engineering, Graz University of Technology, Graz 8010, Styria, Austria
| | | | | | | | | |
Collapse
|
25
|
Cooney C, Korik A, Folli R, Coyle D. Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEG. SENSORS 2020; 20:s20164629. [PMID: 32824559 PMCID: PMC7472624 DOI: 10.3390/s20164629] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Revised: 08/10/2020] [Accepted: 08/13/2020] [Indexed: 01/12/2023]
Abstract
Classification of electroencephalography (EEG) signals corresponding to imagined speech production is important for the development of a direct-speech brain-computer interface (DS-BCI). Deep learning (DL) has been utilized with great success across several domains. However, it remains an open question whether DL methods provide significant advances over traditional machine learning (ML) approaches for classification of imagined speech. Furthermore, hyperparameter (HP) optimization has been neglected in DL-EEG studies, resulting in the significance of its effects remaining uncertain. In this study, we aim to improve classification of imagined speech EEG by employing DL methods while also statistically evaluating the impact of HP optimization on classifier performance. We trained three distinct convolutional neural networks (CNN) on imagined speech EEG using a nested cross-validation approach to HP optimization. Each of the CNNs evaluated was designed specifically for EEG decoding. An imagined speech EEG dataset consisting of both words and vowels facilitated training on both sets independently. CNN results were compared with three benchmark ML methods: Support Vector Machine, Random Forest and regularized Linear Discriminant Analysis. Intra- and inter-subject methods of HP optimization were tested and the effects of HPs statistically analyzed. Accuracies obtained by the CNNs were significantly greater than the benchmark methods when trained on both datasets (words: 24.97%, p < 1 × 10-7, chance: 16.67%; vowels: 30.00%, p < 1 × 10-7, chance: 20%). The effects of varying HP values, and interactions between HPs and the CNNs were both statistically significant. The results of HP optimization demonstrate how critical it is for training CNNs to decode imagined speech.
Collapse
Affiliation(s)
- Ciaran Cooney
- Intelligent Systems Research Centre, Ulster University, Londonderry BT48 7JL, UK; (A.K.); (D.C.)
- Correspondence:
| | - Attila Korik
- Intelligent Systems Research Centre, Ulster University, Londonderry BT48 7JL, UK; (A.K.); (D.C.)
| | - Raffaella Folli
- Institute for Research in Social Sciences, Ulster University, Jordanstown BT37 0QB, UK;
| | - Damien Coyle
- Intelligent Systems Research Centre, Ulster University, Londonderry BT48 7JL, UK; (A.K.); (D.C.)
| |
Collapse
|
26
|
Martinez-Cagigal V, Kobler RJ, Mondini V, Hornero R, Muller-Putz GR. Non-linear online low-frequency EEG decoding of arm movements during a pursuit tracking task. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:2981-2985. [PMID: 33018632 DOI: 10.1109/embc44109.2020.9175723] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Decoding upper-limb movements in invasive recordings has become a reality, but neural tuning in non-invasive low-frequency recordings is still under discussion. Recent studies managed to decode movement positions and velocities using linear decoders, even developing an online system. The decoded signals, however, exhibited smaller amplitudes than actual movements, affecting feedback and user experience. Recently, we showed that a non-linear offline decoder can combine directional (e.g., velocity) and non-directional (e.g., speed) information. In this study, it is assessed if the non-linear decoder can be used online to provide real-time feedback. Five healthy subjects were asked to track a moving target by controlling a robotic arm. Initially, the robot was controlled by their right hand; then, the control was gradually switched until it was entirely controlled by the electroencephalogram (EEG). Correlations between actual and decoded movements were generally above chance level. Results suggest that information about speed was also encoded in the EEG, demonstrating that the proposed non-linear decoder is suitable for decoding real-time arm movements.
Collapse
|
27
|
Lennon O, Tonellato M, Del Felice A, Di Marco R, Fingleton C, Korik A, Guanziroli E, Molteni F, Guger C, Otner R, Coyle D. A Systematic Review Establishing the Current State-of-the-Art, the Limitations, and the DESIRED Checklist in Studies of Direct Neural Interfacing With Robotic Gait Devices in Stroke Rehabilitation. Front Neurosci 2020; 14:578. [PMID: 32714127 PMCID: PMC7344195 DOI: 10.3389/fnins.2020.00578] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2019] [Accepted: 05/12/2020] [Indexed: 01/16/2023] Open
Abstract
Background: Stroke is a disease with a high associated disability burden. Robotic-assisted gait training offers an opportunity for the practice intensity levels associated with good functional walking outcomes in this population. Neural interfacing technology, electroencephalography (EEG), or electromyography (EMG) can offer new strategies for robotic gait re-education after a stroke by promoting more active engagement in movement intent and/or neurophysiological feedback. Objectives: This study identifies the current state-of-the-art and the limitations in direct neural interfacing with robotic gait devices in stroke rehabilitation. Methods: A pre-registered systematic review was conducted using standardized search operators that included the presence of stroke and robotic gait training and neural biosignals (EMG and/or EEG) and was not limited by study type. Results: From a total of 8,899 papers identified, 13 articles were considered for the final selection. Only five of the 13 studies received a strong or moderate quality rating as a clinical study. Three studies recorded EEG activity during robotic gait, two of which used EEG for BCI purposes. While demonstrating utility for decoding kinematic and EMG-related gait data, no EEG study has been identified to close the loop between robot and human. Twelve of the studies recorded EMG activity during or after robotic walking, primarily as an outcome measure. One study used multisource information fusion from EMG, joint angle, and force to modify robotic commands in real time, with higher error rates observed during active movement. A novel study identified used EMG data during robotic gait to derive the optimal, individualized robot-driven step trajectory. Conclusions: Wide heterogeneity in the reporting and the purpose of neurobiosignal use during robotic gait training after a stroke exists. Neural interfacing with robotic gait after a stroke demonstrates promise as a future field of study. However, as a nascent area, direct neural interfacing with robotic gait after a stroke would benefit from a more standardized protocol for biosignal collection and processing and for robotic deployment. Appropriate reporting for clinical studies of this nature is also required with respect to the study type and the participants' characteristics.
Collapse
Affiliation(s)
- Olive Lennon
- School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland
| | - Michele Tonellato
- Department of Neuroscience, Rehabilitation Unit, University of Padova, Padova, Italy
| | - Alessandra Del Felice
- Department of Neuroscience, NEUROMOVE-Rehab Laboratory, University of Padova, Padova, Italy
- Padova Neuroscience Center, University of Padova, Padova, Italy
| | - Roberto Di Marco
- Department of Neuroscience, NEUROMOVE-Rehab Laboratory, University of Padova, Padova, Italy
| | - Caitriona Fingleton
- Department of Physiotherapy, Mater Misericordiae University Hospital, Dublin, Ireland
| | - Attila Korik
- Intelligent Systems Research Centre, School of Computing, Engineering and Intelligent Systems, Ulster University, Derry, United Kingdom
| | | | - Franco Molteni
- Villa Beretta Rehabilitation Center, Valduce Hospital, Costa Masnaga, Italy
| | | | - Rupert Otner
- g.tec Medical Engineering GmbH, Schiedlberg, Austria
| | - Damien Coyle
- Intelligent Systems Research Centre, School of Computing, Engineering and Intelligent Systems, Ulster University, Derry, United Kingdom
| |
Collapse
|
28
|
Distinct cortical networks for hand movement initiation and directional processing: An EEG study. Neuroimage 2020; 220:117076. [PMID: 32585349 PMCID: PMC7573539 DOI: 10.1016/j.neuroimage.2020.117076] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2019] [Revised: 06/15/2020] [Accepted: 06/17/2020] [Indexed: 02/07/2023] Open
Abstract
Movement preparation and initiation have been shown to involve large scale brain networks. Recent findings suggest that movement preparation and initiation are represented in functionally distinct cortical networks. In electroencephalographic (EEG) recordings, movement initiation is reflected as a strong negative potential at medial central channels that is phase-locked to the movement onset - the movement-related cortical potential (MRCP). Movement preparation describes the process of transforming high level movement goals to low level commands. An integral part of this transformation process is directional processing (i.e., where to move). The processing of movement direction during visuomotor and oculomotor tasks is associated with medial parieto-occipital cortex (PO) activity, phase-locked to the presentation of potential movement goals. We surmised that the network generating the MRCP (movement initiation) would encode less information about movement direction than the parieto-occipital network processing movement direction. Here, we studied delta band EEG activity during center-out reaching movements (2D; 4 directions) in visuomotor and oculomotor tasks. In 15 healthy participants, we found a consistent representation of movement direction in PO 300–400 ms after the direction cue irrespective of the task. Despite generating the MRCP, sensorimotor areas (SM) encoded less information about the movement direction than PO. Moreover, the encoded directional information in SM was less consistent across participants and specific to the visuomotor task. In a classification approach, we could infer the four movement directions from the delta band EEG activity with moderate accuracies up to 55.9%. The accuracies for cue-aligned data were significantly higher than for movement onset-aligned data in either task, which also suggests a stronger representation of movement direction during movement preparation. Therefore, we present direct evidence that EEG delta band amplitude modulations carry information about both arm movement initiation and movement direction, and that they are represented in two distinct cortical networks. Delta band EEG carries information about arm movement initiation and direction. Movement initiation and direction are represented in two distinct cortical networks. Information about movement direction is primarily encoded in parieto-occipital areas. The activity in parieto-occipital areas is phase-locked to the direction cue.
Collapse
|
29
|
Jeong JH, Shim KH, Kim DJ, Lee SW. Trajectory Decoding of Arm Reaching Movement Imageries for Brain-Controlled Robot Arm System. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:5544-5547. [PMID: 31947110 DOI: 10.1109/embc.2019.8856312] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Development of noninvasive brain-machine interface (BMI) systems based on electroencephalography (EEG), driven by spontaneous movement intentions, is a useful tool for controlling external devices or supporting a neuro- rehabilitation. In this study, we present the possibility of brain-controlled robot arm system using arm trajectory decoding. To do that, we first constructed the experimental system that can acquire the EEG data for not only movement execution (ME) task but also movement imagery (MI) tasks. Five subjects participated in our experiments and performed four directional reaching tasks (Left, right, forward, and backward) in the 3D plane. For robust arm trajectory decoding, we propose a subject-dependent deep neural network (DNN) architecture. The decoding model applies the principle of bi-directional long short-term memory (LSTM) network. As a result, we confirmed the decoding performance (r-value: >0.8) for all X-, Y-, and Z-axis across all subjects in the MI as well as ME tasks. These results show the feasibility of the EEG-based intuitive robot arm control system for high-level tasks (e.g., drink water or moving some objects). Also, we confirm that the proposed method has no much decoding performance variations between ME and MI tasks for the offline analysis. Hence, we will demonstrate that the decoding model is capable of robust trajectory decoding even in a real-time environment.
Collapse
|
30
|
Jeong JH, Shim KH, Kim DJ, Lee SW. Brain-Controlled Robotic Arm System Based on Multi-Directional CNN-BiLSTM Network Using EEG Signals. IEEE Trans Neural Syst Rehabil Eng 2020; 28:1226-1238. [DOI: 10.1109/tnsre.2020.2981659] [Citation(s) in RCA: 83] [Impact Index Per Article: 20.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
31
|
Amo Usanos C, Boquete L, de Santiago L, Barea Navarro R, Cavaliere C. Induced Gamma-Band Activity during Actual and Imaginary Movements: EEG Analysis. SENSORS 2020; 20:s20061545. [PMID: 32168747 PMCID: PMC7146111 DOI: 10.3390/s20061545] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/09/2020] [Revised: 03/03/2020] [Accepted: 03/10/2020] [Indexed: 11/16/2022]
Abstract
The purpose of this paper is to record and analyze induced gamma-band activity (GBA) (30-60 Hz) in cerebral motor areas during imaginary movement and to compare it quantitatively with activity recorded in the same areas during actual movement using a simplified electroencephalogram (EEG). Brain activity (basal activity, imaginary motor task and actual motor task) is obtained from 12 healthy volunteer subjects using an EEG (Cz channel). GBA is analyzed using the mean power spectral density (PSD) value. Event-related synchronization (ERS) is calculated from the PSD values of the basal GBA (GBAb), the GBA of the imaginary movement (GBAim) and the GBA of the actual movement (GBAac). The mean GBAim and GBAac values for the right and left hands are significantly higher than the GBAb value (p = 0.007). No significant difference is detected between mean GBA values during the imaginary and actual movement (p = 0.242). The mean ERS values for the imaginary movement (ERSimM (%) = 23.52) and for the actual movement (ERSacM = 27.47) do not present any significant difference (p = 0.117). We demonstrated that ERS could provide a useful way of indirectly checking the function of neuronal motor circuits activated by voluntary movement, both imaginary and actual. These results, as a proof of concept, could be applied to physiology studies, brain-computer interfaces, and diagnosis of cognitive or motor pathologies.
Collapse
|
32
|
Kobler RJ, Almeida I, Sburlea AI, Müller-Putz GR. Using machine learning to reveal the population vector from EEG signals. J Neural Eng 2020; 17:026002. [PMID: 32048612 DOI: 10.1088/1741-2552/ab7490] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Since the discovery of the population vector that directly relates neural spiking activity with arm movement direction, it has become feasible to control robotic arms and neuroprostheses using invasively recorded brain signals. For non-invasive approaches, a direct relation between human brain signals and arm movement direction is yet to be established. APPROACH Here, we investigated electroencephalographic (EEG) signals in temporal and spectral domains in a continuous, circular arm movement task. Using machine learning methods that respect the linear mixture of brain activity within EEG signals, we show that directional information is represented in the temporal domain in amplitude modulations of the same frequency as the arm movement, and in the spectral domain in power modulations of the 20-24 Hz frequency band. MAIN RESULTS In the temporal domain, the directional information was mainly expressed in primary sensorimotor cortex (SM1) and posterior parietal cortex (PPC) contralateral to the moving arm, while in the spectral domain SM1 and PPC of both hemispheres predicted arm movement direction. The different cortical representations suggest distinct neural representations in both domains. SIGNIFICANCE This direct relation between neural activity and arm movement direction in both domains demonstrates the potential of machine learning to reveal neuroscientific insights about the dynamics of human arm movements.
Collapse
Affiliation(s)
- Reinmar J Kobler
- Institute of Neural Engineering, Graz University of Technology, Graz, Styria 8010, Austria. These authors contributed equally
| | | | | | | |
Collapse
|
33
|
Sosnik R, Ben Zur O. Reconstruction of hand, elbow and shoulder actual and imagined trajectories in 3D space using EEG slow cortical potentials. J Neural Eng 2020; 17:016065. [DOI: 10.1088/1741-2552/ab59a7] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
|
34
|
Zeng H, Shen Y, Hu X, Song A, Xu B, Li H, Wang Y, Wen P. Semi-Autonomous Robotic Arm Reaching With Hybrid Gaze-Brain Machine Interface. Front Neurorobot 2020; 13:111. [PMID: 32038219 PMCID: PMC6992643 DOI: 10.3389/fnbot.2019.00111] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2019] [Accepted: 12/11/2019] [Indexed: 11/13/2022] Open
Abstract
Recent developments in the non-muscular human-robot interface (HRI) and shared control strategies have shown potential for controlling the assistive robotic arm by people with no residual movement or muscular activity in upper limbs. However, most non-muscular HRIs only produce discrete-valued commands, resulting in non-intuitive and less effective control of the dexterous assistive robotic arm. Furthermore, the user commands and the robot autonomy commands usually switch in the shared control strategies of such applications. This characteristic has been found to yield a reduced sense of agency as well as frustration for the user according to previous user studies. In this study, we firstly propose an intuitive and easy-to-learn-and-use hybrid HRI by combing the Brain-machine interface (BMI) and the gaze-tracking interface. For the proposed hybrid gaze-BMI, the continuous modulation of the movement speed via the motor intention occurs seamlessly and simultaneously to the unconstrained movement direction control with the gaze signals. We then propose a shared control paradigm that always combines user input and the autonomy with the dynamic combination regulation. The proposed hybrid gaze-BMI and shared control paradigm were validated for a robotic arm reaching task performed with healthy subjects. All the users were able to employ the hybrid gaze-BMI for moving the end-effector sequentially to reach the target across the horizontal plane while also avoiding collisions with obstacles. The shared control paradigm maintained as much volitional control as possible, while providing the assistance for the most difficult parts of the task. The presented semi-autonomous robotic system yielded continuous, smooth, and collision-free motion trajectories for the end effector approaching the target. Compared to a system without assistances from robot autonomy, it significantly reduces the rate of failure as well as the time and effort spent by the user to complete the tasks.
Collapse
Affiliation(s)
- Hong Zeng
- School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Yitao Shen
- School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Xuhui Hu
- School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Aiguo Song
- State Key Laboratory of Bioelectronics, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Baoguo Xu
- School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Huijun Li
- School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Yanxin Wang
- School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Pengcheng Wen
- AVIC Aeronautics Computing Technique Research Institute, Xi’an, China
| |
Collapse
|
35
|
Yun YD, Jeong JH, Cho JH, Kim DJ, Lee SW. Reconstructing Degree of Forearm Rotation from Imagined movements for BCI-based Robot Hand Control. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:3014-3017. [PMID: 31946523 DOI: 10.1109/embc.2019.8857674] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Brain-computer interface (BCI) is an important tool for rehabilitation and control of an external device (e.g., robot arm or home appliances). Fully reconstruction of upper limb movement from brain signals is one of the critical issues for intuitive BCI. However, decoding of forearm rotation from imagined movements using electroencephalography (EEG) is difficult to decode degree of rotation accurately. In this paper, we reconstructed imagined forearm rotation from low- frequency (0.3-3 Hz) of EEG signals. We selected 20 EEG channel on motor cortex for analysis. Ten healthy subjects participated in our experiment. The subjects performed actual and imagined forearm rotation to reach different targets. We trained a reconstruction decoder which used the EEG signals measured from actual movements and the kinematic information only. Additionally, we applied a long short-term memory (LSTM) network to enhance decoding performances. As a result, we achieved the high correlation performance (Average: 0.67) to decode imagined forearm rotation angle. This result has demonstrated that the reconstruction decoder which is trained by the EEG data from actual movement has effective to decode robustly for the imagined forearm rotation angle.
Collapse
|
36
|
Jenson D, Bowers AL, Hudock D, Saltuklaroglu T. The Application of EEG Mu Rhythm Measures to Neurophysiological Research in Stuttering. Front Hum Neurosci 2020; 13:458. [PMID: 31998103 PMCID: PMC6965028 DOI: 10.3389/fnhum.2019.00458] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2019] [Accepted: 12/13/2019] [Indexed: 11/29/2022] Open
Abstract
Deficits in basal ganglia-based inhibitory and timing circuits along with sensorimotor internal modeling mechanisms are thought to underlie stuttering. However, much remains to be learned regarding the precise manner how these deficits contribute to disrupting both speech and cognitive functions in those who stutter. Herein, we examine the suitability of electroencephalographic (EEG) mu rhythms for addressing these deficits. We review some previous findings of mu rhythm activity differentiating stuttering from non-stuttering individuals and present some new preliminary findings capturing stuttering-related deficits in working memory. Mu rhythms are characterized by spectral peaks in alpha (8-13 Hz) and beta (14-25 Hz) frequency bands (mu-alpha and mu-beta). They emanate from premotor/motor regions and are influenced by basal ganglia and sensorimotor function. More specifically, alpha peaks (mu-alpha) are sensitive to basal ganglia-based inhibitory signals and sensory-to-motor feedback. Beta peaks (mu-beta) are sensitive to changes in timing and capture motor-to-sensory (i.e., forward model) projections. Observing simultaneous changes in mu-alpha and mu-beta across the time-course of specific events provides a rich window for observing neurophysiological deficits associated with stuttering in both speech and cognitive tasks and can provide a better understanding of the functional relationship between these stuttering symptoms. We review how independent component analysis (ICA) can extract mu rhythms from raw EEG signals in speech production tasks, such that changes in alpha and beta power are mapped to myogenic activity from articulators. We review findings from speech production and auditory discrimination tasks demonstrating that mu-alpha and mu-beta are highly sensitive to capturing sensorimotor and basal ganglia deficits associated with stuttering with high temporal precision. Novel findings from a non-word repetition (working memory) task are also included. They show reduced mu-alpha suppression in a stuttering group compared to a typically fluent group. Finally, we review current limitations and directions for future research.
Collapse
Affiliation(s)
- David Jenson
- Department of Speech and Hearing Sciences, Elson S. Floyd College of Medicine, Washington State University, Spokane, WA, United States
| | - Andrew L. Bowers
- Epley Center for Health Professions, Communication Sciences and Disorders, University of Arkansas, Fayetteville, AR, United States
| | - Daniel Hudock
- Department of Communication Sciences and Disorders, Idaho State University, Pocatello, ID, United States
| | - Tim Saltuklaroglu
- College of Health Professions, Department of Audiology and Speech-Pathology, University of Tennessee Health Science Center, Knoxville, TN, United States
| |
Collapse
|
37
|
Korik A, Sosnik R, Siddique N, Coyle D. Decoding Imagined 3D Arm Movement Trajectories From EEG to Control Two Virtual Arms-A Pilot Study. Front Neurorobot 2019; 13:94. [PMID: 31798438 PMCID: PMC6868122 DOI: 10.3389/fnbot.2019.00094] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2018] [Accepted: 10/28/2019] [Indexed: 11/26/2022] Open
Abstract
Background: Realization of online control of an artificial or virtual arm using information decoded from EEG normally occurs by classifying different activation states or voluntary modulation of the sensorimotor activity linked to different overt actions of the subject. However, using a more natural control scheme, such as decoding the trajectory of imagined 3D arm movements to move a prosthetic, robotic, or virtual arm has been reported in a limited amount of studies, all using offline feed-forward control schemes. Objective: In this study, we report the first attempt to realize online control of two virtual arms generating movements toward three targets/arm in 3D space. The 3D trajectory of imagined arm movements was decoded from power spectral density of mu, low beta, high beta, and low gamma EEG oscillations using multiple linear regression. The analysis was performed on a dataset recorded from three subjects in seven sessions wherein each session comprised three experimental blocks: an offline calibration block and two online feedback blocks. Target classification accuracy using predicted trajectories of the virtual arms was computed and compared with results of a filter-bank common spatial patterns (FBCSP) based multi-class classification method involving mutual information (MI) selection and linear discriminant analysis (LDA) modules. Main Results: Target classification accuracy from predicted trajectory of imagined 3D arm movements in the offline runs for two subjects (mean 45%, std 5%) was significantly higher (p < 0.05) than chance level (33.3%). Nevertheless, the accuracy during real-time control of the virtual arms using the trajectory decoded directly from EEG was in the range of chance level (33.3%). However, the results of two subjects show that false-positive feedback may increase the accuracy in closed-loop. The FBCSP based multi-class classification method distinguished imagined movements of left and right arm with reasonable accuracy for two of the three subjects (mean 70%, std 5% compared to 50% chance level). However, classification of the imagined arm movement toward three targets was not successful with the FBCSP classifier as the achieved accuracy (mean 33%, std 5%) was similar to the chance level (33.3%). Sub-optimal components of the multi-session experimental paradigm were identified, and an improved paradigm proposed.
Collapse
Affiliation(s)
- Attila Korik
- Intelligent Systems Research Centre, Ulster University, Derry, United Kingdom
| | - Ronen Sosnik
- Hybrid BCI Lab, Holon Institute of Technology, Holon, Israel
| | - Nazmul Siddique
- Intelligent Systems Research Centre, Ulster University, Derry, United Kingdom
| | - Damien Coyle
- Intelligent Systems Research Centre, Ulster University, Derry, United Kingdom
| |
Collapse
|
38
|
Schwarz A, Pereira J, Kobler R, Muller-Putz GR. Unimanual and Bimanual Reach-and-Grasp Actions Can Be Decoded From Human EEG. IEEE Trans Biomed Eng 2019; 67:1684-1695. [PMID: 31545707 DOI: 10.1109/tbme.2019.2942974] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
While most tasks of daily life can be handled through a small number of different grasps, many tasks require the action of both hands. In these bimanual tasks, the second hand has either a supporting role (e.g. for fixating a jar) or a more active role (e.g. grasping a pot on both handles). In this study we attempt to discriminate the neural correlates of unimanual (performed with left and right hand) from bimanual reach-and-grasp actions using the low-frequency time-domain electroencephalogram (EEG). In a self-initiated movement task, 15 healthy participants were asked to perform unimanual (palmar and lateral grasps with left and right hand) and bimanual (double lateral, mixed palmar/lateral) reach-and-grasps on objects of daily life. Using EEG time-domain features in the frequency range of 0.3-3 Hz, we achieved multiclass-classification accuracies of 38.6 ± 6.6% (7 classes, 17.1% chance level) for a combination of 6 movements and 1 rest condition. The grand average confusion matrix shows highest true positive rates (TPR) for the rest (63%) condition while TPR for the movement classes varied between 33 to 41%. The underlying movement-related cortical potentials (MRCPs) show significant differences between unimanual (e.g left hand vs. right hand grasps) as well unimanual vs. bimanual conditions which both can be attributed to lateralization effects. We believe that these findings can be exploited and further used for attempts in providing persons with spinal cord injury a form of natural control for bimanual neuroprostheses.
Collapse
|
39
|
Mirzaee MS, Moghimi S. Detection of reaching intention using EEG signals and nonlinear dynamic system identification. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2019; 175:151-161. [PMID: 31104704 DOI: 10.1016/j.cmpb.2019.04.023] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/23/2018] [Revised: 04/19/2019] [Accepted: 04/21/2019] [Indexed: 06/09/2023]
Abstract
BACKGROUND AND OBJECTIVES Low frequency electroencephalography (EEG) signals are associated with preparation of movement and thus provide valuable information for brain-machine interface applications. The purpose of this study was to detect movement intention from EEG signals before execution of self-paced arm reaching movements. METHODS Ten healthy individuals were recruited. Movement onset was determined from surface electromyography recordings time-locked with EEG signals. Unlike previous studies, which employed feature extraction and classification for decoding, a nonlinear dynamic multiple-input/single output (MISO) model was developed. The MISO model consisted of a cascade of Volterra structures and a threshold block, generating the binary output corresponding to intention/no-intention. The modeling process included input selection from a pool of different EEG channels. The predictive performance of the model was evaluated using the receiver operating characteristics curve, from which the optimum threshold was also selected. The Mann-Whitney statistics was employed to select the significant EEG channels for the output by examining the statistical significance of improvement in the predictive capability of the model when the respective channels were included. RESULTS With the proposed approach, movement intention was detected approximately 500 ms before the movement onset and on average, with an accuracy of 96.37 ± 0.94%, a sensitivity of 77.93 ± 4.40% and a specificity of 98.52 ± 1.19%. CONCLUSIONS The model output can be converted to motion commands for neuroprosthetic devices and exoskeletons in future applications.
Collapse
Affiliation(s)
| | - Sahar Moghimi
- Electrical Engineering Department, Ferdowsi University of Mashhad, Mashhad, Iran; Rayan Center for Neuroscience and Behavior, Ferdowsi University of Mashhad, Mashhad, Iran.
| |
Collapse
|
40
|
Kim HH, Jeong J. Decoding electroencephalographic signals for direction in brain-computer interface using echo state network and Gaussian readouts. Comput Biol Med 2019; 110:254-264. [PMID: 31233971 DOI: 10.1016/j.compbiomed.2019.05.024] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2019] [Revised: 05/31/2019] [Accepted: 05/31/2019] [Indexed: 10/26/2022]
Abstract
BACKGROUND Noninvasive brain-computer interfaces (BCI) for movement control via an electroencephalogram (EEG) have been extensively investigated. However, most previous studies decoded user intention for movement directions based on sensorimotor rhythms during motor imagery. BCI systems based on mapping imagery movement of body parts (e.g., left or right hands) to movement directions (left or right directional movement of a machine or cursor) are less intuitive and less convenient due to the complex training procedures. Thus, direct decoding methods for detecting user intention about movement directions are urgently needed. METHODS Here, we describe a novel direct decoding method for user intention about the movement directions using the echo state network and Gaussian readouts. Importantly parameters in the network were optimized using the genetic algorithm method to achieve better decoding performance. We tested the decoding performance of this method with four healthy subjects and an inexpensive wireless EEG system containing 14 channels and then compared the performance outcome with that of a conventional machine learning method. RESULTS We showed that this decoding method successfully classified eight directions of intended movement (approximately 95% of an accuracy). CONCLUSIONS We suggest that the echo state network and Gaussian readouts can be a useful decoding method to directly read user intention of movement directions even using an inexpensive and portable EEG system.
Collapse
Affiliation(s)
- Hoon-Hee Kim
- Department of Bio and Brain Engineering, College of Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea
| | - Jaeseung Jeong
- Department of Bio and Brain Engineering, College of Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea; Program of Brain and Cognitive Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea.
| |
Collapse
|
41
|
Ibáñez‐Marcelo E, Campioni L, Manzoni D, Santarcangelo EL, Petri G. Spectral and topological analyses of the cortical representation of the head position: Does hypnotizability matter? Brain Behav 2019; 9:e01277. [PMID: 31001933 PMCID: PMC6576149 DOI: 10.1002/brb3.1277] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/26/2018] [Revised: 02/25/2019] [Accepted: 02/25/2019] [Indexed: 12/15/2022] Open
Abstract
INTRODUCTION The aim of this exploratory study was to assess the EEG correlates of head positions (which have never been studied in humans) in participants with different psychophysiological characteristics, as encoded by their hypnotizability scores. This choice is motivated by earlier studies suggesting different processing of vestibular/neck proprioceptive information in subjects with high (highs) and low (lows) hypnotizability scores maintaining their head rotated toward one side (RH). METHODS We analyzed EEG signals recorded in 20 highs and 19 lows in basal conditions (head forward) and during RH using spectral analysis, which captures changes localized to specific recording sites, and topological data analysis (TDA), which instead describes large-scale differences in processing and representing sensorimotor information. RESULTS Spectral analysis revealed significant differences related to head position for alpha 1, beta 2, beta 3, and gamma bands, but not to hypnotizability. TDA instead revealed global hypnotizability-related differences in the strengths of the correlations among recording sites during RH. Significant changes were observed in lows on the left parieto-occipital side and in highs in right frontoparietal region. Significant differences between the two groups were found in the occipital region, where changes were larger in lows than in highs. CONCLUSIONS This study reports finding of the EEG correlates of changes in the head posture for the first time, indicating that hypnotizability is related to the head posture representation/processing on large-scale networks and that spectral and topological data analyses provide complementary results.
Collapse
Affiliation(s)
| | - Lisa Campioni
- Department of Translational Research and New Technologies in Medicine and SurgeryUniversity of PisaPisaItaly
| | - Diego Manzoni
- Department of Translational Research and New Technologies in Medicine and SurgeryUniversity of PisaPisaItaly
| | - Enrica L. Santarcangelo
- Department of Translational Research and New Technologies in Medicine and SurgeryUniversity of PisaPisaItaly
| | - Giovanni Petri
- ISI FoundationTurinItaly
- ISI Global Science FoundationNew YorkNYUSA
| |
Collapse
|
42
|
Zeng H, Sun Y, Xu G, Wu C, Song A, Xu B, Li H, Hu C. The Advantage of Low-Delta Electroencephalogram Phase Feature for Reconstructing the Center-Out Reaching Hand Movements. Front Neurosci 2019; 13:480. [PMID: 31156367 PMCID: PMC6530632 DOI: 10.3389/fnins.2019.00480] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2018] [Accepted: 04/29/2019] [Indexed: 11/13/2022] Open
Abstract
It is an emerging frontier of research on the use of neural signals for prosthesis control, in order to restore lost function to amputees and patients after spinal cord injury. Compared to the invasive neural signal based brain-machine interface (BMI), a non-invasive alternative, i.e., the electroencephalogram (EEG)-based BMI would be more widely accepted by the patients above. Ideally, a real-time continuous neuroprosthestic control is required for practical applications. However, conventional EEG-based BMIs mainly deal with the discrete brain activity classification. Until recently, the literature has reported several attempts for achieving the real-time continuous control by reconstructing the continuous movement parameters (e.g., speed, position, etc.) from the EEG recordings, and the low-frequency band EEG is consistently reported to encode the continuous motor control information. Previous studies with executed movement tasks have extensively relied on the amplitude representation of such slow oscillations of EEG signals for building models to decode kinematic parameters. Inspired by the recent successes of instantaneous phase of low-frequency invasive brain signals in the motor control and sensory processing domains, this study examines the extension of such a slow-oscillation phase representation to the reconstructing two-dimensional hand movements, with the non-invasive EEG signals for the first time. The data for analysis are collected on five healthy subjects performing 2D hand center-out reaching along four directions in two sessions. On representative channels over the cortices encoding the execution information of reaching movements, we show that the low-delta EEG phase representation is characterized by higher signal-to-noise ratio and stronger modulation by the movement tasks, compared to the low-delta EEG amplitude representation. Furthermore, we have tested the low-delta EEG phase representation with two commonly used linear decoding models. The results demonstrate that the low-delta EEG phase based decoders lead to superior performance for 2D executed movement reconstruction to its amplitude based counterparts, as well as the other-frequency band amplitude and power based features. Thus, our study contributes to improve the movement reconstruction from EEG by introducing a new feature set based on the low-delta EEG phase patterns, and demonstrates its potential for continuous fine motion control of neuroprostheses.
Collapse
Affiliation(s)
- Hong Zeng
- Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, China.,Mechatronics and Haptics Interfaces Laboratory, Department of Mechanical Engineering, Rice University, Houston, TX, United States
| | - Yuanzi Sun
- Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Guozheng Xu
- College of Automation, Nanjing University of Posts and Telecommunications, Nanjing, China
| | - Changcheng Wu
- College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Aiguo Song
- Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Baoguo Xu
- Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Huijun Li
- Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Cong Hu
- Guangxi Key Laboratory of Automatic Detecting Technology and Instruments, Guilin University of Electronic Technology, Guilin, China
| |
Collapse
|
43
|
Ofner P, Schwarz A, Pereira J, Wyss D, Wildburger R, Müller-Putz GR. Attempted Arm and Hand Movements can be Decoded from Low-Frequency EEG from Persons with Spinal Cord Injury. Sci Rep 2019; 9:7134. [PMID: 31073142 PMCID: PMC6509331 DOI: 10.1038/s41598-019-43594-9] [Citation(s) in RCA: 50] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2019] [Accepted: 04/26/2019] [Indexed: 01/08/2023] Open
Abstract
We show that persons with spinal cord injury (SCI) retain decodable neural correlates of attempted arm and hand movements. We investigated hand open, palmar grasp, lateral grasp, pronation, and supination in 10 persons with cervical SCI. Discriminative movement information was provided by the time-domain of low-frequency electroencephalography (EEG) signals. Based on these signals, we obtained a maximum average classification accuracy of 45% (chance level was 20%) with respect to the five investigated classes. Pattern analysis indicates central motor areas as the origin of the discriminative signals. Furthermore, we introduce a proof-of-concept to classify movement attempts online in a closed loop, and tested it on a person with cervical SCI. We achieved here a modest classification performance of 68.4% with respect to palmar grasp vs hand open (chance level 50%).
Collapse
Affiliation(s)
- Patrick Ofner
- Graz University of Technology, Institute of Neural Engineering, BCI-Lab, Graz, Austria
| | - Andreas Schwarz
- Graz University of Technology, Institute of Neural Engineering, BCI-Lab, Graz, Austria
| | - Joana Pereira
- Graz University of Technology, Institute of Neural Engineering, BCI-Lab, Graz, Austria
| | | | | | - Gernot R Müller-Putz
- Graz University of Technology, Institute of Neural Engineering, BCI-Lab, Graz, Austria.
| |
Collapse
|
44
|
Shakibaee F, Mottaghi E, Kobravi HR, Ghoshuni M. Decoding knee angle trajectory from electroencephalogram signal using NARX neural network and a new channel selection algorithm. Biomed Phys Eng Express 2019. [DOI: 10.1088/2057-1976/aafd48] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
45
|
Wittevrongel B, Khachatryan E, Fahimi Hnazaee M, Camarrone F, Carrette E, De Taeye L, Meurs A, Boon P, Van Roost D, Van Hulle MM. Decoding Steady-State Visual Evoked Potentials From Electrocorticography. Front Neuroinform 2018; 12:65. [PMID: 30319386 PMCID: PMC6168710 DOI: 10.3389/fninf.2018.00065] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2018] [Accepted: 09/06/2018] [Indexed: 12/02/2022] Open
Abstract
We report on a unique electrocorticography (ECoG) experiment in which Steady-State Visual Evoked Potentials (SSVEPs) to frequency- and phase-tagged stimuli were recorded from a large subdural grid covering the entire right occipital cortex of a human subject. The paradigm is popular in EEG-based Brain Computer Interfacing where selectable targets are encoded by different frequency- and/or phase-tagged stimuli. We compare the performance of two state-of-the-art SSVEP decoders on both ECoG- and scalp-recorded EEG signals, and show that ECoG-based decoding is more accurate for very short stimulation lengths (i.e., less than 1 s). Furthermore, whereas the accuracy of scalp-EEG decoding benefits from a multi-electrode approach, to address interfering EEG responses and noise, ECoG decoding enjoys only a marginal improvement as even a single electrode, placed over the posterior part of the primary visual cortex, seems to suffice. This study shows, for the first time, that EEG-based SSVEP decoders can in principle be applied to ECoG, and can be expected to yield faster decoding speeds using less electrodes.
Collapse
Affiliation(s)
- Benjamin Wittevrongel
- Laboratory for Neuro- and Psychophysiology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Elvira Khachatryan
- Laboratory for Neuro- and Psychophysiology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Mansoureh Fahimi Hnazaee
- Laboratory for Neuro- and Psychophysiology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Flavio Camarrone
- Laboratory for Neuro- and Psychophysiology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Evelien Carrette
- Laboratory of Clinical and Experimental Neurophysiology, Neurology Department, Ghent University Hospital, Ghent, Belgium
| | - Leen De Taeye
- Laboratory of Clinical and Experimental Neurophysiology, Neurology Department, Ghent University Hospital, Ghent, Belgium
| | - Alfred Meurs
- Laboratory of Clinical and Experimental Neurophysiology, Neurology Department, Ghent University Hospital, Ghent, Belgium
| | - Paul Boon
- Laboratory of Clinical and Experimental Neurophysiology, Neurology Department, Ghent University Hospital, Ghent, Belgium
| | - Dirk Van Roost
- Department of Neurosurgery, Ghent University Hospital, Ghent, Belgium
| | - Marc M. Van Hulle
- Laboratory for Neuro- and Psychophysiology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| |
Collapse
|