1
|
Wen H, Zhong Y, Yao L, Wang Y. Neural Correlates of Motor/Tactile Imagery and Tactile Sensation in a BCI paradigm: A High-Density EEG Source Imaging Study. CYBORG AND BIONIC SYSTEMS 2024; 5:0118. [PMID: 38912322 PMCID: PMC11192147 DOI: 10.34133/cbsystems.0118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Accepted: 04/01/2024] [Indexed: 06/25/2024] Open
Abstract
Complementary to brain-computer interface (BCI) based on motor imagery (MI) task, sensory imagery (SI) task provides a way for BCI construction using brain activity from somatosensory cortex. The underlying neurophysiological correlation between SI and MI was unclear and difficult to measure through behavior recording. In this study, we investigated the underlying neurodynamic of motor/tactile imagery and tactile sensation tasks through a high-density electroencephalogram (EEG) recording, and EEG source imaging was used to systematically explore the cortical activation differences and correlations between the tasks. In the experiment, participants were instructed to perform the left and right hand tasks in MI paradigm, sensory stimulation (SS) paradigm and SI paradigm. The statistical results demonstrated that the imagined MI and SI tasks differed from each other within ipsilateral sensorimotor scouts, frontal and right temporal areas in α bands, whereas real SS and imagined SI showed a similar activation pattern. The similarity between SS and SI may provide a way to train the BCI system, while the difference between MI and SI may provide a way to integrate the discriminative information between them to enhance BCI performance. The combination of the tasks and its underlying neurodynamic would provide a new approach for BCI designation for a wider application. BCI studies concentrate on the hybrid decoding method combining MI or SI with SS, but the underlining neurophysiological correlates between them were unclear. MI and SI differed from each other within the ipsilateral sensorimotor cortex in alpha bands. This is a first study to investigate the neurophysiological relationship between MI and SI through an EEG source imaging approach from high-density EEG recording.
Collapse
Affiliation(s)
- Huan Wen
- The Department of Neurobiology, Affiliated Mental Health Center & Hangzhou Seventh People’s Hospital,
Zhejiang University School of Medicine, Hangzhou, China
- The Nanhu Brain-Computer Interface Institute, Hangzhou, China
- The MOE Frontiers Science Center for Brain and Brain-Machine Integration,
Zhejiang University, Hangzhou, China
| | - Yucun Zhong
- The Department of Neurobiology, Affiliated Mental Health Center & Hangzhou Seventh People’s Hospital,
Zhejiang University School of Medicine, Hangzhou, China
- The Nanhu Brain-Computer Interface Institute, Hangzhou, China
- The MOE Frontiers Science Center for Brain and Brain-Machine Integration,
Zhejiang University, Hangzhou, China
| | - Lin Yao
- The Department of Neurobiology, Affiliated Mental Health Center & Hangzhou Seventh People’s Hospital,
Zhejiang University School of Medicine, Hangzhou, China
- The Nanhu Brain-Computer Interface Institute, Hangzhou, China
- The MOE Frontiers Science Center for Brain and Brain-Machine Integration,
Zhejiang University, Hangzhou, China
- The College of Computer Science,
Zhejiang University, Hangzhou, China
- The College of Biomedical Engineering & Instrument Science,
Zhejiang University, Hangzhou, China
| | - Yueming Wang
- The MOE Frontiers Science Center for Brain and Brain-Machine Integration,
Zhejiang University, Hangzhou, China
- The College of Computer Science,
Zhejiang University, Hangzhou, China
- The Qiushi Academy for Advanced Studies, Hangzhou, China
| |
Collapse
|
2
|
Della Vedova G, Proverbio AM. Neural signatures of imaginary motivational states: desire for music, movement and social play. Brain Topogr 2024:10.1007/s10548-024-01047-1. [PMID: 38625520 DOI: 10.1007/s10548-024-01047-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 03/12/2024] [Indexed: 04/17/2024]
Abstract
The literature has demonstrated the potential for detecting accurate electrical signals that correspond to the will or intention to move, as well as decoding the thoughts of individuals who imagine houses, faces or objects. This investigation examines the presence of precise neural markers of imagined motivational states through the combining of electrophysiological and neuroimaging methods. 20 participants were instructed to vividly imagine the desire to move, listen to music or engage in social activities. Their EEG was recorded from 128 scalp sites and analysed using individual standardized Low-Resolution Brain Electromagnetic Tomographies (LORETAs) in the N400 time window (400-600 ms). The activation of 1056 voxels was examined in relation to the 3 motivational states. The most active dipoles were grouped in eight regions of interest (ROI), including Occipital, Temporal, Fusiform, Premotor, Frontal, OBF/IF, Parietal, and Limbic areas. The statistical analysis revealed that all motivational imaginary states engaged the right hemisphere more than the left hemisphere. Distinct markers were identified for the three motivational states. Specifically, the right temporal area was more relevant for "Social Play", the orbitofrontal/inferior frontal cortex for listening to music, and the left premotor cortex for the "Movement" desire. This outcome is encouraging in terms of the potential use of neural indicators in the realm of brain-computer interface, for interpreting the thoughts and desires of individuals with locked-in syndrome.
Collapse
Affiliation(s)
- Giada Della Vedova
- Cognitive Electrophysiology lab, Dept. of Psychology, University of Milano, Bicocca, Italy
| | - Alice Mado Proverbio
- Cognitive Electrophysiology lab, Dept. of Psychology, University of Milano, Bicocca, Italy.
- NeuroMI, Milan Center for Neuroscience, Milan, Italy.
- Department of Psychology of University of Milano-Bicocca, Piazza dell'Ateneo nuovo 1, Milan, 20162, Italy.
| |
Collapse
|
3
|
Forenzo D, Zhu H, Shanahan J, Lim J, He B. Continuous tracking using deep learning-based decoding for noninvasive brain-computer interface. PNAS NEXUS 2024; 3:pgae145. [PMID: 38689706 PMCID: PMC11060102 DOI: 10.1093/pnasnexus/pgae145] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 03/28/2024] [Indexed: 05/02/2024]
Abstract
Brain-computer interfaces (BCI) using electroencephalography provide a noninvasive method for users to interact with external devices without the need for muscle activation. While noninvasive BCIs have the potential to improve the quality of lives of healthy and motor-impaired individuals, they currently have limited applications due to inconsistent performance and low degrees of freedom. In this study, we use deep learning (DL)-based decoders for online continuous pursuit (CP), a complex BCI task requiring the user to track an object in 2D space. We developed a labeling system to use CP data for supervised learning, trained DL-based decoders based on two architectures, including a newly proposed adaptation of the PointNet architecture, and evaluated the performance over several online sessions. We rigorously evaluated the DL-based decoders in a total of 28 human participants, and found that the DL-based models improved throughout the sessions as more training data became available and significantly outperformed a traditional BCI decoder by the last session. We also performed additional experiments to test an implementation of transfer learning by pretraining models on data from other subjects, and midsession training to reduce intersession variability. The results from these experiments showed that pretraining did not significantly improve performance, but updating the models' midsession may have some benefit. Overall, these findings support the use of DL-based decoders for improving BCI performance in complex tasks like CP, which can expand the potential applications of BCI devices and help to improve the quality of lives of healthy and motor-impaired individuals.
Collapse
Affiliation(s)
- Dylan Forenzo
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Hao Zhu
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Jenn Shanahan
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Jaehyun Lim
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Bin He
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| |
Collapse
|
4
|
Jung J, Moon H, Yu G, Hwang H. Generative Perturbation Network for Universal Adversarial Attacks on Brain-Computer Interfaces. IEEE J Biomed Health Inform 2023; 27:5622-5633. [PMID: 37556336 DOI: 10.1109/jbhi.2023.3303494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/11/2023]
Abstract
Deep neural networks (DNNs) have successfully classified EEG-based brain-computer interface (BCI) systems. However, recent studies have found that well-designed input samples, known as adversarial examples, can easily fool well-performed deep neural networks model with minor perturbations undetectable by a human. This paper proposes an efficient generative model named generative perturbation network (GPN), which can generate universal adversarial examples with the same architecture for non-targeted and targeted attacks. Furthermore, the proposed model can be efficiently extended to conditionally or simultaneously generate perturbations for various targets and victim models. Our experimental evaluation demonstrates that perturbations generated by the proposed model outperform previous approaches for crafting signal-agnostic perturbations. We demonstrate that the extended network for signal-specific methods also significantly reduces generation time while performing similarly. The transferability across classification networks of the proposed method is superior to the other methods, which shows our perturbations' high level of generality.
Collapse
|
5
|
Zhang M, Huang J, Ni S. Recognition of motor intentions from EEGs of the same upper limb by signal traceability and Riemannian geometry features. Front Neurosci 2023; 17:1270785. [PMID: 38027473 PMCID: PMC10643198 DOI: 10.3389/fnins.2023.1270785] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 10/03/2023] [Indexed: 12/01/2023] Open
Abstract
Introduction The electroencephalographic (EEG) based on the motor imagery task is derived from the physiological electrical signal caused by the autonomous activity of the brain. Its weak potential difference changes make it easy to be overwhelmed by noise, and the EEG acquisition method has a natural limitation of low spatial resolution. These have brought significant obstacles to high-precision recognition, especially the recognition of the motion intention of the same upper limb. Methods This research proposes a method that combines signal traceability and Riemannian geometric features to identify six motor intentions of the same upper limb, including grasping/holding of the palm, flexion/extension of the elbow, and abduction/adduction of the shoulder. First, the EEG data of electrodes irrelevant to the task were screened out by low-resolution brain electromagnetic tomography. Subsequently, tangential spatial features are extracted by the Riemannian geometry framework in the covariance matrix estimated from the reconstructed EEG signals. The learned Riemannian geometric features are used for pattern recognition by a support vector machine with a linear kernel function. Results The average accuracy of the six classifications on the data set of 15 participants is 22.47%, the accuracy is 19.34% without signal traceability, the accuracy is 18.07% when the features are the filter bank common spatial pattern (FBCSP), and the accuracy is 16.7% without signal traceability and characterized by FBCSP. Discussion The results show that the proposed method can significantly improve the accuracy of intent recognition. In addressing the issue of temporal variability in EEG data for active Brain-Machine Interfaces, our method achieved an average standard deviation of 2.98 through model transfer on different days' data.
Collapse
Affiliation(s)
- Meng Zhang
- School of Mechanical and Power Engineering, Nanjing Tech University Nanjing, Nanjing, China
- Research Institute, NeuralEcho Technology Co., Ltd., Beijing, China
| | - Jinfeng Huang
- Faculty of Human Sciences, University of Tsukuba, Ibaraki, Japan
- Research Institute, NeuralEcho Technology Co., Ltd., Beijing, China
| | - Shoudong Ni
- School of Mechanical and Power Engineering, Nanjing Tech University Nanjing, Nanjing, China
| |
Collapse
|
6
|
Khan RA, Rashid N, Shahzaib M, Malik UF, Arif A, Iqbal J, Saleem M, Khan US, Tiwana M. A novel framework for classification of two-class motor imagery EEG signals using logistic regression classification algorithm. PLoS One 2023; 18:e0276133. [PMID: 37682884 PMCID: PMC10490872 DOI: 10.1371/journal.pone.0276133] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 09/29/2022] [Indexed: 09/10/2023] Open
Abstract
Robotics and artificial intelligence have played a significant role in developing assistive technologies for people with motor disabilities. Brain-Computer Interface (BCI) is a communication system that allows humans to communicate with their environment by detecting and quantifying control signals produced from different modalities and translating them into voluntary commands for actuating an external device. For that purpose, classification the brain signals with a very high accuracy and minimization of the errors is of profound importance to the researchers. So in this study, a novel framework has been proposed to classify the binary-class electroencephalogram (EEG) data. The proposed framework is tested on BCI Competition IV dataset 1 and BCI Competition III dataset 4a. Artifact removal from EEG data is done through preprocessing, followed by feature extraction for recognizing discriminative information in the recorded brain signals. Signal preprocessing involves the application of independent component analysis (ICA) on raw EEG data, accompanied by the employment of common spatial pattern (CSP) and log-variance for extracting useful features. Six different classification algorithms, namely support vector machine, linear discriminant analysis, k-nearest neighbor, naïve Bayes, decision trees, and logistic regression, have been compared to classify the EEG data accurately. The proposed framework achieved the best classification accuracies with logistic regression classifier for both datasets. Average classification accuracy of 90.42% has been attained on BCI Competition IV dataset 1 for seven different subjects, while for BCI Competition III dataset 4a, an average accuracy of 95.42% has been attained on five subjects. This indicates that the model can be used in real time BCI systems and provide extra-ordinary results for 2-class Motor Imagery (MI) signals classification applications and with some modifications this framework can also be made compatible for multi-class classification in the future.
Collapse
Affiliation(s)
- Rabia Avais Khan
- Department of Mechatronics Engineering, National University of Sciences & Technology, Islamabad, Pakistan
| | - Nasir Rashid
- Department of Mechatronics Engineering, National University of Sciences & Technology, Islamabad, Pakistan
- Robot Design and Development Lab, National Centre of Robotics and Automation (NCRA), Punjab, Pakistan
| | - Muhammad Shahzaib
- Department of Mechatronics Engineering, National University of Sciences & Technology, Islamabad, Pakistan
| | - Umar Farooq Malik
- Department of Mechatronics Engineering, National University of Sciences & Technology, Islamabad, Pakistan
| | - Arshia Arif
- Department of Mechatronics Engineering, National University of Sciences & Technology, Islamabad, Pakistan
| | - Javaid Iqbal
- Department of Mechatronics Engineering, National University of Sciences & Technology, Islamabad, Pakistan
- Robot Design and Development Lab, National Centre of Robotics and Automation (NCRA), Punjab, Pakistan
| | - Mubasher Saleem
- Department of Mechatronics Engineering, National University of Sciences & Technology, Islamabad, Pakistan
| | - Umar Shahbaz Khan
- Department of Mechatronics Engineering, National University of Sciences & Technology, Islamabad, Pakistan
- Robot Design and Development Lab, National Centre of Robotics and Automation (NCRA), Punjab, Pakistan
| | - Mohsin Tiwana
- Department of Mechatronics Engineering, National University of Sciences & Technology, Islamabad, Pakistan
- Robot Design and Development Lab, National Centre of Robotics and Automation (NCRA), Punjab, Pakistan
| |
Collapse
|
7
|
Rosenfelder MJ, Spiliopoulou M, Hoppenstedt B, Pryss R, Fissler P, della Piedra Walter M, Kolassa IT, Bender A. Stability of mental motor-imagery classification in EEG depends on the choice of classifier model and experiment design, but not on signal preprocessing. Front Comput Neurosci 2023; 17:1142948. [PMID: 37180880 PMCID: PMC10169631 DOI: 10.3389/fncom.2023.1142948] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Accepted: 04/05/2023] [Indexed: 05/16/2023] Open
Abstract
Introduction Modern consciousness research has developed diagnostic tests to improve the diagnostic accuracy of different states of consciousness via electroencephalography (EEG)-based mental motor imagery (MI), which is still challenging and lacks a consensus on how to best analyse MI EEG-data. An optimally designed and analyzed paradigm must detect command-following in all healthy individuals, before it can be applied in patients, e.g., for the diagnosis of disorders of consciousness (DOC). Methods We investigated the effects of two important steps in the raw signal preprocessing on predicting participant performance (F1) and machine-learning classifier performance (area-under-curve, AUC) in eight healthy individuals, that are based solely on MI using high-density EEG (HD-EEG): artifact correction (manual correction with vs. without Independent Component Analysis [ICA]), region of interest (ROI; motor area vs. whole brain), and machine-learning algorithm (support-vector machine [SVM] vs. k-nearest neighbor [KNN]). Results Results revealed no significant effects of artifact correction and ROI on predicting participant performance (F1) and classifier performance (AUC) scores (all ps > 0.05) in the SVM classification model. In the KNN model, ROI had a significant influence on the classifier performance [F(1,8.939) = 7.585, p = 0.023]. There was no evidence for artifact correction and ROI selection changing the prediction of participants performance and classifier performance in EEG-based mental MI if using SVM-based classification (71-100% correct classifications across different signal preprocessing methods). The variance in the prediction of participant performance was significantly higher when the experiment started with a resting-state compared to a mental MI task block [X2(1) = 5.849, p = 0.016]. Discussion Overall, we could show that classification is stable across different modes of EEG signal preprocessing when using SVM models. Exploratory analysis gave a hint toward potential effects of the sequence of task execution on the prediction of participant performance, which should be taken into account in future studies.
Collapse
Affiliation(s)
- Martin Justinus Rosenfelder
- Institute of Psychology and Education, Clinical and Biological Psychology, Ulm University, Ulm, Germany
- Therapiezentrum Burgau, Burgau, Germany
| | - Myra Spiliopoulou
- Knowledge Management and Discovery Lab, Faculty of Computer Science, Otto-von-Guericke-University Magdeburg, Magdeburg, Germany
| | | | - Rüdiger Pryss
- Institute of Databases and Information Systems, Ulm University, Ulm, Germany
- Institute of Clinical Epidemiology and Biometry, University of Würzburg, Würzburg, Germany
| | - Patrick Fissler
- Institute of Psychology and Education, Clinical and Biological Psychology, Ulm University, Ulm, Germany
- Psychiatric Services Thurgau, Münsterlingen, Switzerland
- University Hospital for Psychiatry and Psychotherapy, Paracelsus Medical University, Salzburg, Austria
| | - Mario della Piedra Walter
- Therapiezentrum Burgau, Burgau, Germany
- Faculty 2: Biology/Chemistry, University of Bremen, Bremen, Germany
| | - Iris-Tatjana Kolassa
- Institute of Psychology and Education, Clinical and Biological Psychology, Ulm University, Ulm, Germany
| | - Andreas Bender
- Therapiezentrum Burgau, Burgau, Germany
- Department of Neurology, University of Munich, Munich, Germany
| |
Collapse
|
8
|
Srisrisawang N, Müller-Putz GR. Transfer Learning in Trajectory Decoding: Sensor or Source Space? SENSORS (BASEL, SWITZERLAND) 2023; 23:3593. [PMID: 37050653 PMCID: PMC10098869 DOI: 10.3390/s23073593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/19/2023] [Revised: 03/08/2023] [Accepted: 03/28/2023] [Indexed: 06/19/2023]
Abstract
In this study, across-participant and across-session transfer learning was investigated to minimize the calibration time of the brain-computer interface (BCI) system in the context of continuous hand trajectory decoding. We reanalyzed data from a study with 10 able-bodied participants across three sessions. A leave-one-participant-out (LOPO) model was utilized as a starting model. Recursive exponentially weighted partial least squares regression (REW-PLS) was employed to overcome the memory limitation due to the large pool of training data. We considered four scenarios: generalized with no update (Gen), generalized with cumulative update (GenC), and individual models with cumulative (IndC) and non-cumulative (Ind) updates, with each one trained with sensor-space features or source-space features. The decoding performance in generalized models (Gen and GenC) was lower than the chance level. In individual models, the cumulative update (IndC) showed no significant improvement over the non-cumulative model (Ind). The performance showed the decoder's incapability to generalize across participants and sessions in this task. The results suggested that the best correlation could be achieved with the sensor-space individual model, despite additional anatomical information in the source-space features. The decoding pattern showed a more localized pattern around the precuneus over three sessions in Ind models.
Collapse
Affiliation(s)
- Nitikorn Srisrisawang
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, 8010 Graz, Austria
| | - Gernot R. Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, 8010 Graz, Austria
- BioTechMed Graz, 8010 Graz, Austria
| |
Collapse
|
9
|
Srisrisawang N, Müller-Putz GR. Applying Dimensionality Reduction Techniques in Source-Space Electroencephalography via Template and Magnetic Resonance Imaging-Derived Head Models to Continuously Decode Hand Trajectories. Front Hum Neurosci 2022; 16:830221. [PMID: 35399364 PMCID: PMC8988304 DOI: 10.3389/fnhum.2022.830221] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Accepted: 02/28/2022] [Indexed: 11/13/2022] Open
Abstract
Several studies showed evidence supporting the possibility of hand trajectory decoding from low-frequency electroencephalography (EEG). However, the decoding in the source space via source localization is scarcely investigated. In this study, we tried to tackle the problem of collinearity due to the higher number of signals in the source space by two folds: first, we selected signals in predefined regions of interest (ROIs); second, we applied dimensionality reduction techniques to each ROI. The dimensionality reduction techniques were computing the mean (Mean), principal component analysis (PCA), and locality preserving projections (LPP). We also investigated the effect of decoding between utilizing a template head model and a subject-specific head model during the source localization. The results indicated that applying source-space decoding with PCA yielded slightly higher correlations and signal-to-noise (SNR) ratios than the sensor-space approach. We also observed slightly higher correlations and SNRs when applying the subject-specific head model than the template head model. However, the statistical tests revealed no significant differences between the source-space and sensor-space approaches and no significant differences between subject-specific and template head models. The decoder with Mean and PCA utilizes information mainly from precuneus and cuneus to decode the velocity kinematics similarly in the subject-specific and template head models.
Collapse
Affiliation(s)
| | - Gernot R. Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
- BioTechMed Graz, Graz, Austria
- *Correspondence: Gernot R. Müller-Putz,
| |
Collapse
|
10
|
Yan Y, Zhou H, Huang L, Cheng X, Kuang S. A Novel Two-Stage Refine Filtering Method for EEG-Based Motor Imagery Classification. Front Neurosci 2021; 15:657540. [PMID: 34539326 PMCID: PMC8440963 DOI: 10.3389/fnins.2021.657540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Accepted: 07/27/2021] [Indexed: 11/17/2022] Open
Abstract
Cerebral stroke is a common disease across the world, and it is a promising method to recognize the intention of stroke patients with the help of brain-computer interface (BCI). In the field of motor imagery (MI) classification, appropriate filtering is vital for feature extracting of electroencephalogram (EEG) signals and consequently influences the accuracy of MI classification. In this case, a novel two-stage refine filtering method was proposed, inspired by Gradient-weighted Class Activation Mapping (Grad-CAM), which uses the gradients of any target concept flowing into the final convolutional layer to highlight the important part of training data for predicting the concept. In the first stage, MI classification was carried out and then the frequency band to be filtered was calculated according to the Grad-CAM of the MI classification results. In the second stage, EEG was filtered and classified for a higher classification accuracy. To evaluate the filtering effect, this method was applied to the multi-branch neural network proposed in our previous work. Experiment results revealed that the proposed method reached state-of-the-art classification kappa value levels and acquired at least 3% higher kappa values than other methods This study also proposed some promising application scenarios with this filtering method.
Collapse
Affiliation(s)
- Yuxin Yan
- The First Affiliated Hospital of Soochow University, Soochow University, Suzhou, China
| | - Haifeng Zhou
- College of Mechanical and Electrical Engineering, Soochow University, Suzhou, China
| | - Lixin Huang
- The First Affiliated Hospital of Soochow University, Soochow University, Suzhou, China
| | - Xiao Cheng
- Applied Technology College of Soochow University, Suzhou, China
| | - Shaolong Kuang
- College of Mechanical and Electrical Engineering, Soochow University, Suzhou, China
| |
Collapse
|
11
|
Lun X, Yu Z, Wang F, Chen T, Hou Y. A novel approach of CNN for human motor imagery recognition using the virtual electrode pairs. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2021. [DOI: 10.3233/jifs-202046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
In order to develop an efficient brain-computer interface system, the brain activity measured by electroencephalography needs to be accurately decoded. In this paper, a motor imagery classification approach is proposed, combining virtual electrodes on the cortex layer with a convolutional neural network; this can effectively improve the decoding performance of the brain-computer interface system. A three layer (cortex, skull, and scalp) head volume conduction model was established by using the symmetric boundary element method to map the scalp signal to the cortex area. Nine pairs of virtual electrodes were created on the cortex layer, and the features of the time and frequency sequence from the virtual electrodes were extracted by performing time-frequency analysis. Finally, the convolutional neural network was used to classify motor imagery tasks. The results show that the proposed approach is convergent in both the training model and the test model. Based on the Physionet motor imagery database, the averaged accuracy can reach 98.32% for a single subject, while the averaged values of accuracy, Kappa, precision, recall, and F1-score on the group-wise are 96.23%, 94.83%, 96.21%, 96.13%, and 96.14%, respectively. Based on the High Gamma database, the averaged accuracy has achieved 96.37% and 91.21% at the subject and group levels, respectively. Moreover, this approach is superior to those of other studies on the same database, which suggests robustness and adaptability to individual variability.
Collapse
Affiliation(s)
- Xiangmin Lun
- College of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun, China
- School of Automation Engineering, Northeast Electric Power University, Jilin, China
| | - Zhenglin Yu
- College of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun, China
| | - Fang Wang
- School of Automation Engineering, Northeast Electric Power University, Jilin, China
| | - Tao Chen
- School of Automation Engineering, Northeast Electric Power University, Jilin, China
| | - Yimin Hou
- School of Automation Engineering, Northeast Electric Power University, Jilin, China
| |
Collapse
|
12
|
Saha S, Mamun KA, Ahmed K, Mostafa R, Naik GR, Darvishi S, Khandoker AH, Baumert M. Progress in Brain Computer Interface: Challenges and Opportunities. Front Syst Neurosci 2021; 15:578875. [PMID: 33716680 PMCID: PMC7947348 DOI: 10.3389/fnsys.2021.578875] [Citation(s) in RCA: 82] [Impact Index Per Article: 27.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2020] [Accepted: 01/06/2021] [Indexed: 12/13/2022] Open
Abstract
Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.
Collapse
Affiliation(s)
- Simanto Saha
- School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA, Australia
- Department of Electrical and Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Khondaker A. Mamun
- Advanced Intelligent Multidisciplinary Systems (AIMS) Lab, Department of Computer Science and Engineering, United International University, Dhaka, Bangladesh
| | - Khawza Ahmed
- Department of Electrical and Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Raqibul Mostafa
- Department of Electrical and Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Ganesh R. Naik
- Adelaide Institute for Sleep Health, College of Medicine and Public Health, Flinders University, Adelaide, SA, Australia
| | - Sam Darvishi
- School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA, Australia
| | - Ahsan H. Khandoker
- Healthcare Engineering Innovation Center, Department of Biomedical Engineering, Khalifa University of Science and Technology, Abu Dhabi, United Arab Emirates
| | - Mathias Baumert
- School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA, Australia
| |
Collapse
|
13
|
|
14
|
Roy R, Sikdar D, Mahadevappa M. Chaotic behaviour of EEG responses with an identical grasp posture. Comput Biol Med 2020; 123:103822. [PMID: 32658779 DOI: 10.1016/j.compbiomed.2020.103822] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2019] [Revised: 05/13/2020] [Accepted: 05/13/2020] [Indexed: 11/18/2022]
Abstract
Individuals with severe neuromuscular ailments can benefit from restoring their grasp activities with a brain-controlled upper-limb neuroprosthesis. EEG signals can be utilized as the driving source, and to implement natural human-like grasping abilities. Although good accuracy has already been achieved in classifying the various grasp patterns for specific sets of objects, unseen objects are still a hurdle in real-life implementation. Generalization of grasp patterns should be explored without any prior knowledge of the objects. In this regard, the similarity of motor imagery for different objects requiring similar grasp pattern can be utilized. It is also necessary to identify the brain regions that exhibit prominent distinguishability during different grasp patterns. In this study, we propose a chaos-based method to decode the motor imagery of two quite similar Power grasp patterns-cylindrical and spherical-for holding various objects. Three distinct suitable objects were chosen for each of the two patterns, and a 29-channel EEG was taken of 18 healthy participants to explore motor imagery for grasping the objects. Nonlinear correlation dimension was employed on the EEG data, at sub-band levels α, upper β, and γ, to analyse the distinguishability, as well as the similarity of grasp patterns for the objects. ANOVA was subsequently performed on the obtained CD parameters to identify the contribution of each electrode channel. Furthermore, using an SVM classifier, more than 80% accuracy was obtained in classifying the grasping patterns at the upper β sub-band. The outcome may lead to identification of optimum feature sets of motor imagery from specific brain regions for random objects grasps.
Collapse
Affiliation(s)
- Rinku Roy
- Advanced Technology Development Centre, Indian Institute of Technology Kharagpur, India
| | - Debdeep Sikdar
- School of Medical Science and Technology, Indian Institute of Technology Kharagpur, India
| | - Manjunatha Mahadevappa
- School of Medical Science and Technology, Indian Institute of Technology Kharagpur, India.
| |
Collapse
|
15
|
Hou Y, Zhou L, Jia S, Lun X. A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. J Neural Eng 2020; 17:016048. [PMID: 31585454 DOI: 10.1088/1741-2552/ab4af6] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
OBJECTIVE To develop and implement a novel approach which combines the technique of scout EEG source imaging (ESI) with convolutional neural network (CNN) for the classification of motor imagery (MI) tasks. APPROACH The technique of ESI uses a boundary element method (BEM) and weighted minimum norm estimation (WMNE) to solve the EEG forward and inverse problems, respectively. Ten scouts are then created within the motor cortex to select the region of interest (ROI). We extract features from the time series of scouts using a Morlet wavelet approach. Lastly, CNN is employed for classifying MI tasks. MAIN RESULTS The overall mean accuracy on the Physionet database reaches 94.5% and the individual accuracy of each task reaches 95.3%, 93.3%, 93.6%, 96% for the left fist, right fist, both fists and both feet, correspondingly, validated using ten-fold cross validation. We report an increase of up to 14.4% for overall classification compared with the competitive results from the state-of-the-art MI classification methods. Then, we add four new subjects to verify the validity of the method and the overall mean accuracy is 92.5%. Furthermore, the global classifier was adapted to single subjects improving the overall mean accuracy to 94.54%. SIGNIFICANCE The combination of scout ESI and CNN enhances BCI performance of decoding EEG four-class MI tasks.
Collapse
Affiliation(s)
- Yimin Hou
- School of Automation Engineering, Northeast Electric Power University, Jilin, People's Republic of China
| | | | | | | |
Collapse
|
16
|
Zhao M, Marino M, Samogin J, Swinnen SP, Mantini D. Hand, foot and lip representations in primary sensorimotor cortex: a high-density electroencephalography study. Sci Rep 2019; 9:19464. [PMID: 31857602 PMCID: PMC6923477 DOI: 10.1038/s41598-019-55369-3] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2019] [Accepted: 11/22/2019] [Indexed: 11/09/2022] Open
Abstract
The primary sensorimotor cortex plays a major role in the execution of movements of the contralateral side of the body. The topographic representation of different body parts within this brain region is commonly investigated through functional magnetic resonance imaging (fMRI). However, fMRI does not provide direct information about neuronal activity. In this study, we used high-density electroencephalography (hdEEG) to map the representations of hand, foot, and lip movements in the primary sensorimotor cortex, and to study their neural signatures. Specifically, we assessed the event-related desynchronization (ERD) in the cortical space. We found that the performance of hand, foot, and lip movements elicited an ERD in beta and gamma frequency bands. The primary regions showing significant beta- and gamma-band ERD for hand and foot movements, respectively, were consistent with previously reported using fMRI. We observed relatively weaker ERD for lip movements, which may be explained by the fact that less fine movement control was required. Overall, our study demonstrated that ERD based on hdEEG data can support the study of motor-related neural processes, with relatively high spatial resolution. An interesting avenue may be the use of hdEEG for deeper investigations into the pathophysiology of neuromotor disorders.
Collapse
Affiliation(s)
- Mingqi Zhao
- Research Center for Motor Control and Neuroplasticity, KU Leuven, 3001, Leuven, Belgium
| | - Marco Marino
- Research Center for Motor Control and Neuroplasticity, KU Leuven, 3001, Leuven, Belgium.,Brain Imaging and Neural Dynamics Research Group, IRCCS San Camillo Hospital, 30126, Venice, Italy
| | - Jessica Samogin
- Research Center for Motor Control and Neuroplasticity, KU Leuven, 3001, Leuven, Belgium
| | - Stephan P Swinnen
- Research Center for Motor Control and Neuroplasticity, KU Leuven, 3001, Leuven, Belgium.,Leuven Brain Institute, KU Leuven, 3000, Leuven, Belgium
| | - Dante Mantini
- Research Center for Motor Control and Neuroplasticity, KU Leuven, 3001, Leuven, Belgium. .,Brain Imaging and Neural Dynamics Research Group, IRCCS San Camillo Hospital, 30126, Venice, Italy.
| |
Collapse
|
17
|
Khan AF, Zhang F, Yuan H, Ding L. Dynamic Activation Patterns of the Motor Brain Revealed by Diffuse Optical Tomography . ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2019:6028-6031. [PMID: 31947220 DOI: 10.1109/embc.2019.8857370] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Diffuse optical tomography (DOT), a subset of functional near-infrared spectroscopy (fNIRS), is a noninvasive functional imaging modality for studying the human brain in normal and diseased conditions. It measures changes in concentrations of oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (Hb) in the blood vasculature of the brain. In contrast to functional magnetic resonance imaging (fMRI), the gold standard in human brain imaging, DOT offers the advantage of higher temporal resolution, portability, lower cost, multiple contrasts and usability for persons who cannot otherwise utilize MRI-based imaging modalities, including bedridden patients and infants, etc. The goal of the present study was to evaluate performance of a DOT method in studying dynamic patterns of brain activations involving motor control. CW-fNIRS data were acquired in four sessions from a healthy male participant when he performed a motor task in a block-design experiment. Results from experimental data showed pronounced activity in the primary motor cortex (M1), contralateral to the clenching hand. It was further observed that the M1 activity was consistent over four sessions. Furthermore, temporal dynamics of motor activity at each session further revealed well-sequenced activation patterns among M1, premotor cortex (PMC), and supplementary motor area (SMA). Timed ipsilateral motor activity suppression was also observed several seconds after the onset of contralateral M1 activity. More importantly, these temporal dynamics were similarly observed in all four sessions. These preliminary results suggest that the DOT method has the sensitivity, reliability, and spatio-temporal resolutions to study activities originated from the motor cortices. A full-scope evaluation and validation in more participants on the motor system can establish it as a promising neuroimaging tool to study, such as, infants at the risk of cerebral palsy or elders with Parkinson's due to its portability and usability in clinical environments.
Collapse
|
18
|
Edelman BJ, Meng J, Suma D, Zurn C, Nagarajan E, Baxter BS, Cline CC, He B. Noninvasive neuroimaging enhances continuous neural tracking for robotic device control. Sci Robot 2019; 4:eaaw6844. [PMID: 31656937 PMCID: PMC6814169 DOI: 10.1126/scirobotics.aaw6844] [Citation(s) in RCA: 142] [Impact Index Per Article: 28.4] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
Brain-computer interfaces (BCIs) utilizing signals acquired with intracortical implants have achieved successful high-dimensional robotic device control useful for completing daily tasks. However, the substantial amount of medical and surgical expertise required to correctly implant and operate these systems significantly limits their use beyond a few clinical cases. A noninvasive counterpart requiring less intervention that can provide high-quality control would profoundly impact the integration of BCIs into the clinical and home setting. Here, we present and validate a noninvasive framework utilizing electroencephalography (EEG) to achieve the neural control of a robotic device for continuous random target tracking. This framework addresses and improves upon both the "brain" and "computer" components by respectively increasing user engagement through a continuous pursuit task and associated training paradigm, and the spatial resolution of noninvasive neural data through EEG source imaging. In all, our unique framework enhanced BCI learning by nearly 60% for traditional center-out tasks and by over 500% in the more realistic continuous pursuit task. We further demonstrated an additional enhancement in BCI control of almost 10% by using online noninvasive neuroimaging. Finally, this framework was deployed in a physical task, demonstrating a near seamless transition from the control of an unconstrained virtual cursor to the real-time control of a robotic arm. Such combined advances in the quality of neural decoding and the practical utility of noninvasive robotic arm control will have major implications on the eventual development and implementation of neurorobotics by means of noninvasive BCI.
Collapse
Affiliation(s)
- B. J. Edelman
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - J. Meng
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - D. Suma
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - C. Zurn
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - E. Nagarajan
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | - B. S. Baxter
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - C. C. Cline
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - B. He
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| |
Collapse
|
19
|
Bou Assi E, Rihana S, Nguyen DK, Sawan M. Effective connectivity analysis of iEEG and accurate localization of the epileptogenic focus at the onset of operculo-insular seizures. Epilepsy Res 2019; 152:42-51. [DOI: 10.1016/j.eplepsyres.2019.02.006] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2018] [Revised: 12/23/2018] [Accepted: 02/21/2019] [Indexed: 11/16/2022]
|
20
|
Li MA, Wang YF, Jia SM, Sun YJ, Yang JF. Decoding of motor imagery EEG based on brain source estimation. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2019.02.006] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
21
|
Luo TJ, Zhou CL, Chao F. Exploring spatial-frequency-sequential relationships for motor imagery classification with recurrent neural network. BMC Bioinformatics 2018; 19:344. [PMID: 30268089 PMCID: PMC6162908 DOI: 10.1186/s12859-018-2365-1] [Citation(s) in RCA: 32] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2018] [Accepted: 09/10/2018] [Indexed: 11/27/2022] Open
Abstract
BACKGROUND Conventional methods of motor imagery brain computer interfaces (MI-BCIs) suffer from the limited number of samples and simplified features, so as to produce poor performances with spatial-frequency features and shallow classifiers. METHODS Alternatively, this paper applies a deep recurrent neural network (RNN) with a sliding window cropping strategy (SWCS) to signal classification of MI-BCIs. The spatial-frequency features are first extracted by the filter bank common spatial pattern (FB-CSP) algorithm, and such features are cropped by the SWCS into time slices. By extracting spatial-frequency-sequential relationships, the cropped time slices are then fed into RNN for classification. In order to overcome the memory distractions, the commonly used gated recurrent unit (GRU) and long-short term memory (LSTM) unit are applied to the RNN architecture, and experimental results are used to determine which unit is more suitable for processing EEG signals. RESULTS Experimental results on common BCI benchmark datasets show that the spatial-frequency-sequential relationships outperform all other competing spatial-frequency methods. In particular, the proposed GRU-RNN architecture achieves the lowest misclassification rates on all BCI benchmark datasets. CONCLUSION By introducing spatial-frequency-sequential relationships with cropping time slice samples, the proposed method gives a novel way to construct and model high accuracy and robustness MI-BCIs based on limited trials of EEG signals.
Collapse
Affiliation(s)
- Tian-jian Luo
- Department of Cognitive Science, School of Information Science and Engineering, Xiamen University, 422 Siming South Road, Siming District, Xiamen, 361005 China
| | - Chang-le Zhou
- Department of Cognitive Science, School of Information Science and Engineering, Xiamen University, 422 Siming South Road, Siming District, Xiamen, 361005 China
| | - Fei Chao
- Department of Cognitive Science, School of Information Science and Engineering, Xiamen University, 422 Siming South Road, Siming District, Xiamen, 361005 China
- Department of Computer Science, Institute of Mathematics, Physics and Computer Science, Aberystwyth University, Aberystwyth, Wales, SY23 3DB UK
| |
Collapse
|
22
|
Seeland A, Krell MM, Straube S, Kirchner EA. Empirical Comparison of Distributed Source Localization Methods for Single-Trial Detection of Movement Preparation. Front Hum Neurosci 2018; 12:340. [PMID: 30233341 PMCID: PMC6129768 DOI: 10.3389/fnhum.2018.00340] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2018] [Accepted: 08/09/2018] [Indexed: 11/17/2022] Open
Abstract
The development of technologies for the treatment of movement disorders, like stroke, is still of particular interest in brain-computer interface (BCI) research. In this context, source localization methods (SLMs), that reconstruct the cerebral origin of brain activity measured outside the head, e.g., via electroencephalography (EEG), can add a valuable insight into the current state and progress of the treatment. However, in BCIs SLMs were often solely considered as advanced signal processing methods that are compared against other methods based on the classification performance alone. Though, this approach does not guarantee physiological meaningful results. We present an empirical comparison of three established distributed SLMs with the aim to use one for single-trial movement prediction. The SLMs wMNE, sLORETA, and dSPM were applied on data acquired from eight subjects performing voluntary arm movements. Besides the classification performance as quality measure, a distance metric was used to asses the physiological plausibility of the methods. For the distance metric, which is usually measured to the source position of maximum activity, we further propose a variant based on clusters that is better suited for the single-trial case in which several sources are likely and the actual maximum is unknown. The two metrics showed different results. The classification performance revealed no significant differences across subjects, indicating that all three methods are equally well-suited for single-trial movement prediction. On the other hand, we obtained significant differences in the distance measure, favoring wMNE even after correcting the distance with the number of reconstructed clusters. Further, distance results were inconsistent with the traditional method using the maximum, indicating that for wMNE the point of maximum source activity often did not coincide with the nearest activation cluster. In summary, the presented comparison might help users to select an appropriate SLM and to understand the implications of the selection. The proposed methodology pays attention to the particular properties of distributed SLMs and can serve as a framework for further comparisons.
Collapse
Affiliation(s)
- Anett Seeland
- Robotics Innovation Center, German Research Center for Artificial Intelligence (DFKI GmbH), Bremen, Germany
| | - Mario M Krell
- Robotics Group, Faculty of Mathematics and Computer Science, University of Bremen, Bremen, Germany.,International Computer Science Institute, University of California, Berkeley, Berkeley, CA, United States.,University of California, Berkeley, Berkeley, CA, United States
| | - Sirko Straube
- Robotics Innovation Center, German Research Center for Artificial Intelligence (DFKI GmbH), Bremen, Germany
| | - Elsa A Kirchner
- Robotics Innovation Center, German Research Center for Artificial Intelligence (DFKI GmbH), Bremen, Germany.,Robotics Group, Faculty of Mathematics and Computer Science, University of Bremen, Bremen, Germany
| |
Collapse
|
23
|
Chiesi M, Guermandi M, Placati S, Scarselli EF, Guerrieri R. Creamino: A Cost-Effective, Open-Source EEG-Based BCI System. IEEE Trans Biomed Eng 2018; 66:900-909. [PMID: 30080140 DOI: 10.1109/tbme.2018.2863198] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
This paper presents an open source framework called Creamino. It consists of an Arduino-based cost-effective quick-setup EEG platform built with off-the-shelf components and a set of software modules that easily allow users to connect this system to Simulink or BCI-oriented tools (such as BCI2000 or OpenViBE) and set up a wide number of neuroscientific experiments. Creamino is capable of processing multiple EEG channels in real-time and operates under Windows, Linux, and Mac OS X in real-time on a standard PC. Its objective is to provide a system that can be readily fabricated and used for neurophysiological experiments and, at the same time, can serve as the basis for development of novel BCI platforms by accessing and modifying its open source hardware and software libraries. Schematics, gerber files, bill of materials, source code, software modules, demonstration videos, and instructions on how to use these modules are available free of charge for research and educational purposes online at https://github.com/ArcesUnibo/creamino. Application cases show how the system can be used for neuroscientific or BCI experiments. Thanks to its low production cost and its compatibility with open-source BCI tools, the system presented is particularly suitable for use in BCI research and educational applications.
Collapse
|
24
|
Abstract
Brain activity and connectivity are distributed in the three-dimensional space and evolve in time. It is important to image brain dynamics with high spatial and temporal resolution. Electroencephalography (EEG) and magnetoencephalography (MEG) are noninvasive measurements associated with complex neural activations and interactions that encode brain functions. Electrophysiological source imaging estimates the underlying brain electrical sources from EEG and MEG measurements. It offers increasingly improved spatial resolution and intrinsically high temporal resolution for imaging large-scale brain activity and connectivity on a wide range of timescales. Integration of electrophysiological source imaging and functional magnetic resonance imaging could further enhance spatiotemporal resolution and specificity to an extent that is not attainable with either technique alone. We review methodological developments in electrophysiological source imaging over the past three decades and envision its future advancement into a powerful functional neuroimaging technology for basic and clinical neuroscience applications.
Collapse
Affiliation(s)
- Bin He
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA;
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minnesota 55455, USA
| | - Abbas Sohrabpour
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minnesota 55455, USA
| | - Emery Brown
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
| | - Zhongming Liu
- Weldon School of Biomedical Engineering, School of Electrical and Computer Engineering, and Purdue Institute of Integrative Neuroscience, Purdue University, West Lafayette, Indiana 47906, USA
| |
Collapse
|
25
|
Functional Brain Connectivity during Multiple Motor Imagery Tasks in Spinal Cord Injury. Neural Plast 2018; 2018:9354207. [PMID: 29853852 PMCID: PMC5954936 DOI: 10.1155/2018/9354207] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2017] [Revised: 03/06/2018] [Accepted: 03/21/2018] [Indexed: 12/18/2022] Open
Abstract
Reciprocal communication of the central and peripheral nervous systems is compromised during spinal cord injury due to neurotrauma of ascending and descending pathways. Changes in brain organization after spinal cord injury have been associated with differences in prognosis. Changes in functional connectivity may also serve as injury biomarkers. Most studies on functional connectivity have focused on chronic complete injury or resting-state condition. In our study, ten right-handed patients with incomplete spinal cord injury and ten age- and gender-matched healthy controls performed multiple visual motor imagery tasks of upper extremities and walking under high-resolution electroencephalography recording. Directed transfer function was used to study connectivity at the cortical source space between sensorimotor nodes. Chronic disruption of reciprocal communication in incomplete injury could result in permanent significant decrease of connectivity in a subset of the sensorimotor network, regardless of positive or negative neurological outcome. Cingulate motor areas consistently contributed the larger outflow (right) and received the higher inflow (left) among all nodes, across all motor imagery categories, in both groups. Injured subjects had higher outflow from left cingulate than healthy subjects and higher inflow in right cingulate than healthy subjects. Alpha networks were less dense, showing less integration and more segregation than beta networks. Spinal cord injury patients showed signs of increased local processing as adaptive mechanism. This trial is registered with NCT02443558.
Collapse
|
26
|
Dinh C, Esch L, Rühle J, Bollmann S, Güllmar D, Baumgarten D, Hämäläinen MS, Haueisen J. Real-Time Clustered Multiple Signal Classification (RTC-MUSIC). Brain Topogr 2018; 31:125-128. [PMID: 28879632 PMCID: PMC5773364 DOI: 10.1007/s10548-017-0586-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2016] [Accepted: 08/28/2017] [Indexed: 02/07/2023]
Abstract
Magnetoencephalography (MEG) and electroencephalography provide a high temporal resolution, which allows estimation of the detailed time courses of neuronal activity. However, in real-time analysis of these data two major challenges must be handled: the low signal-to-noise ratio (SNR) and the limited time available for computations. In this work, we present real-time clustered multiple signal classification (RTC-MUSIC) a real-time source localization algorithm, which can handle low SNRs and can reduce the computational effort. It provides correlation information together with sparse source estimation results, which can, e.g., be used to identify evoked responses with high sensitivity. RTC-MUSIC clusters the forward solution based on an anatomical brain atlas and optimizes the scanning process inherent to MUSIC approaches. We evaluated RTC-MUSIC by analyzing MEG auditory and somatosensory data. The results demonstrate that the proposed method localizes sources reliably. For the auditory experiment the most dominant correlated source pair was located bilaterally in the superior temporal gyri. The highest activation in the somatosensory experiment was found in the contra-lateral primary somatosensory cortex.
Collapse
Affiliation(s)
- Christoph Dinh
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital - Massachusetts Institute of Technology - Harvard Medical School, 149 13th St., Charlestown, MA, 02129, USA.
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, Ilmenau, Germany.
| | - Lorenz Esch
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, Ilmenau, Germany
| | - Johannes Rühle
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, Ilmenau, Germany
| | - Steffen Bollmann
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, Ilmenau, Germany
- Centre for Advanced Imaging, University of Queensland, Brisbane, Australia
| | - Daniel Güllmar
- Medical Physics Group, Institute of Diagnostic and Interventional Radiology, University Hospital Jena, Jena, Germany
| | - Daniel Baumgarten
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, Ilmenau, Germany
- Institute of Electrical and Biomedical Engineering, University of Health Sciences, Medical Informatics and Technology (UMIT), Hall in Tirol, Austria
| | - Matti S Hämäläinen
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital - Massachusetts Institute of Technology - Harvard Medical School, 149 13th St., Charlestown, MA, 02129, USA
| | - Jens Haueisen
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, Ilmenau, Germany
- Biomagnetic Center, Clinic for Neurology, Jena University Hospital, Jena, Germany
| |
Collapse
|
27
|
Baxter BS, Edelman BJ, Sohrabpour A, He B. Anodal Transcranial Direct Current Stimulation Increases Bilateral Directed Brain Connectivity during Motor-Imagery Based Brain-Computer Interface Control. Front Neurosci 2017; 11:691. [PMID: 29270110 PMCID: PMC5725434 DOI: 10.3389/fnins.2017.00691] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2017] [Accepted: 11/23/2017] [Indexed: 01/29/2023] Open
Abstract
Transcranial direct current stimulation (tDCS) has been shown to affect motor and cognitive task performance and learning when applied to brain areas involved in the task. Targeted stimulation has also been found to alter connectivity within the stimulated hemisphere during rest. However, the connectivity effect of the interaction of endogenous task specific activity and targeted stimulation is unclear. This study examined the aftereffects of concurrent anodal high-definition tDCS over the left sensorimotor cortex with motor network connectivity during a one-dimensional EEG based sensorimotor rhythm brain-computer interface (SMR-BCI) task. Directed connectivity following anodal tDCS illustrates altered connections bilaterally between frontal and parietal regions, and these alterations occur in a task specific manner; connections between similar cortical regions are altered differentially during left and right imagination trials. During right-hand imagination following anodal tDCS, there was an increase in outflow from the left premotor cortex (PMC) to multiple regions bilaterally in the motor network and increased inflow to the stimulated sensorimotor cortex from the ipsilateral PMC and contralateral sensorimotor cortex. During left-hand imagination following anodal tDCS, there was increased outflow from the stimulated sensorimotor cortex to regions across the motor network. Significant correlations between connectivity and the behavioral measures of total correct trials and time-to-hit (TTH) correct trials were also found, specifically that the input to the left PMC correlated with decreased right hand imagination performance and that flow from the ipsilateral posterior parietal cortex (PPC) to midline sensorimotor cortex correlated with improved performance for both right and left hand imagination. These results indicate that tDCS interacts with task-specific endogenous activity to alter directed connectivity during SMR-BCI. In order to predict and maximize the targeted effect of tDCS, the interaction of stimulation with the dynamics of endogenous activity needs to be examined comprehensively and understood.
Collapse
Affiliation(s)
- Bryan S. Baxter
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, United States
| | - Bradley J. Edelman
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, United States
| | - Abbas Sohrabpour
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, United States
| | - Bin He
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, United States
- Institute for Engineering in Medicine, University of Minnesota, Minneapolis, MN, United States
| |
Collapse
|
28
|
Delisle-Rodriguez D, Villa-Parra AC, Bastos-Filho T, López-Delis A, Frizera-Neto A, Krishnan S, Rocon E. Adaptive Spatial Filter Based on Similarity Indices to Preserve the Neural Information on EEG Signals during On-Line Processing. SENSORS 2017; 17:s17122725. [PMID: 29186848 PMCID: PMC5751387 DOI: 10.3390/s17122725] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/27/2017] [Revised: 11/13/2017] [Accepted: 11/19/2017] [Indexed: 12/20/2022]
Abstract
This work presents a new on-line adaptive filter, which is based on a similarity analysis between standard electrode locations, in order to reduce artifacts and common interferences throughout electroencephalography (EEG) signals, but preserving the useful information. Standard deviation and Concordance Correlation Coefficient (CCC) between target electrodes and its correspondent neighbor electrodes are analyzed on sliding windows to select those neighbors that are highly correlated. Afterwards, a model based on CCC is applied to provide higher values of weight to those correlated electrodes with lower similarity to the target electrode. The approach was applied to brain computer-interfaces (BCIs) based on Canonical Correlation Analysis (CCA) to recognize 40 targets of steady-state visual evoked potential (SSVEP), providing an accuracy (ACC) of 86.44 ± 2.81%. In addition, also using this approach, features of low frequency were selected in the pre-processing stage of another BCI to recognize gait planning. In this case, the recognition was significantly (p<0.01) improved for most of the subjects (ACC≥74.79%), when compared with other BCIs based on Common Spatial Pattern, Filter Bank-Common Spatial Pattern, and Riemannian Geometry.
Collapse
Affiliation(s)
- Denis Delisle-Rodriguez
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo, 29075-910 Vitoria, Brazil.
- Center of Medical Biophysics, University of Oriente, 90500 Santiago de Cuba, Cuba.
| | - Ana Cecilia Villa-Parra
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo, 29075-910 Vitoria, Brazil.
- Biomedical Engineering Research Group GIIB, Universidad Politécnica Salesiana, 010105 Cuenca, Ecuador.
| | - Teodiano Bastos-Filho
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo, 29075-910 Vitoria, Brazil.
| | - Alberto López-Delis
- Center of Medical Biophysics, University of Oriente, 90500 Santiago de Cuba, Cuba.
| | - Anselmo Frizera-Neto
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo, 29075-910 Vitoria, Brazil.
| | - Sridhar Krishnan
- Department of Electrical and Computer Engineering, Ryerson University, Toronto, ON M5B 2K3, Canada.
| | - Eduardo Rocon
- Centre for Automation and Robotics, CSIC-UPM, 28500 Madrid, Spain.
| |
Collapse
|
29
|
Wang K, Wang Z, Guo Y, He F, Qi H, Xu M, Ming D. A brain-computer interface driven by imagining different force loads on a single hand: an online feasibility study. J Neuroeng Rehabil 2017; 14:93. [PMID: 28893295 PMCID: PMC5594542 DOI: 10.1186/s12984-017-0307-1] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2017] [Accepted: 09/05/2017] [Indexed: 11/10/2022] Open
Abstract
Background Motor imagery (MI) induced EEG patterns are widely used as control signals for brain-computer interfaces (BCIs). Kinetic and kinematic factors have been proved to be able to change EEG patterns during motor execution and motor imagery. However, to our knowledge, there is still no literature reporting an effective online MI-BCI using kinetic factor regulated EEG oscillations. This study proposed a novel MI-BCI paradigm in which users can online output multiple commands by imagining clenching their right hand with different force loads. Methods Eleven subjects participated in this study. During the experiment, they were asked to imagine clenching their right hands with two different force loads (30% maximum voluntary contraction (MVC) and 10% MVC). Multi-Common spatial patterns (Multi-CSPs) and support vector machines (SVMs) were used to build the classifier for recognizing three commands corresponding to high load MI, low load MI and relaxed status respectively. EMG were monitored to avoid voluntary muscle activities during the BCI operation. The event-related spectral perturbation (ERSP) method was used to analyse EEG variation during multiple load MI tasks. Results All subjects were able to drive BCI systems using motor imagery of different force loads in online experiments. We achieved an average online accuracy of 70.9%, with the highest accuracy of 83.3%, which was much higher than the chance level (33%). The event-related desynchronization (ERD) phenomenon during high load tasks was significantly higher than it was during low load tasks both in terms of intensity at electrode positions C3 (p < 0.05) and spatial distribution. Conclusions This paper demonstrated the feasibility of the proposed MI-BCI paradigm based on multi-force loads on the same limb through online studies. This paradigm could not only enlarge the command set of MI-BCI, but also provide a promising approach to rehabilitate patients with motor disabilities.
Collapse
Affiliation(s)
- Kun Wang
- Lab of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China.,Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Zhongpeng Wang
- Lab of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China.,Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Yi Guo
- Lab of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China.,Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Feng He
- Lab of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China.,Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Hongzhi Qi
- Lab of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China. .,Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China.
| | - Minpeng Xu
- Lab of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China.,Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Dong Ming
- Lab of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China. .,Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China.
| |
Collapse
|
30
|
Alazrai R, Alwanni H, Baslan Y, Alnuman N, Daoud MI. EEG-Based Brain-Computer Interface for Decoding Motor Imagery Tasks within the Same Hand Using Choi-Williams Time-Frequency Distribution. SENSORS 2017; 17:s17091937. [PMID: 28832513 PMCID: PMC5621048 DOI: 10.3390/s17091937] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/21/2017] [Revised: 08/16/2017] [Accepted: 08/21/2017] [Indexed: 11/24/2022]
Abstract
This paper presents an EEG-based brain-computer interface system for classifying eleven motor imagery (MI) tasks within the same hand. The proposed system utilizes the Choi-Williams time-frequency distribution (CWD) to construct a time-frequency representation (TFR) of the EEG signals. The constructed TFR is used to extract five categories of time-frequency features (TFFs). The TFFs are processed using a hierarchical classification model to identify the MI task encapsulated within the EEG signals. To evaluate the performance of the proposed approach, EEG data were recorded for eighteen intact subjects and four amputated subjects while imagining to perform each of the eleven hand MI tasks. Two performance evaluation analyses, namely channel- and TFF-based analyses, are conducted to identify the best subset of EEG channels and the TFFs category, respectively, that enable the highest classification accuracy between the MI tasks. In each evaluation analysis, the hierarchical classification model is trained using two training procedures, namely subject-dependent and subject-independent procedures. These two training procedures quantify the capability of the proposed approach to capture both intra- and inter-personal variations in the EEG signals for different MI tasks within the same hand. The results demonstrate the efficacy of the approach for classifying the MI tasks within the same hand. In particular, the classification accuracies obtained for the intact and amputated subjects are as high as 88.8% and 90.2%, respectively, for the subject-dependent training procedure, and 80.8% and 87.8%, respectively, for the subject-independent training procedure. These results suggest the feasibility of applying the proposed approach to control dexterous prosthetic hands, which can be of great benefit for individuals suffering from hand amputations.
Collapse
Affiliation(s)
- Rami Alazrai
- Department of Computer Engineering, School of Electrical Engineering and Information Technology, German Jordanian University, Amman 11180, Jordan.
| | - Hisham Alwanni
- Faculty of Engineering, University of Freiburg, Freiburg 79098, Germany.
| | - Yara Baslan
- Department of Biomedical Engineering, School of Applied Medical Sciences, German Jordanian University, Amman 11180, Jordan.
| | - Nasim Alnuman
- Department of Biomedical Engineering, School of Applied Medical Sciences, German Jordanian University, Amman 11180, Jordan.
| | - Mohammad I Daoud
- Department of Computer Engineering, School of Electrical Engineering and Information Technology, German Jordanian University, Amman 11180, Jordan.
| |
Collapse
|
31
|
Abstract
OBJECTIVE In brain-computer interfaces (BCI), measurements of the user's brain activity are classified into commands for the computer. With EEG-based BCIs, the origins of the classified phenomena are often considered to be spatially localized in the cortical volume and mixed in the EEG. We investigate if more accurate BCIs can be obtained by reconstructing the source activities in the volume. APPROACH We contrast the physiology-driven source reconstruction with data-driven representations obtained by statistical machine learning. We explain these approaches in a common linear dictionary framework and review the different ways to obtain the dictionary parameters. We consider the effect of source reconstruction on some major difficulties in BCI classification, namely information loss, feature selection and nonstationarity of the EEG. MAIN RESULTS Our analysis suggests that the approaches differ mainly in their parameter estimation. Physiological source reconstruction may thus be expected to improve BCI accuracy if machine learning is not used or where it produces less optimal parameters. We argue that the considered difficulties of surface EEG classification can remain in the reconstructed volume and that data-driven techniques are still necessary. Finally, we provide some suggestions for comparing approaches. SIGNIFICANCE The present work illustrates the relationships between source reconstruction and machine learning-based approaches for EEG data representation. The provided analysis and discussion should help in understanding, applying, comparing and improving such techniques in the future.
Collapse
|
32
|
Deshpande G, Rangaprakash D, Oeding L, Cichocki A, Hu XP. A New Generation of Brain-Computer Interfaces Driven by Discovery of Latent EEG-fMRI Linkages Using Tensor Decomposition. Front Neurosci 2017. [PMID: 28638316 PMCID: PMC5461249 DOI: 10.3389/fnins.2017.00246] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
A Brain-Computer Interface (BCI) is a setup permitting the control of external devices by decoding brain activity. Electroencephalography (EEG) has been extensively used for decoding brain activity since it is non-invasive, cheap, portable, and has high temporal resolution to allow real-time operation. Due to its poor spatial specificity, BCIs based on EEG can require extensive training and multiple trials to decode brain activity (consequently slowing down the operation of the BCI). On the other hand, BCIs based on functional magnetic resonance imaging (fMRI) are more accurate owing to its superior spatial resolution and sensitivity to underlying neuronal processes which are functionally localized. However, due to its relatively low temporal resolution, high cost, and lack of portability, fMRI is unlikely to be used for routine BCI. We propose a new approach for transferring the capabilities of fMRI to EEG, which includes simultaneous EEG/fMRI sessions for finding a mapping from EEG to fMRI, followed by a BCI run from only EEG data, but driven by fMRI-like features obtained from the mapping identified previously. Our novel data-driven method is likely to discover latent linkages between electrical and hemodynamic signatures of neural activity hitherto unexplored using model-driven methods, and is likely to serve as a template for a novel multi-modal strategy wherein cross-modal EEG-fMRI interactions are exploited for the operation of a unimodal EEG system, leading to a new generation of EEG-based BCIs.
Collapse
Affiliation(s)
- Gopikrishna Deshpande
- AU MRI Research Center, Department of Electrical and Computer Engineering, Auburn UniversityAuburn, AL, USA.,Department of Psychology, Auburn UniversityAuburn, AL, USA.,Alabama Advanced Imaging Consortium, Auburn University and University of Alabama at BirminghamBirmingham, AL, USA
| | - D Rangaprakash
- AU MRI Research Center, Department of Electrical and Computer Engineering, Auburn UniversityAuburn, AL, USA.,Department of Psychiatry and Biobehavioral Sciences, University of California, Los AngelesLos Angeles, CA, USA
| | - Luke Oeding
- Department of Mathematics and Statistics, Auburn UniversityAuburn, AL, USA
| | - Andrzej Cichocki
- Skolkovo Institute of Science and Technology (Skoltech)Moscow, Russia.,Nicolaus Copernicus University (UMK)Torun, Poland.,Systems Research Institute, Polish Academy of ScienceWarsaw, Poland
| | - Xiaoping P Hu
- Department of Bioengineering, University of California, RiversideRiverside, CA, USA
| |
Collapse
|
33
|
Lebedev MA, Nicolelis MAL. Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. Physiol Rev 2017; 97:767-837. [PMID: 28275048 DOI: 10.1152/physrev.00027.2016] [Citation(s) in RCA: 235] [Impact Index Per Article: 33.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023] Open
Abstract
Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.
Collapse
|
34
|
Sun X, Qian C, Chen Z, Wu Z, Luo B, Pan G. Remembered or Forgotten?-An EEG-Based Computational Prediction Approach. PLoS One 2016; 11:e0167497. [PMID: 27973531 PMCID: PMC5156350 DOI: 10.1371/journal.pone.0167497] [Citation(s) in RCA: 30] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2016] [Accepted: 11/15/2016] [Indexed: 01/29/2023] Open
Abstract
Prediction of memory performance (remembered or forgotten) has various potential applications not only for knowledge learning but also for disease diagnosis. Recently, subsequent memory effects (SMEs)—the statistical differences in electroencephalography (EEG) signals before or during learning between subsequently remembered and forgotten events—have been found. This finding indicates that EEG signals convey the information relevant to memory performance. In this paper, based on SMEs we propose a computational approach to predict memory performance of an event from EEG signals. We devise a convolutional neural network for EEG, called ConvEEGNN, to predict subsequently remembered and forgotten events from EEG recorded during memory process. With the ConvEEGNN, prediction of memory performance can be achieved by integrating two main stages: feature extraction and classification. To verify the proposed approach, we employ an auditory memory task to collect EEG signals from scalp electrodes. For ConvEEGNN, the average prediction accuracy was 72.07% by using EEG data from pre-stimulus and during-stimulus periods, outperforming other approaches. It was observed that signals from pre-stimulus period and those from during-stimulus period had comparable contributions to memory performance. Furthermore, the connection weights of ConvEEGNN network can reveal prominent channels, which are consistent with the distribution of SME studied previously.
Collapse
Affiliation(s)
- Xuyun Sun
- College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, China
| | - Cunle Qian
- College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, China
| | - Zhongqin Chen
- The First Affiliated Hospital of Medical School, Zhejiang University, Hangzhou, Zhejiang, China
| | - Zhaohui Wu
- College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, China
| | - Benyan Luo
- The First Affiliated Hospital of Medical School, Zhejiang University, Hangzhou, Zhejiang, China
- * E-mail: (BL); (GP)
| | - Gang Pan
- College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, China
- * E-mail: (BL); (GP)
| |
Collapse
|
35
|
Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks. Sci Rep 2016; 6:38565. [PMID: 27966546 PMCID: PMC5155290 DOI: 10.1038/srep38565] [Citation(s) in RCA: 190] [Impact Index Per Article: 23.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2016] [Accepted: 11/10/2016] [Indexed: 12/27/2022] Open
Abstract
Brain-computer interface (BCI) technologies aim to provide a bridge between the human brain and external devices. Prior research using non-invasive BCI to control virtual objects, such as computer cursors and virtual helicopters, and real-world objects, such as wheelchairs and quadcopters, has demonstrated the promise of BCI technologies. However, controlling a robotic arm to complete reach-and-grasp tasks efficiently using non-invasive BCI has yet to be shown. In this study, we found that a group of 13 human subjects could willingly modulate brain activity to control a robotic arm with high accuracy for performing tasks requiring multiple degrees of freedom by combination of two sequential low dimensional controls. Subjects were able to effectively control reaching of the robotic arm through modulation of their brain rhythms within the span of only a few training sessions and maintained the ability to control the robotic arm over multiple months. Our results demonstrate the viability of human operation of prosthetic limbs using non-invasive BCI technology.
Collapse
|
36
|
Baxter BS, Edelman BJ, Nesbitt N, He B. Sensorimotor Rhythm BCI with Simultaneous High Definition-Transcranial Direct Current Stimulation Alters Task Performance. Brain Stimul 2016; 9:834-841. [PMID: 27522166 PMCID: PMC5143161 DOI: 10.1016/j.brs.2016.07.003] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2016] [Revised: 07/10/2016] [Accepted: 07/12/2016] [Indexed: 01/26/2023] Open
Abstract
BACKGROUND Transcranial direct current stimulation (tDCS) has been used to alter the excitability of neurons within the cerebral cortex. Improvements in motor learning have been found in multiple studies when tDCS was applied to the motor cortex before or during task learning. The motor cortex is also active during the performance of motor imagination, a cognitive task during which a person imagines, but does not execute, a movement. Motor imagery can be used with noninvasive brain computer interfaces (BCIs) to control virtual objects in up to three dimensions, but to master control of such devices requires long training times. OBJECTIVE To evaluate the effect of high-definition tDCS on the performance and underlying electrophysiology of motor imagery based BCI. METHODS We utilize high-definition tDCS to investigate the effect of stimulation on motor imagery-based BCI performance across and within sessions over multiple training days. RESULTS We report a decreased time-to-hit with anodal stimulation both within and across sessions. We also found differing electrophysiological changes of the stimulated sensorimotor cortex during online BCI task performance for left vs. right trials. Cathodal stimulation led to a decrease in alpha and beta band power during task performance compared to sham stimulation for right hand imagination trials. CONCLUSION These results suggest that unilateral tDCS over the sensorimotor motor cortex differentially affects cortical areas based on task specific neural activation.
Collapse
Affiliation(s)
- Bryan S Baxter
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Bradley J Edelman
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Nicholas Nesbitt
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Bin He
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA; Institute for Engineering in Medicine, University of Minnesota, Minneapolis, MN, USA.
| |
Collapse
|
37
|
Shim M, Hwang HJ, Kim DW, Lee SH, Im CH. Machine-learning-based diagnosis of schizophrenia using combined sensor-level and source-level EEG features. Schizophr Res 2016; 176:314-319. [PMID: 27427557 DOI: 10.1016/j.schres.2016.05.007] [Citation(s) in RCA: 71] [Impact Index Per Article: 8.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/30/2016] [Revised: 04/25/2016] [Accepted: 05/05/2016] [Indexed: 11/30/2022]
Abstract
Recently, an increasing number of researchers have endeavored to develop practical tools for diagnosing patients with schizophrenia using machine learning techniques applied to EEG biomarkers. Although a number of studies showed that source-level EEG features can potentially be applied to the differential diagnosis of schizophrenia, most studies have used only sensor-level EEG features such as ERP peak amplitude and power spectrum for machine learning-based diagnosis of schizophrenia. In this study, we used both sensor-level and source-level features extracted from EEG signals recorded during an auditory oddball task for the classification of patients with schizophrenia and healthy controls. EEG signals were recorded from 34 patients with schizophrenia and 34 healthy controls while each subject was asked to attend to oddball tones. Our results demonstrated higher classification accuracy when source-level features were used together with sensor-level features, compared to when only sensor-level features were used. In addition, the selected sensor-level features were mostly found in the frontal area, and the selected source-level features were mostly extracted from the temporal area, which coincide well with the well-known pathological region of cognitive processing in patients with schizophrenia. Our results suggest that our approach would be a promising tool for the computer-aided diagnosis of schizophrenia.
Collapse
Affiliation(s)
- Miseon Shim
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea
| | - Han-Jeong Hwang
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, South Korea
| | - Do-Won Kim
- Berlin Institute of Technology, Machine Learning Group, Marchstrasse 23, Berlin 10587, Germany
| | - Seung-Hwan Lee
- Psychiatry Department, Ilsan Paik Hospital, Inje University, Goyang, South Korea
| | - Chang-Hwan Im
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea.
| |
Collapse
|
38
|
Zhou B, Wu X, Lv Z, Zhang L, Guo X. A Fully Automated Trial Selection Method for Optimization of Motor Imagery Based Brain-Computer Interface. PLoS One 2016; 11:e0162657. [PMID: 27631789 PMCID: PMC5025076 DOI: 10.1371/journal.pone.0162657] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2016] [Accepted: 08/28/2016] [Indexed: 11/24/2022] Open
Abstract
Independent component analysis (ICA) as a promising spatial filtering method can separate motor-related independent components (MRICs) from the multichannel electroencephalogram (EEG) signals. However, the unpredictable burst interferences may significantly degrade the performance of ICA-based brain-computer interface (BCI) system. In this study, we proposed a new algorithm frame to address this issue by combining the single-trial-based ICA filter with zero-training classifier. We developed a two-round data selection method to identify automatically the badly corrupted EEG trials in the training set. The "high quality" training trials were utilized to optimize the ICA filter. In addition, we proposed an accuracy-matrix method to locate the artifact data segments within a single trial and investigated which types of artifacts can influence the performance of the ICA-based MIBCIs. Twenty-six EEG datasets of three-class motor imagery were used to validate the proposed methods, and the classification accuracies were compared with that obtained by frequently used common spatial pattern (CSP) spatial filtering algorithm. The experimental results demonstrated that the proposed optimizing strategy could effectively improve the stability, practicality and classification performance of ICA-based MIBCI. The study revealed that rational use of ICA method may be crucial in building a practical ICA-based MIBCI system.
Collapse
Affiliation(s)
- Bangyan Zhou
- Key Laboratory of Intelligent Computing & Signal Processing, Ministry of Education, Anhui University, Hefei, China
- School of Computer Science and Technology, Anhui University, Hefei, China
| | - Xiaopei Wu
- Key Laboratory of Intelligent Computing & Signal Processing, Ministry of Education, Anhui University, Hefei, China
- School of Computer Science and Technology, Anhui University, Hefei, China
| | - Zhao Lv
- Key Laboratory of Intelligent Computing & Signal Processing, Ministry of Education, Anhui University, Hefei, China
- School of Computer Science and Technology, Anhui University, Hefei, China
| | - Lei Zhang
- Key Laboratory of Intelligent Computing & Signal Processing, Ministry of Education, Anhui University, Hefei, China
- School of Computer Science and Technology, Anhui University, Hefei, China
| | - Xiaojin Guo
- Key Laboratory of Intelligent Computing & Signal Processing, Ministry of Education, Anhui University, Hefei, China
- School of Computer Science and Technology, Anhui University, Hefei, China
| |
Collapse
|
39
|
Wronkiewicz M, Larson E, Lee AKC. Incorporating modern neuroscience findings to improve brain–computer interfaces: tracking auditory attention. J Neural Eng 2016; 13:056017. [DOI: 10.1088/1741-2560/13/5/056017] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
40
|
Hamedi M, Salleh SH, Noor AM. Electroencephalographic Motor Imagery Brain Connectivity Analysis for BCI: A Review. Neural Comput 2016; 28:999-1041. [PMID: 27137671 DOI: 10.1162/neco_a_00838] [Citation(s) in RCA: 97] [Impact Index Per Article: 12.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Recent research has reached a consensus on the feasibility of motor imagery brain-computer interface (MI-BCI) for different applications, especially in stroke rehabilitation. Most MI-BCI systems rely on temporal, spectral, and spatial features of single channels to distinguish different MI patterns. However, no successful communication has been established for a completely locked-in subject. To provide more useful and informative features, it has been recommended to take into account the relationships among electroencephalographic (EEG) sensor/source signals in the form of brain connectivity as an efficient tool of neuroscience. In this review, we briefly report the challenges and limitations of conventional MI-BCIs. Brain connectivity analysis, particularly functional and effective, has been described as one of the most promising approaches for improving MI-BCI performance. An extensive literature on EEG-based MI brain connectivity analysis of healthy subjects is reviewed. We subsequently discuss the brain connectomes during left and right hand, feet, and tongue MI movements. Moreover, key components involved in brain connectivity analysis that considerably affect the results are explained. Finally, possible technical shortcomings that may have influenced the results in previous research are addressed and suggestions are provided.
Collapse
Affiliation(s)
- Mahyar Hamedi
- Center for Biomedical Engineering and Faculty of Bioscience and Medical Engineering, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Malaysia
| | - Sh-Hussain Salleh
- Center for Biomedical Engineering, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Malaysia
| | - Alias Mohd Noor
- Center for Biomedical Engineering, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Malaysia
| |
Collapse
|
41
|
Edelman BJ, Baxter B, He B. EEG Source Imaging Enhances the Decoding of Complex Right-Hand Motor Imagery Tasks. IEEE Trans Biomed Eng 2016; 63:4-14. [PMID: 26276986 PMCID: PMC4716869 DOI: 10.1109/tbme.2015.2467312] [Citation(s) in RCA: 152] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
GOAL Sensorimotor-based brain-computer interfaces (BCIs) have achieved successful control of real and virtual devices in up to three dimensions; however, the traditional sensor-based paradigm limits the intuitive use of these systems. Many control signals for state-of-the-art BCIs involve imagining the movement of body parts that have little to do with the output command, revealing a cognitive disconnection between the user's intent and the action of the end effector. Therefore, there is a need to develop techniques that can identify with high spatial resolution the self-modulated neural activity reflective of the actions of a helpful output device. METHODS We extend previous EEG source imaging (ESI) work to decoding natural hand/wrist manipulations by applying a novel technique to classifying four complex motor imaginations of the right hand: flexion, extension, supination, and pronation. RESULTS We report an increase of up to 18.6% for individual task classification and 12.7% for overall classification using the proposed ESI approach over the traditional sensor-based method. CONCLUSION ESI is able to enhance BCI performance of decoding complex right-hand motor imagery tasks. SIGNIFICANCE This study may lead to the development of BCI systems with naturalistic and intuitive motor imaginations, thus facilitating broad use of noninvasive BCIs.
Collapse
Affiliation(s)
- Bradley J. Edelman
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA ()
| | - Bryan Baxter
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA ()
| | - Bin He
- Department of Biomedical Engineering and Institute for Engineering in Medicine, University of Minnesota, Minneapolis, MN 55455, USA ()
| |
Collapse
|
42
|
Edelman BJ, Johnson N, Sohrabpour A, Tong S, Thakor N, He B. Systems Neuroengineering: Understanding and Interacting with the Brain. ENGINEERING (BEIJING, CHINA) 2015; 1:292-308. [PMID: 34336364 PMCID: PMC8323844 DOI: 10.15302/j-eng-2015078] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/15/2023]
Abstract
In this paper, we review the current state-of-the-art techniques used for understanding the inner workings of the brain at a systems level. The neural activity that governs our everyday lives involves an intricate coordination of many processes that can be attributed to a variety of brain regions. On the surface, many of these functions can appear to be controlled by specific anatomical structures; however, in reality, numerous dynamic networks within the brain contribute to its function through an interconnected web of neuronal and synaptic pathways. The brain, in its healthy or pathological state, can therefore be best understood by taking a systems-level approach. While numerous neuroengineering technologies exist, we focus here on three major thrusts in the field of systems neuroengineering: neuroimaging, neural interfacing, and neuromodulation. Neuroimaging enables us to delineate the structural and functional organization of the brain, which is key in understanding how the neural system functions in both normal and disease states. Based on such knowledge, devices can be used either to communicate with the neural system, as in neural interface systems, or to modulate brain activity, as in neuromodulation systems. The consideration of these three fields is key to the development and application of neuro-devices. Feedback-based neuro-devices require the ability to sense neural activity (via a neuroimaging modality) through a neural interface (invasive or noninvasive) and ultimately to select a set of stimulation parameters in order to alter neural function via a neuromodulation modality. Systems neuroengineering refers to the use of engineering tools and technologies to image, decode, and modulate the brain in order to comprehend its functions and to repair its dysfunction. Interactions between these fields will help to shape the future of systems neuroengineering-to develop neurotechniques for enhancing the understanding of whole-brain function and dysfunction, and the management of neurological and mental disorders.
Collapse
Affiliation(s)
- Bradley J. Edelman
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Nessa Johnson
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Abbas Sohrabpour
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Shanbao Tong
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China
| | - Nitish Thakor
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21205, USA
- SINAPSE Institute, National University of Singapore, Singapore
| | - Bin He
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
- Institute for Engineering in Medicine, University of Minnesota, Minneapolis, MN 55455, USA
| |
Collapse
|
43
|
Yi H, Zhang X, Bai W, Liu T, Tian X. Canonical correlation between LFP network and spike network during working memory task in rat. Behav Brain Res 2015; 289:84-91. [DOI: 10.1016/j.bbr.2015.04.042] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2014] [Revised: 04/18/2015] [Accepted: 04/23/2015] [Indexed: 11/28/2022]
|
44
|
Wronkiewicz M, Larson E, Lee AKC. Leveraging anatomical information to improve transfer learning in brain-computer interfaces. J Neural Eng 2015; 12:046027. [PMID: 26169961 DOI: 10.1088/1741-2560/12/4/046027] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Brain-computer interfaces (BCIs) represent a technology with the potential to rehabilitate a range of traumatic and degenerative nervous system conditions but require a time-consuming training process to calibrate. An area of BCI research known as transfer learning is aimed at accelerating training by recycling previously recorded training data across sessions or subjects. Training data, however, is typically transferred from one electrode configuration to another without taking individual head anatomy or electrode positioning into account, which may underutilize the recycled data. APPROACH We explore transfer learning with the use of source imaging, which estimates neural activity in the cortex. Transferring estimates of cortical activity, in contrast to scalp recordings, provides a way to compensate for variability in electrode positioning and head morphologies across subjects and sessions. MAIN RESULTS Based on simulated and measured electroencephalography activity, we trained a classifier using data transferred exclusively from other subjects and achieved accuracies that were comparable to or surpassed a benchmark classifier (representative of a real-world BCI). Our results indicate that classification improvements depend on the number of trials transferred and the cortical region of interest. SIGNIFICANCE These findings suggest that cortical source-based transfer learning is a principled method to transfer data that improves BCI classification performance and provides a path to reduce BCI calibration time.
Collapse
Affiliation(s)
- Mark Wronkiewicz
- Graduate Program in Neuroscience University of Washington, Box 357270, Seattle, WA 98195, USA
| | | | | |
Collapse
|
45
|
He B, Baxter B, Edelman BJ, Cline CC, Ye W. Noninvasive Brain-Computer Interfaces Based on Sensorimotor Rhythms. PROCEEDINGS OF THE IEEE. INSTITUTE OF ELECTRICAL AND ELECTRONICS ENGINEERS 2015; 103:907-925. [PMID: 34334804 PMCID: PMC8323842 DOI: 10.1109/jproc.2015.2407272] [Citation(s) in RCA: 97] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Brain-computer interfaces (BCIs) have been explored in the field of neuroengineering to investigate how the brain can use these systems to control external devices. We review the principles and approaches we have taken to develop a sensorimotor rhythm EEG based brain-computer interface (BCI). The methods include developing BCI systems incorporating the control of physical devices to increase user engagement, improving BCI systems by inversely mapping scalp-recorded EEG signals to the cortical source domain, integrating BCI with noninvasive neuromodulation strategies to improve learning, and incorporating mind-body awareness training to enhance BCI learning and performance. The challenges and merits of these strategies are discussed, together with recent findings. Our work indicates that the sensorimotor-rhythm-based noninvasive BCI has the potential to provide communication and control capabilities as an alternative to physiological motor pathways.
Collapse
Affiliation(s)
- Bin He
- Department of Biomedical Engineering, University of Minnesota
- Institute for Engineering in Medicine, University of Minnesota
| | - Bryan Baxter
- Department of Biomedical Engineering, University of Minnesota
| | | | | | | |
Collapse
|
46
|
Liu K, Yu ZL, Wu W, Gu Z, Li Y. STRAPS: A Fully Data-Driven Spatio-Temporally Regularized Algorithm for M/EEG Patch Source Imaging. Int J Neural Syst 2015; 25:1550016. [DOI: 10.1142/s0129065715500161] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
For M/EEG-based distributed source imaging, it has been established that the L2-norm-based methods are effective in imaging spatially extended sources, whereas the L1-norm-based methods are more suited for estimating focal and sparse sources. However, when the spatial extents of the sources are unknown a priori, the rationale for using either type of methods is not adequately supported. Bayesian inference by exploiting the spatio-temporal information of the patch sources holds great promise as a tool for adaptive source imaging, but both computational and methodological limitations remain to be overcome. In this paper, based on state-space modeling of the M/EEG data, we propose a fully data-driven and scalable algorithm, termed STRAPS, for M/EEG patch source imaging on high-resolution cortices. Unlike the existing algorithms, the recursive penalized least squares (RPLS) procedure is employed to efficiently estimate the source activities as opposed to the computationally demanding Kalman filtering/smoothing. Furthermore, the coefficients of the multivariate autoregressive (MVAR) model characterizing the spatial-temporal dynamics of the patch sources are estimated in a principled manner via empirical Bayes. Extensive numerical experiments demonstrate STRAPS's excellent performance in the estimation of locations, spatial extents and amplitudes of the patch sources with varying spatial extents.
Collapse
Affiliation(s)
- Ke Liu
- College of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, P. R. China
| | - Zhu Liang Yu
- College of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, P. R. China
| | - Wei Wu
- College of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, P. R. China
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA 94305, USA
| | - Zhenghui Gu
- College of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, P. R. China
| | - Yuanqing Li
- College of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, P. R. China
| |
Collapse
|
47
|
Electroencephalography-based real-time cortical monitoring system that uses hierarchical Bayesian estimations for the brain-machine interface. J Clin Neurophysiol 2015; 31:218-28. [PMID: 24887604 DOI: 10.1097/wnp.0000000000000064] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022] Open
Abstract
In this study, a real-time cortical activity monitoring system was constructed, which could estimate cortical activities every 125 milliseconds over 2,240 vertexes from 64 channel electroencephalography signals through the Hierarchical Bayesian estimation that uses functional magnetic resonance imaging data as its prior information. Recently, functional magnetic resonance imaging has mostly been used in the neurofeedback field because it allows for high spatial resolution. However, in functional magnetic resonance imaging, the time for the neurofeedback information to reach the patient is delayed several seconds because of its poor temporal resolution. Therefore, a number of problems need to be solved to effectively implement feedback training paradigms in patients. To address this issue, this study used a new cortical activity monitoring system that improved both spatial and temporal resolution by using both functional magnetic resonance imaging data and electroencephalography signals in conjunction with one another. This system is advantageous as it can improve applications in the fields of real-time diagnosis, neurofeedback, and the brain-machine interface.
Collapse
|
48
|
Yuan H, He B. Brain-computer interfaces using sensorimotor rhythms: current state and future perspectives. IEEE Trans Biomed Eng 2015; 61:1425-35. [PMID: 24759276 DOI: 10.1109/tbme.2014.2312397] [Citation(s) in RCA: 224] [Impact Index Per Article: 24.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Many studies over the past two decades have shown that people can use brain signals to convey their intent to a computer using brain-computer interfaces (BCIs). BCI systems extract specific features of brain activity and translate them into control signals that drive an output. Recently, a category of BCIs that are built on the rhythmic activity recorded over the sensorimotor cortex, i.e., the sensorimotor rhythm (SMR), has attracted considerable attention among the BCIs that use noninvasive neural recordings, e.g., electroencephalography (EEG), and have demonstrated the capability of multidimensional prosthesis control. This paper reviews the current state and future perspectives of SMR-based BCI and its clinical applications, in particular focusing on the EEG SMR. The characteristic features of SMR from the human brain are described and their underlying neural sources are discussed. The functional components of SMR-based BCI, together with its current clinical applications, are reviewed. Finally, limitations of SMR-BCIs and future outlooks are also discussed.
Collapse
|
49
|
Roset SA, Gant K, Prasad A, Sanchez JC. An adaptive brain actuated system for augmenting rehabilitation. Front Neurosci 2015; 8:415. [PMID: 25565945 PMCID: PMC4264503 DOI: 10.3389/fnins.2014.00415] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2014] [Accepted: 11/25/2014] [Indexed: 11/24/2022] Open
Abstract
For people living with paralysis, restoration of hand function remains the top priority because it leads to independence and improvement in quality of life. In approaches to restore hand and arm function, a goal is to better engage voluntary control and counteract maladaptive brain reorganization that results from non-use. Standard rehabilitation augmented with developments from the study of brain-computer interfaces could provide a combined therapy approach for motor cortex rehabilitation and to alleviate motor impairments. In this paper, an adaptive brain-computer interface system intended for application to control a functional electrical stimulation (FES) device is developed as an experimental test bed for augmenting rehabilitation with a brain-computer interface. The system's performance is improved throughout rehabilitation by passive user feedback and reinforcement learning. By continuously adapting to the user's brain activity, similar adaptive systems could be used to support clinical brain-computer interface neurorehabilitation over multiple days.
Collapse
Affiliation(s)
- Scott A Roset
- Department of Biomedical Engineering, University of Miami Coral Gables, FL, USA
| | - Katie Gant
- Department of Biomedical Engineering, University of Miami Coral Gables, FL, USA
| | - Abhishek Prasad
- Department of Biomedical Engineering, University of Miami Coral Gables, FL, USA
| | - Justin C Sanchez
- Department of Biomedical Engineering, University of Miami Coral Gables, FL, USA ; Miami Project to Cure Paralysis, University of Miami Coral Gables, FL, USA
| |
Collapse
|
50
|
Bulea TC, Prasad S, Kilicarslan A, Contreras-Vidal JL. Sitting and standing intention can be decoded from scalp EEG recorded prior to movement execution. Front Neurosci 2014; 8:376. [PMID: 25505377 PMCID: PMC4243562 DOI: 10.3389/fnins.2014.00376] [Citation(s) in RCA: 78] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2014] [Accepted: 11/04/2014] [Indexed: 12/18/2022] Open
Abstract
Low frequency signals recorded from non-invasive electroencephalography (EEG), in particular movement-related cortical potentials (MRPs), are associated with preparation and execution of movement and thus present a target for use in brain-machine interfaces. We investigated the ability to decode movement intent from delta-band (0.1-4 Hz) EEG recorded immediately before movement execution in healthy volunteers. We used data from epochs starting 1.5 s before movement onset to classify future movements into one of three classes: stand-up, sit-down, or quiet. We assessed classification accuracy in both externally triggered and self-paced paradigms. Movement onset was determined from electromyography (EMG) recordings synchronized with EEG signals. We employed an artifact subspace reconstruction (ASR) algorithm to eliminate high amplitude noise before building our time-embedded EEG features. We applied local Fisher's discriminant analysis to reduce the dimensionality of our spatio-temporal features and subsequently used a Gaussian mixture model classifier for our three class problem. Our results demonstrate significantly better than chance classification accuracy (chance level = 33.3%) for the self-initiated (78.0 ± 2.6%) and triggered (74.7 ± 5.7%) paradigms. Surprisingly, we found no significant difference in classification accuracy between the self-paced and cued paradigms when using the full set of non-peripheral electrodes. However, accuracy was significantly increased for self-paced movements when only electrodes over the primary motor area were used. Overall, this study demonstrates that delta-band EEG recorded immediately before movement carries discriminative information regarding movement type. Our results suggest that EEG-based classifiers could improve lower-limb neuroprostheses and neurorehabilitation techniques by providing earlier detection of movement intent, which could be used in robot-assisted strategies for motor training and recovery of function.
Collapse
Affiliation(s)
- Thomas C Bulea
- Functional and Applied Biomechanics Section, Rehabilitation Medicine Department, National Institutes of Health Bethesda, MD, USA ; Laboratory for Non-invasive Brain-Machine Interface Systems, Department of Electrical and Computer Engineering, University of Houston Houston, TX, USA
| | - Saurabh Prasad
- Laboratory for Non-invasive Brain-Machine Interface Systems, Department of Electrical and Computer Engineering, University of Houston Houston, TX, USA
| | - Atilla Kilicarslan
- Laboratory for Non-invasive Brain-Machine Interface Systems, Department of Electrical and Computer Engineering, University of Houston Houston, TX, USA
| | - Jose L Contreras-Vidal
- Laboratory for Non-invasive Brain-Machine Interface Systems, Department of Electrical and Computer Engineering, University of Houston Houston, TX, USA
| |
Collapse
|