1
|
Kim JS, Kim H, Chung CK, Kim JS. Dual model transfer learning to compensate for individual variability in brain-computer interface. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 254:108294. [PMID: 38943984 DOI: 10.1016/j.cmpb.2024.108294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 06/14/2024] [Accepted: 06/16/2024] [Indexed: 07/01/2024]
Abstract
BACKGROUND AND OBJECTIVE Recent advancements in brain-computer interface (BCI) technology have seen a significant shift towards incorporating complex decoding models such as deep neural networks (DNNs) to enhance performance. These models are particularly crucial for sophisticated tasks such as regression for decoding arbitrary movements. However, these BCI models trained and tested on individual data often face challenges with limited performance and generalizability across different subjects. This limitation is primarily due to a tremendous number of parameters of DNN models. Training complex models demands extensive datasets. Nevertheless, group data from many subjects may not produce sufficient decoding performance because of inherent variability in neural signals both across individuals and over time METHODS: To address these challenges, this study proposed a transfer learning approach that could effectively adapt to subject-specific variability in cortical regions. Our method involved training two separate movement decoding models: one on individual data and another on pooled group data. We then created a salience map for each cortical region from the individual model, which helped us identify the input's contribution variance across subjects. Based on the contribution variance, we combined individual and group models using a modified knowledge distillation framework. This approach allowed the group model to be universally applicable by assigning greater weights to input data, while the individual model was fine-tuned to focus on areas with significant individual variance RESULTS: Our combined model effectively encapsulated individual variability. We validated this approach with nine subjects performing arm-reaching tasks, with our method outperforming (mean correlation coefficient, r = 0.75) both individual (r = 0.70) and group models (r = 0.40) in decoding performance. In particular, there were notable improvements in cases where individual models showed low performances (e.g., r = 0.50 in the individual decoder to r = 0.61 in the proposed decoder) CONCLUSIONS: These results not only demonstrate the potential of our method for robust BCI, but also underscore its ability to generalize individual data for broader applicability.
Collapse
Affiliation(s)
- Jun Su Kim
- Dept. of Brain and Cognitive Sciences, Seoul National University, Seoul, Republic of Korea; Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea
| | - HongJune Kim
- Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea
| | - Chun Kee Chung
- Dept. of Brain and Cognitive Sciences, Seoul National University, Seoul, Republic of Korea; Interdisciplinary Program in Neuroscience, Seoul National University, Seoul, Republic of Korea; Dept. of Neurosurgery, Seoul National University Hospital, Seoul, Republic of Korea; Neuroscience Research Institute, Seoul National University Medical Research Center, Seoul, Republic of Korea
| | - June Sic Kim
- Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea; Research Institute of Biomedical Science & Technology, Konkuk University, Seoul, Republic of Korea.
| |
Collapse
|
2
|
Forenzo D, Liu Y, Kim J, Ding Y, Yoon T, He B. Integrating Simultaneous Motor Imagery and Spatial Attention for EEG-BCI Control. IEEE Trans Biomed Eng 2024; 71:282-294. [PMID: 37494151 PMCID: PMC10803074 DOI: 10.1109/tbme.2023.3298957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/28/2023]
Abstract
OBJECTIVE EEG-based brain-computer interfaces (BCI) are non-invasive approaches for replacing or restoring motor functions in impaired patients, and direct brain-to-device communication in the general population. Motor imagery (MI) is one of the most used BCI paradigms, but its performance varies across individuals and certain users require substantial training to develop control. In this study, we propose to integrate a MI paradigm simultaneously with a recently proposed Overt Spatial Attention (OSA) paradigm, to accomplish BCI control. METHODS We evaluated a cohort of 25 human subjects' ability to control a virtual cursor in one- and two-dimensions over 5 BCI sessions. The subjects used 5 different BCI paradigms: MI alone, OSA alone, MI, and OSA simultaneously towards the same target (MI+OSA), and MI for one axis while OSA controls the other (MI/OSA and OSA/MI). RESULTS Our results show that MI+OSA reached the highest average online performance in 2D tasks at 49% Percent Valid Correct (PVC), and statistically outperforms both MI alone (42%) and OSA alone (45%). MI+OSA had a similar performance to each subject's best individual method between MI alone and OSA alone (50%) and 9 subjects reached their highest average BCI performance using MI+OSA. CONCLUSION Integrating MI and OSA leads to improved performance over both individual methods at the group level and is the best BCI paradigm option for some subjects. SIGNIFICANCE This work proposes a new BCI control paradigm that integrates two existing paradigms and demonstrates its value by showing that it can improve users' BCI performance.
Collapse
Affiliation(s)
- Dylan Forenzo
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Yixuan Liu
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Jeehyun Kim
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Yidan Ding
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Taehyung Yoon
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Bin He
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| |
Collapse
|
3
|
Liang W, Jin J, Daly I, Sun H, Wang X, Cichocki A. Novel channel selection model based on graph convolutional network for motor imagery. Cogn Neurodyn 2023; 17:1283-1296. [PMID: 37786654 PMCID: PMC10542066 DOI: 10.1007/s11571-022-09892-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Revised: 08/03/2022] [Accepted: 09/14/2022] [Indexed: 11/03/2022] Open
Abstract
Multi-channel electroencephalography (EEG) is used to capture features associated with motor imagery (MI) based brain-computer interface (BCI) with a wide spatial coverage across the scalp. However, redundant EEG channels are not conducive to improving BCI performance. Therefore, removing irrelevant channels can help improve the classification performance of BCI systems. We present a new method for identifying relevant EEG channels. Our method is based on the assumption that useful channels share related information and that this can be measured by inter-channel connectivity. Specifically, we treat all candidate EEG channels as a graph and define channel selection as the problem of node classification on a graph. Then we design a graph convolutional neural network (GCN) model for channels classification. Channels are selected based on the outputs of our GCN model. We evaluate our proposed GCN-based channel selection (GCN-CS) method on three MI datasets. On three datasets, GCN-CS achieves performance improvements by reducing the number of channels. Specifically, we achieve classification accuracies of 79.76% on Dataset 1, 89.14% on Dataset 2 and 87.96% on Dataset 3, which outperform competing methods significantly.
Collapse
Affiliation(s)
- Wei Liang
- Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, 200237 China
| | - Jing Jin
- Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, 200237 China
- Shenzhen Research Institute of East China University of Technology, Shenzhen, 518063 China
| | - Ian Daly
- Brain-Computer Interfacing and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK
| | - Hao Sun
- Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, 200237 China
| | - Xingyu Wang
- Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, 200237 China
| | - Andrzej Cichocki
- Skolkovo Institute of Science and Technology, Moscow, Russia 143026
- Systems Research Institute of Polish Academy of Science, Warsaw, Poland
- Department of Informatics, Nicolaus Copernicus University, Torun, Poland
| |
Collapse
|
4
|
Maslova O, Komarova Y, Shusharina N, Kolsanov A, Zakharov A, Garina E, Pyatin V. Non-invasive EEG-based BCI spellers from the beginning to today: a mini-review. Front Hum Neurosci 2023; 17:1216648. [PMID: 37680264 PMCID: PMC10480564 DOI: 10.3389/fnhum.2023.1216648] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Accepted: 07/24/2023] [Indexed: 09/09/2023] Open
Abstract
The defeat of the central motor neuron leads to the motor disorders. Patients lose the ability to control voluntary muscles, for example, of the upper limbs, which introduces a fundamental dissonance in the possibility of daily use of a computer or smartphone. As a result, the patients lose the ability to communicate with other people. The article presents the most popular paradigms used in the brain-computer-interface speller system and designed for typing by people with severe forms of the movement disorders. Brain-computer interfaces (BCIs) have emerged as a promising technology for individuals with communication impairments. BCI-spellers are systems that enable users to spell words by selecting letters on a computer screen using their brain activity. There are three main types of BCI-spellers: P300, motor imagery (MI), and steady-state visual evoked potential (SSVEP). However, each type has its own limitations, which has led to the development of hybrid BCI-spellers that combine the strengths of multiple types. Hybrid BCI-spellers can improve accuracy and reduce the training period required for users to become proficient. Overall, hybrid BCI-spellers have the potential to improve communication for individuals with impairments by combining the strengths of multiple types of BCI-spellers. In conclusion, BCI-spellers are a promising technology for individuals with communication impairments. P300, MI, and SSVEP are the three main types of BCI-spellers, each with their own advantages and limitations. Further research is needed to improve the accuracy and usability of BCI-spellers and to explore their potential applications in other areas such as gaming and virtual reality.
Collapse
Affiliation(s)
- Olga Maslova
- Neurosciences Research Institute, Samara State Medical University, Samara, Russia
| | - Yuliya Komarova
- Neurosciences Research Institute, Samara State Medical University, Samara, Russia
| | - Natalia Shusharina
- Baltic Center for Neurotechnologies and Artificial Intelligence, Immanuel Kant Baltic Federal University, Kaliningrad, Russia
| | - Alexander Kolsanov
- Department of Operative Surgery and Clinical Anatomy with a Course of Innovative Technologies, Samara State Medical University, Samara, Russia
| | - Alexander Zakharov
- Neurosciences Research Institute, Samara State Medical University, Samara, Russia
| | - Evgenia Garina
- Department of Physical Culture, I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia
| | - Vasiliy Pyatin
- Neurosciences Research Institute, Samara State Medical University, Samara, Russia
| |
Collapse
|
5
|
Jiang L, Luo C, Liao Z, Li X, Chen Q, Jin Y, Lu K, Zhang D. SmartRolling: A human–machine interface for wheelchair control using EEG and smart sensing techniques. Inf Process Manag 2023. [DOI: 10.1016/j.ipm.2022.103262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
|
6
|
Cui Z, Li Y, Huang S, Wu X, Fu X, Liu F, Wan X, Wang X, Zhang Y, Qiu H, Chen F, Yang P, Zhu S, Li J, Chen W. BCI system with lower-limb robot improves rehabilitation in spinal cord injury patients through short-term training: a pilot study. Cogn Neurodyn 2022; 16:1283-1301. [PMID: 36408074 PMCID: PMC9666612 DOI: 10.1007/s11571-022-09801-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Revised: 07/27/2021] [Accepted: 11/04/2021] [Indexed: 12/27/2022] Open
Abstract
In the recent years, the increasing applications of brain-computer interface (BCI) in rehabilitation programs have enhanced the chances of functional recovery for patients with neurological disorders. We presented and validated a BCI system with a lower-limb robot for short-term training of patients with spinal cord injury (SCI). The cores of this system included: (1) electroencephalogram (EEG) features related to motor intention reported through experiments and used to drive the robot; (2) a decision tree to determine the training mode provided for patients with different degrees of injuries. Seven SCI patients (one American Spinal Injury Association Impairment Scale (AIS) A, three AIS B, and three AIS C) participated in the short-term training with this system. All patients could learn to use the system rapidly and maintained a high intensity during the training program. The strength of the lower limb key muscles of the patients was improved. Four AIS A/B patients were elevated to AIS C. The cumulative results indicate that clinical application of the BCI system with lower-limb robot is feasible and safe, and has potentially positive effects on SCI patients. Supplementary Information The online version contains supplementary material available at 10.1007/s11571-022-09801-6.
Collapse
Affiliation(s)
- Zhengzhe Cui
- School of Mechanical Engineering, Zhejiang University, Hangzhou, China
| | - Yongqiang Li
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Sisi Huang
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Xixi Wu
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Xiangxiang Fu
- School of Mechanical Engineering, Zhejiang University, Hangzhou, China
| | - Fei Liu
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Xiaojiao Wan
- School of Mechanical Engineering, Zhejiang University, Hangzhou, China
| | - Xue Wang
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Yuting Zhang
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Huaide Qiu
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Fang Chen
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Peijin Yang
- School of Mechanical Engineering, Zhejiang University, Hangzhou, China
| | - Shiqiang Zhu
- School of Mechanical Engineering, Zhejiang University, Hangzhou, China
| | - Jianan Li
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Weidong Chen
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
| |
Collapse
|
7
|
Dong E, Zhang H, Zhu L, Du S, Tong J. A multi-modal brain-computer interface based on threshold discrimination and its application in wheelchair control. Cogn Neurodyn 2022; 16:1123-1133. [PMID: 36237403 PMCID: PMC9508306 DOI: 10.1007/s11571-021-09779-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Revised: 12/02/2021] [Accepted: 12/23/2021] [Indexed: 11/25/2022] Open
Abstract
In this study, we propose a novel multi-modal brain-computer interface (BCI) system based on the threshold discrimination, which is proposed for the first time to distinguish between SSVEP and MI potentials. The system combines these two heterogeneous signals to increase the number of control commands and improve the performance of asynchronous control of external devices. In this research, an electric wheelchair is controlled as an example. The user can continuously control the wheelchair to turn left/right through motion imagination (MI) by imagining left/right-hand movement and generate another 6 commands for the wheelchair control by focusing on the SSVEP stimulation panel. Ten subjects participated in a MI training session and eight of them completed a mobile obstacle-avoidance experiment in a complex environment requesting high control accuracy for successful manipulation. Comparing with the single-modal BCI-controlled wheelchair system, the results demonstrate that the proposed multi-modal method is effective by providing more satisfactory control accuracy, and show the potential of BCI-controlled systems to be applied in complex daily tasks.
Collapse
Affiliation(s)
- Enzeng Dong
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, Tianjin University of Technology, Tianjin, 300384 China
| | - Haoran Zhang
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, Tianjin University of Technology, Tianjin, 300384 China
| | - Lin Zhu
- China North Industries Group 210 Research Institute, Beijing, China
| | - Shengzhi Du
- Department of Electrical Engineering, Tshwane University of Technology, Pretoria, 0001 South Africa
| | - Jigang Tong
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, Tianjin University of Technology, Tianjin, 300384 China
| |
Collapse
|
8
|
Poststroke Cognitive Impairment Research Progress on Application of Brain-Computer Interface. BIOMED RESEARCH INTERNATIONAL 2022; 2022:9935192. [PMID: 35252458 PMCID: PMC8896931 DOI: 10.1155/2022/9935192] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Revised: 12/20/2021] [Accepted: 12/23/2021] [Indexed: 12/19/2022]
Abstract
Brain-computer interfaces (BCIs), a new type of rehabilitation technology, pick up nerve cell signals, identify and classify their activities, and convert them into computer-recognized instructions. This technique has been widely used in the rehabilitation of stroke patients in recent years and appears to promote motor function recovery after stroke. At present, the application of BCI in poststroke cognitive impairment is increasing, which is a common complication that also affects the rehabilitation process. This paper reviews the promise and potential drawbacks of using BCI to treat poststroke cognitive impairment, providing a solid theoretical basis for the application of BCI in this area.
Collapse
|
9
|
Shen X, Wang X, Lu S, Li Z, Shao W, Wu Y. Research on the real-time control system of lower-limb gait movement based on motor imagery and central pattern generator. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.102803] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|
10
|
Xiao R, Huang Y, Xu R, Wang B, Wang X, Jin J. Coefficient-of-variation-based channel selection with a new testing framework for MI-based BCI. Cogn Neurodyn 2021; 16:791-803. [PMID: 35847541 PMCID: PMC9279536 DOI: 10.1007/s11571-021-09752-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2021] [Revised: 09/27/2021] [Accepted: 10/24/2021] [Indexed: 11/29/2022] Open
Abstract
In the motor-imagery (MI) based brain computer interface (BCI), multi-channel electroencephalogram (EEG) is often used to ensure the complete capture of physiological phenomena. With the redundant information and noise, EEG signals cannot be easily converted into separable features through feature extraction algorithms. Channel selection algorithms are proposed to address the issue, in which the filtering technique is widely used with the advantages of low computational cost and strong practicability. In this study, we proposed several improved methods for filtering channel selection algorithm. Specifically, based on the coefficient of variation and inter-class distance, a novel channel classification method was designed, which divided channels into different categories based on their contribution to feature extraction process. Then a filtering channel selection algorithm was proposed according to the previous classification method. Moreover, a new testing framework for filtering channel selection algorithms was proposed, which can better reflect the generalization ability of the algorithm. Experimental results indicated that the proposed channel classification method is effective, and the proposed testing framework is better than the original one. Meanwhile, the proposed channel selection algorithm achieved the accuracy of 87.7% and 81.7% in two BCI competition datasets, respectively, which was superior to competing algorithms.
Collapse
|
11
|
Zhang X, Jin J, Li S, Wang X, Cichocki A. Evaluation of color modulation in visual P300-speller using new stimulus patterns. Cogn Neurodyn 2021; 15:873-886. [PMID: 34603548 DOI: 10.1007/s11571-021-09669-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Revised: 01/04/2021] [Accepted: 02/04/2021] [Indexed: 02/07/2023] Open
Abstract
Objective The stimulus color of P300-BCI systems has been successfully modified. However, the effects of different color combinations have not been widely investigated. In this study, we designed new stimulus patterns to evaluate the influence of color modulation on the BCI performance and waveforms of the evoked related potential (ERP).Methods Comparison was performed for three new stimulus patterns consisting of red face and colored block-shape, namely, red face with a white rectangle (RFW), red face with a blue rectangle (RFB), and red face with a red rectangle (RFR). Bayesian linear discriminant analysis (BLDA) was used to construct the individual classifier model. Repeated-measures ANOVA and Bonferroni correction were applied for statistical analysis. Results The RFW pattern obtained the highest average online accuracy with 96.94%, and those of RFR and RFB patterns were 93.61% and of 92.22% respectively. Significant differences in online accuracy and information transfer rate (ITR) were found between RFW and RFR patterns (p < 0.05). Conclusion Compared with RFR and RFB patterns, RFW yielded the best performance in P300-BCI. These new stimulus patterns with different color combinations have considerable importance to BCI applications and user-friendliness.
Collapse
Affiliation(s)
- Xinru Zhang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, People's Republic of China
| | - Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, People's Republic of China
| | - Shurui Li
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, People's Republic of China
| | - Xingyu Wang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, People's Republic of China
| | - Andrzej Cichocki
- Skolkovo Institute of Science and Technology (Skoltech), 121205 Moscow, Russia.,Nicolaus Copernicus University (UMK), 87-100 Torun, Poland
| |
Collapse
|
12
|
Heo D, Kim M, Kim J, Choi YJ, Kim SP. Effect of Static Posture on Online Performance of P300-Based BCIs for TV Control. SENSORS 2021; 21:s21072278. [PMID: 33805181 PMCID: PMC8036388 DOI: 10.3390/s21072278] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Revised: 03/16/2021] [Accepted: 03/21/2021] [Indexed: 12/31/2022]
Abstract
To implement a practical brain–computer interface (BCI) for daily use, continuing changes in postures while performing daily tasks must be considered in the design of BCIs. To examine whether the performance of a BCI could depend on postures, we compared the online performance of P300-based BCIs built to select TV channels when subjects took sitting, recline, supine, and right lateral recumbent postures during BCI use. Subjects self-reported the degrees of interference, comfort, and familiarity after BCI control in each posture. We found no significant difference in the BCI performance as well as the amplitude and latency of P300 and N200 among the four postures. However, when we compared BCI accuracy outcomes normalized within individuals between two cases where subjects reported relatively more positively or more negatively about using the BCI in a particular posture, we found higher BCI accuracy in those postures for which individual subjects reported more positively. As a result, although the change of postures did not affect the overall performance of P300-based BCIs, the BCI performance varied depending on the degree of postural comfort felt by individual subjects. Our results suggest considering the postural comfort felt by individual BCI users when using a P300-based BCI at home.
Collapse
|
13
|
Saha S, Mamun KA, Ahmed K, Mostafa R, Naik GR, Darvishi S, Khandoker AH, Baumert M. Progress in Brain Computer Interface: Challenges and Opportunities. Front Syst Neurosci 2021; 15:578875. [PMID: 33716680 PMCID: PMC7947348 DOI: 10.3389/fnsys.2021.578875] [Citation(s) in RCA: 82] [Impact Index Per Article: 27.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2020] [Accepted: 01/06/2021] [Indexed: 12/13/2022] Open
Abstract
Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.
Collapse
Affiliation(s)
- Simanto Saha
- School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA, Australia
- Department of Electrical and Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Khondaker A. Mamun
- Advanced Intelligent Multidisciplinary Systems (AIMS) Lab, Department of Computer Science and Engineering, United International University, Dhaka, Bangladesh
| | - Khawza Ahmed
- Department of Electrical and Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Raqibul Mostafa
- Department of Electrical and Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Ganesh R. Naik
- Adelaide Institute for Sleep Health, College of Medicine and Public Health, Flinders University, Adelaide, SA, Australia
| | - Sam Darvishi
- School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA, Australia
| | - Ahsan H. Khandoker
- Healthcare Engineering Innovation Center, Department of Biomedical Engineering, Khalifa University of Science and Technology, Abu Dhabi, United Arab Emirates
| | - Mathias Baumert
- School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA, Australia
| |
Collapse
|
14
|
Luo Z, Jin R, Shi H, Lu X. Research on Recognition of Motor Imagination Based on Connectivity Features of Brain Functional Network. Neural Plast 2021; 2021:6655430. [PMID: 33628220 PMCID: PMC7895585 DOI: 10.1155/2021/6655430] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2020] [Revised: 01/07/2021] [Accepted: 02/05/2021] [Indexed: 02/06/2023] Open
Abstract
Feature extraction is essential for classifying different motor imagery (MI) tasks in a brain-computer interface. To improve classification accuracy, we propose a novel feature extraction method in which the connectivity increment rate (CIR) of the brain function network (BFN) is extracted. First, the BFN is constructed on the basis of the threshold matrix of the Pearson correlation coefficient of the mu rhythm among the channels. In addition, a weighted BFN is constructed and expressed by the sum of the existing edge weights to characterize the cerebral cortex activation degree in different movement patterns. Then, on the basis of the topological structures of seven mental tasks, three regional networks centered on the C3, C4, and Cz channels are constructed, which are consistent with correspondence between limb movement patterns and cerebral cortex in neurophysiology. Furthermore, the CIR of each regional functional network is calculated to form three-dimensional vectors. Finally, we use the support vector machine to learn a classifier for multiclass MI tasks. Experimental results show a significant improvement and demonstrate the success of the extracted feature CIR in dealing with MI classification. Specifically, the average classification performance reaches 88.67% which is higher than other competing methods, indicating that the extracted CIR is effective for MI classification.
Collapse
Affiliation(s)
- Zhizeng Luo
- Institute of Intelligent Control and Robotics, Hangzhou Dizanzi University, Hangzhou, China
| | - Ronghang Jin
- Institute of Intelligent Control and Robotics, Hangzhou Dizanzi University, Hangzhou, China
| | - Hongfei Shi
- The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Xianju Lu
- Institute of Intelligent Control and Robotics, Hangzhou Dizanzi University, Hangzhou, China
| |
Collapse
|
15
|
Mao Y, Jin J, Li S, Miao Y, Cichocki A. Effects of Skin Friction on Tactile P300 Brain-Computer Interface Performance. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2021; 2021:6694310. [PMID: 33628218 PMCID: PMC7886524 DOI: 10.1155/2021/6694310] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/13/2020] [Revised: 01/22/2021] [Accepted: 01/30/2021] [Indexed: 11/18/2022]
Abstract
Tactile perception, the primary sensing channel of the tactile brain-computer interface (BCI), is a complicated process. Skin friction plays a vital role in tactile perception. This study aimed to examine the effects of skin friction on tactile P300 BCI performance. Two kinds of oddball paradigms were designed, silk-stim paradigm (SSP) and linen-stim paradigm (LSP), in which silk and linen were wrapped on target vibration motors, respectively. In both paradigms, the disturbance vibrators were wrapped in cotton. The experimental results showed that LSP could induce stronger event-related potentials (ERPs) and achieved a higher classification accuracy and information transfer rate (ITR) compared with SSP. The findings indicate that high skin friction can achieve high performance in tactile BCI. This work provides a novel research direction and constitutes a viable basis for the future tactile P300 BCI, which may benefit patients with visual impairments.
Collapse
Affiliation(s)
- Ying Mao
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Shurui Li
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Yangyang Miao
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Andrzej Cichocki
- Skolkovo Institute of Science and Technology (SKOLTECH), Moscow 143026, Russia
- Nicolaus Copernicus University (UMK), Torun, Poland
| |
Collapse
|
16
|
|
17
|
Miao Y, Chen S, Zhang X, Jin J, Xu R, Daly I, Jia J, Wang X, Cichocki A, Jung TP. BCI-Based Rehabilitation on the Stroke in Sequela Stage. Neural Plast 2020; 2020:8882764. [PMID: 33414824 PMCID: PMC7752268 DOI: 10.1155/2020/8882764] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Revised: 11/25/2020] [Accepted: 11/30/2020] [Indexed: 11/24/2022] Open
Abstract
Background Stroke is the leading cause of serious and long-term disability worldwide. Survivors may recover some motor functions after rehabilitation therapy. However, many stroke patients missed the best time period for recovery and entered into the sequela stage of chronic stroke. Method Studies have shown that motor imagery- (MI-) based brain-computer interface (BCI) has a positive effect on poststroke rehabilitation. This study used both virtual limbs and functional electrical stimulation (FES) as feedback to provide patients with a closed-loop sensorimotor integration for motor rehabilitation. An MI-based BCI system acquired, analyzed, and classified motor attempts from electroencephalogram (EEG) signals. The FES system would be activated if the BCI detected that the user was imagining wrist dorsiflexion on the instructed side of the body. Sixteen stroke patients in the sequela stage were randomly assigned to a BCI group and a control group. All of them participated in rehabilitation training for four weeks and were assessed by the Fugl-Meyer Assessment (FMA) of motor function. Results The average improvement score of the BCI group was 3.5, which was higher than that of the control group (0.9). The active EEG patterns of the four patients in the BCI group whose FMA scores increased gradually became centralized and shifted to sensorimotor areas and premotor areas throughout the study. Conclusions Study results showed evidence that patients in the BCI group achieved larger functional improvements than those in the control group and that the BCI-FES system is effective in restoring motor function to upper extremities in stroke patients. This study provides a more autonomous approach than traditional treatments used in stroke rehabilitation.
Collapse
Affiliation(s)
- Yangyang Miao
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Shugeng Chen
- Department of Rehabilitation, Huashan Hospital, Fudan University, Shanghai, China
| | - Xinru Zhang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Ren Xu
- Guger Technologies OG, Austria
| | - Ian Daly
- Brain-Computer Interfaces and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, Essex CO4 3SQ, UK
| | - Jie Jia
- Department of Rehabilitation, Huashan Hospital, Fudan University, Shanghai, China
| | - Xingyu Wang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Andrzej Cichocki
- Skolkowo Institute of Science and Technology (SKOLTECH), 143026 Moscow, Russia
- Systems Research Institute PAS, Warsaw, Poland
- Nicolaus Copernicus University (UMK), Torun, Poland
| | - Tzyy-Ping Jung
- Institute for Neural Computation and Institute of Engineering in Medicine, University of California San Diego, La Jolla, CA, USA
| |
Collapse
|
18
|
Xue J, Ren F, Sun X, Yin M, Wu J, Ma C, Gao Z. A Multifrequency Brain Network-Based Deep Learning Framework for Motor Imagery Decoding. Neural Plast 2020; 2020:8863223. [PMID: 33505456 PMCID: PMC7787825 DOI: 10.1155/2020/8863223] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Revised: 10/22/2020] [Accepted: 11/04/2020] [Indexed: 12/11/2022] Open
Abstract
Motor imagery (MI) is an important part of brain-computer interface (BCI) research, which could decode the subject's intention and help remodel the neural system of stroke patients. Therefore, accurate decoding of electroencephalography- (EEG-) based motion imagination has received a lot of attention, especially in the research of rehabilitation training. We propose a novel multifrequency brain network-based deep learning framework for motor imagery decoding. Firstly, a multifrequency brain network is constructed from the multichannel MI-related EEG signals, and each layer corresponds to a specific brain frequency band. The structure of the multifrequency brain network matches the activity profile of the brain properly, which combines the information of channel and multifrequency. The filter bank common spatial pattern (FBCSP) algorithm filters the MI-based EEG signals in the spatial domain to extract features. Further, a multilayer convolutional network model is designed to distinguish different MI tasks accurately, which allows extracting and exploiting the topology in the multifrequency brain network. We use the public BCI competition IV dataset 2a and the public BCI competition III dataset IIIa to evaluate our framework and get state-of-the-art results in the first dataset, i.e., the average accuracy is 83.83% and the value of kappa is 0.784 for the BCI competition IV dataset 2a, and the accuracy is 89.45% and the value of kappa is 0.859 for the BCI competition III dataset IIIa. All these results demonstrate that our framework can classify different MI tasks from multichannel EEG signals effectively and show great potential in the study of remodelling the neural system of stroke patients.
Collapse
Affiliation(s)
- Juntao Xue
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
| | - Feiyue Ren
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
| | - Xinlin Sun
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
| | - Miaomiao Yin
- Department of Neurorehabilitation and Neurology, Tianjin Huanhu Hospital, Tianjin Key Laboratory of Cerebral Vascular and Neurodegenerative Diseases, Tianjin Neurosurgical Institute, Tianjin 300350, China
| | - Jialing Wu
- Department of Neurorehabilitation and Neurology, Tianjin Huanhu Hospital, Tianjin Key Laboratory of Cerebral Vascular and Neurodegenerative Diseases, Tianjin Neurosurgical Institute, Tianjin 300350, China
| | - Chao Ma
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
| | - Zhongke Gao
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
| |
Collapse
|
19
|
Temporal frequency joint sparse optimization and fuzzy fusion for motor imagery-based brain-computer interfaces. J Neurosci Methods 2020; 340:108725. [DOI: 10.1016/j.jneumeth.2020.108725] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2020] [Revised: 04/02/2020] [Accepted: 04/03/2020] [Indexed: 11/20/2022]
|
20
|
Novel channel selection method based on position priori weighted permutation entropy and binary gravity search algorithm. Cogn Neurodyn 2020; 15:141-156. [PMID: 33786085 DOI: 10.1007/s11571-020-09608-3] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2019] [Revised: 05/09/2020] [Accepted: 06/13/2020] [Indexed: 11/27/2022] Open
Abstract
Brain-computer interface (BCI) system based on motor imagery (MI) usually adopts multichannel Electroencephalograph (EEG) signal recording method. However, EEG signals recorded in multi-channel mode usually contain many redundant and artifact information. Therefore, selecting a few effective channels from whole channels may be a means to improve the performance of MI-based BCI systems. We proposed a channel evaluation parameter called position priori weight-permutation entropy (PPWPE), which include amplitude information and position information of a channel. According to the order of PPWPE values, we initially selected half of the channels with large PPWPE value from all sampling electrode channels. Then, the binary gravitational search algorithm (BGSA) was used in searching a channel combination that will be used in determining an optimal channel combination. The features were extracted by common spatial pattern (CSP) method from the final selected channels, and the classifier was trained by support vector machine. The PPWPE + BGSA + CSP channel selection method is validated on two data sets. Results showed that the PPWPE + BGSA + CSP method obtained better mean classification accuracy (88.0% vs. 57.5% for Data set 1 and 91.1% vs. 79.4% for Data set 2) than All-C + CSP method. The PPWPE + BGSA + CSP method can achieve higher classification in fewer channels selected. This method has great potential to improve the performance of MI-based BCI systems.
Collapse
|
21
|
Ren S, Wang W, Hou ZG, Liang X, Wang J, Shi W. Enhanced Motor Imagery Based Brain- Computer Interface via FES and VR for Lower Limbs. IEEE Trans Neural Syst Rehabil Eng 2020; 28:1846-1855. [PMID: 32746291 DOI: 10.1109/tnsre.2020.3001990] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Motor imagery based brain-computer interface (MI-BCI) has been studied for improvement of patients' motor function in neurorehabilitation and motor assistance. However, the difficulties in performing imagery tasks limit its application. To overcome the limitation, an enhanced MI-BCI based on functional electrical stimulation (FES) and virtual reality (VR) is proposed in this study. On one hand, the FES is used to stimulate the subjects' lower limbs before their imagination to make them experience the muscles' contraction and improve their attention on the lower limbs, by which it is supposed that the subjects' motor imagery (MI) abilities can be enhanced. On the other hand, a ball-kicking movement scenario from the first-person perspective is designed to provide visual guidance for performing MI tasks. The combination of FES and VR can be used to reduce the difficulties in performing MI tasks and improve classification accuracy. Finally, the comparison experiments were conducted on twelve healthy subjects to validate the performance of the enhanced MI-BCI. The results show that the classification performance can be improved significantly by using the proposed MI-BCI in terms of the classification accuracy (ACC), the area under the curve (AUC) and the F1 score (paired t-test, ).
Collapse
|