1
|
Kober SE, Wood G, Schuster S, Körner C. Do Miniature Eye Movements Affect Neurofeedback Training Performance? A Combined EEG-Eye Tracking Study. Appl Psychophysiol Biofeedback 2024; 49:313-327. [PMID: 38492124 PMCID: PMC11101551 DOI: 10.1007/s10484-024-09625-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/08/2024] [Indexed: 03/18/2024]
Abstract
EEG-based neurofeedback is a prominent method to modulate one's own brain activity in a desired direction. However, the EEG signal can be disturbed by artifacts, e.g., eye movements, which can consequently confound the neurofeedback performance. Involuntary miniature eye movements can be hardly detected by conventional EEG correction methods such as recording the electro-oculogram (EOG) and subtracting EOG activity from the EEG signal. However, such miniature eye movements can influence EEG activity, especially in the Gamma frequency range, enormously. In the present study, we investigated whether power in different EEG frequencies can be effectively modulated by self-control of brain signals during neurofeedback training and/or whether changes in EEG power are provoked by miniature eye movements during the training. To this end, 24 participants performed one session of SMR and one session of Gamma neurofeedback training. Additionally, in each training session sham feedback was performed. An eye tracker was used to detect miniature eye movements (< 1°) during neurofeedback training. About two thirds of the participants were able to increase their SMR power over the course of NF training, while one third was able to increase Gamma power. Generally, miniature eye movements induced a strong Gamma power increase. The number of eye movements also increased numerically over the course of the NF training. However, we did not find a significant relationship with the NF training performance. This is a first indication that miniature saccades do not affect NF training performance, but should not be neglected during NF training. Our results have to be confirmed in future studies.
Collapse
Affiliation(s)
- Silvia Erika Kober
- Department of Psychology, University of Graz, Universitaetsplatz 2/III, 8010, Graz, Austria.
- BioTechMed-Graz, Graz, Austria.
| | - Guilherme Wood
- Department of Psychology, University of Graz, Universitaetsplatz 2/III, 8010, Graz, Austria
- BioTechMed-Graz, Graz, Austria
| | - Sarah Schuster
- Department of Psychology, University of Graz, Universitaetsplatz 2/III, 8010, Graz, Austria
| | - Christof Körner
- Department of Psychology, University of Graz, Universitaetsplatz 2/III, 8010, Graz, Austria
- BioTechMed-Graz, Graz, Austria
| |
Collapse
|
2
|
Huang D, Wang Y, Fan L, Yu Y, Zhao Z, Zeng P, Wang K, Li N, Shen H. Decoding Subject-Driven Cognitive States from EEG Signals for Cognitive Brain-Computer Interface. Brain Sci 2024; 14:498. [PMID: 38790476 PMCID: PMC11120245 DOI: 10.3390/brainsci14050498] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Revised: 05/06/2024] [Accepted: 05/14/2024] [Indexed: 05/26/2024] Open
Abstract
In this study, we investigated the feasibility of using electroencephalogram (EEG) signals to differentiate between four distinct subject-driven cognitive states: resting state, narrative memory, music, and subtraction tasks. EEG data were collected from seven healthy male participants while performing these cognitive tasks, and the raw EEG signals were transformed into time-frequency maps using continuous wavelet transform. Based on these time-frequency maps, we developed a convolutional neural network model (TF-CNN-CFA) with a channel and frequency attention mechanism to automatically distinguish between these cognitive states. The experimental results demonstrated that the model achieved an average classification accuracy of 76.14% in identifying these four cognitive states, significantly outperforming traditional EEG signal processing methods and other classical image classification algorithms. Furthermore, we investigated the impact of varying lengths of EEG signals on classification performance and found that TF-CNN-CFA demonstrates consistent performance across different window lengths, indicating its strong generalization capability. This study validates the ability of EEG to differentiate higher cognitive states, which could potentially offer a novel BCI paradigm.
Collapse
Affiliation(s)
- Dingyong Huang
- College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China; (D.H.); (L.F.); (Y.Y.); (Z.Z.); (P.Z.); (K.W.)
| | - Yingjie Wang
- College of Physical Education and Health, Hebei Normal University of Science & Technology, Qinhuangdao 066004, China;
| | - Liangwei Fan
- College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China; (D.H.); (L.F.); (Y.Y.); (Z.Z.); (P.Z.); (K.W.)
| | - Yang Yu
- College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China; (D.H.); (L.F.); (Y.Y.); (Z.Z.); (P.Z.); (K.W.)
| | - Ziyu Zhao
- College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China; (D.H.); (L.F.); (Y.Y.); (Z.Z.); (P.Z.); (K.W.)
| | - Pu Zeng
- College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China; (D.H.); (L.F.); (Y.Y.); (Z.Z.); (P.Z.); (K.W.)
| | - Kunqing Wang
- College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China; (D.H.); (L.F.); (Y.Y.); (Z.Z.); (P.Z.); (K.W.)
| | - Na Li
- Radiology Department, Xiangya 3rd Hospital, Central South University, Changsha 410013, China;
| | - Hui Shen
- College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China; (D.H.); (L.F.); (Y.Y.); (Z.Z.); (P.Z.); (K.W.)
| |
Collapse
|
3
|
Gómez DMC, Braidot AAA. Improving motor imagery through a mirror box for BCI users. J Neurophysiol 2024; 131:832-841. [PMID: 38323330 DOI: 10.1152/jn.00121.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Revised: 01/12/2024] [Accepted: 02/04/2024] [Indexed: 02/08/2024] Open
Abstract
The aim of this study was to evaluate mirror visual feedback (MVF) as a training tool for brain-computer interface (BCI) users. This is because approximately 20-30% of subjects require more training to operate a BCI system using motor imagery. Electroencephalograms (EEGs) were recorded from 18 healthy subjects, using event-related desynchronization (ERD) to observe the responses during the movement or movement intention of the hand for the conditions of control, imagination, and the MVF with the mirror box. We constituted two groups: group 1: control, imagination, and MVF; group 2: control, MVF, and imagination. There were significant differences in imagination conditions between groups using MVF before or after imagination (right-hand, P = 0.0403; left-hand, P = 0.00939). The illusion of movement through MVF is not possible in all subjects, but even in those cases, we found an increase in imagination when the subject used the MVF previously. The increase in the r2s of imagination in the right and left hands suggests cross-learning. The increase in motor imagery recorded with EEG after MVF suggests that the mirror box made it easier to imagine movements. Our results provide evidence that the MVF could be used as a training tool to improve motor imagery.NEW & NOTEWORTHY The increase in motor imagery recorded with EEG after MVF (mirror visual feedback) suggests that the mirror box made it easier to imagine movements. Our results demonstrate that MVF could be used as a training tool to improve motor imagery.
Collapse
Affiliation(s)
- Diana Margarita Casas Gómez
- Laboratory of Biomechanics, School of Engineering, National University of Entre Ríos, Entre Ríos, Argentina
- Escuela Ciencias Básicas Tecnología e Ingeniería, Universidad Nacional Abierta y a Distancia, Dosquebradas, Colombia
| | | |
Collapse
|
4
|
Jia T, Sun J, McGeady C, Ji L, Li C. Enhancing Brain-Computer Interface Performance by Incorporating Brain-to-Brain Coupling. CYBORG AND BIONIC SYSTEMS 2024; 5:0116. [PMID: 38680535 PMCID: PMC11052607 DOI: 10.34133/cbsystems.0116] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Accepted: 03/24/2024] [Indexed: 05/01/2024] Open
Abstract
Human cooperation relies on key features of social interaction in order to reach desirable outcomes. Similarly, human-robot interaction may benefit from integration with human-human interaction factors. In this paper, we aim to investigate brain-to-brain coupling during motor imagery (MI)-based brain-computer interface (BCI) training using eye-contact and hand-touch interaction. Twelve pairs of friends (experimental group) and 10 pairs of strangers (control group) were recruited for MI-based BCI tests concurrent with electroencephalography (EEG) hyperscanning. Event-related desynchronization (ERD) was estimated to measure cortical activation, and interbrain functional connectivity was assessed using multilevel statistical analysis. Furthermore, we compared BCI classification performance under different social interaction conditions. In the experimental group, greater ERD was found around the contralateral sensorimotor cortex under social interaction conditions compared with MI without any social interaction. Notably, EEG channels with decreased power were mainly distributed around the frontal, central, and occipital regions. A significant increase in interbrain coupling was also found under social interaction conditions. BCI decoding accuracies were significantly improved in the eye contact condition and eye and hand contact condition compared with the no-interaction condition. However, for the strangers' group, no positive effects were observed in comparisons of cortical activations between interaction and no-interaction conditions. These findings indicate that social interaction can improve the neural synchronization between familiar partners with enhanced brain activations and brain-to-brain coupling. This study may provide a novel method for enhancing MI-based BCI performance in conjunction with neural synchronization between users.
Collapse
Affiliation(s)
- Tianyu Jia
- Lab of Intelligent and Biomimetic Machinery, Department of Mechanical Engineering,
Tsinghua University, Beijing, China
- Department of Bioengineering,
Imperial College London, London, UK
| | - Jingyao Sun
- Lab of Intelligent and Biomimetic Machinery, Department of Mechanical Engineering,
Tsinghua University, Beijing, China
| | - Ciarán McGeady
- Department of Bioengineering,
Imperial College London, London, UK
| | - Linhong Ji
- Lab of Intelligent and Biomimetic Machinery, Department of Mechanical Engineering,
Tsinghua University, Beijing, China
| | - Chong Li
- Lab of Intelligent and Biomimetic Machinery, Department of Mechanical Engineering,
Tsinghua University, Beijing, China
- School of Clinical Medicine,
Tsinghua University, Beijing, China
- Beijing Tsinghua Changgung Hospital,
Tsinghua University, Beijing, China
| |
Collapse
|
5
|
Herbert C. Analyzing and computing humans by means of the brain using Brain-Computer Interfaces - understanding the user - previous evidence, self-relevance and the user's self-concept as potential superordinate human factors of relevance. Front Hum Neurosci 2024; 17:1286895. [PMID: 38435127 PMCID: PMC10904616 DOI: 10.3389/fnhum.2023.1286895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 12/11/2023] [Indexed: 03/05/2024] Open
Abstract
Brain-computer interfaces (BCIs) are well-known instances of how technology can convert a user's brain activity taken from non-invasive electroencephalography (EEG) into computer commands for the purpose of computer-assisted communication and interaction. However, not all users are attaining the accuracy required to use a BCI consistently, despite advancements in technology. Accordingly, previous research suggests that human factors could be responsible for the variance in BCI performance among users. Therefore, the user's internal mental states and traits including motivation, affect or cognition, personality traits, or the user's satisfaction, beliefs or trust in the technology have been investigated. Going a step further, this manuscript aims to discuss which human factors could be potential superordinate factors that influence BCI performance, implicitly, explicitly as well as inter- and intraindividually. Based on the results of previous studies that used comparable protocols to examine the motivational, affective, cognitive state or personality traits of healthy and vulnerable EEG-BCI users within and across well-investigated BCIs (P300-BCIs or SMR-BCIs, respectively), it is proposed that the self-relevance of tasks and stimuli and the user's self-concept provide a huge potential for BCI applications. As potential key human factors self-relevance and the user's self-concept (self-referential knowledge and beliefs about one's self) guide information processing and modulate the user's motivation, attention, or feelings of ownership, agency, and autonomy. Changes in the self-relevance of tasks and stimuli as well as self-referential processing related to one's self (self-concept) trigger changes in neurophysiological activity in specific brain networks relevant to BCI. Accordingly, concrete examples will be provided to discuss how past and future research could incorporate self-relevance and the user's self-concept in the BCI setting - including paradigms, user instructions, and training sessions.
Collapse
Affiliation(s)
- Cornelia Herbert
- Department of Applied Emotion and Motivation Psychology, Institute of Psychology and Education, Ulm University, Ulm, Germany
| |
Collapse
|
6
|
Yamamoto MS, Sadatnejad K, Tanaka T, Islam MR, Dehais F, Tanaka Y, Lotte F. Modeling Complex EEG Data Distribution on the Riemannian Manifold Toward Outlier Detection and Multimodal Classification. IEEE Trans Biomed Eng 2024; 71:377-387. [PMID: 37450357 DOI: 10.1109/tbme.2023.3295769] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/18/2023]
Abstract
OBJECTIVE The usage of Riemannian geometry for Brain-computer interfaces (BCIs) has gained momentum in recent years. Most of the machine learning techniques proposed for Riemannian BCIs consider the data distribution on a manifold to be unimodal. However, the distribution is likely to be multimodal rather than unimodal since high-data variability is a crucial limitation of electroencephalography (EEG). In this paper, we propose a novel data modeling method for considering complex data distributions on a Riemannian manifold of EEG covariance matrices, aiming to improve BCI reliability. METHODS Our method, Riemannian spectral clustering (RiSC), represents EEG covariance matrix distribution on a manifold using a graph with proposed similarity measurement based on geodesic distances, then clusters the graph nodes through spectral clustering. This allows flexibility to model both a unimodal and a multimodal distribution on a manifold. RiSC can be used as a basis to design an outlier detector named outlier detection Riemannian spectral clustering (odenRiSC) and a multimodal classifier named multimodal classifier Riemannian spectral clustering (mcRiSC). All required parameters of odenRiSC/mcRiSC are selected in data-driven manner. Moreover, there is no need to pre-set a threshold for outlier detection and the number of modes for multimodal classification. RESULTS The experimental evaluation revealed odenRiSC can detect EEG outliers more accurately than existing methods and mcRiSC outperformed the standard unimodal classifier, especially on high-variability datasets. CONCLUSION odenRiSC/mcRiSC are anticipated to contribute to making real-life BCIs outside labs and neuroergonomics applications more robust. SIGNIFICANCE RiSC can work as a robust EEG outlier detector and multimodal classifier.
Collapse
|
7
|
Forenzo D, Liu Y, Kim J, Ding Y, Yoon T, He B. Integrating Simultaneous Motor Imagery and Spatial Attention for EEG-BCI Control. IEEE Trans Biomed Eng 2024; 71:282-294. [PMID: 37494151 PMCID: PMC10803074 DOI: 10.1109/tbme.2023.3298957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/28/2023]
Abstract
OBJECTIVE EEG-based brain-computer interfaces (BCI) are non-invasive approaches for replacing or restoring motor functions in impaired patients, and direct brain-to-device communication in the general population. Motor imagery (MI) is one of the most used BCI paradigms, but its performance varies across individuals and certain users require substantial training to develop control. In this study, we propose to integrate a MI paradigm simultaneously with a recently proposed Overt Spatial Attention (OSA) paradigm, to accomplish BCI control. METHODS We evaluated a cohort of 25 human subjects' ability to control a virtual cursor in one- and two-dimensions over 5 BCI sessions. The subjects used 5 different BCI paradigms: MI alone, OSA alone, MI, and OSA simultaneously towards the same target (MI+OSA), and MI for one axis while OSA controls the other (MI/OSA and OSA/MI). RESULTS Our results show that MI+OSA reached the highest average online performance in 2D tasks at 49% Percent Valid Correct (PVC), and statistically outperforms both MI alone (42%) and OSA alone (45%). MI+OSA had a similar performance to each subject's best individual method between MI alone and OSA alone (50%) and 9 subjects reached their highest average BCI performance using MI+OSA. CONCLUSION Integrating MI and OSA leads to improved performance over both individual methods at the group level and is the best BCI paradigm option for some subjects. SIGNIFICANCE This work proposes a new BCI control paradigm that integrates two existing paradigms and demonstrates its value by showing that it can improve users' BCI performance.
Collapse
Affiliation(s)
- Dylan Forenzo
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Yixuan Liu
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Jeehyun Kim
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Yidan Ding
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Taehyung Yoon
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| | - Bin He
- Department of Biomedical Engineering at Carnegie Mellon University, Pittsburgh, PA
| |
Collapse
|
8
|
Ivanov N, Lio A, Chau T. Towards user-centric BCI design: Markov chain-based user assessment for mental imagery EEG-BCIs. J Neural Eng 2023; 20:066037. [PMID: 38128128 DOI: 10.1088/1741-2552/ad17f2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 12/21/2023] [Indexed: 12/23/2023]
Abstract
Objective.While electroencephalography (EEG)-based brain-computer interfaces (BCIs) have many potential clinical applications, their use is impeded by poor performance for many users. To improve BCI performance, either via enhanced signal processing or user training, it is critical to understand and describe each user's ability to perform mental control tasks and produce discernible EEG patterns. While classification accuracy has predominantly been used to assess user performance, limitations and criticisms of this approach have emerged, thus prompting the need to develop novel user assessment approaches with greater descriptive capability. Here, we propose a combination of unsupervised clustering and Markov chain models to assess and describe user skill.Approach.Using unsupervisedK-means clustering, we segmented the EEG signal space into regions representing pattern states that users could produce. A user's movement through these pattern states while performing different tasks was modeled using Markov chains. Finally, using the steady-state distributions and entropy rates of the Markov chains, we proposed two metricstaskDistinctandrelativeTaskInconsistencyto assess, respectively, a user's ability to (i) produce distinct task-specific patterns for each mental task and (ii) maintain consistent patterns during individual tasks.Main results.Analysis of data from 14 adolescents using a three-class BCI revealed significant correlations between thetaskDistinctandrelativeTaskInconsistencymetrics and classification F1 score. Moreover, analysis of the pattern states and Markov chain models yielded descriptive information regarding user performance not immediately apparent from classification accuracy.Significance.Our proposed user assessment method can be used in concert with classifier-based analysis to further understand the extent to which users produce task-specific, time-evolving EEG patterns. In turn, this information could be used to enhance user training or classifier design.
Collapse
Affiliation(s)
- Nicolas Ivanov
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| | - Aaron Lio
- Division of Engineering Science, University of Toronto, Toronto, ON, Canada
| | - Tom Chau
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
9
|
Jorajuria T, Nikulin VV, Kapralov N, Gomez M, Vidaurre C. MEAN SP: How Many Channels are Needed to Predict the Performance of a SMR-Based BCI? IEEE Trans Neural Syst Rehabil Eng 2023; 31:4931-4941. [PMID: 38051627 DOI: 10.1109/tnsre.2023.3339612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
Predicting whether a particular individual would reach an adequate control of a Brain-Computer Interface (BCI) has many practical advantages. On the one hand, participants with low predicted performance could be trained with specifically designed sessions and avoid frustrating experiments; on the other hand, planning time and resources would be more efficient; and finally, the variables related to an accurate prediction could be manipulated to improve the prospective BCI performance. To this end, several predictors have been proposed in the literature, most of them based on the power estimation of EEG signals at the specific frequency bands. Many of these studies evaluate their predictors in relatively small datasets and/or using a relatively high number of channels. In this manuscript, we propose a novel predictor called [Formula: see text] to predict the performance of participants using BCIs that are based on the modulation of sensorimotor rhythms. This novel predictor has been positively evaluated using only 2, 3, 4 or 5 channels. [Formula: see text] has shown to perform as well as or better than other state-of-the-art predictors. The best sets of different number of channels are also provided, which have been tested in two different settings to prove their robustness. The proposed predictor has been successfully evaluated using two large-scale datasets containing 150 and 80 participants, respectively. We also discuss predictor thresholds for users to expect good performance in feedback experiments and show the advantages in comparison to a competing algorithm.
Collapse
|
10
|
Kancheva I, van der Salm SMA, Ramsey NF, Vansteensel MJ. Association between lesion location and sensorimotor rhythms in stroke - a systematic review with narrative synthesis. Neurol Sci 2023; 44:4263-4289. [PMID: 37606742 PMCID: PMC10641054 DOI: 10.1007/s10072-023-06982-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 07/26/2023] [Indexed: 08/23/2023]
Abstract
BACKGROUND Stroke causes alterations in the sensorimotor rhythms (SMRs) of the brain. However, little is known about the influence of lesion location on the SMRs. Understanding this relationship is relevant for the use of SMRs in assistive and rehabilitative therapies, such as Brain-Computer Interfaces (BCIs).. METHODS We reviewed current evidence on the association between stroke lesion location and SMRs through systematically searching PubMed and Embase and generated a narrative synthesis of findings. RESULTS We included 12 articles reporting on 161 patients. In resting-state studies, cortical and pontine damage were related to an overall decrease in alpha (∼8-12 Hz) and increase in delta (∼1-4 Hz) power. In movement paradigm studies, attenuated alpha and beta (∼15-25 Hz) event-related desynchronization (ERD) was shown in stroke patients during (attempted) paretic hand movement, compared to controls. Stronger reductions in alpha and beta ERD in the ipsilesional, compared to contralesional hemisphere, were observed for cortical lesions. Subcortical stroke was found to affect bilateral ERD and ERS, but results were highly variable. CONCLUSIONS Findings suggest a link between stroke lesion location and SMR alterations, but heterogeneity across studies and limited lesion location descriptions precluded a meta-analysis. SIGNIFICANCE Future research would benefit from more uniformly defined outcome measures, homogeneous methodologies, and improved lesion location reporting.
Collapse
Affiliation(s)
- Ivana Kancheva
- UMC Utrecht Brain Center, Department of Neurology & Neurosurgery, University Medical Center Utrecht, P.O. Box 85060, 3508 AB, Utrecht, The Netherlands
| | - Sandra M A van der Salm
- UMC Utrecht Brain Center, Department of Neurology & Neurosurgery, University Medical Center Utrecht, P.O. Box 85060, 3508 AB, Utrecht, The Netherlands
| | - Nick F Ramsey
- UMC Utrecht Brain Center, Department of Neurology & Neurosurgery, University Medical Center Utrecht, P.O. Box 85060, 3508 AB, Utrecht, The Netherlands
| | - Mariska J Vansteensel
- UMC Utrecht Brain Center, Department of Neurology & Neurosurgery, University Medical Center Utrecht, P.O. Box 85060, 3508 AB, Utrecht, The Netherlands.
| |
Collapse
|
11
|
Gu B, Wang K, Chen L, He J, Zhang D, Xu M, Wang Z, Ming D. Study of the Correlation between the Motor Ability of the Individual Upper Limbs and Motor Imagery Induced Neural Activities. Neuroscience 2023; 530:56-65. [PMID: 37652289 DOI: 10.1016/j.neuroscience.2023.08.032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Revised: 07/13/2023] [Accepted: 08/25/2023] [Indexed: 09/02/2023]
Abstract
Motor imagery based brain-computer interfaces (MI-BCIs) have excellent application prospects in motor enhancement and rehabilitation. However, MI-induced electroencephalogram features applied to MI-BCI usually vary from person to person. This study aimed to investigate whether the motor ability of the individual upper limbs was associated with these features, which helps understand the causes of inter-subject variability. We focused on the behavioral and psychological factors reflecting motor abilities. We first obtained the behavioral scale scores from Edinburgh Handedness Questionnaire, Maximum Grip Strength Test, and Purdue Pegboard Test assessments to evaluate the motor execution ability. We also required the subjects to complete the psychological Movement Imagery Questionnaire-3 estimate, representing MI ability. Then we recorded EEG signals from all twenty-two subjects during MI tasks. Pearson correlation coefficient and stepwise regression were used to analyze the relationships between MI-induced relative event-related desynchronization (rERD) patterns and motor abilities. Both Purdue Pegboard Test and Movement Imagery Questionnaire-3 scores had significant correlations with MI-induced neural oscillation patterns. Notably, the Purdue Pegboard Test of the left hand had the most significant correlation with the alpha rERD. The results of stepwise multiple regression analysis showed that the Purdue Pegboard Test and Movement Imagery Questionnaire-3 could best predict the MI-induced rERD. The results demonstrate that hand dexterity and fine motor coordination are significantly related to MI-induced neural activities. In addition, the method of imagining is also relevant to MI features. Therefore, this study is meaningful for understanding individual differences and the design of user-centered MI-BCI.
Collapse
Affiliation(s)
- Bin Gu
- SUISHI (Tianjin) Intelligence Ltd, China; Haihe Laboratory of Brain-computer Interaction and Human-machine Integration, Tianjin, China
| | - Kun Wang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China; Haihe Laboratory of Brain-computer Interaction and Human-machine Integration, Tianjin, China.
| | - Long Chen
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China; Haihe Laboratory of Brain-computer Interaction and Human-machine Integration, Tianjin, China.
| | - Jiatong He
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Dingze Zhang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Minpeng Xu
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China; Haihe Laboratory of Brain-computer Interaction and Human-machine Integration, Tianjin, China
| | - Zhongpeng Wang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China; Haihe Laboratory of Brain-computer Interaction and Human-machine Integration, Tianjin, China
| | - Dong Ming
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China; Haihe Laboratory of Brain-computer Interaction and Human-machine Integration, Tianjin, China
| |
Collapse
|
12
|
Dreyer P, Roc A, Pillette L, Rimbert S, Lotte F. A large EEG database with users' profile information for motor imagery brain-computer interface research. Sci Data 2023; 10:580. [PMID: 37670009 PMCID: PMC10480224 DOI: 10.1038/s41597-023-02445-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Accepted: 08/04/2023] [Indexed: 09/07/2023] Open
Abstract
We present and share a large database containing electroencephalographic signals from 87 human participants, collected during a single day of brain-computer interface (BCI) experiments, organized into 3 datasets (A, B, and C) that were all recorded using the same protocol: right and left hand motor imagery (MI). Each session contains 240 trials (120 per class), which represents more than 20,800 trials, or approximately 70 hours of recording time. It includes the performance of the associated BCI users, detailed information about the demographics, personality profile as well as some cognitive traits and the experimental instructions and codes (executed in the open-source platform OpenViBE). Such database could prove useful for various studies, including but not limited to: (1) studying the relationships between BCI users' profiles and their BCI performances, (2) studying how EEG signals properties varies for different users' profiles and MI tasks, (3) using the large number of participants to design cross-user BCI machine learning algorithms or (4) incorporating users' profile information into the design of EEG signal classification algorithms.
Collapse
Affiliation(s)
- Pauline Dreyer
- Centre Inria de l'université de Bordeaux, Talence, 33405, France
- LaBRI (Univ. Bordeaux/CNRS/Bordeaux INP), Talence, France
| | - Aline Roc
- Centre Inria de l'université de Bordeaux, Talence, 33405, France
- LaBRI (Univ. Bordeaux/CNRS/Bordeaux INP), Talence, France
| | - Léa Pillette
- Inria de l'Université de Rennes, CNRS, IRISA, Rennes, 35042, France
| | - Sébastien Rimbert
- Centre Inria de l'université de Bordeaux, Talence, 33405, France
- LaBRI (Univ. Bordeaux/CNRS/Bordeaux INP), Talence, France
| | - Fabien Lotte
- Centre Inria de l'université de Bordeaux, Talence, 33405, France.
- LaBRI (Univ. Bordeaux/CNRS/Bordeaux INP), Talence, France.
| |
Collapse
|
13
|
Flotaker S, Soler A, Molinas M. Primary color decoding using deep learning on source reconstructed EEG signal responses. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083049 DOI: 10.1109/embc40787.2023.10340033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
The brain's response to visual stimuli of different colors might be used in a brain-computer interface (BCI) paradigm, for letting a user control their surroundings by looking at specific colors. Allowing the user to control certain elements in its environment, such as lighting and doors, by looking at corresponding signs of different colors could serve as an intuitive interface. This paper presents work on the development of an intra-subject classifier for red, green, and blue (RGB) visual evoked potentials (VEPs) in recordings performed with an electroencephalogram (EEG). Three deep neural networks (DNNs), proposed in earlier papers, were employed and tested for data in source- and electrode space. All the tests performed in electrode space yielded better results than those in source space. The best classifier yielded an accuracy of 77% averaged over all subjects, with the best subject having an accuracy of 96%.Clinical relevance- This paper demonstrates that deep learning can be used to classify between red, green and blue visual evoked potentials in EEG recordings with an average accuracy of 77%.
Collapse
|
14
|
Amini Gougeh R, Falk TH. Enhancing motor imagery detection efficacy using multisensory virtual reality priming. FRONTIERS IN NEUROERGONOMICS 2023; 4:1080200. [PMID: 38236517 PMCID: PMC10790854 DOI: 10.3389/fnrgo.2023.1080200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Accepted: 03/23/2023] [Indexed: 01/19/2024]
Abstract
Brain-computer interfaces (BCI) have been developed to allow users to communicate with the external world by translating brain activity into control signals. Motor imagery (MI) has been a popular paradigm in BCI control where the user imagines movements of e.g., their left and right limbs and classifiers are then trained to detect such intent directly from electroencephalography (EEG) signals. For some users, however, it is difficult to elicit patterns in the EEG signal that can be detected with existing features and classifiers. As such, new user control strategies and training paradigms have been highly sought-after to help improve motor imagery performance. Virtual reality (VR) has emerged as one potential tool where improvements in user engagement and level of immersion have shown to improve BCI accuracy. Motor priming in VR, in turn, has shown to further enhance BCI accuracy. In this pilot study, we take the first steps to explore if multisensory VR motor priming, where haptic and olfactory stimuli are present, can improve motor imagery detection efficacy in terms of both improved accuracy and faster detection. Experiments with 10 participants equipped with a biosensor-embedded VR headset, an off-the-shelf scent diffusion device, and a haptic glove with force feedback showed that significant improvements in motor imagery detection could be achieved. Increased activity in the six common spatial pattern filters used were also observed and peak accuracy could be achieved with analysis windows that were 2 s shorter. Combined, the results suggest that multisensory motor priming prior to motor imagery could improve detection efficacy.
Collapse
Affiliation(s)
| | - Tiago H. Falk
- Institut National de la Recherche Scientifique-Energy, Materials and Telecommunications Center, University of Québec, Montreal, QC, Canada
| |
Collapse
|
15
|
Dos Santos EM, San-Martin R, Fraga FJ. Comparison of subject-independent and subject-specific EEG-based BCI using LDA and SVM classifiers. Med Biol Eng Comput 2023; 61:835-845. [PMID: 36626112 DOI: 10.1007/s11517-023-02769-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 01/03/2023] [Indexed: 01/11/2023]
Abstract
Motor imagery brain-computer interface (MI-BCI) is one of the most used paradigms in EEG-based brain-computer interface (BCI). The current state-of-the-art in BCI involves tuning classifiers to subject-specific training data, acquired over several sessions, in order to perform calibration prior to actual use of the so-called subject-specific BCI system (SS-BCI). Herein, the goal is to provide a ready-to-use system requiring minimal effort for setup. Thus, our challenge was to design a subject-independent BCI (SI-BCI) to be used by any new user without the constraint of individual calibration. Outcomes from other studies with the same purpose were used to undertake comparisons and validate our findings. For the EEG signal processing, we used a combination of the delta (0.5-4 Hz), alpha (8-13 Hz), and beta+gamma (13-40 Hz) bands at a stage prior to feature extraction. Next, we extracted features from the 27-channel EEG using common spatial pattern (CSP) and performed binary classification (MI of right- and left-hand) with linear discriminant analysis (LDA) and support vector machine (SVM) classifiers. These analyses were done for both the SS-BCI and SI-BCI models. We employed "leave-one-subject-out" (LOSO) arrangement and 10-fold cross-validation to evaluate our SI-BCI and SS-BCI systems, respectively. Compared with other two studies, our work was the only one that showed higher accuracy for the LDA classifier in SI-BCI as compared to SS-BCI. On the other hand, LDA accuracy was lower than accuracy achieved with SVM in both conditions (SI-BCI and SS-BCI). Our SS-BCI accuracy reached 76.85% using LDA and 94.20% using SVM and for SI-BCI we got 80.30% with LDA and 83.23% with SVM. We conclude that SI-BCI may be a feasible and relevant option, which can be used in scenarios where subjects are not able to submit themselves to long training sessions or to fast evaluation of the so called "BCI illiteracy." Comparatively, our strategy proved to be more efficient, giving us the best result for SI-BCI when faced against the classification performances of other three studies, even considering the caveat that different datasets were used in the comparison of the four studies.
Collapse
|
16
|
Forenzo D, Liu Y, Kim J, Ding Y, Yoon T, He B. Integrating simultaneous motor imagery and spatial attention for EEG-BCI control. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.02.20.529307. [PMID: 36865207 PMCID: PMC9980047 DOI: 10.1101/2023.02.20.529307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
OBJECTIVE EEG-based brain-computer interfaces (BCI) are non-invasive approaches for replacing or restoring motor functions in impaired patients, and direct brain-to-device communication in the general population. Motor imagery (MI) is one of the most used BCI paradigms, but its performance varies across individuals and certain users require substantial training to develop control. In this study, we propose to integrate a MI paradigm simultaneously with a recently proposed Overt Spatial Attention (OSA) paradigm, to accomplish BCI control. METHODS We evaluated a cohort of 25 human subjects' ability to control a virtual cursor in one- and two-dimensions over 5 BCI sessions. The subjects used 5 different BCI paradigms: MI alone, OSA alone, MI and OSA simultaneously towards the same target (MI+OSA), and MI for one axis while OSA controls the other (MI/OSA and OSA/MI). RESULTS Our results show that MI+OSA reached the highest average online performance in 2D tasks at 49% Percent Valid Correct (PVC), statistically outperforms MI alone (42%), and was higher, but not statistically significant, than OSA alone (45%). MI+OSA had a similar performance to each subject's best individual method between MI alone and OSA alone (50%) and 9 subjects reached their highest average BCI performance using MI+OSA. CONCLUSION Integrating MI and OSA leads to improved performance over MI alone at the group level and is the best BCI paradigm option for some subjects. SIGNIFICANCE This work proposes a new BCI control paradigm that integrates two existing paradigms and demonstrates its value by showing that it can improve users' BCI performance.
Collapse
|
17
|
Ivanov N, Chau T. Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training. Front Comput Neurosci 2023; 17:1108889. [PMID: 36860616 PMCID: PMC9968793 DOI: 10.3389/fncom.2023.1108889] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2022] [Accepted: 01/25/2023] [Indexed: 02/15/2023] Open
Abstract
Despite growing interest and research into brain-computer interfaces (BCI), their usage remains limited outside of research laboratories. One reason for this is BCI inefficiency, the phenomenon where a significant number of potential users are unable to produce machine-discernible brain signal patterns to control the devices. To reduce the prevalence of BCI inefficiency, some have advocated for novel user-training protocols that enable users to more effectively modulate their neural activity. Important considerations for the design of these protocols are the assessment measures that are used for evaluating user performance and for providing feedback that guides skill acquisition. Herein, we present three trial-wise adaptations (running, sliding window and weighted average) of Riemannian geometry-based user-performance metrics (classDistinct reflecting the degree of class separability and classStability reflecting the level of within-class consistency) to enable feedback to the user following each individual trial. We evaluated these metrics, along with conventional classifier feedback, using simulated and previously recorded sensorimotor rhythm-BCI data to assess their correlation with and discrimination of broader trends in user performance. Analysis revealed that the sliding window and weighted average variants of our proposed trial-wise Riemannian geometry-based metrics more accurately reflected performance changes during BCI sessions compared to conventional classifier output. The results indicate the metrics are a viable method for evaluating and tracking user performance changes during BCI-user training and, therefore, further investigation into how these metrics may be presented to users during training is warranted.
Collapse
Affiliation(s)
- Nicolas Ivanov
- PRISM Lab, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| | - Tom Chau
- PRISM Lab, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
18
|
Autenrieth M, Kober SE, Wood G. Assessment of the capacity to modulate brain signals in a home-based SMR neurofeedback training setting. Front Hum Neurosci 2023; 16:1032222. [PMID: 36684842 PMCID: PMC9849904 DOI: 10.3389/fnhum.2022.1032222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Accepted: 12/12/2022] [Indexed: 01/07/2023] Open
Abstract
Electroencephalogram (EEG)-based neurofeedback (NF) is mainly used in clinical settings as a therapeutic intervention or to optimize performance in healthy individuals. Home-based NF systems are available and might facilitate general access to NF training, especially when repeated training sessions are necessary. However, it remains an open question whether NF training at home is possible without remote monitoring. In the present study, we assessed the capacity of healthy individuals to modulate their own EEG activity when using a home-based NF training system in a comparable manner as if participants had purchased a commercially available NF system. Participants' face-to-face contact with experimenters was reduced to a minimum, and instructions were provided only in the form of written information or videos. Initially, 38 participants performed 9 sessions of sensorimotor rhythm (SMR) (12-15 Hz) based NF training (three generalization sessions, six training sessions). An active control group (n = 19) received feedback on random EEG frequencies. Because of technical problems, bad EEG data quality, or non-compliance, 21 participants had to be excluded from the final data analysis, providing first evidence for the difficulties of non-supervised home-based NF training. In this study, participants were not able to modulate their own brain activity in a desired direction during NF training. Our results indicate that personal interaction with a NF expert might be of relevance and that remote supervision of the training data and more direct communication with the NF users are necessary to enable successful NF training performance. We provide suggestions for the development and implementation of home-based NF systems.
Collapse
Affiliation(s)
| | - Silvia Erika Kober
- Institute of Psychology, University of Graz, Graz, Austria,BioTechMed-Graz, Graz, Austria
| | - Guilherme Wood
- Institute of Psychology, University of Graz, Graz, Austria,BioTechMed-Graz, Graz, Austria,*Correspondence: Guilherme Wood,
| |
Collapse
|
19
|
Triana-Guzman N, Orjuela-Cañon AD, Jutinico AL, Mendoza-Montoya O, Antelis JM. Decoding EEG rhythms offline and online during motor imagery for standing and sitting based on a brain-computer interface. Front Neuroinform 2022; 16:961089. [PMID: 36120085 PMCID: PMC9481272 DOI: 10.3389/fninf.2022.961089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2022] [Accepted: 08/03/2022] [Indexed: 12/02/2022] Open
Abstract
Motor imagery (MI)-based brain-computer interface (BCI) systems have shown promising advances for lower limb motor rehabilitation. The purpose of this study was to develop an MI-based BCI for the actions of standing and sitting. Thirty-two healthy subjects participated in the study using 17 active EEG electrodes. We used a combination of the filter bank common spatial pattern (FBCSP) method and the regularized linear discriminant analysis (RLDA) technique for decoding EEG rhythms offline and online during motor imagery for standing and sitting. The offline analysis indicated the classification of motor imagery and idle state provided a mean accuracy of 88.51 ± 1.43% and 85.29 ± 1.83% for the sit-to-stand and stand-to-sit transitions, respectively. The mean accuracies of the sit-to-stand and stand-to-sit online experiments were 94.69 ± 1.29% and 96.56 ± 0.83%, respectively. From these results, we believe that the MI-based BCI may be useful to future brain-controlled standing systems.
Collapse
Affiliation(s)
| | | | - Andres L. Jutinico
- Facultad de Ingeniería Mecánica, Electrónica y Biomédica, Universidad Antonio Nariño, Bogota, Colombia
| | - Omar Mendoza-Montoya
- Tecnologico de Monterrey, Escuela de Ingeniería y Ciencias, Monterrey, Mexico
- *Correspondence: Omar Mendoza-Montoya
| | - Javier M. Antelis
- Tecnologico de Monterrey, Escuela de Ingeniería y Ciencias, Monterrey, Mexico
| |
Collapse
|
20
|
Cao L, Wang W, Huang C, Xu Z, Wang H, Jia J, Chen S, Dong Y, Fan C, de Albuquerque VHC. An Effective Fusing Approach by Combining Connectivity Network Pattern and Temporal-Spatial Analysis for EEG-Based BCI Rehabilitation. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2264-2274. [PMID: 35969547 DOI: 10.1109/tnsre.2022.3198434] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Motor-modality-based brain computer interface (BCI) could promote the neural rehabilitation for stroke patients. Temporal-spatial analysis was commonly used for pattern recognition in this task. This paper introduced a novel connectivity network analysis for EEG-based feature selection. The network features of connectivity pattern not only captured the spatial activities responding to motor task, but also mined the interactive pattern among these cerebral regions. Furthermore, the effective combination between temporal-spatial analysis and network analysis was evaluated for improving the performance of BCI classification (81.7%). And the results demonstrated that it could raise the classification accuracies for most of patients (6 of 7 patients). This proposed method was meaningful for developing the effective BCI training program for stroke rehabilitation.
Collapse
|
21
|
Yao L, Jiang N, Mrachacz-Kersting N, Zhu X, Farina D, Wang Y. Performance Variation of a Somatosensory BCI Based on Imagined Sensation: A Large Population Study. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2486-2493. [PMID: 35969546 DOI: 10.1109/tnsre.2022.3198970] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
A proportion of users cannot achieve adequate brain-computer interface (BCI) control. The diversity of BCI modalities provides a way to solve this emerging issue. Here, we investigate the accuracy of a somatosensory BCI based on sensory imagery (SI). During the SI tasks, subjects were instructed to imagine a tactile sensation and to maintain the attention on the corresponding hand, as if there was tactile stimulus on the skin of the wrist. The performance across 106 healthy subjects in left- and right-hand SI discrimination was 78.9±13.2%. In 70.7% of the subjects the performance was above 70%. The SI task induced a contralateral cortical activation, and high-density EEG source localization showed that the real tactile stimulation and imagined tactile stimulation shared similar cortical activations within the somatosensory cortex. The somatosensory BCI based on SI provides a new signal modality for independent BCI development. Moreover, a combination of SI and other BCI modalities, such as motor imagery, may provide new avenues for further improving BCI usage and applicability, especially in those subjects unable to attain adequate BCI control with conventional BCI modalities.
Collapse
|
22
|
Peters B, Eddy B, Galvin-McLaughlin D, Betz G, Oken B, Fried-Oken M. A systematic review of research on augmentative and alternative communication brain-computer interface systems for individuals with disabilities. Front Hum Neurosci 2022; 16:952380. [PMID: 35966988 PMCID: PMC9374067 DOI: 10.3389/fnhum.2022.952380] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Augmentative and alternative communication brain-computer interface (AAC-BCI) systems are intended to offer communication access to people with severe speech and physical impairment (SSPI) without requiring volitional movement. As the field moves toward clinical implementation of AAC-BCI systems, research involving participants with SSPI is essential. Research has demonstrated variability in AAC-BCI system performance across users, and mixed results for comparisons of performance for users with and without disabilities. The aims of this systematic review were to (1) describe study, system, and participant characteristics reported in BCI research, (2) summarize the communication task performance of participants with disabilities using AAC-BCI systems, and (3) explore any differences in performance for participants with and without disabilities. Electronic databases were searched in May, 2018, and March, 2021, identifying 6065 records, of which 73 met inclusion criteria. Non-experimental study designs were common and sample sizes were typically small, with approximately half of studies involving five or fewer participants with disabilities. There was considerable variability in participant characteristics, and in how those characteristics were reported. Over 60% of studies reported an average selection accuracy ≤70% for participants with disabilities in at least one tested condition. However, some studies excluded participants who did not reach a specific system performance criterion, and others did not state whether any participants were excluded based on performance. Twenty-nine studies included participants both with and without disabilities, but few reported statistical analyses comparing performance between the two groups. Results suggest that AAC-BCI systems show promise for supporting communication for people with SSPI, but they remain ineffective for some individuals. The lack of standards in reporting outcome measures makes it difficult to synthesize data across studies. Further research is needed to demonstrate efficacy of AAC-BCI systems for people who experience SSPI of varying etiologies and severity levels, and these individuals should be included in system design and testing. Consensus in terminology and consistent participant, protocol, and performance description will facilitate the exploration of user and system characteristics that positively or negatively affect AAC-BCI use, and support innovations that will make this technology more useful to a broader group of people. Clinical trial registration https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018095345, PROSPERO: CRD42018095345.
Collapse
Affiliation(s)
- Betts Peters
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
| | - Brandon Eddy
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
- Speech and Hearing Sciences Department, Portland State University, Portland, OR, United States
| | - Deirdre Galvin-McLaughlin
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
| | - Gail Betz
- Health Sciences and Human Services Library, University of Maryland, Baltimore, MD, United States
| | - Barry Oken
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- Department of Neurology, Oregon Health & Science University, Portland, OR, United States
| | - Melanie Fried-Oken
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
| |
Collapse
|
23
|
Tibrewal N, Leeuwis N, Alimardani M. Classification of motor imagery EEG using deep learning increases performance in inefficient BCI users. PLoS One 2022; 17:e0268880. [PMID: 35867703 PMCID: PMC9307149 DOI: 10.1371/journal.pone.0268880] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 05/11/2022] [Indexed: 11/19/2022] Open
Abstract
Motor Imagery Brain-Computer Interfaces (MI-BCIs) are AI-driven systems that capture brain activity patterns associated with mental imagination of movement and convert them into commands for external devices. Traditionally, MI-BCIs operate on Machine Learning (ML) algorithms, which require extensive signal processing and feature engineering to extract changes in sensorimotor rhythms (SMR). In recent years, Deep Learning (DL) models have gained popularity for EEG classification as they provide a solution for automatic extraction of spatio-temporal features in the signals. However, past BCI studies that employed DL models, only attempted them with a small group of participants, without investigating the effectiveness of this approach for different user groups such as inefficient users. BCI inefficiency is a known and unsolved problem within BCI literature, generally defined as the inability of the user to produce the desired SMR patterns for the BCI classifier. In this study, we evaluated the effectiveness of DL models in capturing MI features particularly in the inefficient users. EEG signals from 54 subjects who performed a MI task of left- or right-hand grasp were recorded to compare the performance of two classification approaches; a ML approach vs. a DL approach. In the ML approach, Common Spatial Patterns (CSP) was used for feature extraction and then Linear Discriminant Analysis (LDA) model was employed for binary classification of the MI task. In the DL approach, a Convolutional Neural Network (CNN) model was constructed on the raw EEG signals. Additionally, subjects were divided into high vs. low performers based on their online BCI accuracy and the difference between the two classifiers’ performance was compared between groups. Our results showed that the CNN model improved the classification accuracy for all subjects within the range of 2.37 to 28.28%, but more importantly, this improvement was significantly larger for low performers. Our findings show promise for employment of DL models on raw EEG signals in future MI-BCI systems, particularly for BCI inefficient users who are unable to produce desired sensorimotor patterns for conventional ML approaches.
Collapse
Affiliation(s)
- Navneet Tibrewal
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, The Netherlands
| | - Nikki Leeuwis
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, The Netherlands
- Research Department, Unravel Research, Utrecht, The Netherlands
| | - Maryam Alimardani
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, The Netherlands
- * E-mail:
| |
Collapse
|
24
|
Kasahara K, DaSalla CS, Honda M, Hanakawa T. Basal ganglia-cortical connectivity underlies self-regulation of brain oscillations in humans. Commun Biol 2022; 5:712. [PMID: 35842523 PMCID: PMC9288463 DOI: 10.1038/s42003-022-03665-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Accepted: 06/30/2022] [Indexed: 11/09/2022] Open
Abstract
Brain-computer interfaces provide an artificial link by which the brain can directly interact with the environment. To achieve fine brain-computer interface control, participants must modulate the patterns of the cortical oscillations generated from the motor and somatosensory cortices. However, it remains unclear how humans regulate cortical oscillations, the controllability of which substantially varies across individuals. Here, we performed simultaneous electroencephalography (to assess brain-computer interface control) and functional magnetic resonance imaging (to measure brain activity) in healthy participants. Self-regulation of cortical oscillations induced activity in the basal ganglia-cortical network and the neurofeedback control network. Successful self-regulation correlated with striatal activity in the basal ganglia-cortical network, through which patterns of cortical oscillations were likely modulated. Moreover, basal ganglia-cortical network and neurofeedback control network connectivity correlated with strong and weak self-regulation, respectively. The findings indicate that the basal ganglia-cortical network is important for self-regulation, the understanding of which should help advance brain-computer interface technology. Simultaneous fMRI-EEG in 26 healthy participants indicate that the basal ganglia cortical network and the neurofeedback control network play different roles in self-regulation, providing further insight into the neural correlates for brain-machine interface control and feedback.
Collapse
Affiliation(s)
- Kazumi Kasahara
- Department of Advanced Neuroimaging, Integrative Brain Imaging Center, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan.,Department of Functional Brain Research, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan.,Human Informatics and Interaction Research Institute, National Institute of Advanced Industrial Science and Technology, Ibaraki, 305-8566, Japan
| | - Charles S DaSalla
- Department of Advanced Neuroimaging, Integrative Brain Imaging Center, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan.,Department of Functional Brain Research, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan
| | - Manabu Honda
- Department of Functional Brain Research, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan
| | - Takashi Hanakawa
- Department of Advanced Neuroimaging, Integrative Brain Imaging Center, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan. .,Department of Functional Brain Research, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan. .,Integrated Neuroanatomy and Neuroimaging, Kyoto University Graduate School of Medicine, Kyoto, 606-8501, Japan.
| |
Collapse
|
25
|
Le Franc S, Herrera Altamira G, Guillen M, Butet S, Fleck S, Lécuyer A, Bougrain L, Bonan I. Toward an Adapted Neurofeedback for Post-stroke Motor Rehabilitation: State of the Art and Perspectives. Front Hum Neurosci 2022; 16:917909. [PMID: 35911589 PMCID: PMC9332194 DOI: 10.3389/fnhum.2022.917909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Accepted: 06/20/2022] [Indexed: 11/28/2022] Open
Abstract
Stroke is a severe health issue, and motor recovery after stroke remains an important challenge in the rehabilitation field. Neurofeedback (NFB), as part of a brain–computer interface, is a technique for modulating brain activity using on-line feedback that has proved to be useful in motor rehabilitation for the chronic stroke population in addition to traditional therapies. Nevertheless, its use and applications in the field still leave unresolved questions. The brain pathophysiological mechanisms after stroke remain partly unknown, and the possibilities for intervention on these mechanisms to promote cerebral plasticity are limited in clinical practice. In NFB motor rehabilitation, the aim is to adapt the therapy to the patient’s clinical context using brain imaging, considering the time after stroke, the localization of brain lesions, and their clinical impact, while taking into account currently used biomarkers and technical limitations. These modern techniques also allow a better understanding of the physiopathology and neuroplasticity of the brain after stroke. We conducted a narrative literature review of studies using NFB for post-stroke motor rehabilitation. The main goal was to decompose all the elements that can be modified in NFB therapies, which can lead to their adaptation according to the patient’s context and according to the current technological limits. Adaptation and individualization of care could derive from this analysis to better meet the patients’ needs. We focused on and highlighted the various clinical and technological components considering the most recent experiments. The second goal was to propose general recommendations and enhance the limits and perspectives to improve our general knowledge in the field and allow clinical applications. We highlighted the multidisciplinary approach of this work by combining engineering abilities and medical experience. Engineering development is essential for the available technological tools and aims to increase neuroscience knowledge in the NFB topic. This technological development was born out of the real clinical need to provide complementary therapeutic solutions to a public health problem, considering the actual clinical context of the post-stroke patient and the practical limits resulting from it.
Collapse
Affiliation(s)
- Salomé Le Franc
- Rehabilitation Medicine Unit, University Hospital of Rennes, Rennes, France
- Hybrid Team, Inria, University of Rennes, Irisa, UMR CNRS 6074, Rennes, France
- *Correspondence: Salomé Le Franc,
| | | | - Maud Guillen
- Hybrid Team, Inria, University of Rennes, Irisa, UMR CNRS 6074, Rennes, France
- Neurology Unit, University Hospital of Rennes, Rennes, France
| | - Simon Butet
- Rehabilitation Medicine Unit, University Hospital of Rennes, Rennes, France
- Empenn Unit U1228, Inserm, Inria, University of Rennes, Irisa, UMR CNRS 6074, Rennes, France
| | - Stéphanie Fleck
- Université de Lorraine, CNRS, LORIA, Nancy, France
- EA7312 Laboratoire de Psychologie Ergonomique et Sociale pour l’Expérience Utilisateurs (PERSEUS), Metz, France
| | - Anatole Lécuyer
- Hybrid Team, Inria, University of Rennes, Irisa, UMR CNRS 6074, Rennes, France
| | | | - Isabelle Bonan
- Rehabilitation Medicine Unit, University Hospital of Rennes, Rennes, France
- Empenn Unit U1228, Inserm, Inria, University of Rennes, Irisa, UMR CNRS 6074, Rennes, France
| |
Collapse
|
26
|
Chen L, Zhang L, Wang Z, Gu B, Zhang X, Ming D. The Effects of Sensory Threshold Somatosensory Electrical Stimulation on Users With Different MI-BCI Performance. Front Neurosci 2022; 16:909434. [PMID: 35784856 PMCID: PMC9247255 DOI: 10.3389/fnins.2022.909434] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 05/23/2022] [Indexed: 11/13/2022] Open
Abstract
Motor imagery-based brain-computer interface (MI-BCI) has been largely studied to improve motor learning and promote motor recovery. However, the difficulty in performing MI limits the widespread application of MI-BCI. It has been suggested that the usage of sensory threshold somatosensory electrical stimulation (st-SES) is a promising way to guide participants on MI tasks, but it is still unclear whether st-SES is effective for all users. In the present study, we aimed to examine the effects of st-SES on the MI-BCI performance in two BCI groups (High Performers and Low Performers). Twenty healthy participants were recruited to perform MI and resting tasks with EEG recordings. These tasks were modulated with or without st-SES. We demonstrated that st-SES improved the performance of MI-BCI in the Low Performers, but led to a decrease in the accuracy of MI-BCI in the High Performers. Furthermore, for the Low Performers, the combination of st-SES and MI resulted in significantly greater event-related desynchronization (ERD) and sample entropy of sensorimotor rhythm than MI alone. However, the ERD and sample entropy values of MI did not change significantly during the st-SES intervention in the High Performers. Moreover, we found that st-SES had an effect on the functional connectivity of the fronto-parietal network in the alpha band of Low Performers and the beta band of High Performers, respectively. Our results demonstrated that somatosensory input based on st-SES was only beneficial for sensorimotor cortical activation and MI-BCI performance in the Low Performers, but not in the High Performers. These findings help to optimize guidance strategies to adapt to different categories of users in the practical application of MI-BCI.
Collapse
Affiliation(s)
- Long Chen
- Department of Biomedical Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Lei Zhang
- Department of Biomedical Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Zhongpeng Wang
- Department of Biomedical Engineering, College of Precision Instruments & Optoelectronics Engineering, Tianjin University, Tianjin, China
| | - Bin Gu
- Department of Biomedical Engineering, College of Precision Instruments & Optoelectronics Engineering, Tianjin University, Tianjin, China
| | - Xin Zhang
- Department of Biomedical Engineering, College of Precision Instruments & Optoelectronics Engineering, Tianjin University, Tianjin, China
| | - Dong Ming
- Department of Biomedical Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
- Department of Biomedical Engineering, College of Precision Instruments & Optoelectronics Engineering, Tianjin University, Tianjin, China
| |
Collapse
|
27
|
Blanco-Mora D, Aldridge A, Jorge C, Vourvopoulos A, Figueiredo P, Bermúdez I Badia S. Impact of age, VR, immersion, and spatial resolution on classifier performance for a MI-based BCI. BRAIN-COMPUTER INTERFACES 2022. [DOI: 10.1080/2326263x.2022.2054606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
- D.A. Blanco-Mora
- NeuroRehabLab, Madeira Interactive Techonologies Institute, Universidade da Madeira, Funchal, Portugal
| | - A. Aldridge
- Department of Computer Science and Engineering, Mississippi State University, Starkville, Missippi, USA
| | - C. Jorge
- NeuroRehabLab, Madeira Interactive Techonologies Institute, Universidade da Madeira, Funchal, Portugal
| | - A. Vourvopoulos
- Institute for Systems and Robotics, Lisboa,Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal
| | - P. Figueiredo
- Institute for Systems and Robotics, Lisboa,Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal
| | - S. Bermúdez I Badia
- Faculdade de Ciências Exatas e da Engenharia, Universidade da Madeira, Funchal, Portugal
| |
Collapse
|
28
|
Pandarinath C, Bensmaia SJ. The science and engineering behind sensitized brain-controlled bionic hands. Physiol Rev 2022; 102:551-604. [PMID: 34541898 PMCID: PMC8742729 DOI: 10.1152/physrev.00034.2020] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Revised: 09/07/2021] [Accepted: 09/13/2021] [Indexed: 12/13/2022] Open
Abstract
Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.
Collapse
Affiliation(s)
- Chethan Pandarinath
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia
- Department of Neurosurgery, Emory University, Atlanta, Georgia
| | - Sliman J Bensmaia
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, Illinois
- Committee on Computational Neuroscience, University of Chicago, Chicago, Illinois
- Grossman Institute for Neuroscience, Quantitative Biology, and Human Behavior, University of Chicago, Chicago, Illinois
| |
Collapse
|
29
|
Nascimben M, Wang YK, King JT, Jung TP, Touryan J, Lance BJ, Lin CT. Alpha Correlates of Practice During Mental Preparation for Motor Imagery. IEEE Trans Cogn Dev Syst 2022. [DOI: 10.1109/tcds.2020.3026530] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
30
|
Gantenbein J, Dittli J, Meyer JT, Gassert R, Lambercy O. Intention Detection Strategies for Robotic Upper-Limb Orthoses: A Scoping Review Considering Usability, Daily Life Application, and User Evaluation. Front Neurorobot 2022; 16:815693. [PMID: 35264940 PMCID: PMC8900616 DOI: 10.3389/fnbot.2022.815693] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2021] [Accepted: 01/24/2022] [Indexed: 11/13/2022] Open
Abstract
Wearable robotic upper limb orthoses (ULO) are promising tools to assist or enhance the upper-limb function of their users. While the functionality of these devices has continuously increased, the robust and reliable detection of the user's intention to control the available degrees of freedom remains a major challenge and a barrier for acceptance. As the information interface between device and user, the intention detection strategy (IDS) has a crucial impact on the usability of the overall device. Yet, this aspect and the impact it has on the device usability is only rarely evaluated with respect to the context of use of ULO. A scoping literature review was conducted to identify non-invasive IDS applied to ULO that have been evaluated with human participants, with a specific focus on evaluation methods and findings related to functionality and usability and their appropriateness for specific contexts of use in daily life. A total of 93 studies were identified, describing 29 different IDS that are summarized and classified according to a four-level classification scheme. The predominant user input signal associated with the described IDS was electromyography (35.6%), followed by manual triggers such as buttons, touchscreens or joysticks (16.7%), as well as isometric force generated by residual movement in upper-limb segments (15.1%). We identify and discuss the strengths and weaknesses of IDS with respect to specific contexts of use and highlight a trade-off between performance and complexity in selecting an optimal IDS. Investigating evaluation practices to study the usability of IDS, the included studies revealed that, primarily, objective and quantitative usability attributes related to effectiveness or efficiency were assessed. Further, it underlined the lack of a systematic way to determine whether the usability of an IDS is sufficiently high to be appropriate for use in daily life applications. This work highlights the importance of a user- and application-specific selection and evaluation of non-invasive IDS for ULO. For technology developers in the field, it further provides recommendations on the selection process of IDS as well as to the design of corresponding evaluation protocols.
Collapse
Affiliation(s)
- Jessica Gantenbein
- Rehabilitation Engineering Laboratory, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
| | - Jan Dittli
- Rehabilitation Engineering Laboratory, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
| | - Jan Thomas Meyer
- Rehabilitation Engineering Laboratory, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
| | - Roger Gassert
- Rehabilitation Engineering Laboratory, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
- Future Health Technologies, Singapore-ETH Centre, Campus for Research Excellence and Technological Enterprise (CREATE), Singapore, Singapore
| | - Olivier Lambercy
- Rehabilitation Engineering Laboratory, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
- Future Health Technologies, Singapore-ETH Centre, Campus for Research Excellence and Technological Enterprise (CREATE), Singapore, Singapore
| |
Collapse
|
31
|
Zhang X, Hou W, Wu X, Feng S, Chen L. A Novel Online Action Observation-Based Brain-Computer Interface That Enhances Event-Related Desynchronization. IEEE Trans Neural Syst Rehabil Eng 2021; 29:2605-2614. [PMID: 34878977 DOI: 10.1109/tnsre.2021.3133853] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Brain-computer interface (BCI)-based stroke rehabilitation is an emerging field in which different studies have reported variable outcomes. Among the BCI paradigms, motor imagery (MI)-based closed-loop BCI is still the main pattern in rehabilitation training. It can estimate a patient' motor intention and provide corresponding feedback. However, the individual difference in the ability to generate event-related desynchronization (ERD) and the low classification accuracy of the multi-class scenario restrict the application of MI-based BCI. In the current study, a novel online action observation (AO)-based BCI was proposed. The visual stimuli of four types of hand movements were designed to simultaneously induce steady-state motion visual evoked potential (SSMVEP) in the occipital region and to activate the sensorimotor region. Task-related component analysis was performed to identify the SSMVEP. Results showed that the amplitude of the induced frequency in the SSMVEP had a negative relationship with the stimulus frequency. The classification accuracy in the four-class scenario reached 72.81 ± 13.55% within 2.5s. Importantly, the AO-based closed-loop BCI, which provided visual feedback based on the SSMVEP, could enhance ERD compared with AO-alone. The increased attentiveness might be one key factor for the enhancement of the ERD in the designed AO-based BCI. In summary, the proposed AO-based BCI provides a new insight for BCI-based rehabilitation.
Collapse
|
32
|
Nagels-Coune L, Riecke L, Benitez-Andonegui A, Klinkhammer S, Goebel R, De Weerd P, Lührs M, Sorger B. See, Hear, or Feel - to Speak: A Versatile Multiple-Choice Functional Near-Infrared Spectroscopy-Brain-Computer Interface Feasible With Visual, Auditory, or Tactile Instructions. Front Hum Neurosci 2021; 15:784522. [PMID: 34899223 PMCID: PMC8656940 DOI: 10.3389/fnhum.2021.784522] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Accepted: 11/05/2021] [Indexed: 11/13/2022] Open
Abstract
Severely motor-disabled patients, such as those suffering from the so-called "locked-in" syndrome, cannot communicate naturally. They may benefit from brain-computer interfaces (BCIs) exploiting brain signals for communication and therewith circumventing the muscular system. One BCI technique that has gained attention recently is functional near-infrared spectroscopy (fNIRS). Typically, fNIRS-based BCIs allow for brain-based communication via voluntarily modulation of brain activity through mental task performance guided by visual or auditory instructions. While the development of fNIRS-BCIs has made great progress, the reliability of fNIRS-BCIs across time and environments has rarely been assessed. In the present fNIRS-BCI study, we tested six healthy participants across three consecutive days using a straightforward four-choice fNIRS-BCI communication paradigm that allows answer encoding based on instructions using various sensory modalities. To encode an answer, participants performed a motor imagery task (mental drawing) in one out of four time periods. Answer encoding was guided by either the visual, auditory, or tactile sensory modality. Two participants were tested outside the laboratory in a cafeteria. Answers were decoded from the time course of the most-informative fNIRS channel-by-chromophore combination. Across the three testing days, we obtained mean single- and multi-trial (joint analysis of four consecutive trials) accuracies of 62.5 and 85.19%, respectively. Obtained multi-trial accuracies were 86.11% for visual, 80.56% for auditory, and 88.89% for tactile sensory encoding. The two participants that used the fNIRS-BCI in a cafeteria obtained the best single- (72.22 and 77.78%) and multi-trial accuracies (100 and 94.44%). Communication was reliable over the three recording sessions with multi-trial accuracies of 86.11% on day 1, 86.11% on day 2, and 83.33% on day 3. To gauge the trade-off between number of optodes and decoding accuracy, averaging across two and three promising fNIRS channels was compared to the one-channel approach. Multi-trial accuracy increased from 85.19% (one-channel approach) to 91.67% (two-/three-channel approach). In sum, the presented fNIRS-BCI yielded robust decoding results using three alternative sensory encoding modalities. Further, fNIRS-BCI communication was stable over the course of three consecutive days, even in a natural (social) environment. Therewith, the developed fNIRS-BCI demonstrated high flexibility, reliability and robustness, crucial requirements for future clinical applicability.
Collapse
Affiliation(s)
- Laurien Nagels-Coune
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
- Zorggroep Sint-Kamillus, Bierbeek, Belgium
| | - Lars Riecke
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
| | - Amaia Benitez-Andonegui
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
- MEG Core Facility, National Institutes of Mental Health, Bethesda, MD, United States
| | - Simona Klinkhammer
- Department of Psychiatry and Neuropsychology, Faculty of Health Medicine and Life Sciences, Maastricht University, Maastricht, Netherlands
- School for Mental Health and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Rainer Goebel
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
- Brain Innovation B.V., Maastricht, Netherlands
| | - Peter De Weerd
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
- Maastricht Centre for Systems Biology, Maastricht University, Maastricht, Netherlands
| | | | - Bettina Sorger
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
| |
Collapse
|
33
|
Park S, Ha J, Kim DH, Kim L. Improving Motor Imagery-Based Brain-Computer Interface Performance Based on Sensory Stimulation Training: An Approach Focused on Poorly Performing Users. Front Neurosci 2021; 15:732545. [PMID: 34803582 PMCID: PMC8602688 DOI: 10.3389/fnins.2021.732545] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Accepted: 10/20/2021] [Indexed: 11/13/2022] Open
Abstract
The motor imagery (MI)-based brain-computer interface (BCI) is an intuitive interface that provides control over computer applications directly from brain activity. However, it has shown poor performance compared to other BCI systems such as P300 and SSVEP BCI. Thus, this study aimed to improve MI-BCI performance by training participants in MI with the help of sensory inputs from tangible objects (i.e., hard and rough balls), with a focus on poorly performing users. The proposed method is a hybrid of training and imagery, combining motor execution and somatosensory sensation from a ball-type stimulus. Fourteen healthy participants participated in the somatosensory-motor imagery (SMI) experiments (within-subject design) involving EEG data classification with a three-class system (signaling with left hand, right hand, or right foot). In the scenario of controlling a remote robot to move it to the target point, the participants performed MI when faced with a three-way intersection. The SMI condition had a better classification performance than did the MI condition, achieving a 68.88% classification performance averaged over all participants, which was 6.59% larger than that in the MI condition (p < 0.05). In poor performers, the classification performance in SMI was 10.73% larger than in the MI condition (62.18% vs. 51.45%). However, good performers showed a slight performance decrement (0.86%) in the SMI condition compared to the MI condition (80.93% vs. 81.79%). Combining the brain signals from the motor and somatosensory cortex, the proposed hybrid MI-BCI system demonstrated improved classification performance, this phenomenon was predominant in poor performers (eight out of nine subjects). Hybrid MI-BCI systems may significantly contribute to reducing the proportion of BCI-inefficiency users and closing the performance gap with other BCI systems.
Collapse
Affiliation(s)
- Sangin Park
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
| | - Jihyeon Ha
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea
| | - Da-Hye Kim
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
| | - Laehyun Kim
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
- Department of HY-KIST Bio-Convergence, Hanyang University, Seoul, South Korea
| |
Collapse
|
34
|
Jia T, Mo L, Li C, Liu A, Li Z, Ji L. 5 Hz rTMS improves motor-imagery based BCI classification performance. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:6116-6120. [PMID: 34892512 DOI: 10.1109/embc46164.2021.9630102] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Brain-computer interface (BCI) based rehabilitation has been proven a promising method facilitating motor recovery. Recognizing motor intention is crucial for realizing BCI rehabilitation training. Event-related desynchronization (ERD) is a kind of electroencephalogram (EEG) inherent characteristics associated with motor intention. However, due to brain deficits poststroke, some patients are not able to generate ERD, which discourages them to be involved in BCI rehabilitation training. To boost ERD during motor imagery (MI), this paper investigates the effects of high-frequency repetitive transcranial magnetic stimulation (rTMS) on BCI classification performance. Eleven subjects participated in this study. The experiment consisted of two conditions: rTMS + MI versus sham rTMS + MI, which were arranged on different days. MI tests with 64-channel EEG recording were arranged immediately before and after rTMS and sham rTMS. Time-frequency analysis were utilized to measure ERD changes. Common spatial pattern was used to extract features and linear discriminant analysis was used to calculate offline classification accuracies. Paired-sample t-test and Wilcoxon signed rank tests with post-hoc analysis were used to compare performance before and after stimulation. Statistically stronger ERD (-13.93±12.99%) was found after real rTMS compared with ERD (-5.71±21.25%) before real rTMS (p<0.05). Classification accuracy after real rTMS (70.71±10.32%) tended to be higher than that before real rTMS (66.50±8.48%) (p<0.1). However, no statistical differences were found after sham stimulation. This research provides an effective method in improving BCI performance by utilizing neural modulation.Clinical Relevance- This study offers a promising treatment for patients who cannot be recruited in BCI rehabilitation training due to poor BCI classification performance.
Collapse
|
35
|
Leeuwis N, Yoon S, Alimardani M. Functional Connectivity Analysis in Motor-Imagery Brain Computer Interfaces. Front Hum Neurosci 2021; 15:732946. [PMID: 34720907 PMCID: PMC8555469 DOI: 10.3389/fnhum.2021.732946] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Accepted: 09/03/2021] [Indexed: 11/25/2022] Open
Abstract
Motor Imagery BCI systems have a high rate of users that are not capable of modulating their brain activity accurately enough to communicate with the system. Several studies have identified psychological, cognitive, and neurophysiological measures that might explain this MI-BCI inefficiency. Traditional research had focused on mu suppression in the sensorimotor area in order to classify imagery, but this does not reflect the true dynamics that underlie motor imagery. Functional connectivity reflects the interaction between brain regions during the MI task and resting-state network and is a promising tool in improving MI-BCI classification. In this study, 54 novice MI-BCI users were split into two groups based on their accuracy and their functional connectivity was compared in three network scales (Global, Large and Local scale) during the resting-state, left vs. right-hand motor imagery task, and the transition between the two phases. Our comparison of High and Low BCI performers showed that in the alpha band, functional connectivity in the right hemisphere was increased in High compared to Low aptitude MI-BCI users during motor imagery. These findings contribute to the existing literature that indeed connectivity might be a valuable feature in MI-BCI classification and in solving the MI-BCI inefficiency problem.
Collapse
Affiliation(s)
- Nikki Leeuwis
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, Netherlands
| | | | | |
Collapse
|
36
|
Hehenberger L, Batistic L, Sburlea AI, Müller-Putz GR. Directional Decoding From EEG in a Center-Out Motor Imagery Task With Visual and Vibrotactile Guidance. Front Hum Neurosci 2021; 15:687252. [PMID: 34630055 PMCID: PMC8497713 DOI: 10.3389/fnhum.2021.687252] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Accepted: 08/30/2021] [Indexed: 11/13/2022] Open
Abstract
Motor imagery is a popular technique employed as a motor rehabilitation tool, or to control assistive devices to substitute lost motor function. In both said areas of application, artificial somatosensory input helps to mirror the sensorimotor loop by providing kinesthetic feedback or guidance in a more intuitive fashion than via visual input. In this work, we study directional and movement-related information in electroencephalographic signals acquired during a visually guided center-out motor imagery task in two conditions, i.e., with and without additional somatosensory input in the form of vibrotactile guidance. Imagined movements to the right and forward could be discriminated in low-frequency electroencephalographic amplitudes with group level peak accuracies of 70% with vibrotactile guidance, and 67% without vibrotactile guidance. The peak accuracies with and without vibrotactile guidance were not significantly different. Furthermore, the motor imagery could be classified against a resting baseline with group level accuracies between 76 and 83%, using either low-frequency amplitude features or μ and β power spectral features. On average, accuracies were higher with vibrotactile guidance, while this difference was only significant in the latter set of features. Our findings suggest that directional information in low-frequency electroencephalographic amplitudes is retained in the presence of vibrotactile guidance. Moreover, they hint at an enhancing effect on motor-related μ and β spectral features when vibrotactile guidance is provided.
Collapse
Affiliation(s)
- Lea Hehenberger
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Luka Batistic
- Laboratory for Application of Information Technologies, Faculty of Engineering, Department of Computer Engineering, University of Rijeka, Rijeka, Croatia
| | - Andreea I Sburlea
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria.,BioTechMed Graz, Graz, Austria
| |
Collapse
|
37
|
Guthrie MD, Herrera AJ, Downey JE, Brane LJ, Boninger ML, Collinger JL. The impact of distractions on intracortical brain–computer interface control of a robotic arm. BRAIN-COMPUTER INTERFACES 2021. [DOI: 10.1080/2326263x.2021.1980292] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Affiliation(s)
- Michael D. Guthrie
- Rehab Neural Engineering Labs, Department of Bioengineering, Pittsburgh, PA, USA
- Department of Physical Medicine and Rehabilitation, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Angelica J Herrera
- Rehab Neural Engineering Labs, Department of Bioengineering, Pittsburgh, PA, USA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
- Center for the Neural Basis of Cognition, Department of Neuroscience, University of Pittsburgh, Pittsburgh, PA, USA
| | - John E. Downey
- Rehab Neural Engineering Labs, Department of Bioengineering, Pittsburgh, PA, USA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL, USA
| | - Lucas J. Brane
- Rehab Neural Engineering Labs, Department of Bioengineering, Pittsburgh, PA, USA
- Department of Physical Medicine and Rehabilitation, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Michael L. Boninger
- Rehab Neural Engineering Labs, Department of Bioengineering, Pittsburgh, PA, USA
- Department of Physical Medicine and Rehabilitation, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
- Department of Veterans Affairs, Human Engineering Research Laboratories, Va Center of Excellence, Pittsburgh, Pa, USA
| | - Jennifer L. Collinger
- Rehab Neural Engineering Labs, Department of Bioengineering, Pittsburgh, PA, USA
- Department of Physical Medicine and Rehabilitation, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
- Center for the Neural Basis of Cognition, Department of Neuroscience, University of Pittsburgh, Pittsburgh, PA, USA
- Department of Veterans Affairs, Human Engineering Research Laboratories, Va Center of Excellence, Pittsburgh, Pa, USA
| |
Collapse
|
38
|
Martinek R, Ladrova M, Sidikova M, Jaros R, Behbehani K, Kahankova R, Kawala-Sterniuk A. Advanced Bioelectrical Signal Processing Methods: Past, Present and Future Approach-Part II: Brain Signals. SENSORS (BASEL, SWITZERLAND) 2021; 21:6343. [PMID: 34640663 PMCID: PMC8512967 DOI: 10.3390/s21196343] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Revised: 09/12/2021] [Accepted: 09/14/2021] [Indexed: 12/14/2022]
Abstract
As it was mentioned in the previous part of this work (Part I)-the advanced signal processing methods are one of the quickest and the most dynamically developing scientific areas of biomedical engineering with their increasing usage in current clinical practice. In this paper, which is a Part II work-various innovative methods for the analysis of brain bioelectrical signals were presented and compared. It also describes both classical and advanced approaches for noise contamination removal such as among the others digital adaptive and non-adaptive filtering, signal decomposition methods based on blind source separation, and wavelet transform.
Collapse
Affiliation(s)
- Radek Martinek
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.L.); (M.S.); (R.J.); (R.K.)
| | - Martina Ladrova
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.L.); (M.S.); (R.J.); (R.K.)
| | - Michaela Sidikova
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.L.); (M.S.); (R.J.); (R.K.)
| | - Rene Jaros
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.L.); (M.S.); (R.J.); (R.K.)
| | - Khosrow Behbehani
- College of Engineering, The University of Texas in Arlington, Arlington, TX 76019, USA;
| | - Radana Kahankova
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.L.); (M.S.); (R.J.); (R.K.)
| | - Aleksandra Kawala-Sterniuk
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland
| |
Collapse
|
39
|
Palumbo A, Gramigna V, Calabrese B, Ielpo N. Motor-Imagery EEG-Based BCIs in Wheelchair Movement and Control: A Systematic Literature Review. SENSORS (BASEL, SWITZERLAND) 2021; 21:6285. [PMID: 34577493 PMCID: PMC8473300 DOI: 10.3390/s21186285] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/07/2021] [Revised: 09/09/2021] [Accepted: 09/14/2021] [Indexed: 02/07/2023]
Abstract
The pandemic emergency of the coronavirus disease 2019 (COVID-19) shed light on the need for innovative aids, devices, and assistive technologies to enable people with severe disabilities to live their daily lives. EEG-based Brain-Computer Interfaces (BCIs) can lead individuals with significant health challenges to improve their independence, facilitate participation in activities, thus enhancing overall well-being and preventing impairments. This systematic review provides state-of-the-art applications of EEG-based BCIs, particularly those using motor-imagery (MI) data, to wheelchair control and movement. It presents a thorough examination of the different studies conducted since 2010, focusing on the algorithm analysis, features extraction, features selection, and classification techniques used as well as on wheelchair components and performance evaluation. The results provided in this paper could highlight the limitations of current biomedical instrumentations applied to people with severe disabilities and bring focus to innovative research topics.
Collapse
Affiliation(s)
- Arrigo Palumbo
- Department of Medical and Surgical Sciences, “Magna Græcia” University, 88100 Catanzaro, Italy; (A.P.); (B.C.); (N.I.)
| | - Vera Gramigna
- Neuroscience Research Center, Magna Græcia University, 88100 Catanzaro, Italy
| | - Barbara Calabrese
- Department of Medical and Surgical Sciences, “Magna Græcia” University, 88100 Catanzaro, Italy; (A.P.); (B.C.); (N.I.)
| | - Nicola Ielpo
- Department of Medical and Surgical Sciences, “Magna Græcia” University, 88100 Catanzaro, Italy; (A.P.); (B.C.); (N.I.)
| |
Collapse
|
40
|
Simon C, Bolton DAE, Kennedy NC, Soekadar SR, Ruddy KL. Challenges and Opportunities for the Future of Brain-Computer Interface in Neurorehabilitation. Front Neurosci 2021; 15:699428. [PMID: 34276299 PMCID: PMC8282929 DOI: 10.3389/fnins.2021.699428] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Accepted: 06/08/2021] [Indexed: 12/18/2022] Open
Abstract
Brain-computer interfaces (BCIs) provide a unique technological solution to circumvent the damaged motor system. For neurorehabilitation, the BCI can be used to translate neural signals associated with movement intentions into tangible feedback for the patient, when they are unable to generate functional movement themselves. Clinical interest in BCI is growing rapidly, as it would facilitate rehabilitation to commence earlier following brain damage and provides options for patients who are unable to partake in traditional physical therapy. However, substantial challenges with existing BCI implementations have prevented its widespread adoption. Recent advances in knowledge and technology provide opportunities to facilitate a change, provided that researchers and clinicians using BCI agree on standardisation of guidelines for protocols and shared efforts to uncover mechanisms. We propose that addressing the speed and effectiveness of learning BCI control are priorities for the field, which may be improved by multimodal or multi-stage approaches harnessing more sensitive neuroimaging technologies in the early learning stages, before transitioning to more practical, mobile implementations. Clarification of the neural mechanisms that give rise to improvement in motor function is an essential next step towards justifying clinical use of BCI. In particular, quantifying the unknown contribution of non-motor mechanisms to motor recovery calls for more stringent control conditions in experimental work. Here we provide a contemporary viewpoint on the factors impeding the scalability of BCI. Further, we provide a future outlook for optimal design of the technology to best exploit its unique potential, and best practices for research and reporting of findings.
Collapse
Affiliation(s)
- Colin Simon
- Trinity College Institute of Neuroscience and School of Psychology, Trinity College Dublin, Dublin, Ireland
| | - David A. E. Bolton
- Department of Kinesiology and Health Science, Utah State University, Logan, UT, United States
| | - Niamh C. Kennedy
- School of Psychology, Ulster University, Coleraine, United Kingdom
| | - Surjo R. Soekadar
- Clinical Neurotechnology Laboratory, Neurowissenschaftliches Forschungszentrum, Department of Psychiatry and Neurosciences, Charité – Universitätsmedizin Berlin, Berlin, Germany
| | - Kathy L. Ruddy
- Trinity College Institute of Neuroscience and School of Psychology, Trinity College Dublin, Dublin, Ireland
| |
Collapse
|
41
|
Cattai T, Colonnese S, Corsi MC, Bassett DS, Scarano G, De Vico Fallani F. Phase/Amplitude Synchronization of Brain Signals During Motor Imagery BCI Tasks. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1168-1177. [PMID: 34115589 DOI: 10.1109/tnsre.2021.3088637] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
In the last decade, functional connectivity (FC) has been increasingly adopted based on its ability to capture statistical dependencies between multivariate brain signals. However, the role of FC in the context of brain-computer interface applications is still poorly understood. To address this gap in knowledge, we considered a group of 20 healthy subjects during an EEG-based hand motor imagery (MI) task. We studied two well-established FC estimators, i.e. spectral- and imaginary-coherence, and we investigated how they were modulated by the MI task. We characterized the resulting FC networks by extracting the strength of connectivity of each EEG sensor and we compared the discriminant power with respect to standard power spectrum features. At the group level, results showed that while spectral-coherence based network features were increasing in the sensorimotor areas, those based on imaginary-coherence were significantly decreasing. We demonstrated that this opposite, but complementary, behavior was respectively determined by the increase in amplitude and phase synchronization between the brain signals. At the individual level, we eventually assessed the potential of these network connectivity features in a simple off-line classification scenario. Taken together, our results provide fresh insights into the oscillatory mechanisms subserving brain network changes during MI and offer new perspectives to improve BCI performance.
Collapse
|
42
|
Using Brain Activity Patterns to Differentiate Real and Virtual Attended Targets during Augmented Reality Scenarios. INFORMATION 2021. [DOI: 10.3390/info12060226] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023] Open
Abstract
Augmented reality is the fusion of virtual components and our real surroundings. The simultaneous visibility of generated and natural objects often requires users to direct their selective attention to a specific target that is either real or virtual. In this study, we investigated whether this target is real or virtual by using machine learning techniques to classify electroencephalographic (EEG) and eye tracking data collected in augmented reality scenarios. A shallow convolutional neural net classified 3 second EEG data windows from 20 participants in a person-dependent manner with an average accuracy above 70% if the testing data and training data came from different trials. This accuracy could be significantly increased to 77% using a multimodal late fusion approach that included the recorded eye tracking data. Person-independent EEG classification was possible above chance level for 6 out of 20 participants. Thus, the reliability of such a brain–computer interface is high enough for it to be treated as a useful input mechanism for augmented reality applications.
Collapse
|
43
|
Trambaiolli LR, Tiwari A, Falk TH. Affective Neurofeedback Under Naturalistic Conditions: A Mini-Review of Current Achievements and Open Challenges. FRONTIERS IN NEUROERGONOMICS 2021; 2:678981. [PMID: 38235228 PMCID: PMC10790905 DOI: 10.3389/fnrgo.2021.678981] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/10/2021] [Accepted: 04/28/2021] [Indexed: 01/19/2024]
Abstract
Affective neurofeedback training allows for the self-regulation of the putative circuits of emotion regulation. This approach has recently been studied as a possible additional treatment for psychiatric disorders, presenting positive effects in symptoms and behaviors. After neurofeedback training, a critical aspect is the transference of the learned self-regulation strategies to outside the laboratory and how to continue reinforcing these strategies in non-controlled environments. In this mini-review, we discuss the current achievements of affective neurofeedback under naturalistic setups. For this, we first provide a brief overview of the state-of-the-art for affective neurofeedback protocols. We then discuss virtual reality as a transitional step toward the final goal of "in-the-wild" protocols and current advances using mobile neurotechnology. Finally, we provide a discussion of open challenges for affective neurofeedback protocols in-the-wild, including topics such as convenience and reliability, environmental effects in attention and workload, among others.
Collapse
Affiliation(s)
- Lucas R. Trambaiolli
- Basic Neuroscience Division, McLean Hospital–Harvard Medical School, Belmont, MA, United States
| | - Abhishek Tiwari
- Institut National de la Recherche Scientifique, University of Quebec, Montreal, QC, Canada
| | - Tiago H. Falk
- Institut National de la Recherche Scientifique, University of Quebec, Montreal, QC, Canada
| |
Collapse
|
44
|
Zhang K, Xu G, Du C, Wu Y, Zheng X, Zhang S, Han C, Liang R, Chen R. Weak Feature Extraction and Strong Noise Suppression for SSVEP-EEG Based on Chaotic Detection Technology. IEEE Trans Neural Syst Rehabil Eng 2021; 29:862-871. [PMID: 33872154 DOI: 10.1109/tnsre.2021.3073918] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Brain computer interface (BCI) is a novel communication method that does not rely on the normal neural pathway between the brain and muscle of human. It can transform mental activities into relevant commands to control external equipment and establish direct communication pathway. Among different paradigms, steady-state visual evoked potential (SSVEP) is widely used due to its certain periodicity and stability of control. However, electroencephalogram (EEG) of SSVEP is extremely weak and companied with multi-scale and strong noise. Existing algorithms for classification are based on the principle of template matching and spatial filtering, which cannot obtain satisfied performance of feature extraction under the multi-scale noise. Especially for the subjects produce weak response for external stimuli in EEG representation, i.e., BCI-Illiteracy subject, traditional algorithms are difficult to recognize the internal patterns of brain. To address this issue, a novel method based on Chaos theory is proposed to extract feature of SSVEP. The rule of this method is applying the peculiarity of nonlinear dynamics system to detect feature of SSVEP by judging the state changes of chaotic systems after adding weak EEG. To evaluate the validity of proposed method, this research recruit 32 subjects to participate the experiment. All subjects are divided into two groups according to the preliminary classification accuracy (mean acc >70% or < 70%) by canonical correlation analysis and we define the accuracy above 70% as group A (normal subjects), below 70% as group B (BCI-Illiteracy). Then, the classification accuracy and information transmission rate of two groups are verified using Chaotic theory. Experimental results show that all classification methods using in our study achieve good performance for normal subjects while chaos obtain excellent performance and significant improvements than traditional methods for BCI-Illiteracy.
Collapse
|
45
|
Leeuwis N, Paas A, Alimardani M. Vividness of Visual Imagery and Personality Impact Motor-Imagery Brain Computer Interfaces. Front Hum Neurosci 2021; 15:634748. [PMID: 33889080 PMCID: PMC8055841 DOI: 10.3389/fnhum.2021.634748] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2020] [Accepted: 03/08/2021] [Indexed: 12/19/2022] Open
Abstract
Brain-computer interfaces (BCIs) are communication bridges between a human brain and external world, enabling humans to interact with their environment without muscle intervention. Their functionality, therefore, depends on both the BCI system and the cognitive capacities of the user. Motor-imagery BCIs (MI-BCI) rely on the users' mental imagination of body movements. However, not all users have the ability to sufficiently modulate their brain activity for control of a MI-BCI; a problem known as BCI illiteracy or inefficiency. The underlying mechanism of this phenomenon and the cause of such difference among users is yet not fully understood. In this study, we investigated the impact of several cognitive and psychological measures on MI-BCI performance. Fifty-five novice BCI-users participated in a left- versus right-hand motor imagery task. In addition to their BCI classification error rate and demographics, psychological measures including personality factors, affinity for technology, and motivation during the experiment, as well as cognitive measures including visuospatial memory and spatial ability and Vividness of Visual Imagery were collected. Factors that were found to have a significant impact on MI-BCI performance were Vividness of Visual Imagery, and the personality factors of orderliness and autonomy. These findings shed light on individual traits that lead to difficulty in BCI operation and hence can help with early prediction of inefficiency among users to optimize training for them.
Collapse
Affiliation(s)
- Nikki Leeuwis
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, Netherlands
| | | | | |
Collapse
|
46
|
Cattan G. The Use of Brain–Computer Interfaces in Games Is Not Ready for the General Public. FRONTIERS IN COMPUTER SCIENCE 2021. [DOI: 10.3389/fcomp.2021.628773] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
|
47
|
Neo PSH, Mayne T, Fu X, Huang Z, Franz EA. Crosstalk disrupts the production of motor imagery brain signals in brain-computer interfaces. Health Inf Sci Syst 2021; 9:13. [PMID: 33786162 DOI: 10.1007/s13755-021-00142-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2020] [Accepted: 02/24/2021] [Indexed: 10/21/2022] Open
Abstract
Brain-computer interfaces (BCIs) target specific brain activity for neuropsychological rehabilitation, and also allow patients with motor disabilities to control mobility and communication devices. Motor imagery of single-handed actions is used in BCIs but many users cannot control the BCIs effectively, limiting applications in the health systems. Crosstalk is unintended brain activations that interfere with bimanual actions and could also occur during motor imagery. To test if crosstalk impaired BCI user performance, we recorded EEG in 46 participants while they imagined movements in four experimental conditions using motor imagery: left hand (L), right hand (R), tongue (T) and feet (F). Pairwise classification accuracies of the tasks were compared (LR, LF, LT, RF, RT, FT), using common spatio-spectral filters and linear discriminant analysis. We hypothesized that LR classification accuracy would be lower than every other combination that included a hand imagery due to crosstalk. As predicted, classification accuracy for LR (58%) was reliably the lowest. Interestingly, participants who showed poor LR classification also demonstrated at least one good TR, TL, FR or FL classification; and good LR classification was detected in 16% of the participants. For the first time, we showed that crosstalk occurred in motor imagery, and affected BCI performance negatively. Such effects are effector-sensitive regardless of the BCI methods used; and likely not apparent to the user or the BCI developer. This means that tasks choice is crucial when designing BCI. Critically, the effects of crosstalk appear mitigatable. We conclude that understanding crosstalk mitigation is important for improving BCI applicability. Supplementary Information The online version of this article contains supplementary material available (10.1007/s13755-021-00142-y).
Collapse
Affiliation(s)
- Phoebe S-H Neo
- Department of Computer Science, University of Otago, Dunedin, New Zealand.,Department of Psychology, University of Otago, Dunedin, New Zealand
| | - Terence Mayne
- Department of Computer Science, University of Otago, Dunedin, New Zealand.,Department of Psychology, University of Otago, Dunedin, New Zealand
| | - Xiping Fu
- Department of Computer Science, University of Otago, Dunedin, New Zealand
| | - Zhiyi Huang
- Department of Computer Science, University of Otago, Dunedin, New Zealand
| | - Elizabeth A Franz
- Department of Psychology, University of Otago, Dunedin, New Zealand.,fMRI Otago, Dunedin, New Zealand
| |
Collapse
|
48
|
Velasquez-Martinez LF, Zapata-Castano F, Castellanos-Dominguez G. Dynamic Modeling of Common Brain Neural Activity in Motor Imagery Tasks. Front Neurosci 2020; 14:714. [PMID: 33328839 PMCID: PMC7711077 DOI: 10.3389/fnins.2020.00714] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2019] [Accepted: 06/12/2020] [Indexed: 12/17/2022] Open
Abstract
Evaluation of brain dynamics elicited by motor imagery (MI) tasks can contribute to clinical and learning applications. The multi-subject analysis is to make inferences on the group/population level about the properties of MI brain activity. However, intrinsic neurophysiological variability of neural dynamics poses a challenge for devising efficient MI systems. Here, we develop a time-frequency model for estimating the spatial relevance of common neural activity across subjects employing an introduced statistical thresholding rule. In deriving multi-subject spatial maps, we present a comparative analysis of three feature extraction methods: Common Spatial Patterns, Functional Connectivity, and Event-Related De/Synchronization. In terms of interpretability, we evaluate the effectiveness in gathering MI data from collective populations by introducing two assumptions: (i) Non-linear assessment of the similarity between multi-subject data originating the subject-level dynamics; (ii) Assessment of time-varying brain network responses according to the ranking of individual accuracy performed in distinguishing distinct motor imagery tasks (left-hand vs. right-hand). The obtained validation results indicate that the estimated collective dynamics differently reflect the flow of sensorimotor cortex activation, providing new insights into the evolution of MI responses.
Collapse
Affiliation(s)
| | - Frank Zapata-Castano
- Signal Processing and Recognition Group, Universidad Nacional de Colombia, Manizales, Colombia
| | | |
Collapse
|
49
|
Wriessnegger SC, Müller-Putz GR, Brunner C, Sburlea AI. Inter- and Intra-individual Variability in Brain Oscillations During Sports Motor Imagery. Front Hum Neurosci 2020; 14:576241. [PMID: 33192406 PMCID: PMC7662155 DOI: 10.3389/fnhum.2020.576241] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2020] [Accepted: 09/24/2020] [Indexed: 11/23/2022] Open
Abstract
The aim of this work was to re-evaluate electrophysiological data from a previous study on motor imagery (MI) with a special focus on observed inter- and intra-individual differences. More concretely, we investigated event-related desynchronization/synchronization patterns during sports MI (playing tennis) compared with simple MI (squeezing a ball) and discovered high variability across participants. Thirty healthy volunteers were divided in two groups; the experimental group (EG) performed a physical exercise between two imagery sessions, and the control group (CG) watched a landscape movie without physical activity. We computed inter-individual differences by assessing the dissimilarities among subjects for each group, condition, time period, and frequency band. In the alpha band, we observe some clustering in the ranking of the subjects, therefore showing smaller distances than others. Moreover, in our statistical evaluation, we observed a consistency in ranking across time periods both for the EG and for the CG. For the latter, we also observed similar rankings across conditions. On the contrary, in the beta band, the ranking of the subjects was more similar for the EG across conditions and time periods than for the subjects of the CG. With this study, we would like to draw attention to variability measures instead of primarily focusing on the identification of common patterns across participants, which often do not reflect the whole neurophysiological reality.
Collapse
Affiliation(s)
- Selina C Wriessnegger
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria.,BioTechMed-Graz, Graz, Austria
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria.,BioTechMed-Graz, Graz, Austria
| | | | - Andreea I Sburlea
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| |
Collapse
|
50
|
McFarland DJ. Brain-computer interfaces for amyotrophic lateral sclerosis. Muscle Nerve 2020; 61:702-707. [PMID: 32034787 DOI: 10.1002/mus.26828] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2019] [Revised: 01/30/2020] [Accepted: 02/01/2020] [Indexed: 12/11/2022]
Abstract
A brain-computer interface (BCI) is a device that detects signals from the brain and transforms them into useful commands. Researchers have developed BCIs that utilize different kinds of brain signals. These different BCI systems have differing characteristics, such as the amount of training required and the degree to which they are or are not invasive. Much of the research on BCIs to date has involved healthy individuals and evaluation of classification algorithms. Some BCIs have been shown to have potential benefit for users with minimal muscular function as a result of amyotrophic lateral sclerosis. However, there are still several challenges that need to be successfully addressed before BCIs can be clinically useful.
Collapse
|