1
|
Dussard C, Pillette L, Dumas C, Pierrieau E, Hugueville L, Lau B, Jeunet-Kelway C, George N. Influence of feedback transparency on motor imagery neurofeedback performance: the contribution of agency. J Neural Eng 2024; 21:056029. [PMID: 39321834 DOI: 10.1088/1741-2552/ad7f88] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2024] [Accepted: 09/25/2024] [Indexed: 09/27/2024]
Abstract
Objective.Neurofeedback (NF) is a cognitive training procedure based on real-time feedback (FB) of a participant's brain activity that they must learn to self-regulate. A classical visual FB delivered in a NF task is a filling gauge reflecting a measure of brain activity. This abstract visual FB is not transparently linked-from the subject's perspective-to the task performed (e.g., motor imagery (MI)). This may decrease the sense of agency, that is, the participants' reported control over FB. Here, we assessed the influence of FB transparency on NF performance and the role of agency in this relationship.Approach.Participants performed a NF task using MI to regulate brain activity measured using electroencephalography. In separate blocks, participants experienced three different conditions designed to vary transparency: FB was presented as either (1) a swinging pendulum, (2) a clenching virtual hand, (3) a clenching virtual hand combined with a motor illusion induced by tendon vibration. We measured self-reported agency and user experience after each NF block.Main results. We found that FB transparency influences NF performance. Transparent visual FB provided by the virtual hand resulted in significantly better NF performance than the abstract FB of the pendulum. Surprisingly, adding a motor illusion to the virtual hand significantly decreased performance relative to the virtual hand alone. When introduced in incremental linear mixed effect models, self-reported agency was significantly associated with NF performance and it captured the variance related to the effect of FB transparency on NF performance.Significance. Our results highlight the relevance of transparent FB in relation to the sense of agency. This is likely an important consideration in designing FB to improve NF performance and learning outcomes.
Collapse
Affiliation(s)
- Claire Dussard
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France
| | - Léa Pillette
- Université de Rennes, CNRS, IRISA, UMR 6074, 35000 Rennes, France
| | - Cassandra Dumas
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France
| | | | - Laurent Hugueville
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France
- Institut du Cerveau, ICM, Inserm, U1127, CNRS, UMR 7225, Sorbonne Université, CENIR, Centre MEG-EEG, Paris, France
| | - Brian Lau
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France
| | | | - Nathalie George
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France
- Institut du Cerveau, ICM, Inserm, U1127, CNRS, UMR 7225, Sorbonne Université, CENIR, Centre MEG-EEG, Paris, France
| |
Collapse
|
2
|
Xu X, Fan X, Dong J, Zhang X, Song Z, Bai D, Pu F. Enhancing motor imagery in the third-person perspective by manipulating sense of body ownership with virtual reality. Eur J Neurosci 2024; 60:5750-5763. [PMID: 39210784 DOI: 10.1111/ejn.16515] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2023] [Revised: 07/31/2024] [Accepted: 08/12/2024] [Indexed: 09/04/2024]
Abstract
Virtual reality (VR)-guided motor imagery (MI) is a widely used approach for motor rehabilitation, especially for patients with severe motor impairments. Most approaches provide visual guidance from the first-person perspective (1PP). MI training with visual guidance from the third-person perspective (3PP) remains largely unexplored. We argue that 3PP MI training has its own advantages and can supplement 1PP MI. For some movements beyond the view of 1PP, such as shoulder shrugging and other axial movements, MI are suitable performed under 3PP. However, the efficiency of existing paradigms for 3PP MI is unsatisfactory. We speculate that the absence of sense of body ownership (SOO) from 3PP could be one possible factor and hypothesize that 3PP MI could be enhanced by eliciting SOO over a 3PP avatar. Based on our hypothesis, a novel paradigm was proposed to enhance 3PP MI by inducing full-body illusion (FBI) from 3PP, which is similar to the so-called out-of-body experience (OBE), using synchronous visuo-tactile stimulus with VR. The event-related Electroencephalograph (EEG) desynchronization (ERD) at motor-related regions from 31 healthy participants were calculated and compared with a control paradigm without "OBE" FBI induction. This study attempts to enhance 3PP MI with FBI induction. It offers an opportunity to perform MI guided by action observation from 3PP with elicited SOO to the observed avatar. We believe that 3PP MI could provide more possibilities for effective rehabilitation training, when SOO could be elicited to a virtual avatar and the present work demonstrates its viability and effectiveness.
Collapse
Affiliation(s)
- Xiaotian Xu
- Key Laboratory of Human Motion Analysis and Rehabilitation Technology of the Ministry of Civil Affairs, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Xiaoya Fan
- Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, School of Software, Dalian University of Technology, Dalian, Liaoning, China
| | - Jiaoyang Dong
- Key Laboratory of Human Motion Analysis and Rehabilitation Technology of the Ministry of Civil Affairs, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Xiting Zhang
- Key Laboratory of Human Motion Analysis and Rehabilitation Technology of the Ministry of Civil Affairs, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Zhe Song
- Key Laboratory of Human Motion Analysis and Rehabilitation Technology of the Ministry of Civil Affairs, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Dingqun Bai
- Department of Rehabilitation Medicine, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Fang Pu
- Key Laboratory of Human Motion Analysis and Rehabilitation Technology of the Ministry of Civil Affairs, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
- State Key Laboratory of Virtual Reality Technology and System, Beihang University, Beijing, China
- Research Unit of Virtual Body and Virtual Surgery Technologies, Chinese Academy of Medical Sciences, Beijing, China
| |
Collapse
|
3
|
Fodor MA, Herschel H, Cantürk A, Heisenberg G, Volosyak I. Evaluation of Different Visual Feedback Methods for Brain-Computer Interfaces (BCI) Based on Code-Modulated Visual Evoked Potentials (cVEP). Brain Sci 2024; 14:846. [PMID: 39199537 PMCID: PMC11352856 DOI: 10.3390/brainsci14080846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2024] [Revised: 08/08/2024] [Accepted: 08/19/2024] [Indexed: 09/01/2024] Open
Abstract
Brain-computer interfaces (BCIs) enable direct communication between the brain and external devices using electroencephalography (EEG) signals. BCIs based on code-modulated visual evoked potentials (cVEPs) are based on visual stimuli, thus appropriate visual feedback on the interface is crucial for an effective BCI system. Many previous studies have demonstrated that implementing visual feedback can improve information transfer rate (ITR) and reduce fatigue. This research compares a dynamic interface, where target boxes change their sizes based on detection certainty, with a threshold bar interface in a three-step cVEP speller. In this study, we found that both interfaces perform well, with slight variations in accuracy, ITR, and output characters per minute (OCM). Notably, some participants showed significant performance improvements with the dynamic interface and found it less distracting compared to the threshold bars. These results suggest that while average performance metrics are similar, the dynamic interface can provide significant benefits for certain users. This study underscores the potential for personalized interface choices to enhance BCI user experience and performance. By improving user friendliness, performance, and reducing distraction, dynamic visual feedback could optimize BCI technology for a broader range of users.
Collapse
Affiliation(s)
- Milán András Fodor
- Faculty of Technology and Bionics, Rhine-Waal University of Applied Sciences, 47533 Kleve, Germany
| | - Hannah Herschel
- Faculty of Technology and Bionics, Rhine-Waal University of Applied Sciences, 47533 Kleve, Germany
| | - Atilla Cantürk
- Faculty of Technology and Bionics, Rhine-Waal University of Applied Sciences, 47533 Kleve, Germany
| | - Gernot Heisenberg
- Institute of Information Science, Technical University of Applied Sciences Cologne, 50678 Cologne, Germany
| | - Ivan Volosyak
- Faculty of Technology and Bionics, Rhine-Waal University of Applied Sciences, 47533 Kleve, Germany
| |
Collapse
|
4
|
Tortolani AF, Kunigk NG, Sobinov AR, Boninger ML, Bensmaia SJ, Collinger JL, Hatsopoulos NG, Downey JE. How different immersive environments affect intracortical brain computer interfaces. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.30.605911. [PMID: 39131333 PMCID: PMC11312620 DOI: 10.1101/2024.07.30.605911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 08/13/2024]
Abstract
As brain-computer interface (BCI) research advances, many new applications are being developed. Tasks can be performed in different environments, and whether a BCI user can switch environments seamlessly will influence the ultimate utility of a clinical device. Here we investigate the importance of the immersiveness of the virtual environment used to train BCI decoders on the resulting decoder and its generalizability between environments. Two participants who had intracortical electrodes implanted in their precentral gyrus used a BCI to control a virtual arm, either viewed immersively through virtual reality goggles or at a distance on a flat television monitor. Each participant performed better with a decoder trained and tested in the environment they had used the most prior to the study, one for each environment type. The neural tuning to the desired movement was minimally influenced by the immersiveness of the environment. Finally, in further testing with one of the participants, we found that decoders trained in one environment generalized well to the other environment, but the order in which the environments were experienced within a session mattered. Overall, experience with an environment was more influential on performance than the immersiveness of the environment, but BCI performance generalized well after accounting for experience.
Collapse
Affiliation(s)
- Ariana F Tortolani
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL
| | - Nicolas G Kunigk
- Rehab Neural Engineering Labs, University of Pittsburgh, Pittsburgh, PA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA
| | - Anton R Sobinov
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL
| | - Michael L Boninger
- Rehab Neural Engineering Labs, University of Pittsburgh, Pittsburgh, PA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA
- Department of Physical Medicine and Rehabilitation, University of Pittsburgh, Pittsburgh, PA
| | - Sliman J Bensmaia
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL
- Neuroscience Institute, University of Chicago, Chicago, IL
| | - Jennifer L Collinger
- Rehab Neural Engineering Labs, University of Pittsburgh, Pittsburgh, PA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA
- Department of Physical Medicine and Rehabilitation, University of Pittsburgh, Pittsburgh, PA
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA
| | - Nicholas G Hatsopoulos
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL
- Neuroscience Institute, University of Chicago, Chicago, IL
| | - John E Downey
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL
| |
Collapse
|
5
|
von Groll VG, Leeuwis N, Rimbert S, Roc A, Pillette L, Lotte F, Alimardani M. Large scale investigation of the effect of gender on mu rhythm suppression in motor imagery brain-computer interfaces. BRAIN-COMPUTER INTERFACES 2024; 11:87-97. [PMID: 39355516 PMCID: PMC11441392 DOI: 10.1080/2326263x.2024.2345449] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 04/16/2024] [Indexed: 10/03/2024]
Abstract
The utmost issue in Motor Imagery Brain-Computer Interfaces (MI-BCI) is the BCI poor performance known as 'BCI inefficiency'. Although past research has attempted to find a solution by investigating factors influencing users' MI-BCI performance, the issue persists. One of the factors that has been studied in relation to MI-BCI performance is gender. Research regarding the influence of gender on a user's ability to control MI-BCIs remains inconclusive, mainly due to the small sample size and unbalanced gender distribution in past studies. To address these issues and obtain reliable results, this study combined four MI-BCI datasets into one large dataset with 248 subjects and equal gender distribution. The datasets included EEG signals from healthy subjects from both gender groups who had executed a right- vs. left-hand motor imagery task following the Graz protocol. The analysis consisted of extracting the Mu Suppression Index from C3 and C4 electrodes and comparing the values between female and male participants. Unlike some of the previous findings which reported an advantage for female BCI users in modulating mu rhythm activity, our results did not show any significant difference between the Mu Suppression Index of both groups, indicating that gender may not be a predictive factor for BCI performance.
Collapse
Affiliation(s)
| | - Nikki Leeuwis
- Department of Cognitive Science and AI, Tilburg University, Tilburg, Netherlands
| | | | - Aline Roc
- Inria Center at the University of Bordeaux / LaBRI, Talence, France
| | - Léa Pillette
- Department of Virtual Reality, Virtual Humans, Interactions and Robotics, University of Rennes, Inria, CNRS, France
| | - Fabien Lotte
- Inria Center at the University of Bordeaux / LaBRI, Talence, France
| | - Maryam Alimardani
- Department of Cognitive Science and AI, Tilburg University, Tilburg, Netherlands
| |
Collapse
|
6
|
Lakshminarayanan K, Shah R, Daulat SR, Moodley V, Yao Y, Madathil D. The effect of combining action observation in virtual reality with kinesthetic motor imagery on cortical activity. Front Neurosci 2023; 17:1201865. [PMID: 37383098 PMCID: PMC10299830 DOI: 10.3389/fnins.2023.1201865] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 05/25/2023] [Indexed: 06/30/2023] Open
Abstract
Introduction In the past, various techniques have been used to improve motor imagery (MI), such as immersive virtual-reality (VR) and kinesthetic rehearsal. While electroencephalography (EEG) has been used to study the differences in brain activity between VR-based action observation and kinesthetic motor imagery (KMI), there has been no investigation into their combined effect. Prior research has demonstrated that VR-based action observation can enhance MI by providing both visual information and embodiment, which is the perception of oneself as part of the observed entity. Additionally, KMI has been found to produce similar brain activity to physically performing a task. Therefore, we hypothesized that utilizing VR to offer an immersive visual scenario for action observation while participants performed kinesthetic motor imagery would significantly improve cortical activity related to MI. Methods In this study, 15 participants (9 male, 6 female) performed kinesthetic motor imagery of three hand tasks (drinking, wrist flexion-extension, and grabbing) both with and without VR-based action observation. Results Our results indicate that combining VR-based action observation with KMI enhances brain rhythmic patterns and provides better task differentiation compared to KMI without action observation. Discussion These findings suggest that using VR-based action observation alongside kinesthetic motor imagery can improve motor imagery performance.
Collapse
Affiliation(s)
- Kishor Lakshminarayanan
- Neuro-Rehabilitation Lab, School of Electronics Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India
| | - Rakshit Shah
- Department of Chemical and Biomedical Engineering, Cleveland State University, Cleveland, OH, United States
| | - Sohail R. Daulat
- Department of Physiology, University of Arizona College of Medicine – Tucson, Tucson, AZ, United States
| | - Viashen Moodley
- Arizona Center for Hand to Shoulder Surgery, Phoenix, AZ, United States
| | - Yifei Yao
- Soft Tissue Biomechanics Laboratory, School of Biomedical Engineering, Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China
| | - Deepa Madathil
- Jindal Institute of Behavioural Sciences, O.P. Jindal Global University, Sonipat, Haryana, India
| |
Collapse
|
7
|
Farabbi A, Figueiredo P, Ghiringhelli F, Mainardi L, Sanches JM, Moreno P, Santos-Victor J, Vourvopoulos A. Investigating the impact of visual perspective in a motor imagery-based brain-robot interaction: A pilot study with healthy participants. FRONTIERS IN NEUROERGONOMICS 2023; 4:1080794. [PMID: 38234500 PMCID: PMC10790830 DOI: 10.3389/fnrgo.2023.1080794] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Accepted: 03/08/2023] [Indexed: 01/19/2024]
Abstract
Introduction Motor Imagery (MI)-based Brain Computer Interfaces (BCI) have raised gained attention for their use in rehabilitation therapies since they allow controlling an external device by using brain activity, in this way promoting brain plasticity mechanisms that could lead to motor recovery. Specifically, rehabilitation robotics can provide precision and consistency for movement exercises, while embodied robotics could provide sensory feedback that can help patients improve their motor skills and coordination. However, it is still not clear whether different types of visual feedback may affect the elicited brain response and hence the effectiveness of MI-BCI for rehabilitation. Methods In this paper, we compare two visual feedback strategies based on controlling the movement of robotic arms through a MI-BCI system: 1) first-person perspective, with visual information that the user receives when they view the robot arms from their own perspective; and 2) third-person perspective, whereby the subjects observe the robot from an external perspective. We studied 10 healthy subjects over three consecutive sessions. The electroencephalographic (EEG) signals were recorded and evaluated in terms of the power of the sensorimotor rhythms, as well as their lateralization, and spatial distribution. Results Our results show that both feedback perspectives can elicit motor-related brain responses, but without any significant differences between them. Moreover, the evoked responses remained consistent across all sessions, showing no significant differences between the first and the last session. Discussion Overall, these results suggest that the type of perspective may not influence the brain responses during a MI- BCI task based on a robotic feedback, although, due to the limited sample size, more evidence is required. Finally, this study resulted into the production of 180 labeled MI EEG datasets, publicly available for research purposes.
Collapse
Affiliation(s)
- Andrea Farabbi
- B3Lab, Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Milan, Italy
| | - Patricia Figueiredo
- Institute for Systems and Robotics-Lisboa, Instituto Superior Tecnico, Lisbon, Portugal
| | - Fabiola Ghiringhelli
- B3Lab, Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Milan, Italy
| | - Luca Mainardi
- B3Lab, Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Milan, Italy
| | - Joao Miguel Sanches
- Institute for Systems and Robotics-Lisboa, Instituto Superior Tecnico, Lisbon, Portugal
| | - Plinio Moreno
- Institute for Systems and Robotics-Lisboa, Instituto Superior Tecnico, Lisbon, Portugal
| | - Jose Santos-Victor
- Institute for Systems and Robotics-Lisboa, Instituto Superior Tecnico, Lisbon, Portugal
| | | |
Collapse
|
8
|
Lai D, Wan Z, Lin J, Pan L, Ren F, Zhu J, Zhang J, Wang Y, Hao Y, Xu K. Neuronal representation of bimanual arm motor imagery in the motor cortex of a tetraplegia human, a pilot study. Front Neurosci 2023; 17:1133928. [PMID: 36937679 PMCID: PMC10014804 DOI: 10.3389/fnins.2023.1133928] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Accepted: 02/13/2023] [Indexed: 03/05/2023] Open
Abstract
Introduction How the human brain coordinates bimanual movements is not well-established. Methods Here, we recorded neural signals from a paralyzed individual's left motor cortex during both unimanual and bimanual motor imagery tasks and quantified the representational interaction between arms by analyzing the tuning parameters of each neuron. Results We found a similar proportion of neurons preferring each arm during unimanual movements, however, when switching to bimanual movements, the proportion of contralateral preference increased to 71.8%, indicating contralateral lateralization. We also observed a decorrelation process for each arm's representation across the unimanual and bimanual tasks. We further confined that these changes in bilateral relationships are mainly caused by the alteration of tuning parameters, such as the increased bilateral preferred direction (PD) shifts and the significant suppression in bilateral modulation depths (MDs), especially the ipsilateral side. Discussion These results contribute to the knowledge of bimanual coordination and thus the design of cutting-edge bimanual brain-computer interfaces.
Collapse
Affiliation(s)
- Dongrong Lai
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- Key Laboratory of Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou, China
| | - Zijun Wan
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- Key Laboratory of Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou, China
| | - Jiafan Lin
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- Key Laboratory of Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou, China
| | - Li Pan
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- Key Laboratory of Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou, China
| | - Feixiao Ren
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- College of Computer Science and Technology, Zhejiang University, Hangzhou, China
| | - Junming Zhu
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
- MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University, Hangzhou, China
| | - Jianmin Zhang
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
- MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University, Hangzhou, China
| | - Yueming Wang
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- College of Computer Science and Technology, Zhejiang University, Hangzhou, China
- MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University, Hangzhou, China
| | - Yaoyao Hao
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou, China
- College of Computer Science and Technology, Zhejiang University, Hangzhou, China
- MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University, Hangzhou, China
- *Correspondence: Yaoyao Hao,
| | - Kedi Xu
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- Key Laboratory of Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou, China
- MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University, Hangzhou, China
- Kedi Xu,
| |
Collapse
|
9
|
The Effects of Subthreshold Vibratory Noise on Cortical Activity During Motor Imagery. Motor Control 2023:1-14. [PMID: 36801814 DOI: 10.1123/mc.2022-0061] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Revised: 12/04/2022] [Accepted: 01/08/2023] [Indexed: 02/19/2023]
Abstract
Previous studies have demonstrated that both visual and proprioceptive feedback play vital roles in mental practice of movements. Tactile sensation has been shown to improve with peripheral sensory stimulation via imperceptible vibratory noise by stimulating the sensorimotor cortex. With both proprioception and tactile sensation sharing the same population of posterior parietal neurons encoding within high-level spatial representations, the effect of imperceptible vibratory noise on motor imagery-based brain-computer interface is unknown. The objective of this study was to investigate the effects of this sensory stimulation via imperceptible vibratory noise applied to the index fingertip in improving motor imagery-based brain-computer interface performance. Fifteen healthy adults (nine males and six females) were studied. Each subject performed three motor imagery tasks, namely drinking, grabbing, and flexion-extension of the wrist, with and without sensory stimulation while being presented a rich immersive visual scenario through a virtual reality headset. Results showed that vibratory noise increased event-related desynchronization during motor imagery compared with no vibration. Furthermore, the task classification percentage was higher with vibration when the tasks were discriminated using a machine learning algorithm. In conclusion, subthreshold random frequency vibration affected motor imagery-related event-related desynchronization and improved task classification performance.
Collapse
|
10
|
Patel HH, Berlinberg EJ, Nwachukwu B, Williams RJ, Mandelbaum B, Sonkin K, Forsythe B. Quadriceps Weakness is Associated with Neuroplastic Changes Within Specific Corticospinal Pathways and Brain Areas After Anterior Cruciate Ligament Reconstruction: Theoretical Utility of Motor Imagery-Based Brain-Computer Interface Technology for Rehabilitation. Arthrosc Sports Med Rehabil 2022; 5:e207-e216. [PMID: 36866306 PMCID: PMC9971910 DOI: 10.1016/j.asmr.2022.11.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2022] [Accepted: 11/09/2022] [Indexed: 12/29/2022] Open
Abstract
Persistent quadriceps weakness is a problematic sequela of anterior cruciate ligament reconstruction (ACLR). The purposes of this review are to summarize neuroplastic changes after ACL reconstruction; provide an overview of a promising interventions, motor imagery (MI), and its utility in muscle activation; and propose a framework using a brain-computer interface (BCI) to augment quadriceps activation. A literature review of neuroplastic changes, MI training, and BCI-MI technology in postoperative neuromuscular rehabilitation was conducted in PubMed, Embase, and Scopus. Combinations of the following search terms were used to identify articles: "quadriceps muscle," "neurofeedback," "biofeedback," "muscle activation," "motor learning," "anterior cruciate ligament," and "cortical plasticity." We found that ACLR disrupts sensory input from the quadriceps, which results in reduced sensitivity to electrochemical neuronal signals, an increase in central inhibition of neurons regulating quadriceps control and dampening of reflexive motor activity. MI training consists of visualizing an action, without physically engaging in muscle activity. Imagined motor output during MI training increases the sensitivity and conductivity of corticospinal tracts emerging from the primary motor cortex, which helps "exercise" the connections between the brain and target muscle tissues. Motor rehabilitation studies using BCI-MI technology have demonstrated increased excitability of the motor cortex, corticospinal tract, spinal motor neurons, and disinhibition of inhibitory interneurons. This technology has been validated and successfully applied in the recovery of atrophied neuromuscular pathways in stroke patients but has yet to be investigated in peripheral neuromuscular insults, such as ACL injury and reconstruction. Well-designed clinical studies may assess the impact of BCI on clinical outcomes and recovery time. Quadriceps weakness is associated with neuroplastic changes within specific corticospinal pathways and brain areas. BCI-MI shows strong potential for facilitating recovery of atrophied neuromuscular pathways after ACLR and may offer an innovative, multidisciplinary approach to orthopaedic care. Level of Evidence V, expert opinion.
Collapse
Affiliation(s)
- Harsh H. Patel
- Department of Orthopaedic Surgery, Midwest Orthopaedics at Rush, Chicago, Illinois
| | - Elyse J. Berlinberg
- Department of Orthopaedic Surgery, Midwest Orthopaedics at Rush, Chicago, Illinois
| | - Benedict Nwachukwu
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York City, New York
| | - Riley J. Williams
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York City, New York
| | - Bert Mandelbaum
- Department of Orthopaedic Surgery, Cedars-Sinai Kerlan-Jobe Institute, Santa Monica, California, U.S.A
| | | | - Brian Forsythe
- Department of Orthopaedic Surgery, Midwest Orthopaedics at Rush, Chicago, Illinois,Address correspondence to Brian Forsythe, M.D., 1611 W. Harrison St, Suite 360, Chicago, IL 60621
| |
Collapse
|
11
|
Leeuwis N, van Bommel T, Alimardani M. A framework for application of consumer neuroscience in pro-environmental behavior change interventions. Front Hum Neurosci 2022; 16:886600. [PMID: 36188183 PMCID: PMC9520489 DOI: 10.3389/fnhum.2022.886600] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Accepted: 08/23/2022] [Indexed: 11/15/2022] Open
Abstract
Most consumers are aware that climate change is a growing problem and admit that action is needed. However, research shows that consumers' behavior often does not conform to their value and orientations. This value-behavior gap is due to contextual factors such as price, product design, and social norms as well as individual factors such as personal and hedonic values, environmental beliefs, and the workload capacity an individual can handle. Because of this conflict of interest, consumers have a hard time identifying the true drivers of their behavior, as they are either unaware of or unwilling to acknowledge the processes at play. Therefore, consumer neuroscience methods might provide a valuable tool to uncover the implicit measurements of pro-environmental behavior (PEB). Several studies have already defined neurophysiological differences between green and non-green individuals; however, a behavior change intervention must be developed to motivate PEB among consumers. Motivating behavior with reward or punishment will most likely get users engaged in climate change action via brain structures related to the reward system, such as the amygdala, nucleus accumbens, and (pre)frontal cortex, where the reward information and subsequent affective responses are encoded. The intensity of the reward experience can be increased when the consumer is consciously considering the action to achieve it. This makes goal-directed behavior the potential aim of behavior change interventions. This article provides an extensive review of the neuroscientific evidence for consumer attitude, behavior, and decision-making processes in the light of sustainability incentives for behavior change interventions. Based on this review, we aim to unite the current theories and provide future research directions to exploit the power of affective conditioning and neuroscience methods for promoting PEB engagement.
Collapse
Affiliation(s)
- Nikki Leeuwis
- Department of Cognitive Science and Artificial Intelligence, Tilburg School of Humanities and Digital Sciences, Tilburg University, Tilburg, Netherlands
- Unravel Research, Utrecht, Netherlands
| | | | - Maryam Alimardani
- Department of Cognitive Science and Artificial Intelligence, Tilburg School of Humanities and Digital Sciences, Tilburg University, Tilburg, Netherlands
| |
Collapse
|
12
|
Tibrewal N, Leeuwis N, Alimardani M. Classification of motor imagery EEG using deep learning increases performance in inefficient BCI users. PLoS One 2022; 17:e0268880. [PMID: 35867703 PMCID: PMC9307149 DOI: 10.1371/journal.pone.0268880] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 05/11/2022] [Indexed: 11/19/2022] Open
Abstract
Motor Imagery Brain-Computer Interfaces (MI-BCIs) are AI-driven systems that capture brain activity patterns associated with mental imagination of movement and convert them into commands for external devices. Traditionally, MI-BCIs operate on Machine Learning (ML) algorithms, which require extensive signal processing and feature engineering to extract changes in sensorimotor rhythms (SMR). In recent years, Deep Learning (DL) models have gained popularity for EEG classification as they provide a solution for automatic extraction of spatio-temporal features in the signals. However, past BCI studies that employed DL models, only attempted them with a small group of participants, without investigating the effectiveness of this approach for different user groups such as inefficient users. BCI inefficiency is a known and unsolved problem within BCI literature, generally defined as the inability of the user to produce the desired SMR patterns for the BCI classifier. In this study, we evaluated the effectiveness of DL models in capturing MI features particularly in the inefficient users. EEG signals from 54 subjects who performed a MI task of left- or right-hand grasp were recorded to compare the performance of two classification approaches; a ML approach vs. a DL approach. In the ML approach, Common Spatial Patterns (CSP) was used for feature extraction and then Linear Discriminant Analysis (LDA) model was employed for binary classification of the MI task. In the DL approach, a Convolutional Neural Network (CNN) model was constructed on the raw EEG signals. Additionally, subjects were divided into high vs. low performers based on their online BCI accuracy and the difference between the two classifiers’ performance was compared between groups. Our results showed that the CNN model improved the classification accuracy for all subjects within the range of 2.37 to 28.28%, but more importantly, this improvement was significantly larger for low performers. Our findings show promise for employment of DL models on raw EEG signals in future MI-BCI systems, particularly for BCI inefficient users who are unable to produce desired sensorimotor patterns for conventional ML approaches.
Collapse
Affiliation(s)
- Navneet Tibrewal
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, The Netherlands
| | - Nikki Leeuwis
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, The Netherlands
- Research Department, Unravel Research, Utrecht, The Netherlands
| | - Maryam Alimardani
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, The Netherlands
- * E-mail:
| |
Collapse
|
13
|
Ziadeh H, Gulyas D, Nielsen LD, Lehmann S, Nielsen TB, Kjeldsen TKK, Hougaard BI, Jochumsen M, Knoche H. "Mine Works Better": Examining the Influence of Embodiment in Virtual Reality on the Sense of Agency During a Binary Motor Imagery Task With a Brain-Computer Interface. Front Psychol 2022; 12:806424. [PMID: 35002899 PMCID: PMC8741301 DOI: 10.3389/fpsyg.2021.806424] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2021] [Accepted: 12/06/2021] [Indexed: 11/13/2022] Open
Abstract
Motor imagery-based brain-computer interfaces (MI-BCI) have been proposed as a means for stroke rehabilitation, which combined with virtual reality allows for introducing game-based interactions into rehabilitation. However, the control of the MI-BCI may be difficult to obtain and users may face poor performance which frustrates them and potentially affects their motivation to use the technology. Decreases in motivation could be reduced by increasing the users' sense of agency over the system. The aim of this study was to understand whether embodiment (ownership) of a hand depicted in virtual reality can enhance the sense of agency to reduce frustration in an MI-BCI task. Twenty-two healthy participants participated in a within-subject study where their sense of agency was compared in two different embodiment experiences: 1) avatar hand (with body), or 2) abstract blocks. Both representations closed with a similar motion for spatial congruency and popped a balloon as a result. The hand/blocks were controlled through an online MI-BCI. Each condition consisted of 30 trials of MI-activation of the avatar hand/blocks. After each condition a questionnaire probed the participants' sense of agency, ownership, and frustration. Afterwards, a semi-structured interview was performed where the participants elaborated on their ratings. Both conditions supported similar levels of MI-BCI performance. A significant correlation between ownership and agency was observed (r = 0.47, p = 0.001). As intended, the avatar hand yielded much higher ownership than the blocks. When controlling for performance, ownership increased sense of agency. In conclusion, designers of BCI-based rehabilitation applications can draw on anthropomorphic avatars for the visual mapping of the trained limb to improve ownership. While not While not reducing frustration ownership can improve perceived agency given sufficient BCI performance. In future studies the findings should be validated in stroke patients since they may perceive agency and ownership differently than able-bodied users.
Collapse
Affiliation(s)
- Hamzah Ziadeh
- Human Machine Interaction Lab, Department of Architecture, Design, and Media Technology, Institute for Architecture and Media Technology, Aalborg University, Aalborg, Denmark
| | - David Gulyas
- Human Machine Interaction Lab, Department of Architecture, Design, and Media Technology, Institute for Architecture and Media Technology, Aalborg University, Aalborg, Denmark
| | - Louise Dørr Nielsen
- Human Machine Interaction Lab, Department of Architecture, Design, and Media Technology, Institute for Architecture and Media Technology, Aalborg University, Aalborg, Denmark
| | - Steffen Lehmann
- Human Machine Interaction Lab, Department of Architecture, Design, and Media Technology, Institute for Architecture and Media Technology, Aalborg University, Aalborg, Denmark
| | - Thomas Bendix Nielsen
- Human Machine Interaction Lab, Department of Architecture, Design, and Media Technology, Institute for Architecture and Media Technology, Aalborg University, Aalborg, Denmark
| | - Thomas Kim Kroman Kjeldsen
- Human Machine Interaction Lab, Department of Architecture, Design, and Media Technology, Institute for Architecture and Media Technology, Aalborg University, Aalborg, Denmark
| | - Bastian Ilsø Hougaard
- Human Machine Interaction Lab, Department of Architecture, Design, and Media Technology, Institute for Architecture and Media Technology, Aalborg University, Aalborg, Denmark
| | - Mads Jochumsen
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Hendrik Knoche
- Human Machine Interaction Lab, Department of Architecture, Design, and Media Technology, Institute for Architecture and Media Technology, Aalborg University, Aalborg, Denmark
| |
Collapse
|
14
|
Park S, Ha J, Kim DH, Kim L. Improving Motor Imagery-Based Brain-Computer Interface Performance Based on Sensory Stimulation Training: An Approach Focused on Poorly Performing Users. Front Neurosci 2021; 15:732545. [PMID: 34803582 PMCID: PMC8602688 DOI: 10.3389/fnins.2021.732545] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Accepted: 10/20/2021] [Indexed: 11/13/2022] Open
Abstract
The motor imagery (MI)-based brain-computer interface (BCI) is an intuitive interface that provides control over computer applications directly from brain activity. However, it has shown poor performance compared to other BCI systems such as P300 and SSVEP BCI. Thus, this study aimed to improve MI-BCI performance by training participants in MI with the help of sensory inputs from tangible objects (i.e., hard and rough balls), with a focus on poorly performing users. The proposed method is a hybrid of training and imagery, combining motor execution and somatosensory sensation from a ball-type stimulus. Fourteen healthy participants participated in the somatosensory-motor imagery (SMI) experiments (within-subject design) involving EEG data classification with a three-class system (signaling with left hand, right hand, or right foot). In the scenario of controlling a remote robot to move it to the target point, the participants performed MI when faced with a three-way intersection. The SMI condition had a better classification performance than did the MI condition, achieving a 68.88% classification performance averaged over all participants, which was 6.59% larger than that in the MI condition (p < 0.05). In poor performers, the classification performance in SMI was 10.73% larger than in the MI condition (62.18% vs. 51.45%). However, good performers showed a slight performance decrement (0.86%) in the SMI condition compared to the MI condition (80.93% vs. 81.79%). Combining the brain signals from the motor and somatosensory cortex, the proposed hybrid MI-BCI system demonstrated improved classification performance, this phenomenon was predominant in poor performers (eight out of nine subjects). Hybrid MI-BCI systems may significantly contribute to reducing the proportion of BCI-inefficiency users and closing the performance gap with other BCI systems.
Collapse
Affiliation(s)
- Sangin Park
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
| | - Jihyeon Ha
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea
| | - Da-Hye Kim
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
| | - Laehyun Kim
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
- Department of HY-KIST Bio-Convergence, Hanyang University, Seoul, South Korea
| |
Collapse
|
15
|
Grigorev NA, Savosenkov AO, Lukoyanov MV, Udoratina A, Shusharina NN, Kaplan AY, Hramov AE, Kazantsev VB, Gordleeva S. A BCI-Based Vibrotactile Neurofeedback Training Improves Motor Cortical Excitability During Motor Imagery. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1583-1592. [PMID: 34343094 DOI: 10.1109/tnsre.2021.3102304] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
In this study, we address the issue of whether vibrotactile feedback can enhance the motor cortex excitability translated into the plastic changes in local cortical areas during motor imagery (MI) BCI-based training. For this purpose, we focused on two of the most notable neurophysiological effects of MI - the event-related desynchronization (ERD) level and the increase in cortical excitability assessed with navigated transcranial magnetic stimulation (nTMS). For TMS navigation, we used individual high-resolution 3D brain MRIs. Ten BCI-naive and healthy adults participated in this study. The MI (rest or left/right hand imagery using Graz-BCI paradigm) tasks were performed separately in the presence and absence of feedback. To investigate how much the presence/absence of vibrotactile feedback in MI BCI-based training could contribute to the sensorimotor cortical activations, we compared the MEPs amplitude during MI after training with and without feedback. In addition, the ERD levels during MI BCI-based training were investigated. Our findings provide evidence that applying vibrotactile feedback during MI training leads to (i) an enhancement of the desynchronization level of mu-rhythm EEG patterns over the contralateral motor cortex area corresponding to the MI of the non-dominant hand; (ii) an increase in motor cortical excitability in hand muscle representation corresponding to a muscle engaged by the MI.
Collapse
|
16
|
Leeuwis N, Paas A, Alimardani M. Vividness of Visual Imagery and Personality Impact Motor-Imagery Brain Computer Interfaces. Front Hum Neurosci 2021; 15:634748. [PMID: 33889080 PMCID: PMC8055841 DOI: 10.3389/fnhum.2021.634748] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2020] [Accepted: 03/08/2021] [Indexed: 12/19/2022] Open
Abstract
Brain-computer interfaces (BCIs) are communication bridges between a human brain and external world, enabling humans to interact with their environment without muscle intervention. Their functionality, therefore, depends on both the BCI system and the cognitive capacities of the user. Motor-imagery BCIs (MI-BCI) rely on the users' mental imagination of body movements. However, not all users have the ability to sufficiently modulate their brain activity for control of a MI-BCI; a problem known as BCI illiteracy or inefficiency. The underlying mechanism of this phenomenon and the cause of such difference among users is yet not fully understood. In this study, we investigated the impact of several cognitive and psychological measures on MI-BCI performance. Fifty-five novice BCI-users participated in a left- versus right-hand motor imagery task. In addition to their BCI classification error rate and demographics, psychological measures including personality factors, affinity for technology, and motivation during the experiment, as well as cognitive measures including visuospatial memory and spatial ability and Vividness of Visual Imagery were collected. Factors that were found to have a significant impact on MI-BCI performance were Vividness of Visual Imagery, and the personality factors of orderliness and autonomy. These findings shed light on individual traits that lead to difficulty in BCI operation and hence can help with early prediction of inefficiency among users to optimize training for them.
Collapse
Affiliation(s)
- Nikki Leeuwis
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, Netherlands
| | | | | |
Collapse
|
17
|
Bennett JD, John SE, Grayden DB, Burkitt AN. A neurophysiological approach to spatial filter selection for adaptive brain–computer interfaces. J Neural Eng 2021; 18. [DOI: 10.1088/1741-2552/abd51f] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2020] [Accepted: 12/18/2020] [Indexed: 11/12/2022]
Abstract
Abstract
Objective. The common spatial patterns (CSP) algorithm is an effective method to extract discriminatory features from electroencephalography (EEG) to be used by a brain–computer interface (BCI). However, informed selection of CSP filters typically requires oversight from a BCI expert to accept or reject filters based on the neurophysiological plausibility of their activation patterns. Our goal was to identify, analyze and automatically classify prototypical CSP patterns to enhance the prediction of motor imagery states in a BCI. Approach. A data-driven approach that used four publicly available EEG datasets was adopted. Cluster analysis revealed recurring, visually similar CSP patterns and a convolutional neural network was developed to distinguish between established CSP pattern classes. Furthermore, adaptive spatial filtering schemes that utilize the categorization of CSP patterns were proposed and evaluated. Main results. Classes of common neurophysiologically probable and improbable CSP patterns were established. Analysis of the relationship between these categories of CSP patterns and classification performance revealed discarding neurophysiologically improbable filters can decrease decoder performance. Further analysis revealed that the spatial orientation of EEG modulations can evolve over time, and that the features extracted from the original CSP filters can become inseparable. Importantly, it was shown through a novel adaptive CSP technique that adaptation in response to these emerging patterns can restore feature separability. Significance. These findings highlight the importance of considering and reporting on spatial filter activation patterns in both online and offline studies. They also emphasize to researchers in the field the importance of spatial filter adaptation in BCI decoder design, particularly for online studies with a focus on training users to develop stable and suitable brain patterns.
Collapse
|
18
|
Alchalabi B, Faubert J, Labbé D. A multi-modal modified feedback self-paced BCI to control the gait of an avatar. J Neural Eng 2021; 18. [PMID: 33711832 DOI: 10.1088/1741-2552/abee51] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Accepted: 03/12/2021] [Indexed: 11/12/2022]
Abstract
Brain-computer interfaces (BCI) have been used to control the gait of a virtual self-avatar with a proposed application in the field of gait rehabilitation. OBJECTIVE to develop a high performance multi-modal BCI to control single steps and forward walking of an immersive virtual reality avatar. This system will overcome the limitation of existing systems. APPROACH This system used MI of these actions, in cue-paced and self-paced modes. Twenty healthy participants participated in this 4 sessions study across 4 different days. They were cued to imagine a single step forward with their right or left foot, or to imagine walking forward. They were instructed to reach a target by using the MI of multiple steps (self-paced switch-control mode) or by maintaining MI of forward walking (continuous-control mode). The movement of the avatar was controlled by two calibrated RLDA classifiers that used the µ power spectral density (PSD) over the foot area of the motor cortex as a feature. The classifiers were retrained after every session. For a subset of the trials, positive modified feedback was presented to half of the participants. MAIN RESULTS All participants were able to operate the BCI. Their average offline performance, after retraining the classifiers was 86.0 ± 6.1%, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9 ± 8.4% showing that modified feedback enhanced BCI performance (p =0.001). The average performance was 83% at self-paced switch control and 92% at continuous control mode. SIGNIFICANCE This study reports on the first novel integration of different design approaches, different control modes and different performance enhancement techniques, all in parallel in one single high performance and multi-modal BCI system, to control single steps and forward walking of an immersive virtual reality avatar.
Collapse
Affiliation(s)
- Bilal Alchalabi
- biomedical engineering, University of Montreal, 2900 Boulevard Edouard mon Petit, Montreal, Quebec, H3C 3J7, CANADA
| | - Jocelyn Faubert
- Université de Montréal, 3744 Rue Jean Brillant, Montreal, Quebec, H3T 1P1, CANADA
| | - David Labbé
- École de technologie supérieure, 1100 Rue Notre-Dame ouest, Montreal, Quebec, H3C 1K3, CANADA
| |
Collapse
|
19
|
Zhang JJ, Fong KNK. The Effects of Priming Intermittent Theta Burst Stimulation on Movement-Related and Mirror Visual Feedback-Induced Sensorimotor Desynchronization. Front Hum Neurosci 2021; 15:626887. [PMID: 33584232 PMCID: PMC7878678 DOI: 10.3389/fnhum.2021.626887] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2020] [Accepted: 01/06/2021] [Indexed: 11/24/2022] Open
Abstract
The potential benefits of priming intermittent theta burst stimulation (iTBS) with continuous theta burst stimulation (cTBS) have not been examined in regard to sensorimotor oscillatory activities recorded in electroencephalography (EEG). The objective of this study was to investigate the modulatory effect of priming iTBS (cTBS followed by iTBS) delivered to the motor cortex on movement-related and mirror visual feedback (MVF)-induced sensorimotor event-related desynchronization (ERD), compared with iTBS alone, on healthy adults. Twenty participants were randomly allocated into Group 1: priming iTBS—cTBS followed by iTBS, and Group 2: non-priming iTBS—sham cTBS followed by iTBS. The stimulation was delivered to the right primary motor cortex daily for 4 consecutive days. EEG was measured before and after 4 sessions of stimulation. Movement-related ERD was evaluated during left-index finger tapping and MVF-induced sensorimotor ERD was evaluated by comparing the difference between right-index finger tapping with and without MVF. After stimulation, both protocols increased movement-related ERD and MVF-induced sensorimotor ERD in high mu and low beta bands, indicated by significant time effects. A significant interaction effect favoring Group 1 in enhancing movement-related ERD was observed in the high mu band [F(1,18) = 4.47, p = 0.049], compared with Group 2. Our experiment suggests that among healthy adults priming iTBS with cTBS delivered to the motor cortex yields similar effects with iTBS alone on enhancing ERD induced by MVF-based observation, while movement-related ERD was more enhanced in the priming iTBS condition, specifically in the high mu band.
Collapse
Affiliation(s)
- Jack Jiaqi Zhang
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong
| | - Kenneth N K Fong
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong
| |
Collapse
|
20
|
Improving performance in motor imagery BCI-based control applications via virtually embodied feedback. Comput Biol Med 2020; 127:104079. [PMID: 33126130 DOI: 10.1016/j.compbiomed.2020.104079] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Revised: 09/30/2020] [Accepted: 10/20/2020] [Indexed: 12/16/2022]
Abstract
OBJECTIVE Brain-computer interfaces (BCIs) based on motor imagery (MI) are commonly used for control applications. However, these applications require strong and discriminant neural patterns for which extensive experience in MI may be necessary. Inspired by the field of rehabilitation where embodiment is a key element for improving cortical activity, our study proposes a novel control scheme in which virtually embodiable feedback is provided during control to enhance performance. METHODS Subjects underwent two immersive virtual reality control scenarios in which they controlled the two-dimensional movement of a device using electroencephalography (EEG). The two scenarios only differ on whether embodiable feedback, which mirrors the movement of the classified intention, is provided. After undergoing each scenario, subjects also answered a questionnaire in which they rated how immersive the scenario and embodiable the feedback were. RESULTS Subjects exhibited higher control performance, greater discriminability in brain activity patterns, and enhanced cortical activation when using our control scheme compared to the standard control scheme in which embodiable feedback is absent. Moreover, the self-rated embodiment and presence scores showed significantly positive linear relationships with performance. SIGNIFICANCE The findings in our study provide evidence that providing embodiable feedback as guidance on how intention is classified may be effective for control applications by inducing enhanced neural activity and patterns with greater discriminability. By applying embodiable feedback to immersive virtual reality, our study also serves as another instance in which virtual reality is shown to be a promising tool for improving MI.
Collapse
|
21
|
Nostadt N, Abbink DA, Christ O, Beckerle P. Embodiment, Presence, and Their Intersections. ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION 2020. [DOI: 10.1145/3389210] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
Subjective experience of human control over remote, artificial, or virtual limbs has traditionally been investigated from two separate angles: presence research originates from teleoperation, aiming to capture to what extent the user feels like actually being in the remote or virtual environment. Embodiment captures to what extent a virtual or artificial limb is perceived as one’s own limb. Unfortunately, the two research fields have not interacted much. This survey intends to provide a coherent overview of the literature at the intersection of these two fields to further that interaction. Two rounds of systematic research in topic-related data bases resulted in 414 related articles, 14 of which satisfy the deliberately strict inclusion criteria: 2 theoretical frameworks that highlighted intersections and 12 experimental studies that evaluated subjective measures for both concepts. Considering the surrounding literature as well, theoretical and experimental potential of embodiment and presence are discussed and suggestions to apply them in teleoperation research are derived. While increased publication activity is observed between 2016 and 2018, potentially caused by affordable virtual reality technologies, various open questions remain. To tackle them, human-in-the-loop experiments and three guiding principles for teleoperation system design (mechanical fidelity, spatial bodily awareness, and self-identification) are suggested.
Collapse
Affiliation(s)
| | - David A. Abbink
- Delft Haptics Lab, Department of Cognitive Robotics, Faculty 3mE, Delft University of Technology, The Netherlands
| | - Oliver Christ
- Institute Humans in Complex Systems, School of Applied Psychology, University of Applied Sciences and Arts Northwestern Switzerland, Switzerland
| | - Philipp Beckerle
- Elastic Lightweight Robotics Group, Robotics Research Institute, Technische Universität Dortmund, German and Institute for Mechatronic Systems in Mechanical Engineering, Technische Universität Darmstadt, Darmstadt, Germany
| |
Collapse
|
22
|
de Castro-Cros M, Sebastian-Romagosa M, Rodríguez-Serrano J, Opisso E, Ochoa M, Ortner R, Guger C, Tost D. Effects of Gamification in BCI Functional Rehabilitation. Front Neurosci 2020; 14:882. [PMID: 32973435 PMCID: PMC7472985 DOI: 10.3389/fnins.2020.00882] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2020] [Accepted: 07/28/2020] [Indexed: 12/25/2022] Open
Abstract
Objective To evaluate whether introducing gamification in BCI rehabilitation of the upper limbs of post-stroke patients has a positive impact on their experience without altering their efficacy in creating motor mental images (MI). Design A game was designed purposely adapted to the pace and goals of an established BCI-rehabilitation protocol. Rehabilitation was based on a double feedback: functional electrostimulation and animation of a virtual avatar of the patient’s limbs. The game introduced a narrative on top of this visual feedback with an external goal to achieve (protecting bits of cheese from a rat character). A pilot study was performed with 10 patients and a control group of six volunteers. Two rehabilitation sessions were done, each made up of one stage of calibration and two training stages, some stages with the game and others without. The accuracy of the classification computed was taken as a measure to compare the efficacy of MI. Users’ opinions were gathered through a questionnaire. No potentially identifiable human images or data are presented in this study. Results The gamified rehabilitation presented in the pilot study does not impact on the efficacy of MI, but it improves users experience making it more fun. Conclusion These preliminary results are encouraging to continue investigating how game narratives can be introduced in BCI rehabilitation to make it more gratifying and engaging.
Collapse
Affiliation(s)
| | | | | | | | | | - Rupert Ortner
- g.tec medical engineering Spain S.L., Barcelona, Spain
| | - Christoph Guger
- g.tec medical engineering Spain S.L., Barcelona, Spain.,g.tec medical engineering GmbH, Schiedlberg, Austria.,Guger Technologies (Austria), Graz, Austria
| | - Dani Tost
- Universitat Politecnica de Catalunya, Barcelona, Spain.,Research Center in Biomedical Engineering (CREB), Barcelona, Spain.,Sant Joan de Déu Research Institute, Esplugues de Llobregat, Spain
| |
Collapse
|
23
|
Choi JW, Kim BH, Huh S, Jo S. Observing Actions Through Immersive Virtual Reality Enhances Motor Imagery Training. IEEE Trans Neural Syst Rehabil Eng 2020; 28:1614-1622. [DOI: 10.1109/tnsre.2020.2998123] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
24
|
Brain-Computer Interface-Based Humanoid Control: A Review. SENSORS 2020; 20:s20133620. [PMID: 32605077 PMCID: PMC7374399 DOI: 10.3390/s20133620] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/25/2020] [Revised: 06/12/2020] [Accepted: 06/17/2020] [Indexed: 11/17/2022]
Abstract
A Brain-Computer Interface (BCI) acts as a communication mechanism using brain signals to control external devices. The generation of such signals is sometimes independent of the nervous system, such as in Passive BCI. This is majorly beneficial for those who have severe motor disabilities. Traditional BCI systems have been dependent only on brain signals recorded using Electroencephalography (EEG) and have used a rule-based translation algorithm to generate control commands. However, the recent use of multi-sensor data fusion and machine learning-based translation algorithms has improved the accuracy of such systems. This paper discusses various BCI applications such as tele-presence, grasping of objects, navigation, etc. that use multi-sensor fusion and machine learning to control a humanoid robot to perform a desired task. The paper also includes a review of the methods and system design used in the discussed applications.
Collapse
|
25
|
Juliano JM, Spicer RP, Vourvopoulos A, Lefebvre S, Jann K, Ard T, Santarnecchi E, Krum DM, Liew SL. Embodiment Is Related to Better Performance on a Brain-Computer Interface in Immersive Virtual Reality: A Pilot Study. SENSORS (BASEL, SWITZERLAND) 2020; 20:E1204. [PMID: 32098317 PMCID: PMC7070491 DOI: 10.3390/s20041204] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/21/2019] [Revised: 02/19/2020] [Accepted: 02/19/2020] [Indexed: 01/25/2023]
Abstract
Electroencephalography (EEG)-based brain-computer interfaces (BCIs) for motor rehabilitation aim to "close the loop" between attempted motor commands and sensory feedback by providing supplemental information when individuals successfully achieve specific brain patterns. Existing EEG-based BCIs use various displays to provide feedback, ranging from displays considered more immersive (e.g., head-mounted display virtual reality (HMD-VR)) to displays considered less immersive (e.g., computer screens). However, it is not clear whether more immersive displays improve neurofeedback performance and whether there are individual performance differences in HMD-VR versus screen-based neurofeedback. In this pilot study, we compared neurofeedback performance in HMD-VR versus a computer screen in 12 healthy individuals and examined whether individual differences on two measures (i.e., presence, embodiment) were related to neurofeedback performance in either environment. We found that, while participants' performance on the BCI was similar between display conditions, the participants' reported levels of embodiment were significantly different. Specifically, participants experienced higher levels of embodiment in HMD-VR compared to a computer screen. We further found that reported levels of embodiment positively correlated with neurofeedback performance only in HMD-VR. Overall, these preliminary results suggest that embodiment may relate to better performance on EEG-based BCIs and that HMD-VR may increase embodiment compared to computer screens.
Collapse
Affiliation(s)
- Julia M. Juliano
- Neural Plasticity and Neurorehabilitation Laboratory, Neuroscience Graduate Program, University of Southern California, Los Angeles, CA 90089, USA;
| | - Ryan P. Spicer
- Institute for Creative Technologies, University of Southern California, Playa Vista, CA 90094, USA; (R.P.S.); (D.M.K.)
| | - Athanasios Vourvopoulos
- Neural Plasticity and Neurorehabilitation Laboratory, Division of Occupational Science and Occupational Therapy, University of Southern California, Los Angeles, CA 90089, USA; (A.V.); (S.L.)
| | - Stephanie Lefebvre
- Neural Plasticity and Neurorehabilitation Laboratory, Division of Occupational Science and Occupational Therapy, University of Southern California, Los Angeles, CA 90089, USA; (A.V.); (S.L.)
| | - Kay Jann
- USC Stevens Neuroimaging and Informatics Institute, Department of Neurology, University of Southern California, Los Angeles, CA 90033, USA; (K.J.); (T.A.)
| | - Tyler Ard
- USC Stevens Neuroimaging and Informatics Institute, Department of Neurology, University of Southern California, Los Angeles, CA 90033, USA; (K.J.); (T.A.)
| | - Emiliano Santarnecchi
- Berenson-Allen Center for Non-Invasive Brain Stimulation and Division of Cognitive Neurology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA 02215, USA;
| | - David M. Krum
- Institute for Creative Technologies, University of Southern California, Playa Vista, CA 90094, USA; (R.P.S.); (D.M.K.)
| | - Sook-Lei Liew
- Neural Plasticity and Neurorehabilitation Laboratory, Division of Occupational Science and Occupational Therapy, University of Southern California, Los Angeles, CA 90089, USA; (A.V.); (S.L.)
- USC Stevens Neuroimaging and Informatics Institute, Department of Neurology, University of Southern California, Los Angeles, CA 90033, USA; (K.J.); (T.A.)
| |
Collapse
|
26
|
Leeb R, Pérez-Marcos D. Brain-computer interfaces and virtual reality for neurorehabilitation. HANDBOOK OF CLINICAL NEUROLOGY 2020; 168:183-197. [PMID: 32164852 DOI: 10.1016/b978-0-444-63934-9.00014-7] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
Brain-computer interfaces (BCIs) and virtual reality (VR) are two technologic advances that are changing our way of interacting with the world. BCIs can be used to influence and can serve as a control mechanism in navigation tasks, communication, or other assistive functions. VR can create ad hoc interactive scenarios that involve all our senses, stimulate the brain in a multisensory fashion, and increase the motivation and fun with game-like environments. VR and motion tracking enable natural human-computer interaction at cognitive and physical levels. This includes both brain and body in the design of meaningful VR experiences; these cases in which participants feel naturally present could help augment the benefits of BCIs for assistive and neurorehabilitation applications for the relearning of motor and cognitive skills. VR technology is now available at the consumer level thanks to the proliferation of affordable head-mounted displays (HMDs). Merging both technologies into simplified, practical devices may help democratize these technologies.
Collapse
|
27
|
Škola F, Tinková S, Liarokapis F. Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment. Front Hum Neurosci 2019; 13:329. [PMID: 31616269 PMCID: PMC6775193 DOI: 10.3389/fnhum.2019.00329] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2019] [Accepted: 09/06/2019] [Indexed: 12/28/2022] Open
Abstract
This paper presents a gamified motor imagery brain-computer interface (MI-BCI) training in immersive virtual reality. The aim of the proposed training method is to increase engagement, attention, and motivation in co-adaptive event-driven MI-BCI training. This was achieved using gamification, progressive increase of the training pace, and virtual reality design reinforcing body ownership transfer (embodiment) into the avatar. From the 20 healthy participants performing 6 runs of 2-class MI-BCI training (left/right hand), 19 were trained for a basic level of MI-BCI operation, with average peak accuracy in the session = 75.84%. This confirms the proposed training method succeeded in improvement of the MI-BCI skills; moreover, participants were leaving the session in high positive affect. Although the performance was not directly correlated to the degree of embodiment, subjective magnitude of the body ownership transfer illusion correlated with the ability to modulate the sensorimotor rhythm.
Collapse
Affiliation(s)
- Filip Škola
- Faculty of Informatics, Masaryk University, Brno, Czechia
| | | | | |
Collapse
|
28
|
Hülsmann F, Frank C, Senna I, Ernst MO, Schack T, Botsch M. Superimposed Skilled Performance in a Virtual Mirror Improves Motor Performance and Cognitive Representation of a Full Body Motor Action. Front Robot AI 2019; 6:43. [PMID: 33501059 PMCID: PMC7805859 DOI: 10.3389/frobt.2019.00043] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2018] [Accepted: 05/28/2019] [Indexed: 11/16/2022] Open
Abstract
Feedback is essential for skill acquisition as it helps identifying and correcting performance errors. Nowadays, Virtual Reality can be used as a tool to guide motor learning, and to provide innovative types of augmented feedback that exceed real world opportunities. Concurrent feedback has shown to be especially beneficial for novices. Moreover, watching skilled performances helps novices to acquire a motor skill, and this effect depends on the perspective taken by the observer. To date, however, the impact of watching one's own performance together with full body superimposition of a skilled performance, either from the front or from the side, remains to be explored. Here we used an immersive, state-of-the-art, low-latency cave automatic virtual environment (CAVE), and we asked novices to perform squat movements in front of a virtual mirror. Participants were assigned to one of three concurrent visual feedback groups: participants either watched their own avatar performing full body movements or were presented with the movement of a skilled individual superimposed on their own performance during movement execution, either from a frontal or from a side view. Motor performance and cognitive representation were measured in order to track changes in movement quality as well as motor memory across time. Consistent with our hypotheses, results showed an advantage of the groups that observed their own avatar performing the squat together with the superimposed skilled performance for some of the investigated parameters, depending on perspective. Specifically, for the deepest point of the squat, participants watching the squat from the front adapted their height, while those watching from the side adapted their backward movement. In a control experiment, we ruled out the possibility that the observed improvements were due to the mere fact of performing the squat movements—irrespective of the type of visual feedback. The present findings indicate that it can be beneficial for novices to watch themselves together with a skilled performance during execution, and that improvement depends on the perspective chosen.
Collapse
Affiliation(s)
- Felix Hülsmann
- Computer Graphics and Geometry Processing, Bielefeld University, Bielefeld, Germany.,Social Cognitive Systems, Bielefeld University, Bielefeld, Germany
| | - Cornelia Frank
- Neurocognition and Action, Bielefeld University, Bielefeld, Germany
| | - Irene Senna
- Applied Cognitive Psychology, Ulm University, Ulm, Germany
| | - Marc O Ernst
- Applied Cognitive Psychology, Ulm University, Ulm, Germany
| | - Thomas Schack
- Neurocognition and Action, Bielefeld University, Bielefeld, Germany
| | - Mario Botsch
- Computer Graphics and Geometry Processing, Bielefeld University, Bielefeld, Germany
| |
Collapse
|
29
|
Salazar-Ramirez A, Martin JI, Martinez R, Arruti A, Muguerza J, Sierra B. A hierarchical architecture for recognising intentionality in mental tasks on a brain-computer interface. PLoS One 2019; 14:e0218181. [PMID: 31211812 PMCID: PMC6581259 DOI: 10.1371/journal.pone.0218181] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2019] [Accepted: 05/28/2019] [Indexed: 11/19/2022] Open
Abstract
A brain-computer interface (BCI), based on motor imagery EEG, uses information extracted from the electroencephalography signals generated by a person who intends to perform any action. One of the most important issues of current research is how to detect automatically whether the user intends to send some message to a certain device. This study presents a proposal, based on a hierarchical structured system, for recognising intentional and non-intentional mental tasks on a BCI system by applying machine learning techniques to the EEG signals. First-level clustering is performed to distinguish between intentional control (IC) and non-intentional control (NC) state patterns. Then, the patterns recognised as IC are passed on to a second stage where supervised learning techniques are used to classify them. In BCI applications, it is critical to correctly classify NC states with a low false positive rate (FPR) to avoid undesirable effects. According to the literature, we selected a maximum FPR of 10%. Under these conditions, our proposal achieved an average test accuracy of 66.6%, with an 8.2% FPR, for the BCI competition IIIa dataset. The main contribution of this paper is the hierarchical approach, based on machine learning paradigms, which performs intentional and non-intentional discrimination and, depending on the case, classifies the intended command selected by the user.
Collapse
Affiliation(s)
- Asier Salazar-Ramirez
- Department of Computer Architecture and Technology, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
| | - Jose I. Martin
- Department of Computer Architecture and Technology, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
- * E-mail:
| | - Raquel Martinez
- Department of System Engineering and Automation, University of the Basque Country (UPV/EHU), Bilbao, Spain
| | - Andoni Arruti
- Department of Computer Architecture and Technology, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
| | - Javier Muguerza
- Department of Computer Architecture and Technology, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
| | - Basilio Sierra
- Department of Computer Science and Artificial Intelligence, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
| |
Collapse
|
30
|
Lukoyanov MV, Gordleeva SY, Pimashkin AS, Grigor’ev NA, Savosenkov AV, Motailo A, Kazantsev VB, Kaplan AY. The Efficiency of the Brain-Computer Interfaces Based on Motor Imagery with Tactile and Visual Feedback. ACTA ACUST UNITED AC 2018. [DOI: 10.1134/s0362119718030088] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
31
|
Cervera MA, Soekadar SR, Ushiba J, Millán JDR, Liu M, Birbaumer N, Garipelli G. Brain-computer interfaces for post-stroke motor rehabilitation: a meta-analysis. Ann Clin Transl Neurol 2018; 5:651-663. [PMID: 29761128 PMCID: PMC5945970 DOI: 10.1002/acn3.544] [Citation(s) in RCA: 210] [Impact Index Per Article: 35.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2017] [Accepted: 01/28/2018] [Indexed: 11/10/2022] Open
Abstract
Brain‐computer interfaces (BCIs) can provide sensory feedback of ongoing brain oscillations, enabling stroke survivors to modulate their sensorimotor rhythms purposefully. A number of recent clinical studies indicate that repeated use of such BCIs might trigger neurological recovery and hence improvement in motor function. Here, we provide a first meta‐analysis evaluating the clinical effectiveness of BCI‐based post‐stroke motor rehabilitation. Trials were identified using MEDLINE, CENTRAL, PEDro and by inspection of references in several review articles. We selected randomized controlled trials that used BCIs for post‐stroke motor rehabilitation and provided motor impairment scores before and after the intervention. A random‐effects inverse variance method was used to calculate the summary effect size. We initially identified 524 articles and, after removing duplicates, we screened titles and abstracts of 473 articles. We found 26 articles corresponding to BCI clinical trials, of these, there were nine studies that involved a total of 235 post‐stroke survivors that fulfilled the inclusion criterion (randomized controlled trials that examined motor performance as an outcome measure) for the meta‐analysis. Motor improvements, mostly quantified by the upper limb Fugl‐Meyer Assessment (FMA‐UE), exceeded the minimal clinically important difference (MCID=5.25) in six BCI studies, while such improvement was reached only in three control groups. Overall, the BCI training was associated with a standardized mean difference of 0.79 (95% CI: 0.37 to 1.20) in FMA‐UE compared to control conditions, which is in the range of medium to large summary effect size. In addition, several studies indicated BCI‐induced functional and structural neuroplasticity at a subclinical level. This suggests that BCI technology could be an effective intervention for post‐stroke upper limb rehabilitation. However, more studies with larger sample size are required to increase the reliability of these results.
Collapse
Affiliation(s)
- María A Cervera
- Life Sciences and Technology École polytechnique fédérale de Lausanne (EPFL) Lausanne Switzerland
| | - Surjo R Soekadar
- Applied Neurotechnology Laboratory Department of Psychiatry and Psychotherapy University Hospital of Tübingen Tübingen Germany
| | - Junichi Ushiba
- Department of Biosciences and Informatics Faculty of Science and Technology Keio University Yokohama Japan
| | - José Del R Millán
- Defitech Chair in Brain-Machine Interface Center for Neuroprosthetics École polytechnique fédérale de Lausanne (EPFL) Lausanne Switzerland
| | - Meigen Liu
- Department of Rehabilitation Medicine Keio University School of Medicine Tokyo Japan
| | - Niels Birbaumer
- Institute for Medical Psychology and Behavioural Neurobiology University Tübingen Tübingen Germany.,WYSS Center for Bio and Neuroengineering Geneva Switzerland
| | | |
Collapse
|
32
|
Virtual and Actual Humanoid Robot Control with Four-Class Motor-Imagery-Based Optical Brain-Computer Interface. BIOMED RESEARCH INTERNATIONAL 2017; 2017:1463512. [PMID: 28804712 PMCID: PMC5539938 DOI: 10.1155/2017/1463512] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/01/2017] [Accepted: 06/06/2017] [Indexed: 12/11/2022]
Abstract
Motor-imagery tasks are a popular input method for controlling brain-computer interfaces (BCIs), partially due to their similarities to naturally produced motor signals. The use of functional near-infrared spectroscopy (fNIRS) in BCIs is still emerging and has shown potential as a supplement or replacement for electroencephalography. However, studies often use only two or three motor-imagery tasks, limiting the number of available commands. In this work, we present the results of the first four-class motor-imagery-based online fNIRS-BCI for robot control. Thirteen participants utilized upper- and lower-limb motor-imagery tasks (left hand, right hand, left foot, and right foot) that were mapped to four high-level commands (turn left, turn right, move forward, and move backward) to control the navigation of a simulated or real robot. A significant improvement in classification accuracy was found between the virtual-robot-based BCI (control of a virtual robot) and the physical-robot BCI (control of the DARwIn-OP humanoid robot). Differences were also found in the oxygenated hemoglobin activation patterns of the four tasks between the first and second BCI. These results corroborate previous findings that motor imagery can be improved with feedback and imply that a four-class motor-imagery-based fNIRS-BCI could be feasible with sufficient subject training.
Collapse
|
33
|
Sorbello R, Tramonte S, Giardina ME, La Bella V, Spataro R, Allison B, Guger C, Chella A. A Human-Humanoid Interaction Through the Use of BCI for Locked-In ALS Patients Using Neuro-Biological Feedback Fusion. IEEE Trans Neural Syst Rehabil Eng 2017; 26:487-497. [PMID: 28727554 DOI: 10.1109/tnsre.2017.2728140] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
This paper illustrates a new architecture for a human-humanoid interaction based on EEG-brain computer interface (EEG-BCI) for patients affected by locked-in syndrome caused by Amyotrophic Lateral Sclerosis (ALS). The proposed architecture is able to recognise users' mental state accordingly to the biofeedback factor , based on users' attention, intention, and focus, that is used to elicit a robot to perform customised behaviours. Experiments have been conducted with a population of eight subjects: four ALS patients in a near locked-in status with normal ocular movement and four healthy control subjects enrolled for age, education, and computer expertise. The results showed as three ALS patients have completed the task with 96.67% success; the healthy controls with 100% success; the fourth ALS has been excluded from the results for his low general attention during the task; the analysis of factor highlights as ALS subjects have shown stronger (81.20%) than healthy controls (76.77%). Finally, a post-hoc analysis is provided to show how robotic feedback helps in maintaining focus on expected task. These preliminary data suggest that ALS patients could successfully control a humanoid robot through a BCI architecture, potentially enabling them to conduct some everyday tasks and extend their presence in the environment.
Collapse
|