1
|
Phang CR, Su KH, Cheng YY, Chen CH, Ko LW. Time synchronization between parietal-frontocentral connectivity with MRCP and gait in post-stroke bipedal tasks. J Neuroeng Rehabil 2024; 21:101. [PMID: 38872209 PMCID: PMC11170849 DOI: 10.1186/s12984-024-01330-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2023] [Accepted: 06/20/2023] [Indexed: 06/15/2024] Open
Abstract
BACKGROUND In post-stroke rehabilitation, functional connectivity (FC), motor-related cortical potential (MRCP), and gait activities are common measures related to recovery outcomes. However, the interrelationship between FC, MRCP, gait activities, and bipedal distinguishability have yet to be investigated. METHODS Ten participants were equipped with EEG devices and inertial measurement units (IMUs) while performing lower limb motor preparation (MP) and motor execution (ME) tasks. MRCP, FCs, and bipedal distinguishability were extracted from the EEG signals, while the change in knee degree during the ME phase was calculated from the gait data. FCs were analyzed with pairwise Pearson's correlation, and the brain-wide FC was fed into support vector machine (SVM) for bipedal classification. RESULTS Parietal-frontocentral connectivity (PFCC) dysconnection and MRCP desynchronization were related to the MP and ME phases, respectively. Hemiplegic limb movement exhibited higher PFCC strength than nonhemiplegic limb movement. Bipedal classification had a short-lived peak of 75.1% in the pre-movement phase. These results contribute to a better understanding of the neurophysiological functions during motor tasks, with respect to localized MRCP and nonlocalized FC activities. The difference in PFCCs between both limbs could be a marker to understand the motor function of the brain of post-stroke patients. CONCLUSIONS In this study, we discovered that PFCCs are temporally dependent on lower limb gait movement and MRCP. The PFCCs are also related to the lower limb motor performance of post-stroke patients. The detection of motor intentions allows the development of bipedal brain-controlled exoskeletons for lower limb active rehabilitation.
Collapse
Affiliation(s)
- Chun-Ren Phang
- International Ph.D. Program in Interdisciplinary Neuroscience (UST), College of Biological Science and Technology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
- Center for Intelligent Drug Systems and Smart Bio-devices (IDS2B), College of Biological Science and Technology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Kai-Hsiang Su
- Center for Intelligent Drug Systems and Smart Bio-devices (IDS2B), College of Biological Science and Technology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
- Department of Biological Science and Technology, College of Biological Science and Technology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Yuan-Yang Cheng
- Department of Physical Medicine and Rehabilitation, Taichung Veterans General Hospital, Taichung, Taiwan
- Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taichung, Taiwan
- School of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan
| | - Chia-Hsin Chen
- Department of Physical Medicine and Rehabilitation, Kaohsiung Medical University Hospital, Kaohsiung, Taiwan
- School of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
- Regenerative Medicine and Cell Therapy Research Center, Kaohsiung Medical University, Kaohsiung, Taiwan
| | - Li-Wei Ko
- International Ph.D. Program in Interdisciplinary Neuroscience (UST), College of Biological Science and Technology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan.
- Center for Intelligent Drug Systems and Smart Bio-devices (IDS2B), College of Biological Science and Technology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan.
- Department of Biological Science and Technology, College of Biological Science and Technology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan.
- Institute of Electrical and Control Engineering, National Yang Ming Chiao Tung University, Hsinchu, Taiwan.
- Department of Biomedical Science and Environment Biology, Kaohsiung Medical University, Kaohsiung, Taiwan.
- Drug Development and Value Creation Research Center, Kaohsiung Medical University, Kaohsiung, Taiwan.
| |
Collapse
|
2
|
Pérez-Velasco S, Marcos-Martínez D, Santamaría-Vázquez E, Martínez-Cagigal V, Moreno-Calderón S, Hornero R. Unraveling motor imagery brain patterns using explainable artificial intelligence based on Shapley values. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 246:108048. [PMID: 38308997 DOI: 10.1016/j.cmpb.2024.108048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 01/22/2024] [Accepted: 01/23/2024] [Indexed: 02/05/2024]
Abstract
BACKGROUND AND OBJECTIVE Motor imagery (MI) based brain-computer interfaces (BCIs) are widely used in rehabilitation due to the close relationship that exists between MI and motor execution (ME). However, the underlying brain mechanisms of MI remain not well understood. Most MI-BCIs use the sensorimotor rhythms elicited in the primary motor cortex (M1) and somatosensory cortex (S1), which consist of an event-related desynchronization followed by an event-related synchronization. Consequently, this has resulted in systems that only record signals around M1 and S1. However, MI could involve a more complex network including sensory, association, and motor areas. In this study, we hypothesize that the superior accuracies achieved by new deep learning (DL) models applied to MI decoding rely on focusing on a broader MI activation of the brain. Parallel to the success of DL, the field of explainable artificial intelligence (XAI) has seen continuous development to provide explanations for DL networks success. The goal of this study is to use XAI in combination with DL to extract information about MI brain activation patterns from non-invasive electroencephalography (EEG) signals. METHODS We applied an adaptation of Shapley additive explanations (SHAP) to EEGSym, a state-of-the-art DL network with exceptional transfer learning capabilities for inter-subject MI classification. We obtained the SHAP values from two public databases comprising 171 users generating left and right hand MI instances with and without real-time feedback. RESULTS We found that EEGSym based most of its prediction on the signal of the frontal electrodes, i.e. F7 and F8, and on the first 1500 ms of the analyzed imagination period. We also found that MI involves a broad network not only based on M1 and S1, but also on the prefrontal cortex (PFC) and the posterior parietal cortex (PPC). We further applied this knowledge to select a 8-electrode configuration that reached inter-subject accuracies of 86.5% ± 10.6% on the Physionet dataset and 88.7% ± 7.0% on the Carnegie Mellon University's dataset. CONCLUSION Our results demonstrate the potential of combining DL and SHAP-based XAI to unravel the brain network involved in producing MI. Furthermore, SHAP values can optimize the requirements for out-of-laboratory BCI applications involving real users.
Collapse
Affiliation(s)
- Sergio Pérez-Velasco
- Biomedical Engineering Group, E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Spain.
| | - Diego Marcos-Martínez
- Biomedical Engineering Group, E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Spain
| | - Eduardo Santamaría-Vázquez
- Biomedical Engineering Group, E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Spain
| | - Víctor Martínez-Cagigal
- Biomedical Engineering Group, E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Spain
| | - Selene Moreno-Calderón
- Biomedical Engineering Group, E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain
| | - Roberto Hornero
- Biomedical Engineering Group, E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Spain
| |
Collapse
|
3
|
Mora MC, García-Ortiz JV, Cerdá-Boluda J. sEMG-Based Robust Recognition of Grasping Postures with a Machine Learning Approach for Low-Cost Hand Control. SENSORS (BASEL, SWITZERLAND) 2024; 24:2063. [PMID: 38610275 PMCID: PMC11013908 DOI: 10.3390/s24072063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/13/2024] [Revised: 03/10/2024] [Accepted: 03/20/2024] [Indexed: 04/14/2024]
Abstract
The design and control of artificial hands remains a challenge in engineering. Popular prostheses are bio-mechanically simple with restricted manipulation capabilities, as advanced devices are pricy or abandoned due to their difficult communication with the hand. For social robots, the interpretation of human intention is key for their integration in daily life. This can be achieved with machine learning (ML) algorithms, which are barely used for grasping posture recognition. This work proposes an ML approach to recognize nine hand postures, representing 90% of the activities of daily living in real time using an sEMG human-robot interface (HRI). Data from 20 subjects wearing a Myo armband (8 sEMG signals) were gathered from the NinaPro DS5 and from experimental tests with the YCB Object Set, and they were used jointly in the development of a simple multi-layer perceptron in MATLAB, with a global percentage success of 73% using only two features. GPU-based implementations were run to select the best architecture, with generalization capabilities, robustness-versus-electrode shift, low memory expense, and real-time performance. This architecture enables the implementation of grasping posture recognition in low-cost devices, aimed at the development of affordable functional prostheses and HRI for social robots.
Collapse
Affiliation(s)
- Marta C. Mora
- Department of Mechanical Engineering and Construction, Universitat Jaume I, Avda de Vicent Sos Baynat s/n, 12071 Castelló de la Plana, Spain;
| | - José V. García-Ortiz
- Department of Mechanical Engineering and Construction, Universitat Jaume I, Avda de Vicent Sos Baynat s/n, 12071 Castelló de la Plana, Spain;
| | - Joaquín Cerdá-Boluda
- Instituto de Instrumentación para Imagen Molecular (I3M), Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain;
| |
Collapse
|
4
|
Zhang J, Liu D, Chen W, Pei Z, Wang J. Boosting lower-limb motor imagery performance through an ensemble method for gait rehabilitation. Comput Biol Med 2024; 169:107910. [PMID: 38183703 DOI: 10.1016/j.compbiomed.2023.107910] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2023] [Revised: 12/05/2023] [Accepted: 12/24/2023] [Indexed: 01/08/2024]
Abstract
Lower-limb exoskeletons have been used extensively in many rehabilitation applications to assist disabled people with their therapies. Brain-machine interfaces (BMIs) further provide effective and natural control schemes. However, the limited performance of brain signal decoding from lower-limb kinematics restricts the broad growth of both BMI and rehabilitation industry. To address these challenges, we propose an ensemble method for lower-limb motor imagery (MI) classification. The proposed model employs multiple techniques to boost performance, including deep and shallow parts. Traditional wavelet transformation followed by filter-bank common spatial pattern (CSP) employs neurophysiologically reasonable patterns, while multi-head self-attention (MSA) followed by temporal convolutional network (TCN) extracts deeper encoded generalized patterns. Experimental results in a customized lower-limb exoskeleton on 8 subjects in 3 consecutive sessions showed that the proposed method achieved 60.27% and 64.20% for three (MI of left leg, MI of right leg, and rest) and two classes (lower-limb MI vs. rest), respectively. Besides, the proposed model achieves improvements of up to 4% and 2% accuracy for the subject-specific and subject-independent modes compared to the current state-of-the-art (SOTA) techniques, respectively. Finally, feature analysis was conducted to show discriminative brain patterns in each MI task and sessions with different feedback modalities. The proposed models integrated in the brain-actuated lower-limb exoskeleton established a potential BMI for gait training and neuroprosthesis.
Collapse
Affiliation(s)
- Jing Zhang
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China; Hangzhou Innovation Institute, Beihang University, Hangzhou, Zhejiang 310052, China.
| | - Dong Liu
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China.
| | - Weihai Chen
- School of Electrical Engineering and Automation, Anhui University, Hefei, Anhui 230601, China.
| | - Zhongcai Pei
- Hangzhou Innovation Institute, Beihang University, Hangzhou, Zhejiang 310052, China.
| | - Jianhua Wang
- Hangzhou Innovation Institute, Beihang University, Hangzhou, Zhejiang 310052, China.
| |
Collapse
|
5
|
Kosmyna N, Hauptmann E, Hmaidan Y. A Brain-Controlled Quadruped Robot: A Proof-of-Concept Demonstration. SENSORS (BASEL, SWITZERLAND) 2023; 24:80. [PMID: 38202942 PMCID: PMC10780665 DOI: 10.3390/s24010080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Revised: 12/15/2023] [Accepted: 12/19/2023] [Indexed: 01/12/2024]
Abstract
Coupling brain-computer interfaces (BCIs) and robotic systems in the future can enable seamless personal assistant systems in everyday life, with the requests that can be performed in a discrete manner, using one's brain activity only. These types of systems might be of a particular interest for people with locked-in syndrome (LIS) or amyotrophic lateral sclerosis (ALS) because they can benefit from communicating with robotic assistants using brain sensing interfaces. In this proof-of-concept work, we explored how a wireless and wearable BCI device can control a quadruped robot-Boston Dynamics' Spot. The device measures the user's electroencephalography (EEG) and electrooculography (EOG) activity of the user from the electrodes embedded in the glasses' frame. The user responds to a series of questions with YES/NO answers by performing a brain-teaser activity of mental calculus. Each question-answer pair has a pre-configured set of actions for Spot. For instance, Spot was prompted to walk across a room, pick up an object, and retrieve it for the user (i.e., bring a bottle of water) when a sequence resolved to a YES response. Our system achieved at a success rate of 83.4%. To the best of our knowledge, this is the first integration of wireless, non-visual-based BCI systems with Spot in the context of personal assistant use cases. While this BCI quadruped robot system is an early prototype, future iterations may embody friendly and intuitive cues similar to regular service dogs. As such, this project aims to pave a path towards future developments in modern day personal assistant robots powered by wireless and wearable BCI systems in everyday living conditions.
Collapse
Affiliation(s)
- Nataliya Kosmyna
- Media Lab, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | | | - Yasmeen Hmaidan
- Psychology Department, University of Toronto, Toronto, ON M5S 3E4, Canada;
| |
Collapse
|
6
|
Vidaurre C, Irastorza-Landa N, Sarasola-Sanz A, Insausti-Delgado A, Ray AM, Bibián C, Helmhold F, Mahmoud WJ, Ortego-Isasa I, López-Larraz E, Lozano Peiteado H, Ramos-Murguialday A. Challenges of neural interfaces for stroke motor rehabilitation. Front Hum Neurosci 2023; 17:1070404. [PMID: 37789905 PMCID: PMC10543821 DOI: 10.3389/fnhum.2023.1070404] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 08/28/2023] [Indexed: 10/05/2023] Open
Abstract
More than 85% of stroke survivors suffer from different degrees of disability for the rest of their lives. They will require support that can vary from occasional to full time assistance. These conditions are also associated to an enormous economic impact for their families and health care systems. Current rehabilitation treatments have limited efficacy and their long-term effect is controversial. Here we review different challenges related to the design and development of neural interfaces for rehabilitative purposes. We analyze current bibliographic evidence of the effect of neuro-feedback in functional motor rehabilitation of stroke patients. We highlight the potential of these systems to reconnect brain and muscles. We also describe all aspects that should be taken into account to restore motor control. Our aim with this work is to help researchers designing interfaces that demonstrate and validate neuromodulation strategies to enforce a contingent and functional neural linkage between the central and the peripheral nervous system. We thus give clues to design systems that can improve or/and re-activate neuroplastic mechanisms and open a new recovery window for stroke patients.
Collapse
Affiliation(s)
- Carmen Vidaurre
- TECNALIA, Basque Research and Technology Alliance (BRTA), San Sebastian, Spain
- Ikerbasque Science Foundation, Bilbao, Spain
| | | | | | | | - Andreas M. Ray
- Institute for Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Carlos Bibián
- Institute for Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Florian Helmhold
- Institute for Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Wala J. Mahmoud
- Institute for Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Iñaki Ortego-Isasa
- TECNALIA, Basque Research and Technology Alliance (BRTA), San Sebastian, Spain
| | - Eduardo López-Larraz
- Institute for Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
- Bitbrain, Zaragoza, Spain
| | | | - Ander Ramos-Murguialday
- TECNALIA, Basque Research and Technology Alliance (BRTA), San Sebastian, Spain
- Institute for Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| |
Collapse
|
7
|
Batistić L, Sušanj D, Pinčić D, Ljubic S. Motor Imagery Classification Based on EEG Sensing with Visual and Vibrotactile Guidance. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23115064. [PMID: 37299791 DOI: 10.3390/s23115064] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Revised: 05/15/2023] [Accepted: 05/23/2023] [Indexed: 06/12/2023]
Abstract
Motor imagery (MI) is a technique of imagining the performance of a motor task without actually using the muscles. When employed in a brain-computer interface (BCI) supported by electroencephalographic (EEG) sensors, it can be used as a successful method of human-computer interaction. In this paper, the performance of six different classifiers, namely linear discriminant analysis (LDA), support vector machine (SVM), random forest (RF), and three classifiers from the family of convolutional neural networks (CNN), is evaluated using EEG MI datasets. The study investigates the effectiveness of these classifiers on MI, guided by a static visual cue, dynamic visual guidance, and a combination of dynamic visual and vibrotactile (somatosensory) guidance. The effect of filtering passband during data preprocessing was also investigated. The results show that the ResNet-based CNN significantly outperforms the competing classifiers on both vibrotactile and visually guided data when detecting different directions of MI. Preprocessing the data using low-frequency signal features proves to be a better solution to achieve higher classification accuracy. It has also been shown that vibrotactile guidance has a significant impact on classification accuracy, with the associated improvement particularly evident for architecturally simpler classifiers. These findings have important implications for the development of EEG-based BCIs, as they provide valuable insight into the suitability of different classifiers for different contexts of use.
Collapse
Affiliation(s)
- Luka Batistić
- University of Rijeka, Faculty of Engineering, Vukovarska 58, HR-51000 Rijeka, Croatia
- Center for Artificial Intelligence and Cybersecurity, University of Rijeka, R. Matejcic 2, HR-51000 Rijeka, Croatia
| | - Diego Sušanj
- University of Rijeka, Faculty of Engineering, Vukovarska 58, HR-51000 Rijeka, Croatia
| | - Domagoj Pinčić
- University of Rijeka, Faculty of Engineering, Vukovarska 58, HR-51000 Rijeka, Croatia
| | - Sandi Ljubic
- University of Rijeka, Faculty of Engineering, Vukovarska 58, HR-51000 Rijeka, Croatia
- Center for Artificial Intelligence and Cybersecurity, University of Rijeka, R. Matejcic 2, HR-51000 Rijeka, Croatia
| |
Collapse
|
8
|
Catalán JM, Trigili E, Nann M, Blanco-Ivorra A, Lauretti C, Cordella F, Ivorra E, Armstrong E, Crea S, Alcañiz M, Zollo L, Soekadar SR, Vitiello N, García-Aracil N. Hybrid brain/neural interface and autonomous vision-guided whole-arm exoskeleton control to perform activities of daily living (ADLs). J Neuroeng Rehabil 2023; 20:61. [PMID: 37149621 PMCID: PMC10164333 DOI: 10.1186/s12984-023-01185-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2022] [Accepted: 04/26/2023] [Indexed: 05/08/2023] Open
Abstract
BACKGROUND The aging of the population and the progressive increase of life expectancy in developed countries is leading to a high incidence of age-related cerebrovascular diseases, which affect people's motor and cognitive capabilities and might result in the loss of arm and hand functions. Such conditions have a detrimental impact on people's quality of life. Assistive robots have been developed to help people with motor or cognitive disabilities to perform activities of daily living (ADLs) independently. Most of the robotic systems for assisting on ADLs proposed in the state of the art are mainly external manipulators and exoskeletal devices. The main objective of this study is to compare the performance of an hybrid EEG/EOG interface to perform ADLs when the user is controlling an exoskeleton rather than using an external manipulator. METHODS Ten impaired participants (5 males and 5 females, mean age 52 ± 16 years) were instructed to use both systems to perform a drinking task and a pouring task comprising multiple subtasks. For each device, two modes of operation were studied: synchronous mode (the user received a visual cue indicating the sub-tasks to be performed at each time) and asynchronous mode (the user started and finished each of the sub-tasks independently). Fluent control was assumed when the time for successful initializations ranged below 3 s and a reliable control in case it remained below 5 s. NASA-TLX questionnaire was used to evaluate the task workload. For the trials involving the use of the exoskeleton, a custom Likert-Scale questionnaire was used to evaluate the user's experience in terms of perceived comfort, safety, and reliability. RESULTS All participants were able to control both systems fluently and reliably. However, results suggest better performances of the exoskeleton over the external manipulator (75% successful initializations remain below 3 s in case of the exoskeleton and bellow 5s in case of the external manipulator). CONCLUSIONS Although the results of our study in terms of fluency and reliability of EEG control suggest better performances of the exoskeleton over the external manipulator, such results cannot be considered conclusive, due to the heterogeneity of the population under test and the relatively limited number of participants.
Collapse
Affiliation(s)
- José M Catalán
- Robotics and Artificial Intelligence Group of the Bioengineering Institute, Miguel Hernandez University, 03202, Elche, Spain.
| | - Emilio Trigili
- BioRobotics Institute, Scuola Superiore Sant'Anna, 56025, Pontedera, Italy.
- Department of Excellence in Robotics & AI, Scuola Superiore Sant'Anna, Pisa, Italy.
| | - Marius Nann
- Clinical Neurotechnology Laboratory, Charité, Universitätsmedizin Berlin, 10117, Belin, Germany
| | - Andrea Blanco-Ivorra
- Robotics and Artificial Intelligence Group of the Bioengineering Institute, Miguel Hernandez University, 03202, Elche, Spain
| | - Clemente Lauretti
- Laboratory of Biomedical Robotics and Biomicrosystems, Università Campus Bio-Medico di Roma, 00128, Rome, Italy
| | - Francesca Cordella
- Laboratory of Biomedical Robotics and Biomicrosystems, Università Campus Bio-Medico di Roma, 00128, Rome, Italy
| | - Eugenio Ivorra
- University Institute for Human-Centered Technology Research (Human-Tech), Universitat Politècnica de València, 46022, Valencia, Spain
| | | | - Simona Crea
- BioRobotics Institute, Scuola Superiore Sant'Anna, 56025, Pontedera, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant'Anna, Pisa, Italy
- IRCCS, Fondazione Don Carlo Gnocchi, Milan, Italy
| | - Mariano Alcañiz
- University Institute for Human-Centered Technology Research (Human-Tech), Universitat Politècnica de València, 46022, Valencia, Spain
| | - Loredana Zollo
- Laboratory of Biomedical Robotics and Biomicrosystems, Università Campus Bio-Medico di Roma, 00128, Rome, Italy
| | - Surjo R Soekadar
- Clinical Neurotechnology Laboratory, Charité, Universitätsmedizin Berlin, 10117, Belin, Germany
| | - Nicola Vitiello
- BioRobotics Institute, Scuola Superiore Sant'Anna, 56025, Pontedera, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant'Anna, Pisa, Italy
- IRCCS, Fondazione Don Carlo Gnocchi, Milan, Italy
| | - Nicolás García-Aracil
- Robotics and Artificial Intelligence Group of the Bioengineering Institute, Miguel Hernandez University, 03202, Elche, Spain
| |
Collapse
|
9
|
Fu J, Jiang Z, Shu X, Chen S, Jia J. Correlation between the ERD in grasp/open tasks of BCIs and hand function of stroke patients: a cross-sectional study. Biomed Eng Online 2023; 22:36. [PMID: 37061673 PMCID: PMC10105926 DOI: 10.1186/s12938-023-01091-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2022] [Accepted: 03/02/2023] [Indexed: 04/17/2023] Open
Abstract
BACKGROUND AND AIMS Brain-computer interfaces (BCIs) are emerging as a promising tool for upper limb recovery after stroke, and motor tasks are an essential part of BCIs for patient training and control of rehabilitative/assistive BCIs. However, the correlation between brain activation with different levels of motor impairment and motor tasks in BCIs is still not so clear. Thus, we aim to compare the brain activation of different levels of motor impairment in performing the hand grasping and opening tasks in BCIs. METHODS We instructed stroke patients to perform motor attempts (MA) to grasp and open the affected hand for 30 trials, respectively. During this period, they underwent EEG acquisition and BCIs accuracy recordings. They also received detailed history records and behavioral scale assessments (the Fugl-Meyer assessment of upper limb, FMA-UE). RESULTS The FMA-UE was negatively correlated with the event-related desynchronization (ERD) of the affected hemisphere during open MA (R = - 0.423, P = 0.009) but not with grasp MA (R = - 0.058, P = 0.733). Then we divided the stroke patients into group 1 (Brunnstrom recovery stages between I to II, n = 19) and group 2 (Brunnstrom recovery stages between III to VI, n = 23). No difference during the grasping task (t = 0.091, P = 0.928), but a significant difference during the open task (t = 2.156, P = 0.037) was found between the two groups on the affected hemisphere. No significant difference was found in the unaffected hemisphere. CONCLUSIONS The study indicated that brain activation is positively correlated with the hand function of stroke in open-hand tasks. In the grasping task, the patients in the different groups have a similar brain response, while in the open task, mildly injured patients have more brain activation in open the hand than the poor hand function patients.
Collapse
Affiliation(s)
- Jianghong Fu
- Department of Rehabilitation Medicine, Huashan Hospital, Fudan University, 12 Mid-Wulumuqi Road, Jing'an District, Shanghai, 200040, China
| | - ZeWu Jiang
- Department of Rehabilitation Medicine, Huashan Hospital, Fudan University, 12 Mid-Wulumuqi Road, Jing'an District, Shanghai, 200040, China
| | - Xiaokang Shu
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Shugeng Chen
- Department of Rehabilitation Medicine, Huashan Hospital, Fudan University, 12 Mid-Wulumuqi Road, Jing'an District, Shanghai, 200040, China.
| | - Jie Jia
- Department of Rehabilitation Medicine, Huashan Hospital, Fudan University, 12 Mid-Wulumuqi Road, Jing'an District, Shanghai, 200040, China.
- National Clinical Research Center for Aging and Medicine, Huashan Hospital, Fudan University, Shanghai, China.
- National Center for Neurological Disorders, Shanghai, China.
| |
Collapse
|
10
|
Tao T, Jia Y, Xu G, Liang R, Zhang Q, Chen L, Gao Y, Chen R, Zheng X, Yu Y. Enhancement of motor imagery training efficiency by an online adaptive training paradigm integrated with error related potential. J Neural Eng 2023; 20. [PMID: 36608339 DOI: 10.1088/1741-2552/acb102] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Accepted: 01/06/2023] [Indexed: 01/07/2023]
Abstract
Objective. Motor imagery (MI) is a process of autonomously modulating the motor area to rehearse action mentally without actual execution. Based on the neuroplasticity of the cerebral cortex, MI can promote the functional rehabilitation of the injured cerebral cortex motor area. However, it usually takes several days to a few months to train individuals to acquire the necessary MI ability to control rehabilitation equipment in current studies, which greatly limits the clinical application of rehabilitation training systems based on the MI brain-computer interface (BCI).Approach. A novel MI training paradigm combined with the error related potential (ErrP) is proposed, and online adaptive training of the MI classifier was performed using ErrP. ErrP is used to correct the output of the MI classification to obtain a higher accuracy of kinesthetic feedback based on the imagination intention of subjects while generating simulated labels for MI online adaptive training. In this way, we improved the MI training efficiency. Thirteen subjects were randomly divided into an experimental group using the proposed paradigm and a control group using the traditional MI training paradigm to participate in six MI training experiments.Main results. The proposed paradigm enabled the experimental group to obtain a higher event-related desynchronization modulation level in the contralateral brain region compared with the control group and 69.76% online classification accuracy of MI after three MI training experiments. The online classification accuracy reached 72.76% and the whole system recognized the MI intention of the subjects with an online accuracy of 82.61% after six experiments.Significance. Compared with the conventional unimodal MI training strategy, the proposed approach enables subjects to use the MI-BCI based system directly and achieve a better performance after only three training experiments with training left and right hands simultaneously. This greatly improves the usability of the MI-BCI-based rehabilitation system and makes it more convenient for clinical use.
Collapse
Affiliation(s)
- Tangfei Tao
- Key Laboratory of Education Ministry for Modern Design and Rotor-Bearing System, Xi'an Jiaotong University, Xi'an, People's Republic of China.,School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Yagang Jia
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Guanghua Xu
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China.,State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China.,The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Renghao Liang
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Qiuxiang Zhang
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Longting Chen
- School of Mechanical and Electrical Engineering, Central South University, Changsha, People's Republic of China
| | - Yuxiang Gao
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Ruiquan Chen
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Xiaowei Zheng
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Yunhui Yu
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| |
Collapse
|
11
|
Cajigas I, Davis KC, Prins NW, Gallo S, Naeem JA, Fisher L, Ivan ME, Prasad A, Jagid JR. Brain-Computer interface control of stepping from invasive electrocorticography upper-limb motor imagery in a patient with quadriplegia. Front Hum Neurosci 2023; 16:1077416. [PMID: 36776220 PMCID: PMC9912159 DOI: 10.3389/fnhum.2022.1077416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2022] [Accepted: 12/19/2022] [Indexed: 01/11/2023] Open
Abstract
Introduction: Most spinal cord injuries (SCI) result in lower extremities paralysis, thus diminishing ambulation. Using brain-computer interfaces (BCI), patients may regain leg control using neural signals that actuate assistive devices. Here, we present a case of a subject with cervical SCI with an implanted electrocorticography (ECoG) device and determined whether the system is capable of motor-imagery-initiated walking in an assistive ambulator. Methods: A 24-year-old male subject with cervical SCI (C5 ASIA A) was implanted before the study with an ECoG sensing device over the sensorimotor hand region of the brain. The subject used motor-imagery (MI) to train decoders to classify sensorimotor rhythms. Fifteen sessions of closed-loop trials followed in which the subject ambulated for one hour on a robotic-assisted weight-supported treadmill one to three times per week. We evaluated the stability of the best-performing decoder over time to initiate walking on the treadmill by decoding upper-limb (UL) MI. Results: An online bagged trees classifier performed best with an accuracy of 84.15% averaged across 9 weeks. Decoder accuracy remained stable following throughout closed-loop data collection. Discussion: These results demonstrate that decoding UL MI is a feasible control signal for use in lower-limb motor control. Invasive BCI systems designed for upper-extremity motor control can be extended for controlling systems beyond upper extremity control alone. Importantly, the decoders used were able to use the invasive signal over several weeks to accurately classify MI from the invasive signal. More work is needed to determine the long-term consequence between UL MI and the resulting lower-limb control.
Collapse
Affiliation(s)
- Iahn Cajigas
- Department of Neurological Surgery, University of Pennsylvania, Philadelphia, PA, United States
| | - Kevin C. Davis
- Department of Biomedical Engineering, University of Miami, Miami, FL, United States
| | - Noeline W. Prins
- Department of Electrical and Information Engineering, University of Ruhana, Hapugala, Sri Lanka
| | - Sebastian Gallo
- Department of Biomedical Engineering, University of Miami, Miami, FL, United States
| | - Jasim A. Naeem
- Department of Biomedical Engineering, University of Miami, Miami, FL, United States
| | - Letitia Fisher
- Department of Neurological Surgery, University of Miami, Miami, FL, United States
- Miami Project to Cure Paralysis, University of Miami, Miami, FL, United States
| | - Michael E. Ivan
- Department of Neurological Surgery, University of Miami, Miami, FL, United States
| | - Abhishek Prasad
- Department of Biomedical Engineering, University of Miami, Miami, FL, United States
- Miami Project to Cure Paralysis, University of Miami, Miami, FL, United States
| | - Jonathan R. Jagid
- Department of Neurological Surgery, University of Miami, Miami, FL, United States
- Miami Project to Cure Paralysis, University of Miami, Miami, FL, United States
| |
Collapse
|
12
|
Cho W, Vidaurre C, An J, Birbaumer N, Ramos-Murguialday A. Cortical processing during robot and functional electrical stimulation. Front Syst Neurosci 2023; 17:1045396. [PMID: 37025164 PMCID: PMC10070684 DOI: 10.3389/fnsys.2023.1045396] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Accepted: 02/28/2023] [Indexed: 04/08/2023] Open
Abstract
Introduction Like alpha rhythm, the somatosensory mu rhythm is suppressed in the presence of somatosensory inputs by implying cortical excitation. Sensorimotor rhythm (SMR) can be classified into two oscillatory frequency components: mu rhythm (8-13 Hz) and beta rhythm (14-25 Hz). The suppressed/enhanced SMR is a neural correlate of cortical activation related to efferent and afferent movement information. Therefore, it would be necessary to understand cortical information processing in diverse movement situations for clinical applications. Methods In this work, the EEG of 10 healthy volunteers was recorded while fingers were moved passively under different kinetic and kinematic conditions for proprioceptive stimulation. For the kinetics aspect, afferent brain activity (no simultaneous volition) was compared under two conditions of finger extension: (1) generated by an orthosis and (2) generated by the orthosis simultaneously combined and assisted with functional electrical stimulation (FES) applied at the forearm muscles related to finger extension. For the kinematic aspect, the finger extension was divided into two phases: (1) dynamic extension and (2) static extension (holding the extended position). Results In the kinematic aspect, both mu and beta rhythms were more suppressed during a dynamic than a static condition. However, only the mu rhythm showed a significant difference between kinetic conditions (with and without FES) affected by attention to proprioception after transitioning from dynamic to static state, but the beta rhythm was not. Discussion Our results indicate that mu rhythm was influenced considerably by muscle kinetics during finger movement produced by external devices, which has relevant implications for the design of neuromodulation and neurorehabilitation interventions.
Collapse
Affiliation(s)
- Woosang Cho
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
- g.tec Medical Engineering GmbH, Schiedlberg, Austria
- *Correspondence: Woosang Cho,
| | - Carmen Vidaurre
- TECNALIA, Basque Research and Technology Alliance, Neurotechnology Laboratory, San Sebastián, Spain
- Ikerbasque-Basque Foundation for Science, Bilbao, Spain
| | - Jinung An
- Interdisciplinary Studies, Graduate School, Daegu Gyeongbuk Institute of Science and Technology, Daegu, Republic of Korea
| | - Niels Birbaumer
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
- San Camillo Hospital, Institute for Hospitalization and Scientific Care, Venice Lido, Italy
| | - Ander Ramos-Murguialday
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
- TECNALIA, Basque Research and Technology Alliance, Neurotechnology Laboratory, San Sebastián, Spain
| |
Collapse
|
13
|
EEG in Neurorehabilitation: A Bibliometric Analysis and Content Review. Neurol Int 2022; 14:1046-1061. [PMID: 36548189 PMCID: PMC9782188 DOI: 10.3390/neurolint14040084] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 12/13/2022] [Accepted: 12/14/2022] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND There is increasing interest in the role of EEG in neurorehabilitation. We primarily aimed to identify the knowledge base through highly influential studies. Our secondary aims were to imprint the relevant thematic hotspots, research trends, and social networks within the scientific community. METHODS We performed an electronic search in Scopus, looking for studies reporting on rehabilitation in patients with neurological disabilities. We used the most influential papers to outline the knowledge base and carried out a word co-occurrence analysis to identify the research hotspots. We also used depicted collaboration networks between universities, authors, and countries after analyzing the cocitations. The results were presented in summary tables, plots, and maps. Finally, a content review based on the top-20 most cited articles completed our study. RESULTS Our current bibliometric study was based on 874 records from 420 sources. There was vivid research interest in EEG use for neurorehabilitation, with an annual growth rate as high as 14.3%. The most influential paper was the study titled "Brain-computer interfaces, a review" by L.F. Nicolas-Alfonso and J. Gomez-Gill, with 997 citations, followed by "Brain-computer interfaces in neurological rehabilitation" by J. Daly and J.R. Wolpaw (708 citations). The US, Italy, and Germany were among the most productive countries. The research hotspots shifted with time from the use of functional magnetic imaging to EEG-based brain-machine interface, motor imagery, and deep learning. CONCLUSIONS EEG constitutes the most significant input in brain-computer interfaces (BCIs) and can be successfully used in the neurorehabilitation of patients with stroke symptoms, amyotrophic lateral sclerosis, and traumatic brain and spinal injuries. EEG-based BCI facilitates the training, communication, and control of wheelchair and exoskeletons. However, research is limited to specific scientific groups from developed countries. Evidence is expected to change with the broader availability of BCI and improvement in EEG-filtering algorithms.
Collapse
|
14
|
Insausti-Delgado A, López-Larraz E, Nishimura Y, Ziemann U, Ramos-Murguialday A. Non-invasive brain-spine interface: Continuous control of trans-spinal magnetic stimulation using EEG. Front Bioeng Biotechnol 2022; 10:975037. [PMID: 36394044 PMCID: PMC9659618 DOI: 10.3389/fbioe.2022.975037] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 09/23/2022] [Indexed: 08/22/2023] Open
Abstract
Brain-controlled neuromodulation has emerged as a promising tool to promote functional recovery in patients with motor disorders. Brain-machine interfaces exploit this neuromodulatory strategy and could be used for restoring voluntary control of lower limbs. In this work, we propose a non-invasive brain-spine interface (BSI) that processes electroencephalographic (EEG) activity to volitionally control trans-spinal magnetic stimulation (ts-MS), as an approach for lower-limb neurorehabilitation. This novel platform allows to contingently connect motor cortical activation during leg motor imagery with the activation of leg muscles via ts-MS. We tested this closed-loop system in 10 healthy participants using different stimulation conditions. This BSI efficiently removed stimulation artifacts from EEG regardless of ts-MS intensity used, allowing continuous monitoring of cortical activity and real-time closed-loop control of ts-MS. Our BSI induced afferent and efferent evoked responses, being this activation ts-MS intensity-dependent. We demonstrated the feasibility, safety and usability of this non-invasive BSI. The presented system represents a novel non-invasive means of brain-controlled neuromodulation and opens the door towards its integration as a therapeutic tool for lower-limb rehabilitation.
Collapse
Affiliation(s)
- Ainhoa Insausti-Delgado
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
- International Max Planck Research School (IMPRS) for Cognitive and Systems Neuroscience, Tübingen, Germany
- IKERBASQUE, Basque Foundation for Science, Bilbao, Spain
- TECNALIA, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain
| | - Eduardo López-Larraz
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
- Bitbrain, Zaragoza, Spain
| | - Yukio Nishimura
- Neural Prosthetics Project, Department of Brain and Neuroscience, Tokyo Metropolitan Institute of Medical Science, Tokyo, Japan
| | - Ulf Ziemann
- Department of Neurology and Stroke, University of Tübingen, Tübingen, Germany
- Hertie Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany
| | - Ander Ramos-Murguialday
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
- TECNALIA, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain
| |
Collapse
|
15
|
De Miguel-Rubio A, Muñoz-Pérez L, Alba-Rueda A, Arias-Avila M, Rodrigues-de-Souza DP. A Therapeutic Approach Using the Combined Application of Virtual Reality with Robotics for the Treatment of Patients with Spinal Cord Injury: A Systematic Review. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19148772. [PMID: 35886624 PMCID: PMC9322038 DOI: 10.3390/ijerph19148772] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Revised: 07/14/2022] [Accepted: 07/16/2022] [Indexed: 02/04/2023]
Abstract
Spinal cord injury (SCI) has been associated with high mortality rates. Thanks to the multidisciplinary vision and approach of SCI, including the application of new technologies in the field of neurorehabilitation, people with SCI can survive and prosper after injury. The main aim of this systematic review was to analyze the effectiveness of the combined use of VR and robotics in the treatment of patients with SCI. The literature search was performed between May and July 2021 in the Cochrane Central Register of Controlled Trials, Physiotherapy Evidence Database (PEDro), PubMed, and Web of Science. The methodological quality of each study was assessed using the SCIRE system and the PEDro scale, whereas the risk of bias was analyzed using the Cochrane Collaboration’s tool. A total of six studies, involving 63 participants, were included in this systematic review. Relevant changes were found in the upper limbs, with improvements of shoulder and upper arm mobility, as well as the strengthening of weaker muscles. Combined rehabilitation may be a valuable approach to improve motor function in SCI patients. Nonetheless, further research is necessary, with a larger patient sample and a longer duration.
Collapse
Affiliation(s)
- Amaranta De Miguel-Rubio
- Department of Nursing, Pharmacology and Physiotherapy, University of Cordoba, 14004 Cordoba, Spain; (L.M.-P.); (A.A.-R.)
- Correspondence: ; Tel.: +34-957-218-220
| | - Lorena Muñoz-Pérez
- Department of Nursing, Pharmacology and Physiotherapy, University of Cordoba, 14004 Cordoba, Spain; (L.M.-P.); (A.A.-R.)
| | - Alvaro Alba-Rueda
- Department of Nursing, Pharmacology and Physiotherapy, University of Cordoba, 14004 Cordoba, Spain; (L.M.-P.); (A.A.-R.)
| | - Mariana Arias-Avila
- Physical Therapy Department, Universidade Federal de São Carlos, São Carlos, São Paulo 13565-905, Brazil;
| | | |
Collapse
|
16
|
Perez-Velasco S, Santamaria-Vazquez E, Martinez-Cagigal V, Marcos-Martinez D, Hornero R. EEGSym: Overcoming Inter-Subject Variability in Motor Imagery Based BCIs With Deep Learning. IEEE Trans Neural Syst Rehabil Eng 2022; 30:1766-1775. [PMID: 35759578 DOI: 10.1109/tnsre.2022.3186442] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
In this study, we present a new Deep Learning (DL) architecture for Motor Imagery (MI) based Brain Computer Interfaces (BCIs) called EEGSym. Our implementation aims to improve previous state-of-the-art performances on MI classification by overcoming inter-subject variability and reducing BCI inefficiency, which has been estimated to affect 10-50% of the population. This convolutional neural network includes the use of inception modules, residual connections and a design that introduces the symmetry of the brain through the mid-sagittal plane into the network architecture. It is complemented with a data augmentation technique that improves the generalization of the model and with the use of transfer learning across different datasets. We compare EEGSym's performance on inter-subject MI classification with ShallowConvNet, DeepConvNet, EEGNet and EEG-Inception. This comparison is performed on 5 publicly available datasets that include left or right hand motor imagery of 280 subjects. This population is the largest that has been evaluated in similar studies to date. EEGSym significantly outperforms the baseline models reaching accuracies of 88.6±9.0 on Physionet, 83.3±9.3 on OpenBMI, 85.1±9.5 on Kaya2018, 87.4±8.0 on Meng2019 and 90.2±6.5 on Stieger2021. At the same time, it allows 95.7% of the tested population (268 out of 280 users) to reach BCI control (≥70% accuracy). Furthermore, these results are achieved using only 16 electrodes of the more than 60 available on some datasets. Our implementation of EEGSym, which includes new advances for EEG processing with DL, outperforms previous state-of-the-art approaches on inter-subject MI classification.
Collapse
|
17
|
Mascolini A, Niazi IK, Mesin L. Non-linear optimized spatial filter for single-trial identification of movement related cortical potential. Biocybern Biomed Eng 2022. [DOI: 10.1016/j.bbe.2022.02.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
18
|
Meng J, Wu Z, Li S, Zhu X. Effects of Gaze Fixation on the Performance of a Motor Imagery-Based Brain-Computer Interface. Front Hum Neurosci 2022; 15:773603. [PMID: 35140593 PMCID: PMC8818858 DOI: 10.3389/fnhum.2021.773603] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2021] [Accepted: 12/08/2021] [Indexed: 11/13/2022] Open
Abstract
Motor imagery-based brain-computer interfaces (BCIs) have been studied without controlling subjects’ gaze fixation position previously. The effect of gaze fixation and covert attention on the behavioral performance of BCI is still unknown. This study designed a gaze fixation controlled experiment. Subjects were required to conduct a secondary task of gaze fixation when performing the primary task of motor imagination. Subjects’ performance was analyzed according to the relationship between motor imagery target and the gaze fixation position, resulting in three BCI control conditions, i.e., congruent, incongruent, and center cross trials. A group of fourteen subjects was recruited. The average group performances of three different conditions did not show statistically significant differences in terms of BCI control accuracy, feedback duration, and trajectory length. Further analysis of gaze shift response time revealed a significantly shorter response time for congruent trials compared to incongruent trials. Meanwhile, the parietal occipital cortex also showed active neural activities for congruent and incongruent trials, and this was revealed by a contrast analysis of R-square values and lateralization index. However, the lateralization index computed from the parietal and occipital areas was not correlated with the BCI behavioral performance. Subjects’ BCI behavioral performance was not affected by the position of gaze fixation and covert attention. This indicated that motor imagery-based BCI could be used freely in robotic arm control without sacrificing performance.
Collapse
Affiliation(s)
- Jianjun Meng
- Department of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
- *Correspondence: Jianjun Meng,
| | - Zehan Wu
- Department of Neurosurgery, Huashan Hospital, Fudan University, Shanghai, China
| | - Songwei Li
- Department of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Xiangyang Zhu
- Department of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
19
|
|
20
|
Bibián C, Irastorza-Landa N, Schönauer M, Birbaumer N, López-Larraz E, Ramos-Murguialday A. On the Extraction of Purely Motor EEG Neural Correlates during an Upper Limb Visuomotor Task. Cereb Cortex 2021; 32:4243-4254. [PMID: 34969088 DOI: 10.1093/cercor/bhab479] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Revised: 11/11/2021] [Accepted: 11/12/2021] [Indexed: 11/14/2022] Open
Abstract
Deciphering and analyzing the neural correlates of different movements from the same limb using electroencephalography (EEG) would represent a notable breakthrough in the field of sensorimotor neurophysiology. Functional movements involve concurrent posture co-ordination and head and eye movements, which create electrical activity that affects EEG recordings. In this paper, we revisit the identification of brain signatures of different reaching movements using EEG and present, test, and validate a protocol to separate the effect of head and eye movements from a reaching task-related visuomotor brain activity. Ten healthy participants performed reaching movements under two different conditions: avoiding head and eye movements and moving with no constrains. Reaching movements can be identified from EEG with unconstrained eye and head movement, whereas the discriminability of the signals drops to chance level otherwise. These results show that neural patterns associated with different arm movements could only be extracted from EEG if the eye and head movements occurred concurrently with the task, polluting the recordings. Although these findings do not imply that brain correlates of reaching directions cannot be identified from EEG, they show the consequences that ignoring these events can have in any EEG study that includes a visuomotor task.
Collapse
Affiliation(s)
- Carlos Bibián
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen 72076, Germany
- International Max Planck Research School for Cognitive and Systems Neuroscience, Tübingen 72074, Germany
| | - Nerea Irastorza-Landa
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen 72076, Germany
- TECNALIA, Basque Research and Technology Alliance (BRTA), Neurotechnology Laboratory, San Sebastián 20009, Spain
| | - Monika Schönauer
- Institute of Psychology, Neuropsychology, University of Freiburg, Freiburg 79085, Germany
| | - Niels Birbaumer
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen 72076, Germany
| | - Eduardo López-Larraz
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen 72076, Germany
- Bitbrain, Zaragoza 50008, Spain
| | - Ander Ramos-Murguialday
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen 72076, Germany
- TECNALIA, Basque Research and Technology Alliance (BRTA), Neurotechnology Laboratory, San Sebastián 20009, Spain
| |
Collapse
|
21
|
Le Bars S, Chokron S, Balp R, Douibi K, Waszak F. Theoretical Perspective on an Ideomotor Brain-Computer Interface: Toward a Naturalistic and Non-invasive Brain-Computer Interface Paradigm Based on Action-Effect Representation. Front Hum Neurosci 2021; 15:732764. [PMID: 34776904 PMCID: PMC8581635 DOI: 10.3389/fnhum.2021.732764] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Accepted: 10/11/2021] [Indexed: 11/13/2022] Open
Abstract
Recent years have been marked by the fulgurant expansion of non-invasive Brain-Computer Interface (BCI) devices and applications in various contexts (medical, industrial etc.). This technology allows agents "to directly act with thoughts," bypassing the peripheral motor system. Interestingly, it is worth noting that typical non-invasive BCI paradigms remain distant from neuroscientific models of human voluntary action. Notably, bidirectional links between action and perception are constantly ignored in BCI experiments. In the current perspective article, we proposed an innovative BCI paradigm that is directly inspired by the ideomotor principle, which postulates that voluntary actions are driven by the anticipated representation of forthcoming perceptual effects. We believe that (1) adapting BCI paradigms could allow simple action-effect bindings and consequently action-effect predictions and (2) using neural underpinnings of those action-effect predictions as features of interest in AI methods, could lead to more accurate and naturalistic BCI-mediated actions.
Collapse
Affiliation(s)
- Solène Le Bars
- Altran Lab, Capgemini Engineering, Paris, France.,Université de Paris, INCC UMR 8002, CNRS, Paris, France
| | - Sylvie Chokron
- Université de Paris, INCC UMR 8002, CNRS, Paris, France.,Fondation Ophtalmologique Adolphe de Rothschild, Paris, France
| | - Rodrigo Balp
- Altran Lab, Capgemini Engineering, Paris, France
| | | | | |
Collapse
|
22
|
Park S, Ha J, Kim DH, Kim L. Improving Motor Imagery-Based Brain-Computer Interface Performance Based on Sensory Stimulation Training: An Approach Focused on Poorly Performing Users. Front Neurosci 2021; 15:732545. [PMID: 34803582 PMCID: PMC8602688 DOI: 10.3389/fnins.2021.732545] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Accepted: 10/20/2021] [Indexed: 11/13/2022] Open
Abstract
The motor imagery (MI)-based brain-computer interface (BCI) is an intuitive interface that provides control over computer applications directly from brain activity. However, it has shown poor performance compared to other BCI systems such as P300 and SSVEP BCI. Thus, this study aimed to improve MI-BCI performance by training participants in MI with the help of sensory inputs from tangible objects (i.e., hard and rough balls), with a focus on poorly performing users. The proposed method is a hybrid of training and imagery, combining motor execution and somatosensory sensation from a ball-type stimulus. Fourteen healthy participants participated in the somatosensory-motor imagery (SMI) experiments (within-subject design) involving EEG data classification with a three-class system (signaling with left hand, right hand, or right foot). In the scenario of controlling a remote robot to move it to the target point, the participants performed MI when faced with a three-way intersection. The SMI condition had a better classification performance than did the MI condition, achieving a 68.88% classification performance averaged over all participants, which was 6.59% larger than that in the MI condition (p < 0.05). In poor performers, the classification performance in SMI was 10.73% larger than in the MI condition (62.18% vs. 51.45%). However, good performers showed a slight performance decrement (0.86%) in the SMI condition compared to the MI condition (80.93% vs. 81.79%). Combining the brain signals from the motor and somatosensory cortex, the proposed hybrid MI-BCI system demonstrated improved classification performance, this phenomenon was predominant in poor performers (eight out of nine subjects). Hybrid MI-BCI systems may significantly contribute to reducing the proportion of BCI-inefficiency users and closing the performance gap with other BCI systems.
Collapse
Affiliation(s)
- Sangin Park
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
| | - Jihyeon Ha
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea
| | - Da-Hye Kim
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
| | - Laehyun Kim
- Center for Bionics, Korea Institute of Science and Technology, Seoul, South Korea
- Department of HY-KIST Bio-Convergence, Hanyang University, Seoul, South Korea
| |
Collapse
|
23
|
Burke JF, Kunwar N, Yaroshinsky MS, Louie KH, Shirvalkar P, Su P, Henry M, Pasvankas G, Poree L, Jacques L, Wang DD. Epidural Spinal Electrogram Provides Direct Spinal Recordings in Awake Human Participants. Front Hum Neurosci 2021; 15:721076. [PMID: 34764858 PMCID: PMC8577539 DOI: 10.3389/fnhum.2021.721076] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2021] [Accepted: 10/04/2021] [Indexed: 11/13/2022] Open
Abstract
Little is known about the electrophysiological activity of the spinal cord during voluntary movement control in humans. We present a novel method for recording electrophysiological activity from the human spinal cord using implanted epidural electrodes during naturalistic movements including overground walking. Spinal electrograms (SEGs) were recorded from epidural electrodes implanted as part of a test trial for patients with chronic pain undergoing evaluation for spinal cord stimulation. Externalized ends of the epidural leads were connected to an external amplifier to capture SEGs. Electromyographic and accelerometry data from the upper and lower extremities were collected using wireless sensors and synchronized to the SEG data. Patients were instructed to perform various arm and leg movements while SEG and kinematic data were collected. This study proves the safety and feasibility of performing epidural spinal recordings from human subjects performing movement tasks.
Collapse
Affiliation(s)
- John F. Burke
- Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, United States
| | - Nikhita Kunwar
- Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, United States
| | - Maria S. Yaroshinsky
- Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, United States
| | - Kenneth H. Louie
- Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, United States
| | - Prasad Shirvalkar
- Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, United States
- Department of Anesthesia and Pain Management, University of California, San Francisco, San Francisco, CA, United States
- Department of Neurology, University of California, San Francisco, San Francisco, CA, United States
| | - Paul Su
- Department of Anesthesia and Pain Management, University of California, San Francisco, San Francisco, CA, United States
| | - Melanie Henry
- Department of Anesthesia and Pain Management, University of California, San Francisco, San Francisco, CA, United States
| | - George Pasvankas
- Department of Anesthesia and Pain Management, University of California, San Francisco, San Francisco, CA, United States
| | - Lawrence Poree
- Department of Anesthesia and Pain Management, University of California, San Francisco, San Francisco, CA, United States
| | - Lines Jacques
- Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, United States
| | - Doris D. Wang
- Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, United States
| |
Collapse
|
24
|
Hehenberger L, Batistic L, Sburlea AI, Müller-Putz GR. Directional Decoding From EEG in a Center-Out Motor Imagery Task With Visual and Vibrotactile Guidance. Front Hum Neurosci 2021; 15:687252. [PMID: 34630055 PMCID: PMC8497713 DOI: 10.3389/fnhum.2021.687252] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Accepted: 08/30/2021] [Indexed: 11/13/2022] Open
Abstract
Motor imagery is a popular technique employed as a motor rehabilitation tool, or to control assistive devices to substitute lost motor function. In both said areas of application, artificial somatosensory input helps to mirror the sensorimotor loop by providing kinesthetic feedback or guidance in a more intuitive fashion than via visual input. In this work, we study directional and movement-related information in electroencephalographic signals acquired during a visually guided center-out motor imagery task in two conditions, i.e., with and without additional somatosensory input in the form of vibrotactile guidance. Imagined movements to the right and forward could be discriminated in low-frequency electroencephalographic amplitudes with group level peak accuracies of 70% with vibrotactile guidance, and 67% without vibrotactile guidance. The peak accuracies with and without vibrotactile guidance were not significantly different. Furthermore, the motor imagery could be classified against a resting baseline with group level accuracies between 76 and 83%, using either low-frequency amplitude features or μ and β power spectral features. On average, accuracies were higher with vibrotactile guidance, while this difference was only significant in the latter set of features. Our findings suggest that directional information in low-frequency electroencephalographic amplitudes is retained in the presence of vibrotactile guidance. Moreover, they hint at an enhancing effect on motor-related μ and β spectral features when vibrotactile guidance is provided.
Collapse
Affiliation(s)
- Lea Hehenberger
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Luka Batistic
- Laboratory for Application of Information Technologies, Faculty of Engineering, Department of Computer Engineering, University of Rijeka, Rijeka, Croatia
| | - Andreea I Sburlea
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria.,BioTechMed Graz, Graz, Austria
| |
Collapse
|
25
|
Tabernig CB, Carrere LC, Manresa JB, Spaich EG. Does feedback based on FES-evoked nociceptive withdrawal reflex condition event-related desynchronization? An exploratory study with brain-computer interfaces. Biomed Phys Eng Express 2021; 7. [PMID: 34431480 DOI: 10.1088/2057-1976/ac2077] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Accepted: 08/24/2021] [Indexed: 11/11/2022]
Abstract
Introduction.Event-related desynchronization (ERD) is used in brain-computer interfaces (BCI) to detect the user's motor intention (MI) and convert it into a command for an actuator to provide sensory feedback or mobility, for example by means of functional electrical stimulation (FES). Recent studies have proposed to evoke the nociceptive withdrawal reflex (NWR) using FES, in order to evoke synergistic movements of the lower limb and to facilitate the gait rehabilitation of stroke patients. The use of NWR to provide sensorimotor feedback in ERD-based BCI is novel; thererfore, the conditioning effect that nociceptive stimuli might have on MI is still unknown.Objetive.To assess the ERD produced during the MI after FES-evoked NWR, in order to evaluate if nociceptive stimuli condition subsequent ERDs.Methods. Data from 528 electroencephalography trials of 8 healthy volunteers were recorded and analyzed. Volunteers used an ERD-based BCI, which provided two types of feedback: intrisic by the FES-evoked NWR and extrinsic by virtual reality. The electromyogram of the tibialis anterior muscle was also recorded. The main outcome variables were the normalized root mean square of the evoked electromyogram (RMSnorm), the average electroencephalogram amplitude at the ERD frequency during MI (A¯MI) and the percentage decrease ofA¯MIrelative to rest (ERD%) at the first MI subsequent to the activation of the BCI.Results.No evidence of changes of theRMSnormon both theA¯MI(p = 0.663) and theERD%(p = 0.252) of the subsequent MI was detected. A main effect of the type of feedback was found in the subsequentA¯MI(p < 0.001), with intrinsic feedback resulting in a largerA¯MI.Conclusions.No evidence of ERD conditioning was observed using BCI feedback based on FES-evoked NWR .Significance.FES-evoked NWR could constitute a potential feedback modality in an ERD-based BCI to facilitate motor recovery of stroke people.
Collapse
Affiliation(s)
- Carolina B Tabernig
- Laboratory of Rehabilitation Engineering and Neuromuscular and Sensory Research (LIRINS), Faculty of Engineering, National University of Entre Ríos, Oro Verde, Argentina
| | - L Carolina Carrere
- Laboratory of Rehabilitation Engineering and Neuromuscular and Sensory Research (LIRINS), Faculty of Engineering, National University of Entre Ríos, Oro Verde, Argentina
| | - José Biurrun Manresa
- Laboratory of Rehabilitation Engineering and Neuromuscular and Sensory Research (LIRINS), Faculty of Engineering, National University of Entre Ríos, Oro Verde, Argentina.,Institute for Research and Development in Bioengineering and Bioinformatics (IBB), CONICET-UNER, Oro Verde, Argentina
| | - Erika G Spaich
- Neurorehabilitation Systems Group, Department of Health Science and Technology, Aalborg University, Fredrik Bajers Vej 7D2, 9220 Aalborg, Denmark
| |
Collapse
|
26
|
Abstract
Abstract
Neurofeedback (NF) is a versatile non-invasive neuromodulation technique. In combination with motor imagery (MI), NF has considerable potential for enhancing motor performance or supplementing motor rehabilitation. However, not all users achieve reliable NF control. While research has focused on various brain signal properties and the optimisation of signal processing to solve this issue, the impact of context, i.e. the conditions in which NF motor tasks occur, is comparatively unknown. We review current research on the impact of context on MI NF and related motor domains. We identify long-term factors that act at the level of the individual or of the intervention, and short-term factors, with levels before/after and during a session. The reviewed literature indicates that context plays a significant role. We propose considering context factors as well as within-level and across-level interactions when studying MI NF.
Collapse
|
27
|
Grigorev NA, Savosenkov AO, Lukoyanov MV, Udoratina A, Shusharina NN, Kaplan AY, Hramov AE, Kazantsev VB, Gordleeva S. A BCI-Based Vibrotactile Neurofeedback Training Improves Motor Cortical Excitability During Motor Imagery. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1583-1592. [PMID: 34343094 DOI: 10.1109/tnsre.2021.3102304] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
In this study, we address the issue of whether vibrotactile feedback can enhance the motor cortex excitability translated into the plastic changes in local cortical areas during motor imagery (MI) BCI-based training. For this purpose, we focused on two of the most notable neurophysiological effects of MI - the event-related desynchronization (ERD) level and the increase in cortical excitability assessed with navigated transcranial magnetic stimulation (nTMS). For TMS navigation, we used individual high-resolution 3D brain MRIs. Ten BCI-naive and healthy adults participated in this study. The MI (rest or left/right hand imagery using Graz-BCI paradigm) tasks were performed separately in the presence and absence of feedback. To investigate how much the presence/absence of vibrotactile feedback in MI BCI-based training could contribute to the sensorimotor cortical activations, we compared the MEPs amplitude during MI after training with and without feedback. In addition, the ERD levels during MI BCI-based training were investigated. Our findings provide evidence that applying vibrotactile feedback during MI training leads to (i) an enhancement of the desynchronization level of mu-rhythm EEG patterns over the contralateral motor cortex area corresponding to the MI of the non-dominant hand; (ii) an increase in motor cortical excitability in hand muscle representation corresponding to a muscle engaged by the MI.
Collapse
|
28
|
Vidaurre C, Jorajuría T, Ramos-Murguialday A, Müller KR, Gómez M, Nikulin VV. Improving motor imagery classification during induced motor perturbations. J Neural Eng 2021; 18. [PMID: 34233305 DOI: 10.1088/1741-2552/ac123f] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2021] [Accepted: 07/07/2021] [Indexed: 11/11/2022]
Abstract
Objective.Motor imagery is the mental simulation of movements. It is a common paradigm to design brain-computer interfaces (BCIs) that elicits the modulation of brain oscillatory activity similar to real, passive and induced movements. In this study, we used peripheral stimulation to provoke movements of one limb during the performance of motor imagery tasks. Unlike other works, in which induced movements are used to support the BCI operation, our goal was to test and improve the robustness of motor imagery based BCI systems to perturbations caused by artificially generated movements.Approach.We performed a BCI session with ten participants who carried out motor imagery of three limbs. In some of the trials, one of the arms was moved by neuromuscular stimulation. We analysed 2-class motor imagery classifications with and without movement perturbations. We investigated the performance decrease produced by these disturbances and designed different computational strategies to attenuate the observed classification accuracy drop.Main results.When the movement was induced in a limb not coincident with the motor imagery classes, extracting oscillatory sources of the movement imagination tasks resulted in BCI performance being similar to the control (undisturbed) condition; when the movement was induced in a limb also involved in the motor imagery tasks, the performance drop was significantly alleviated by spatially filtering out the neural noise caused by the stimulation. We also show that the loss of BCI accuracy was accompanied by weaker power of the sensorimotor rhythm. Importantly, this residual power could be used to predict whether a BCI user will perform with sufficient accuracy under the movement disturbances.Significance.We provide methods to ameliorate and even eliminate motor related afferent disturbances during the performance of motor imagery tasks. This can help improving the reliability of current motor imagery based BCI systems.
Collapse
Affiliation(s)
- C Vidaurre
- Department of Statistics, Computer Science and Mathematics, Public University of Navarre, Pamplona, Spain.,Machine Learning Group, Computer Science Faculty, Berlin Institute of Technology, Berlin, Germany.,Both authors contributed equally
| | - T Jorajuría
- Department of Statistics, Computer Science and Mathematics, Public University of Navarre, Pamplona, Spain.,Both authors contributed equally
| | - A Ramos-Murguialday
- Institute for Medical Psychology and Behavioral Neurobiology (IMP), University of Tübingen, 72076 Tübingen, Germany.,Neurotechnology Laboratory, TECNALIA, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain
| | - K-R Müller
- Machine Learning Group, Computer Science Faculty, Berlin Institute of Technology, Berlin, Germany.,BIFOLD Berlin Institute for the Foundations of Learning and Data, Berlin, Germany.,Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany.,Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea.,Max Planck Institute for Informatics, Saarbrücken, Germany
| | - M Gómez
- Department of Statistics, Computer Science and Mathematics, Public University of Navarre, Pamplona, Spain
| | - V V Nikulin
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.,Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russia
| |
Collapse
|
29
|
Horowitz AJ, Guger C, Korostenskaja M. What Internal Variables Affect Sensorimotor Rhythm Brain-Computer Interface (SMR-BCI) Performance? HCA HEALTHCARE JOURNAL OF MEDICINE 2021; 2:163-179. [PMID: 37427003 PMCID: PMC10324829 DOI: 10.36518/2689-0216.1196] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/11/2023]
Abstract
Description In this review article, we aimed to create a summary of the effects of internal variables on the performance of sensorimotor rhythm-based brain computer interfaces (SMR-BCIs). SMR-BCIs can be potentially used for interfacing between the brain and devices, bypassing usual central nervous system output, such as muscle activity. The careful consideration of internal factors, affecting SMR-BCI performance, can maximize BCI application in both healthy and disabled people. Internal variables may be generalized as descriptors of the processes mainly dependent on the BCI user and/or originating within the user. The current review aimed to critically evaluate and summarize the currently accumulated body of knowledge regarding the effect of internal variables on SMR-BCI performance. The examples of such internal variables include motor imagery, hand coordination, attention, motivation, quality of life, mood and neurophysiological signals other than SMR. We will conclude our review with the discussion about the future developments regarding the research on the effects of internal variables on SMR-BCI performance. The end-goal of this review paper is to provide current BCI users and researchers with the reference guide that can help them optimize the SMR-BCI performance by accounting for possible influences of various internal factors.
Collapse
Affiliation(s)
- Alex J. Horowitz
- Functional Brain Mapping and Brain Computer Interface Lab, Neuroscience Institute, AdventHealth Orlando, Orlando, FL,
USA
- University of Central Florida/HCA Healthcare GME Consortium, Gainesville, Florida
| | | | - Milena Korostenskaja
- Functional Brain Mapping and Brain Computer Interface Lab, Neuroscience Institute, AdventHealth Orlando, Orlando, FL,
USA
- MEG Lab, AdventHealth for Children, Orlando, FL,
USA
- Department of Psychology, College of Arts and Sciences, University of North Florida, Jacksonville, FL,
USA
- Comprehensive Epilepsy Center, AdventHealth Orlando, Orlando, FL,
USA
| |
Collapse
|
30
|
Horowitz AJ, Guger C, Korostenskaja M. What External Variables Affect Sensorimotor Rhythm Brain-Computer Interface (SMR-BCI) Performance? HCA HEALTHCARE JOURNAL OF MEDICINE 2021; 2:143-162. [PMID: 37427002 PMCID: PMC10324824 DOI: 10.36518/2689-0216.1188] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/11/2023]
Abstract
Description Sensorimotor rhythm-based brain-computer interfaces (SMR-BCIs) are used for the acquisition and translation of motor imagery-related brain signals into machine control commands, bypassing the usual central nervous system output. The selection of optimal external variable configuration can maximize SMR-BCI performance in both healthy and disabled people. This performance is especially important now when the BCI is targeted for everyday use in the environment beyond strictly regulated laboratory settings. In this review article, we summarize and critically evaluate the current body of knowledge pertaining to the effect of the external variables on SMR-BCI performance. When assessing the relationship between SMR-BCI performance and external variables, we broadly characterize them as elements that are less dependent on the BCI user and originate from beyond the user. These elements include such factors as BCI type, distractors, training, visual and auditory feedback, virtual reality and magneto electric feedback, proprioceptive and haptic feedback, carefulness of electroencephalography (EEG) system assembling and positioning of EEG electrodes as well as recording-related artifacts. At the end of this review paper, future developments are proposed regarding the research into the effects of external variables on SMR-BCI performance. We believe that our critical review will be of value for academic BCI scientists and developers and clinical professionals working in the field of BCIs as well as for SMR-BCI users.
Collapse
Affiliation(s)
- Alex J. Horowitz
- Functional Brain Mapping and Brain Computer Interface Lab, Neuroscience Institute, AdventHealth Orlando, Orlando, FL,
USA
- University of Central Florida/HCA Healthcare GME Consortium, Gainesville, Florida
| | | | - Milena Korostenskaja
- Functional Brain Mapping and Brain Computer Interface Lab, Neuroscience Institute, AdventHealth Orlando, Orlando, FL,
USA
- MEG Lab, AdventHealth for Children, Orlando, FL,
USA
- Department of Psychology, College of Arts and Sciences, University of North Florida, Jacksonville, FL,
USA
- Comprehensive Epilepsy Center, AdventHealth Orlando, Orlando, FL,
USA
| |
Collapse
|
31
|
Zapała D, Hossaini A, Kianpour M, Sahonero-Alvarez G, Ayesh A. A functional BCI model by the P2731 working group: psychology. BRAIN-COMPUTER INTERFACES 2021. [DOI: 10.1080/2326263x.2021.1935124] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Dariusz Zapała
- Department of Experimental Psychology, The John Paul II Catholic University of Lublin, Lublin, Poland
| | - Ali Hossaini
- Department of Engineering, King’s College London, London, UK
| | - Mazaher Kianpour
- Department of Information Security and Communication Technology, Norwegian University of Science and Technology, Gjøvik, Norway
| | - Guillermo Sahonero-Alvarez
- Center for Research, Development and Innovation in Mechatronics Engineering,Department of Mechatronics Engineering, Universidad Catolica Boliviana San Pablo, La Paz, Bolivia
| | - Aladdin Ayesh
- Faculty of Computing,Engineering and Media,De Montfort University, Leicester, UK
| |
Collapse
|
32
|
Exploring the Use of Brain-Computer Interfaces in Stroke Neurorehabilitation. BIOMED RESEARCH INTERNATIONAL 2021; 2021:9967348. [PMID: 34239936 PMCID: PMC8235968 DOI: 10.1155/2021/9967348] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/20/2021] [Accepted: 06/04/2021] [Indexed: 11/17/2022]
Abstract
With the continuous development of artificial intelligence technology, "brain-computer interfaces" are gradually entering the field of medical rehabilitation. As a result, brain-computer interfaces (BCIs) have been included in many countries' strategic plans for innovating this field, and subsequently, major funding and talent have been invested in this technology. In neurological rehabilitation for stroke patients, the use of BCIs opens up a new chapter in "top-down" rehabilitation. In our study, we first reviewed the latest BCI technologies, then presented recent research advances and landmark findings in BCI-based neurorehabilitation for stroke patients. Neurorehabilitation was focused on the areas of motor, sensory, speech, cognitive, and environmental interactions. Finally, we summarized the shortcomings of BCI use in the field of stroke neurorehabilitation and the prospects for BCI technology development for rehabilitation.
Collapse
|
33
|
Guerrero Moreno J, Biazoli CE, Baptista AF, Trambaiolli LR. Closed-loop neurostimulation for affective symptoms and disorders: An overview. Biol Psychol 2021; 161:108081. [PMID: 33757806 DOI: 10.1016/j.biopsycho.2021.108081] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2020] [Revised: 03/12/2021] [Accepted: 03/15/2021] [Indexed: 12/28/2022]
Abstract
Affective and anxiety disorders are the most prevalent and incident psychiatric disorders worldwide. Therapeutic approaches to these disorders using non-invasive brain stimulation (NIBS) and analogous techniques have been extensively investigated. In this paper, we discuss the combination of NIBS and neurofeedback in closed-loop setups and its application for affective symptoms and disorders. For this, we first provide a rationale for this combination by presenting some of the main original findings of NIBS, with a primary focus on transcranial magnetic stimulation (TMS), and neurofeedback, including protocols based on electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). Then, we provide a scope review of studies combining real-time neurofeedback with NIBS protocols in the so-called closed-loop brain state-dependent neuromodulation (BSDS). Finally, we discuss the concomitant use of TMS and real-time functional near-infrared spectroscopy (fNIRS) as a possible solution to the current limitations of BSDS-based protocols for affective and anxiety disorders.
Collapse
Affiliation(s)
- Javier Guerrero Moreno
- Center of Mathematics, Computation and Cognition, Universidade Federal do ABC, Santo André, Brazil
| | - Claudinei Eduardo Biazoli
- Center of Mathematics, Computation and Cognition, Universidade Federal do ABC, Santo André, Brazil; Department of Psychology, School of Biological and Chemical Sciences, Queen Mary University of London, UK
| | - Abrahão Fontes Baptista
- Center of Mathematics, Computation and Cognition, Universidade Federal do ABC, Santo André, Brazil; Laboratory of Medical Investigations 54 (LIM-54), Universidade de São Paulo, São Paulo, Brazil; NAPeN Network (Rede de Núcleos de Assistência e Pesquisa em Neuromodulação), Brazil; Brazilian Institute of Neuroscience and Neurotechnology (BRAINN/CEPID-FAPESP), University of Campinas, Campinas, São Paulo, Brazil
| | - Lucas Remoaldo Trambaiolli
- McLean Hospital, Harvard Medical School, Boston, USA; School of Medicine and Dentistry, University of Rochester, Rochester, USA.
| |
Collapse
|
34
|
Baniqued PDE, Stanyer EC, Awais M, Alazmani A, Jackson AE, Mon-Williams MA, Mushtaq F, Holt RJ. Brain-computer interface robotics for hand rehabilitation after stroke: a systematic review. J Neuroeng Rehabil 2021; 18:15. [PMID: 33485365 PMCID: PMC7825186 DOI: 10.1186/s12984-021-00820-8] [Citation(s) in RCA: 57] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2019] [Accepted: 01/12/2021] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND Hand rehabilitation is core to helping stroke survivors regain activities of daily living. Recent studies have suggested that the use of electroencephalography-based brain-computer interfaces (BCI) can promote this process. Here, we report the first systematic examination of the literature on the use of BCI-robot systems for the rehabilitation of fine motor skills associated with hand movement and profile these systems from a technical and clinical perspective. METHODS A search for January 2010-October 2019 articles using Ovid MEDLINE, Embase, PEDro, PsycINFO, IEEE Xplore and Cochrane Library databases was performed. The selection criteria included BCI-hand robotic systems for rehabilitation at different stages of development involving tests on healthy participants or people who have had a stroke. Data fields include those related to study design, participant characteristics, technical specifications of the system, and clinical outcome measures. RESULTS 30 studies were identified as eligible for qualitative review and among these, 11 studies involved testing a BCI-hand robot on chronic and subacute stroke patients. Statistically significant improvements in motor assessment scores relative to controls were observed for three BCI-hand robot interventions. The degree of robot control for the majority of studies was limited to triggering the device to perform grasping or pinching movements using motor imagery. Most employed a combination of kinaesthetic and visual response via the robotic device and display screen, respectively, to match feedback to motor imagery. CONCLUSION 19 out of 30 studies on BCI-robotic systems for hand rehabilitation report systems at prototype or pre-clinical stages of development. We identified large heterogeneity in reporting and emphasise the need to develop a standard protocol for assessing technical and clinical outcomes so that the necessary evidence base on efficiency and efficacy can be developed.
Collapse
Affiliation(s)
| | - Emily C Stanyer
- School of Psychology, University of Leeds, Leeds, LS2 9JZ, UK
| | - Muhammad Awais
- School of Psychology, University of Leeds, Leeds, LS2 9JZ, UK
| | - Ali Alazmani
- School of Mechanical Engineering, University of Leeds, Leeds, LS2 9JT, UK
| | - Andrew E Jackson
- School of Mechanical Engineering, University of Leeds, Leeds, LS2 9JT, UK
| | | | - Faisal Mushtaq
- School of Psychology, University of Leeds, Leeds, LS2 9JZ, UK.
| | - Raymond J Holt
- School of Mechanical Engineering, University of Leeds, Leeds, LS2 9JT, UK
| |
Collapse
|
35
|
Hehenberger L, Sburlea AI, Müller-Putz GR. Assessing the impact of vibrotactile kinaesthetic feedback on electroencephalographic signals in a center-out task. J Neural Eng 2020; 17:056032. [DOI: 10.1088/1741-2552/abb069] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
36
|
Fleury M, Lioi G, Barillot C, Lécuyer A. A Survey on the Use of Haptic Feedback for Brain-Computer Interfaces and Neurofeedback. Front Neurosci 2020; 14:528. [PMID: 32655347 PMCID: PMC7325479 DOI: 10.3389/fnins.2020.00528] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2020] [Accepted: 04/28/2020] [Indexed: 11/23/2022] Open
Abstract
Neurofeedback (NF) and brain-computer interface (BCI) applications rely on the registration and real-time feedback of individual patterns of brain activity with the aim of achieving self-regulation of specific neural substrates or control of external devices. These approaches have historically employed visual stimuli. However, in some cases vision is unsuitable or inadequately engaging. Other sensory modalities, such as auditory or haptic feedback have been explored, and multisensory stimulation is expected to improve the quality of the interaction loop. Moreover, for motor imagery tasks, closing the sensorimotor loop through haptic feedback may be relevant for motor rehabilitation applications, as it can promote plasticity mechanisms. This survey reviews the various haptic technologies and describes their application to BCIs and NF. We identify major trends in the use of haptic interfaces for BCI and NF systems and discuss crucial aspects that could motivate further studies.
Collapse
Affiliation(s)
- Mathis Fleury
- University of Rennes 1, INRIA, EMPENN & HYBRID, Rennes, France
| | | | | | | |
Collapse
|
37
|
Neural Representation of Observed, Imagined, and Attempted Grasping Force in Motor Cortex of Individuals with Chronic Tetraplegia. Sci Rep 2020; 10:1429. [PMID: 31996696 PMCID: PMC6989675 DOI: 10.1038/s41598-020-58097-1] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2019] [Accepted: 01/07/2020] [Indexed: 12/15/2022] Open
Abstract
Hybrid kinetic and kinematic intracortical brain-computer interfaces (iBCIs) have the potential to restore functional grasping and object interaction capabilities in individuals with tetraplegia. This requires an understanding of how kinetic information is represented in neural activity, and how this representation is affected by non-motor parameters such as volitional state (VoS), namely, whether one observes, imagines, or attempts an action. To this end, this work investigates how motor cortical neural activity changes when three human participants with tetraplegia observe, imagine, and attempt to produce three discrete hand grasping forces with the dominant hand. We show that force representation follows the same VoS-related trends as previously shown for directional arm movements; namely, that attempted force production recruits more neural activity compared to observed or imagined force production. Additionally, VoS-modulated neural activity to a greater extent than grasping force. Neural representation of forces was lower than expected, possibly due to compromised somatosensory pathways in individuals with tetraplegia, which have been shown to influence motor cortical activity. Nevertheless, attempted forces (but not always observed or imagined forces) could be decoded significantly above chance, thereby potentially providing relevant information towards the development of a hybrid kinetic and kinematic iBCI.
Collapse
|
38
|
Missiroli F, Barsotti M, Leonardis D, Gabardi M, Rosati G, Frisoli A. Haptic Stimulation for Improving Training of a Motor Imagery BCI Developed for a Hand-Exoskeleton in Rehabilitation. IEEE Int Conf Rehabil Robot 2020; 2019:1127-1132. [PMID: 31374781 DOI: 10.1109/icorr.2019.8779370] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
The use of robotic devices to provide active motor support and sensory feedback of ongoing motor intention, by means of a Brain Computer Interface (BCI), has received growing support by recent literature, with particular focus on neurorehabilitation therapies. At the same time, performance in the use of the BCI has become a more critical factor, since it directly influences congruency and consistency of the provided sensory feedback. As motor imagery is the mental simulation of a given movement without depending on residual function, training of patients in the use of motor imagery BCI can be extended beyond each rehabilitation session, and practiced by using simpler devices than rehabilitation robots available in the hospital. In this work, we investigated the use of haptic stimulation provided by vibrating electromagnetic motors to enhance BCI system training. The BCI is based on motor imagery of hand grasping and designed to operate a hand exoskeleton. We investigated whether haptic stimulation at fingerpads proves to be more effective than stimulation at wrist, already experimented in literature, due to the higher density of mechano-receptors. Our results did not show significant differences between the two body locations in BCI performance, yet a wider and more stable event-relateddesynchronization appeared for the finger-located stimulation. Future investigations will put in relation training with haptic feedback at fingerpads with BCI performance using the handexoskeleton, in grasping tasks that naturally involve haptic feedback at fingerpads.
Collapse
|
39
|
Ray AM, Figueiredo TDC, López-Larraz E, Birbaumer N, Ramos-Murguialday A. Brain oscillatory activity as a biomarker of motor recovery in chronic stroke. Hum Brain Mapp 2019; 41:1296-1308. [PMID: 31778265 PMCID: PMC7268060 DOI: 10.1002/hbm.24876] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2019] [Revised: 10/21/2019] [Accepted: 11/13/2019] [Indexed: 12/31/2022] Open
Abstract
In the present work, we investigated the relationship of oscillatory sensorimotor brain activity to motor recovery. The neurophysiological data of 30 chronic stroke patients with severe upper‐limb paralysis are the basis of the observational study presented here. These patients underwent an intervention including movement training based on combined brain–machine interfaces and physiotherapy of several weeks recorded in a double‐blinded randomized clinical trial. We analyzed the alpha oscillations over the motor cortex of 22 of these patients employing multilevel linear predictive modeling. We identified a significant correlation between the evolution of the alpha desynchronization during rehabilitative intervention and clinical improvement. Moreover, we observed that the initial alpha desynchronization conditions its modulation during intervention: Patients showing a strong alpha desynchronization at the beginning of the training improved if they increased their alpha desynchronization. Patients showing a small alpha desynchronization at initial training stages improved if they decreased it further on both hemispheres. In all patients, a progressive shift of desynchronization toward the ipsilesional hemisphere correlates significantly with clinical improvement regardless of lesion location. The results indicate that initial alpha desynchronization might be key for stratification of patients undergoing BMI interventions and that its interhemispheric balance plays an important role in motor recovery.
Collapse
Affiliation(s)
- Andreas M Ray
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Thiago D C Figueiredo
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Eduardo López-Larraz
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Niels Birbaumer
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Ander Ramos-Murguialday
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany.,TECNALIA, Health Department, Neural Engineering Laboratory, San Sebastián, Spain
| |
Collapse
|
40
|
Calabrò RS, Accorinti M, Porcari B, Carioti L, Ciatto L, Billeri L, Andronaco VA, Galletti F, Filoni S, Naro A. Does hand robotic rehabilitation improve motor function by rebalancing interhemispheric connectivity after chronic stroke? Encouraging data from a randomised-clinical-trial. Clin Neurophysiol 2019; 130:767-780. [DOI: 10.1016/j.clinph.2019.02.013] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2018] [Revised: 01/28/2019] [Accepted: 02/13/2019] [Indexed: 01/16/2023]
|
41
|
Penaloza CI, Alimardani M, Nishio S. Android Feedback-Based Training Modulates Sensorimotor Rhythms During Motor Imagery. IEEE Trans Neural Syst Rehabil Eng 2019. [PMID: 29522410 DOI: 10.1109/tnsre.2018.2792481] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
EEG-based brain computer interface (BCI) systems have demonstrated potential to assist patients with devastating motor paralysis conditions. However, there is great interest in shifting the BCI trend toward applications aimed at healthy users. Although BCI operation depends on technological factors (i.e., EEG pattern classification algorithm) and human factors (i.e., how well the person can generate good quality EEG patterns), it is the latter that is least investigated. In order to control a motor imagery-based BCI, users need to learn to modulate their sensorimotor brain rhythms by practicing motor imagery using a classical training protocol with an abstract visual feedback. In this paper, we investigate a different BCI training protocol using a human-like android robot (Geminoid HI-2) to provide realistic visual feedback. The proposed training protocol addresses deficiencies of the classical approach and takes the advantage of body-abled user capabilities. Experimental results suggest that android feedback-based BCI training improves the modulation of sensorimotor rhythms during motor imagery task. Moreover, we discuss how the influence of body ownership transfer illusion toward the android might have an effect on the modulation of event-related desynchronization/synchronization activity.
Collapse
|
42
|
Ramos-Murguialday A, Curado MR, Broetz D, Yilmaz Ö, Brasil FL, Liberati G, Garcia-Cossio E, Cho W, Caria A, Cohen LG, Birbaumer N. Brain-Machine Interface in Chronic Stroke: Randomized Trial Long-Term Follow-up. Neurorehabil Neural Repair 2019; 33:188-198. [PMID: 30722727 DOI: 10.1177/1545968319827573] [Citation(s) in RCA: 47] [Impact Index Per Article: 9.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND Brain-machine interfaces (BMIs) have been recently proposed as a new tool to induce functional recovery in stroke patients. OBJECTIVE Here we evaluated long-term effects of BMI training and physiotherapy in motor function of severely paralyzed chronic stroke patients 6 months after intervention. METHODS A total of 30 chronic stroke patients with severe hand paresis from our previous study were invited, and 28 underwent follow-up assessments. BMI training included voluntary desynchronization of ipsilesional EEG-sensorimotor rhythms triggering paretic upper-limb movements via robotic orthoses (experimental group, n = 16) or random orthoses movements (sham group, n = 12). Both groups received identical physiotherapy following BMI sessions and a home-based training program after intervention. Upper-limb motor assessment scores, electromyography (EMG), and functional magnetic resonance imaging (fMRI) were assessed before (Pre), immediately after (Post1), and 6 months after intervention (Post2). RESULTS The experimental group presented with upper-limb Fugl-Meyer assessment (cFMA) scores significantly higher in Post2 (13.44 ± 1.96) as compared with the Pre session (11.16 ± 1.73; P = .015) and no significant changes between Post1 and Post2 sessions. The Sham group showed no significant changes on cFMA scores. Ashworth scores and EMG activity in both groups increased from Post1 to Post2. Moreover, fMRI-BOLD laterality index showed no significant difference from Pre or Post1 to Post2 sessions. CONCLUSIONS BMI-based rehabilitation promotes long-lasting improvements in motor function of chronic stroke patients with severe paresis and represents a promising strategy in severe stroke neurorehabilitation.
Collapse
Affiliation(s)
- Ander Ramos-Murguialday
- 1 University of Tubingen, Tübingen, Germany.,2 TECNALIA Health Technologies, Neurotechnology Laboratory, San Sebastian, Spain
| | - Marco R Curado
- 1 University of Tubingen, Tübingen, Germany.,3 AbbVie Pharmaceuticals, Ludwigshafen, Germany
| | | | - Özge Yilmaz
- 1 University of Tubingen, Tübingen, Germany.,4 Bahcesehir University, Istanbul, Turkey
| | - Fabricio L Brasil
- 1 University of Tubingen, Tübingen, Germany.,5 Santos Dumont Institute, Macaiba, Brazil
| | - Giulia Liberati
- 1 University of Tubingen, Tübingen, Germany.,6 Université catholique de Louvain, Brussels, Belgium
| | - Eliana Garcia-Cossio
- 1 University of Tubingen, Tübingen, Germany.,7 NeuroCare Group, Mental Health Care, Munich, Germany
| | - Woosang Cho
- 1 University of Tubingen, Tübingen, Germany.,8 g.tec Medical Engineering GmbH, Schiedlberg, Austria
| | | | | | - Niels Birbaumer
- 1 University of Tubingen, Tübingen, Germany.,10 WYSS-Center of Bio- and Neuroengineering, Geneva, Switzerland
| |
Collapse
|
43
|
Abiri R, Borhani S, Sellers EW, Jiang Y, Zhao X. A comprehensive review of EEG-based brain–computer interface paradigms. J Neural Eng 2019; 16:011001. [DOI: 10.1088/1741-2552/aaf12e] [Citation(s) in RCA: 270] [Impact Index Per Article: 54.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
|
44
|
Chowdhury A, Raza H, Meena YK, Dutta A, Prasad G. Online Covariate Shift Detection-Based Adaptive Brain–Computer Interface to Trigger Hand Exoskeleton Feedback for Neuro-Rehabilitation. IEEE Trans Cogn Dev Syst 2018. [DOI: 10.1109/tcds.2017.2787040] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
45
|
Spüler M, López-Larraz E, Ramos-Murguialday A. On the design of EEG-based movement decoders for completely paralyzed stroke patients. J Neuroeng Rehabil 2018; 15:110. [PMID: 30458838 PMCID: PMC6247630 DOI: 10.1186/s12984-018-0438-z] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2018] [Accepted: 10/17/2018] [Indexed: 11/24/2022] Open
Abstract
Background Brain machine interface (BMI) technology has demonstrated its efficacy for rehabilitation of paralyzed chronic stroke patients. The critical component in BMI-training consists of the associative connection (contingency) between the intention and the feedback provided. However, the relationship between the BMI design and its performance in stroke patients is still an open question. Methods In this study we compare different methodologies to design a BMI for rehabilitation and evaluate their effects on movement intention decoding performance. We analyze the data of 37 chronic stroke patients who underwent 4 weeks of BMI intervention with different types of association between their brain activity and the proprioceptive feedback. We simulate the pseudo-online performance that a BMI would have under different conditions, varying: (1) the cortical source of activity (i.e., ipsilesional, contralesional, bihemispheric), (2) the type of spatial filter applied, (3) the EEG frequency band, (4) the type of classifier; and also evaluated the use of residual EMG activity to decode the movement intentions. Results We observed a significant influence of the different BMI designs on the obtained performances. Our results revealed that using bihemispheric beta activity with a common average reference and an adaptive support vector machine led to the best classification results. Furthermore, the decoding results based on brain activity were significantly higher than those based on muscle activity. Conclusions This paper underscores the relevance of the different parameters used to decode movement, using EEG in severely paralyzed stroke patients. We demonstrated significant differences in performance for the different designs, which supports further research that should elucidate if those approaches leading to higher accuracies also induce higher motor recovery in paralyzed stroke patients.
Collapse
Affiliation(s)
- Martin Spüler
- Department of Computer Engineering, Wilhelm-Schickard-Institute, University of Tübingen, Sand 14, 72076, Tübingen, Germany
| | - Eduardo López-Larraz
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Silcherstr. 5, 72076, Tübingen, Germany
| | - Ander Ramos-Murguialday
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Silcherstr. 5, 72076, Tübingen, Germany. .,TECNALIA, Health Technologies, Neural Enginering Laboratory, Mikeletegi Pasalekua 1, 20009, San Sebastian, Spain.
| |
Collapse
|
46
|
López-Larraz E, Sarasola-Sanz A, Irastorza-Landa N, Birbaumer N, Ramos-Murguialday A. Brain-machine interfaces for rehabilitation in stroke: A review. NeuroRehabilitation 2018; 43:77-97. [PMID: 30056435 DOI: 10.3233/nre-172394] [Citation(s) in RCA: 55] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
BACKGROUND Motor paralysis after stroke has devastating consequences for the patients, families and caregivers. Although therapies have improved in the recent years, traditional rehabilitation still fails in patients with severe paralysis. Brain-machine interfaces (BMI) have emerged as a promising tool to guide motor rehabilitation interventions as they can be applied to patients with no residual movement. OBJECTIVE This paper reviews the efficiency of BMI technologies to facilitate neuroplasticity and motor recovery after stroke. METHODS We provide an overview of the existing rehabilitation therapies for stroke, the rationale behind the use of BMIs for motor rehabilitation, the current state of the art and the results achieved so far with BMI-based interventions, as well as the future perspectives of neural-machine interfaces. RESULTS Since the first pilot study by Buch and colleagues in 2008, several controlled clinical studies have been conducted, demonstrating the efficacy of BMIs to facilitate functional recovery in completely paralyzed stroke patients with noninvasive technologies such as the electroencephalogram (EEG). CONCLUSIONS Despite encouraging results, motor rehabilitation based on BMIs is still in a preliminary stage, and further improvements are required to boost its efficacy. Invasive and hybrid approaches are promising and might set the stage for the next generation of stroke rehabilitation therapies.
Collapse
Affiliation(s)
- E López-Larraz
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - A Sarasola-Sanz
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany.,International Max Planck Research School (IMPRS) for Cognitive and Systems Neuroscience, University of Tübingen, Tübingen, Germany.,Neurotechnology, Tecnalia Research & Innovation, San Sebastián, Spain
| | - N Irastorza-Landa
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany.,International Max Planck Research School (IMPRS) for Cognitive and Systems Neuroscience, University of Tübingen, Tübingen, Germany.,IKERBASQUE, Basque Foundation for Science, Bilbao, Spain
| | - N Birbaumer
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany.,Wyss Center for Bio and Neuro Engineering, Geneva, Switzerland
| | - A Ramos-Murguialday
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany.,Neurotechnology, Tecnalia Research & Innovation, San Sebastián, Spain
| |
Collapse
|
47
|
EEG-Based Control for Upper and Lower Limb Exoskeletons and Prostheses: A Systematic Review. SENSORS 2018; 18:s18103342. [PMID: 30301238 PMCID: PMC6211123 DOI: 10.3390/s18103342] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/14/2018] [Revised: 09/12/2018] [Accepted: 09/28/2018] [Indexed: 12/13/2022]
Abstract
Electroencephalography (EEG) signals have great impact on the development of assistive rehabilitation devices. These signals are used as a popular tool to investigate the functions and the behavior of the human motion in recent research. The study of EEG-based control of assistive devices is still in early stages. Although the EEG-based control of assistive devices has attracted a considerable level of attention over the last few years, few studies have been carried out to systematically review these studies, as a means of offering researchers and experts a comprehensive summary of the present, state-of-the-art EEG-based control techniques used for assistive technology. Therefore, this research has three main goals. The first aim is to systematically gather, summarize, evaluate and synthesize information regarding the accuracy and the value of previous research published in the literature between 2011 and 2018. The second goal is to extensively report on the holistic, experimental outcomes of this domain in relation to current research. It is systematically performed to provide a wealthy image and grounded evidence of the current state of research covering EEG-based control for assistive rehabilitation devices to all the experts and scientists. The third goal is to recognize the gap of knowledge that demands further investigation and to recommend directions for future research in this area.
Collapse
|
48
|
López-Larraz E, Figueiredo TC, Insausti-Delgado A, Ziemann U, Birbaumer N, Ramos-Murguialday A. Event-related desynchronization during movement attempt and execution in severely paralyzed stroke patients: An artifact removal relevance analysis. Neuroimage Clin 2018; 20:972-986. [PMID: 30312940 PMCID: PMC6180341 DOI: 10.1016/j.nicl.2018.09.035] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2018] [Revised: 07/27/2018] [Accepted: 09/25/2018] [Indexed: 02/04/2023]
Abstract
The electroencephalogram (EEG) constitutes a relevant tool to study neural dynamics and to develop brain-machine interfaces (BMI) for rehabilitation of patients with paralysis due to stroke. However, the EEG is easily contaminated by artifacts of physiological origin, which can pollute the measured cortical activity and bias the interpretations of such data. This is especially relevant when recording EEG of stroke patients while they try to move their paretic limbs, since they generate more artifacts due to compensatory activity. In this paper, we study how physiological artifacts (i.e., eye movements, motion artifacts, muscle artifacts and compensatory movements with the other limb) can affect EEG activity of stroke patients. Data from 31 severely paralyzed stroke patients performing/attempting grasping movements with their healthy/paralyzed hand were analyzed offline. We estimated the cortical activation as the event-related desynchronization (ERD) of sensorimotor rhythms and used it to detect the movements with a pseudo-online simulated BMI. Automated state-of-the-art methods (linear regression to remove ocular contaminations and statistical thresholding to reject the other types of artifacts) were used to minimize the influence of artifacts. The effect of artifact reduction was quantified in terms of ERD and BMI performance. The results reveal a significant contamination affecting the EEG, being involuntary muscle activity the main source of artifacts. Artifact reduction helped extracting the oscillatory signatures of motor tasks, isolating relevant information from noise and revealing a more prominent ERD activity. Lower BMI performances were obtained when artifacts were eliminated from the training datasets. This suggests that artifacts produce an optimistic bias that improves theoretical accuracy but may result in a poor link between task-related oscillatory activity and BMI peripheral feedback. With a clinically relevant dataset of stroke patients, we evidence the need of appropriate methodologies to remove artifacts from EEG datasets to obtain accurate estimations of the motor brain activity.
Collapse
Affiliation(s)
- Eduardo López-Larraz
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Germany.
| | - Thiago C Figueiredo
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Germany
| | - Ainhoa Insausti-Delgado
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Germany; International Max Planck Research School (IMPRS) for Cognitive and Systems Neuroscience, Tübingen, Germany; IKERBASQUE, Basque Foundation for Science, Bilbao, Spain
| | - Ulf Ziemann
- Department of Neurology & Stroke, and Hertie Institute for Clinical Brain Research, University of Tübingen, Germany
| | - Niels Birbaumer
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Germany; Wyss Institute for Bio- and Neuroengineering, Genève, Switzerland
| | - Ander Ramos-Murguialday
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Germany; Neural Engineering Laboratory, Health Department, TECNALIA, San Sebastián, Spain
| |
Collapse
|
49
|
Norman SL, McFarland DJ, Miner A, Cramer SC, Wolbrecht ET, Wolpaw JR, Reinkensmeyer DJ. Controlling pre-movement sensorimotor rhythm can improve finger extension after stroke. J Neural Eng 2018; 15:056026. [PMID: 30063219 PMCID: PMC6158016 DOI: 10.1088/1741-2552/aad724] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Brain-computer interface (BCI) technology is attracting increasing interest as a tool for enhancing recovery of motor function after stroke, yet the optimal way to apply this technology is unknown. Here, we studied the immediate and therapeutic effects of BCI-based training to control pre-movement sensorimotor rhythm (SMR) amplitude on robot-assisted finger extension in people with stroke. APPROACH Eight people with moderate to severe hand impairment due to chronic stroke completed a four-week three-phase protocol during which they practiced finger extension with assistance from the FINGER robotic exoskeleton. In Phase 1, we identified spatiospectral SMR features for each person that correlated with the intent to extend the index and/or middle finger(s). In Phase 2, the participants learned to increase or decrease SMR features given visual feedback, without movement. In Phase 3, the participants were cued to increase or decrease their SMR features, and when successful, were then cued to immediately attempt to extend the finger(s) with robot assistance. MAIN RESULTS Of the four participants that achieved SMR control in Phase 2, three initiated finger extensions with a reduced reaction time after decreasing (versus increasing) pre-movement SMR amplitude during Phase 3. Two also extended at least one of their fingers more forcefully after decreasing pre-movement SMR amplitude. Hand function, measured by the box and block test (BBT), improved by 7.3 ± 7.5 blocks versus 3.5 ± 3.1 blocks in those with and without SMR control, respectively. Higher BBT scores at baseline correlated with a larger change in BBT score. SIGNIFICANCE These results suggest that learning to control person-specific pre-movement SMR features associated with finger extension can improve finger extension ability after stroke for some individuals. These results merit further investigation in a rehabilitation context.
Collapse
Affiliation(s)
- S L Norman
- University of California Irvine, Irvine, CA, United States of America
| | | | | | | | | | | | | |
Collapse
|
50
|
Tabernig CB, Lopez CA, Carrere LC, Spaich EG, Ballario CH. Neurorehabilitation therapy of patients with severe stroke based on functional electrical stimulation commanded by a brain computer interface. J Rehabil Assist Technol Eng 2018; 5:2055668318789280. [PMID: 31191948 PMCID: PMC6453036 DOI: 10.1177/2055668318789280] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2017] [Accepted: 06/21/2018] [Indexed: 02/06/2023] Open
Abstract
Introduction Brain computer interface is an emerging technology to treat the sequelae of stroke. The purpose of this study was to explore the motor imagery related desynchronization of sensorimotor rhythms of stroke patients and to assess the efficacy of an upper limb neurorehabilitation therapy based on functional electrical stimulation controlled by a brain computer interface. Methods Eight severe chronic stroke patients were recruited. The study consisted of two stages: screening and therapy. During screening, the ability of patients to desynchronize the contralateral oscillatory sensorimotor rhythms by motor imagery of the most affected hand was assessed. In the second stage, a therapeutic intervention was performed. It involved 20 sessions where an electrical stimulator was activated when the patient's cerebral activity related to motor imagery was detected. The upper limb was assessed, before and after the intervention, by the Fugl-Meyer score (primary outcome). Spasticity, motor activity, range of movement and quality of life were also evaluated (secondary outcomes). Results Desynchronization was identified in all screened patients. Significant post-treatment improvement (p < 0.05) was detected in the primary outcome measure and in the majority of secondary outcome scores. Conclusions The results suggest that the proposed therapy could be beneficial in the neurorehabilitation of stroke individuals.
Collapse
Affiliation(s)
- Carolina B Tabernig
- Laboratorio de Ingeniería en Rehabilitación e Investigaciones Neuromusculares y Sensoriales (LIRINS), Facultad de Ingeniería, Universidad Nacional de Entre Ríos, Oro Verde, Argentina
| | - Camila A Lopez
- Fundación Rosarina de Neuro-rehabilitación, Rosario, Argentina
| | - Lucía C Carrere
- Laboratorio de Ingeniería en Rehabilitación e Investigaciones Neuromusculares y Sensoriales (LIRINS), Facultad de Ingeniería, Universidad Nacional de Entre Ríos, Oro Verde, Argentina
| | - Erika G Spaich
- SMI®, Department of Health Science and Technology, Aalborg University, Denmark
| | | |
Collapse
|