1
|
Liu M, Li T, Zhang X, Yang Y, Zhou Z, Fu T. IMH-Net: a convolutional neural network for end-to-end EEG motor imagery classification. Comput Methods Biomech Biomed Engin 2024; 27:2175-2188. [PMID: 37936533 DOI: 10.1080/10255842.2023.2275244] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 10/15/2023] [Accepted: 10/17/2023] [Indexed: 11/09/2023]
Abstract
As the main component of Brain-computer interface (BCI) technology, the classification algorithm based on EEG has developed rapidly. The previous algorithms were often based on subject-dependent settings, resulting in BCI needing to be calibrated for new users. In this work, we propose IMH-Net, an end-to-end subject-independent model. The model first uses Inception blocks extracts the frequency domain features of the data, then further compresses the feature vectors to extract the spatial domain features, and finally learns the global information and classification through Multi-Head Attention mechanism. On the OpenBMI dataset, IMH-Net obtained 73.90 ± 13.10% accuracy and 73.09 ± 14.99% F1-score in subject-independent manner, which improved the accuracy by 1.96% compared with the comparison model. On the BCI competition IV dataset 2a, this model also achieved the highest accuracy and F1-score in subject-dependent manner. The IMH-Net model we proposed can improve the accuracy of subject-independent Motor Imagery (MI), and the robustness of the algorithm is high, which has strong practical value in the field of BCI.
Collapse
Affiliation(s)
- Menghao Liu
- Mechanical College, Shanghai Dianji University, Shanghai, China
| | - Tingting Li
- Department of Anesthesiology, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Xu Zhang
- Mechanical College, Shanghai Dianji University, Shanghai, China
| | - Yang Yang
- Shanghai Lanhui Medical Technology Co., Ltd, Shanghai, China
| | - Zhiyong Zhou
- Mechanical College, Shanghai Dianji University, Shanghai, China
| | - Tianhao Fu
- Mechanical College, Shanghai Dianji University, Shanghai, China
| |
Collapse
|
2
|
Blanco-Díaz CF, Guerrero-Mendez CD, Delisle-Rodriguez D, Jaramillo-Isaza S, Ruiz-Olaya AF, Frizera-Neto A, Ferreira de Souza A, Bastos-Filho T. Evaluation of temporal, spatial and spectral filtering in CSP-based methods for decoding pedaling-based motor tasks using EEG signals. Biomed Phys Eng Express 2024; 10:035003. [PMID: 38417162 DOI: 10.1088/2057-1976/ad2e35] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Accepted: 02/28/2024] [Indexed: 03/01/2024]
Abstract
Stroke is a neurological syndrome that usually causes a loss of voluntary control of lower/upper body movements, making it difficult for affected individuals to perform Activities of Daily Living (ADLs). Brain-Computer Interfaces (BCIs) combined with robotic systems, such as Motorized Mini Exercise Bikes (MMEB), have enabled the rehabilitation of people with disabilities by decoding their actions and executing a motor task. However, Electroencephalography (EEG)-based BCIs are affected by the presence of physiological and non-physiological artifacts. Thus, movement discrimination using EEG become challenging, even in pedaling tasks, which have not been well explored in the literature. In this study, Common Spatial Patterns (CSP)-based methods were proposed to classify pedaling motor tasks. To address this, Filter Bank Common Spatial Patterns (FBCSP) and Filter Bank Common Spatial-Spectral Patterns (FBCSSP) were implemented with different spatial filtering configurations by varying the time segment with different filter bank combinations for the three methods to decode pedaling tasks. An in-house EEG dataset during pedaling tasks was registered for 8 participants. As results, the best configuration corresponds to a filter bank with two filters (8-19 Hz and 19-30 Hz) using a time window between 1.5 and 2.5 s after the cue and implementing two spatial filters, which provide accuracy of approximately 0.81, False Positive Rates lower than 0.19, andKappaindex of 0.61. This work implies that EEG oscillatory patterns during pedaling can be accurately classified using machine learning. Therefore, our method can be applied in the rehabilitation context, such as MMEB-based BCIs, in the future.
Collapse
Affiliation(s)
- Cristian Felipe Blanco-Díaz
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo (UFES), 29075-910 Vitória, Brazil
- Faculty of Mechanical, Electronics and Biomedical Engineering, Antonio Nariño University, Bogotá D.C, Colombia
| | - Cristian David Guerrero-Mendez
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo (UFES), 29075-910 Vitória, Brazil
- Faculty of Mechanical, Electronics and Biomedical Engineering, Antonio Nariño University, Bogotá D.C, Colombia
| | | | | | - Andrés Felipe Ruiz-Olaya
- Faculty of Mechanical, Electronics and Biomedical Engineering, Antonio Nariño University, Bogotá D.C, Colombia
| | - Anselmo Frizera-Neto
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo (UFES), 29075-910 Vitória, Brazil
| | | | - Teodiano Bastos-Filho
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo (UFES), 29075-910 Vitória, Brazil
| |
Collapse
|
3
|
Ha J, Baek SC, Lim Y, Chung JH. Validation of cost-efficient EEG experimental setup for neural tracking in an auditory attention task. Sci Rep 2023; 13:22682. [PMID: 38114579 PMCID: PMC10730561 DOI: 10.1038/s41598-023-49990-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 12/14/2023] [Indexed: 12/21/2023] Open
Abstract
When individuals listen to speech, their neural activity phase-locks to the slow temporal rhythm, which is commonly referred to as "neural tracking". The neural tracking mechanism allows for the detection of an attended sound source in a multi-talker situation by decoding neural signals obtained by electroencephalography (EEG), known as auditory attention decoding (AAD). Neural tracking with AAD can be utilized as an objective measurement tool for diverse clinical contexts, and it has potential to be applied to neuro-steered hearing devices. To effectively utilize this technology, it is essential to enhance the accessibility of EEG experimental setup and analysis. The aim of the study was to develop a cost-efficient neural tracking system and validate the feasibility of neural tracking measurement by conducting an AAD task using an offline and real-time decoder model outside the soundproof environment. We devised a neural tracking system capable of conducting AAD experiments using an OpenBCI and Arduino board. Nine participants were recruited to assess the performance of the AAD using the developed system, which involved presenting competing speech signals in an experiment setting without soundproofing. As a result, the offline decoder model demonstrated an average performance of 90%, and real-time decoder model exhibited a performance of 78%. The present study demonstrates the feasibility of implementing neural tracking and AAD using cost-effective devices in a practical environment.
Collapse
Affiliation(s)
- Jiyeon Ha
- Department of HY-KIST Bio-Convergence, Hanyang University, Seoul, 04763, Korea
- Center for Intelligent & Interactive Robotics, Artificial Intelligence and Robot Institute, Korea Institute of Science and Technology, Seoul, 02792, Korea
| | - Seung-Cheol Baek
- Center for Intelligent & Interactive Robotics, Artificial Intelligence and Robot Institute, Korea Institute of Science and Technology, Seoul, 02792, Korea
- Research Group Neurocognition of Music and Language, Max Planck Institute for Empirical Aesthetics, 60322, Frankfurt\ Main, Germany
| | - Yoonseob Lim
- Department of HY-KIST Bio-Convergence, Hanyang University, Seoul, 04763, Korea.
- Center for Intelligent & Interactive Robotics, Artificial Intelligence and Robot Institute, Korea Institute of Science and Technology, Seoul, 02792, Korea.
| | - Jae Ho Chung
- Department of HY-KIST Bio-Convergence, Hanyang University, Seoul, 04763, Korea.
- Center for Intelligent & Interactive Robotics, Artificial Intelligence and Robot Institute, Korea Institute of Science and Technology, Seoul, 02792, Korea.
- Department of Otolaryngology-Head and Neck Surgery, College of Medicine, Hanyang University, Seoul, 04763, Korea.
- Department of Otolaryngology-Head and Neck Surgery, School of Medicine, Hanyang University, 222-Wangshimni-ro, Seongdong-gu, Seoul, 133-792, Korea.
| |
Collapse
|
4
|
Guerrero-Mendez CD, Blanco-Diaz CF, Rivera-Flor H, De Souza AF, Jaramillo-Isaza S, Ruiz-Olaya AF, Bastos-Filho TF. Coupling Effects of Cross-Corticomuscular Association during Object Manipulation Tasks on Different Haptic Sensations. NEUROSCI 2023; 4:195-210. [PMID: 39483199 PMCID: PMC11523752 DOI: 10.3390/neurosci4030018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2023] [Revised: 08/03/2023] [Accepted: 08/11/2023] [Indexed: 11/03/2024] Open
Abstract
The effects of corticomuscular connectivity during object manipulation tasks with different haptic sensations have not been quantitatively investigated. Connectivity analyses enable the study of cortical effects and muscle responses during movements, revealing communication pathways between the brain and muscles. This study aims to examine the corticomuscular connectivity of three Electroencephalography (EEG) channels and five muscles during object manipulation tasks involving contact surfaces of Sandpaper, Suede, and Silk. The analyses included 12 healthy subjects performing tasks with their right hand. Power-Based Connectivity (PBC) and Mutual Information (MI) measures were utilized to evaluate significant differences in connectivity between contact surfaces, EEG channels, muscles, and frequency bands. The research yielded the following findings: Suede contact surface exhibited higher connectivity; Mu and Gamma frequency bands exerted greater influence; significant connectivity was observed between the three EEG channels (C 3 ,C z ,C 4 ) and the Anterior Deltoid (AD) and Brachioradialis (B) muscles; and connectivity was primarily involved during active movement in the AD muscle compared to the resting state. These findings suggest potential implementation in motor rehabilitation for more complex movements using novel alternative training systems with high effectiveness.
Collapse
Affiliation(s)
- Cristian D Guerrero-Mendez
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo (UFES), Vitória 29075-910, Brazil; (C.F.B.-D.); (H.R.-F.); (T.F.B.-F.)
| | - Cristian F Blanco-Diaz
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo (UFES), Vitória 29075-910, Brazil; (C.F.B.-D.); (H.R.-F.); (T.F.B.-F.)
| | - Hamilton Rivera-Flor
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo (UFES), Vitória 29075-910, Brazil; (C.F.B.-D.); (H.R.-F.); (T.F.B.-F.)
| | - Alberto F De Souza
- Department of Informatics, Federal University of Espírito Santo (UFES), Vitória 29075-910, Brazil;
| | | | - Andres F Ruiz-Olaya
- Faculty of Mechanical, Electronic and Biomedical Engineering, Antonio Nariño University (UAN), Bogotá 110231, Colombia;
| | - Teodiano F Bastos-Filho
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo (UFES), Vitória 29075-910, Brazil; (C.F.B.-D.); (H.R.-F.); (T.F.B.-F.)
| |
Collapse
|
5
|
Saibene A, Caglioni M, Corchs S, Gasparini F. EEG-Based BCIs on Motor Imagery Paradigm Using Wearable Technologies: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:2798. [PMID: 36905004 PMCID: PMC10007053 DOI: 10.3390/s23052798] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Revised: 02/21/2023] [Accepted: 02/28/2023] [Indexed: 06/18/2023]
Abstract
In recent decades, the automatic recognition and interpretation of brain waves acquired by electroencephalographic (EEG) technologies have undergone remarkable growth, leading to a consequent rapid development of brain-computer interfaces (BCIs). EEG-based BCIs are non-invasive systems that allow communication between a human being and an external device interpreting brain activity directly. Thanks to the advances in neurotechnologies, and especially in the field of wearable devices, BCIs are now also employed outside medical and clinical applications. Within this context, this paper proposes a systematic review of EEG-based BCIs, focusing on one of the most promising paradigms based on motor imagery (MI) and limiting the analysis to applications that adopt wearable devices. This review aims to evaluate the maturity levels of these systems, both from the technological and computational points of view. The selection of papers has been performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), leading to 84 publications considered in the last ten years (from 2012 to 2022). Besides technological and computational aspects, this review also aims to systematically list experimental paradigms and available datasets in order to identify benchmarks and guidelines for the development of new applications and computational models.
Collapse
Affiliation(s)
- Aurora Saibene
- Department of Informatics, Systems and Communication, University of Milano-Bicocca, Viale Sarca 336, 20126 Milano, Italy
- NeuroMI, Milan Center for Neuroscience, Piazza dell’Ateneo Nuovo 1, 20126 Milano, Italy
| | - Mirko Caglioni
- Department of Informatics, Systems and Communication, University of Milano-Bicocca, Viale Sarca 336, 20126 Milano, Italy
| | - Silvia Corchs
- NeuroMI, Milan Center for Neuroscience, Piazza dell’Ateneo Nuovo 1, 20126 Milano, Italy
- Department of Theoretical and Applied Sciences, University of Insubria, Via J. H. Dunant 3, 21100 Varese, Italy
| | - Francesca Gasparini
- Department of Informatics, Systems and Communication, University of Milano-Bicocca, Viale Sarca 336, 20126 Milano, Italy
- NeuroMI, Milan Center for Neuroscience, Piazza dell’Ateneo Nuovo 1, 20126 Milano, Italy
| |
Collapse
|
6
|
Muñoz VF. Sensors Technology for Medical Robotics. SENSORS (BASEL, SWITZERLAND) 2022; 22:9290. [PMID: 36501991 PMCID: PMC9736968 DOI: 10.3390/s22239290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Accepted: 11/21/2022] [Indexed: 06/17/2023]
Abstract
There are many definitions for the concept of a robot, perhaps too many; it has even been said that we do not know how to define them, but when we see a robot, we identify it [...].
Collapse
Affiliation(s)
- Víctor F Muñoz
- Department of System Engineering and Automation C/Severo Ochoa 4, Universidad de Malaga, 29590 Malaga, Spain
| |
Collapse
|
7
|
A Comprehensive Review of Endogenous EEG-Based BCIs for Dynamic Device Control. SENSORS 2022; 22:s22155802. [PMID: 35957360 PMCID: PMC9370865 DOI: 10.3390/s22155802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 07/23/2022] [Accepted: 07/30/2022] [Indexed: 11/28/2022]
Abstract
Electroencephalogram (EEG)-based brain–computer interfaces (BCIs) provide a novel approach for controlling external devices. BCI technologies can be important enabling technologies for people with severe mobility impairment. Endogenous paradigms, which depend on user-generated commands and do not need external stimuli, can provide intuitive control of external devices. This paper discusses BCIs to control various physical devices such as exoskeletons, wheelchairs, mobile robots, and robotic arms. These technologies must be able to navigate complex environments or execute fine motor movements. Brain control of these devices presents an intricate research problem that merges signal processing and classification techniques with control theory. In particular, obtaining strong classification performance for endogenous BCIs is challenging, and EEG decoder output signals can be unstable. These issues present myriad research questions that are discussed in this review paper. This review covers papers published until the end of 2021 that presented BCI-controlled dynamic devices. It discusses the devices controlled, EEG paradigms, shared control, stabilization of the EEG signal, traditional machine learning and deep learning techniques, and user experience. The paper concludes with a discussion of open questions and avenues for future work.
Collapse
|
8
|
Covantes-Osuna C, López JB, Paredes O, Vélez-Pérez H, Romo-Vázquez R. Multilayer Network Approach in EEG Motor Imagery with an Adaptive Threshold. SENSORS 2021; 21:s21248305. [PMID: 34960399 PMCID: PMC8704651 DOI: 10.3390/s21248305] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/02/2021] [Revised: 12/05/2021] [Accepted: 12/06/2021] [Indexed: 12/15/2022]
Abstract
The brain has been understood as an interconnected neural network generally modeled as a graph to outline the functional topology and dynamics of brain processes. Classic graph modeling is based on single-layer models that constrain the traits conveyed to trace brain topologies. Multilayer modeling, in contrast, makes it possible to build whole-brain models by integrating features of various kinds. The aim of this work was to analyze EEG dynamics studies while gathering motor imagery data through single-layer and multilayer network modeling. The motor imagery database used consists of 18 EEG recordings of four motor imagery tasks: left hand, right hand, feet, and tongue. Brain connectivity was estimated by calculating the coherence adjacency matrices from each electrophysiological band (δ, θ, α and β) from brain areas and then embedding them by considering each band as a single-layer graph and a layer of the multilayer brain models. Constructing a reliable multilayer network topology requires a threshold that distinguishes effective connections from spurious ones. For this reason, two thresholds were implemented, the classic fixed (average) one and Otsu’s version. The latter is a new proposal for an adaptive threshold that offers reliable insight into brain topology and dynamics. Findings from the brain network models suggest that frontal and parietal brain regions are involved in motor imagery tasks.
Collapse
|