1
|
Zolfaghari S, Yousefi Rezaii T, Meshgini S. Applying Common Spatial Pattern and Convolutional Neural Network to Classify Movements via EEG Signals. Clin EEG Neurosci 2024:15500594241234836. [PMID: 38523306 DOI: 10.1177/15500594241234836] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/26/2024]
Abstract
Developing an electroencephalography (EEG)-based brain-computer interface (BCI) system is crucial to enhancing the control of external prostheses by accurately distinguishing various movements through brain signals. This innovation can provide comfortable circumstances for the populace who have movement disabilities. This study combined the most prospering methods used in BCI systems, including one-versus-rest common spatial pattern (OVR-CSP) and convolutional neural network (CNN), to automatically extract features and classify eight different movements of the shoulder, wrist, and elbow via EEG signals. The number of subjects who participated in the experiment was 10, and their EEG signals were recorded while performing movements at fast and slow speeds. We used preprocessing techniques before transforming EEG signals into another space by OVR-CSP, followed by sending signals into the CNN architecture consisting of four convolutional layers. Moreover, we extracted feature vectors after applying OVR-CSP and considered them as inputs to KNN, SVM, and MLP classifiers. Then, the performance of these classifiers was compared with the CNN method. The results demonstrated that the classification of eight movements using the proposed CNN architecture obtained an average accuracy of 97.65% for slow movements and 96.25% for fast movements in the subject-independent model. This method outperformed other classifiers with a substantial difference; ergo, it can be useful in improving BCI systems for better control of prostheses.
Collapse
Affiliation(s)
- Sepideh Zolfaghari
- Department of Biomedical Engineering, Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran
| | - Tohid Yousefi Rezaii
- Department of Biomedical Engineering, Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran
| | - Saeed Meshgini
- Department of Biomedical Engineering, Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran
| |
Collapse
|
2
|
Khan H, Khadka R, Sultan MS, Yazidi A, Ombao H, Mirtaheri P. Unleashing the potential of fNIRS with machine learning: classification of fine anatomical movements to empower future brain-computer interface. Front Hum Neurosci 2024; 18:1354143. [PMID: 38435744 PMCID: PMC10904609 DOI: 10.3389/fnhum.2024.1354143] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2023] [Accepted: 01/31/2024] [Indexed: 03/05/2024] Open
Abstract
In this study, we explore the potential of using functional near-infrared spectroscopy (fNIRS) signals in conjunction with modern machine-learning techniques to classify specific anatomical movements to increase the number of control commands for a possible fNIRS-based brain-computer interface (BCI) applications. The study focuses on novel individual finger-tapping, a well-known task in fNIRS and fMRI studies, but limited to left/right or few fingers. Twenty-four right-handed participants performed the individual finger-tapping task. Data were recorded by using sixteen sources and detectors placed over the motor cortex according to the 10-10 international system. The event's average oxygenated Δ HbO and deoxygenated Δ HbR hemoglobin data were utilized as features to assess the performance of diverse machine learning (ML) models in a challenging multi-class classification setting. These methods include LDA, QDA, MNLR, XGBoost, and RF. A new DL-based model named "Hemo-Net" has been proposed which consists of multiple parallel convolution layers with different filters to extract the features. This paper aims to explore the efficacy of using fNRIS along with ML/DL methods in a multi-class classification task. Complex models like RF, XGBoost, and Hemo-Net produce relatively higher test set accuracy when compared to LDA, MNLR, and QDA. Hemo-Net has depicted a superior performance achieving the highest test set accuracy of 76%, however, in this work, we do not aim at improving the accuracies of models rather we are interested in exploring if fNIRS has the neural signatures to help modern ML/DL methods in multi-class classification which can lead to applications like brain-computer interfaces. Multi-class classification of fine anatomical movements, such as individual finger movements, is difficult to classify with fNIRS data. Traditional ML models like MNLR and LDA show inferior performance compared to the ensemble-based methods of RF and XGBoost. DL-based method Hemo-Net outperforms all methods evaluated in this study and demonstrates a promising future for fNIRS-based BCI applications.
Collapse
Affiliation(s)
- Haroon Khan
- Department of Mechanical, Electronics and Chemical Engineering, OsloMet - Oslo Metropolitan University, Oslo, Norway
| | - Rabindra Khadka
- Department of Information Technology, Oslomet - Oslo Metropolitan University, Oslo, Norway
| | - Malik Shahid Sultan
- Department of Computer, Electrical and Mathematical Science and Engineering, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia
| | - Anis Yazidi
- Department of Information Technology, Oslomet - Oslo Metropolitan University, Oslo, Norway
| | - Hernando Ombao
- Department of Computer, Electrical and Mathematical Science and Engineering, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia
| | - Peyman Mirtaheri
- Department of Mechanical, Electronics and Chemical Engineering, OsloMet - Oslo Metropolitan University, Oslo, Norway
| |
Collapse
|
3
|
Bu Y, Harrington DL, Lee RR, Shen Q, Angeles-Quinto A, Ji Z, Hansen H, Hernandez-Lucas J, Baumgartner J, Song T, Nichols S, Baker D, Rao R, Lerman I, Lin T, Tu XM, Huang M. Magnetoencephalogram-based brain-computer interface for hand-gesture decoding using deep learning. Cereb Cortex 2023:7161766. [PMID: 37183188 DOI: 10.1093/cercor/bhad173] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 04/28/2023] [Accepted: 04/29/2023] [Indexed: 05/16/2023] Open
Abstract
Advancements in deep learning algorithms over the past decade have led to extensive developments in brain-computer interfaces (BCI). A promising imaging modality for BCI is magnetoencephalography (MEG), which is a non-invasive functional imaging technique. The present study developed a MEG sensor-based BCI neural network to decode Rock-Paper-scissors gestures (MEG-RPSnet). Unique preprocessing pipelines in tandem with convolutional neural network deep-learning models accurately classified gestures. On a single-trial basis, we found an average of 85.56% classification accuracy in 12 subjects. Our MEG-RPSnet model outperformed two state-of-the-art neural network architectures for electroencephalogram-based BCI as well as a traditional machine learning method, and demonstrated equivalent and/or better performance than machine learning methods that have employed invasive, electrocorticography-based BCI using the same task. In addition, MEG-RPSnet classification performance using an intra-subject approach outperformed a model that used a cross-subject approach. Remarkably, we also found that when using only central-parietal-occipital regional sensors or occipitotemporal regional sensors, the deep learning model achieved classification performances that were similar to the whole-brain sensor model. The MEG-RSPnet model also distinguished neuronal features of individual hand gestures with very good accuracy. Altogether, these results show that noninvasive MEG-based BCI applications hold promise for future BCI developments in hand-gesture decoding.
Collapse
Affiliation(s)
- Yifeng Bu
- Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093, USA
| | - Deborah L Harrington
- Radiology, Research Services, VA, San Diego Healthcare System, San Diego, CA 92161, USA
- Department of Radiology, University of California San Diego, La Jolla, CA 92093, USA
| | - Roland R Lee
- Radiology, Research Services, VA, San Diego Healthcare System, San Diego, CA 92161, USA
- Department of Radiology, University of California San Diego, La Jolla, CA 92093, USA
| | - Qian Shen
- Radiology, Research Services, VA, San Diego Healthcare System, San Diego, CA 92161, USA
- Department of Radiology, University of California San Diego, La Jolla, CA 92093, USA
| | - Annemarie Angeles-Quinto
- Radiology, Research Services, VA, San Diego Healthcare System, San Diego, CA 92161, USA
- Department of Radiology, University of California San Diego, La Jolla, CA 92093, USA
| | - Zhengwei Ji
- Department of Radiology, University of California San Diego, La Jolla, CA 92093, USA
| | - Hayden Hansen
- Radiology, Research Services, VA, San Diego Healthcare System, San Diego, CA 92161, USA
| | | | - Jared Baumgartner
- Department of Radiology, University of California San Diego, La Jolla, CA 92093, USA
| | - Tao Song
- Department of Radiology, University of California San Diego, La Jolla, CA 92093, USA
| | - Sharon Nichols
- Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA
| | - Dewleen Baker
- VA Center of Excellence for Stress and Mental Health, VA San Diego Healthcare System, San Diego, CA 92161, USA
- Department of Psychiatry, University of California San Diego, La Jolla, CA 92093, USA
| | - Ramesh Rao
- Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093, USA
| | - Imanuel Lerman
- Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093, USA
- VA Center of Excellence for Stress and Mental Health, VA San Diego Healthcare System, San Diego, CA 92161, USA
- Department of Anesthesiology, University of California San Diego, La Jolla, CA 92093, USA
| | - Tuo Lin
- Division of Biostatistics and Bioinformatics, University of California, San Diego, CA 92093, USA
| | - Xin Ming Tu
- Division of Biostatistics and Bioinformatics, University of California, San Diego, CA 92093, USA
| | - Mingxiong Huang
- Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093, USA
- Radiology, Research Services, VA, San Diego Healthcare System, San Diego, CA 92161, USA
- Department of Radiology, University of California San Diego, La Jolla, CA 92093, USA
| |
Collapse
|
4
|
EEG-Based Empathic Safe Cobot. MACHINES 2022. [DOI: 10.3390/machines10080603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/10/2022]
Abstract
An empathic collaborative robot (cobot) was realized through the transmission of fear from a human agent to a robot agent. Such empathy was induced through an electroencephalographic (EEG) sensor worn by the human agent, thus realizing an empathic safe brain-computer interface (BCI). The empathic safe cobot reacts to the fear and in turn transmits it to the human agent, forming a social circle of empathy and safety. A first randomized, controlled experiment involved two groups of 50 healthy subjects (100 total subjects) to measure the EEG signal in the presence or absence of a frightening event. The second randomized, controlled experiment on two groups of 50 different healthy subjects (100 total subjects) exposed the subjects to comfortable and uncomfortable movements of a collaborative robot (cobot) while the subjects’ EEG signal was acquired. The result was that a spike in the subject’s EEG signal was observed in the presence of uncomfortable movement. The questionnaires were distributed to the subjects, and confirmed the results of the EEG signal measurement. In a controlled laboratory setting, all experiments were found to be statistically significant. In the first experiment, the peak EEG signal measured just after the activating event was greater than the resting EEG signal (p < 10−3). In the second experiment, the peak EEG signal measured just after the uncomfortable movement of the cobot was greater than the EEG signal measured under conditions of comfortable movement of the cobot (p < 10−3). In conclusion, within the isolated and constrained experimental environment, the results were satisfactory.
Collapse
|
5
|
Saeidi M, Karwowski W, Farahani FV, Fiok K, Taiar R, Hancock PA, Al-Juaid A. Neural Decoding of EEG Signals with Machine Learning: A Systematic Review. Brain Sci 2021; 11:1525. [PMID: 34827524 PMCID: PMC8615531 DOI: 10.3390/brainsci11111525] [Citation(s) in RCA: 44] [Impact Index Per Article: 14.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Revised: 11/04/2021] [Accepted: 11/11/2021] [Indexed: 11/16/2022] Open
Abstract
Electroencephalography (EEG) is a non-invasive technique used to record the brain's evoked and induced electrical activity from the scalp. Artificial intelligence, particularly machine learning (ML) and deep learning (DL) algorithms, are increasingly being applied to EEG data for pattern analysis, group membership classification, and brain-computer interface purposes. This study aimed to systematically review recent advances in ML and DL supervised models for decoding and classifying EEG signals. Moreover, this article provides a comprehensive review of the state-of-the-art techniques used for EEG signal preprocessing and feature extraction. To this end, several academic databases were searched to explore relevant studies from the year 2000 to the present. Our results showed that the application of ML and DL in both mental workload and motor imagery tasks has received substantial attention in recent years. A total of 75% of DL studies applied convolutional neural networks with various learning algorithms, and 36% of ML studies achieved competitive accuracy by using a support vector machine algorithm. Wavelet transform was found to be the most common feature extraction method used for all types of tasks. We further examined the specific feature extraction methods and end classifier recommendations discovered in this systematic review.
Collapse
Affiliation(s)
- Maham Saeidi
- Computational Neuroergonomics Laboratory, Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL 32816, USA; (F.V.F.); (K.F.)
| | - Waldemar Karwowski
- Computational Neuroergonomics Laboratory, Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL 32816, USA; (F.V.F.); (K.F.)
| | - Farzad V. Farahani
- Computational Neuroergonomics Laboratory, Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL 32816, USA; (F.V.F.); (K.F.)
- Department of Biostatistics, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Krzysztof Fiok
- Computational Neuroergonomics Laboratory, Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL 32816, USA; (F.V.F.); (K.F.)
| | - Redha Taiar
- MATIM, Moulin de la Housse, Université de Reims Champagne Ardenne, CEDEX 02, 51687 Reims, France;
| | - P. A. Hancock
- Department of Psychology, University of Central Florida, Orlando, FL 32816, USA;
| | - Awad Al-Juaid
- Industrial Engineering Department, Taif University, Taif 26571, Saudi Arabia;
| |
Collapse
|
6
|
Esposito D, Centracchio J, Andreozzi E, Gargiulo GD, Naik GR, Bifulco P. Biosignal-Based Human-Machine Interfaces for Assistance and Rehabilitation: A Survey. SENSORS 2021; 21:s21206863. [PMID: 34696076 PMCID: PMC8540117 DOI: 10.3390/s21206863] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Revised: 09/30/2021] [Accepted: 10/12/2021] [Indexed: 12/03/2022]
Abstract
As a definition, Human–Machine Interface (HMI) enables a person to interact with a device. Starting from elementary equipment, the recent development of novel techniques and unobtrusive devices for biosignals monitoring paved the way for a new class of HMIs, which take such biosignals as inputs to control various applications. The current survey aims to review the large literature of the last two decades regarding biosignal-based HMIs for assistance and rehabilitation to outline state-of-the-art and identify emerging technologies and potential future research trends. PubMed and other databases were surveyed by using specific keywords. The found studies were further screened in three levels (title, abstract, full-text), and eventually, 144 journal papers and 37 conference papers were included. Four macrocategories were considered to classify the different biosignals used for HMI control: biopotential, muscle mechanical motion, body motion, and their combinations (hybrid systems). The HMIs were also classified according to their target application by considering six categories: prosthetic control, robotic control, virtual reality control, gesture recognition, communication, and smart environment control. An ever-growing number of publications has been observed over the last years. Most of the studies (about 67%) pertain to the assistive field, while 20% relate to rehabilitation and 13% to assistance and rehabilitation. A moderate increase can be observed in studies focusing on robotic control, prosthetic control, and gesture recognition in the last decade. In contrast, studies on the other targets experienced only a small increase. Biopotentials are no longer the leading control signals, and the use of muscle mechanical motion signals has experienced a considerable rise, especially in prosthetic control. Hybrid technologies are promising, as they could lead to higher performances. However, they also increase HMIs’ complexity, so their usefulness should be carefully evaluated for the specific application.
Collapse
Affiliation(s)
- Daniele Esposito
- Department of Electrical Engineering and Information Technologies, Polytechnic and Basic Sciences School, University of Naples “Federico II”, 80125 Naples, Italy; (D.E.); (J.C.); (E.A.); (P.B.)
| | - Jessica Centracchio
- Department of Electrical Engineering and Information Technologies, Polytechnic and Basic Sciences School, University of Naples “Federico II”, 80125 Naples, Italy; (D.E.); (J.C.); (E.A.); (P.B.)
| | - Emilio Andreozzi
- Department of Electrical Engineering and Information Technologies, Polytechnic and Basic Sciences School, University of Naples “Federico II”, 80125 Naples, Italy; (D.E.); (J.C.); (E.A.); (P.B.)
| | - Gaetano D. Gargiulo
- School of Engineering, Design and Built Environment, Western Sydney University, Penrith, NSW 2747, Australia;
- The MARCS Institute, Western Sydney University, Penrith, NSW 2751, Australia
| | - Ganesh R. Naik
- School of Engineering, Design and Built Environment, Western Sydney University, Penrith, NSW 2747, Australia;
- The Adelaide Institute for Sleep Health, Flinders University, Bedford Park, SA 5042, Australia
- Correspondence:
| | - Paolo Bifulco
- Department of Electrical Engineering and Information Technologies, Polytechnic and Basic Sciences School, University of Naples “Federico II”, 80125 Naples, Italy; (D.E.); (J.C.); (E.A.); (P.B.)
| |
Collapse
|