1
|
Chen Y, Wang F, Li T, Zhao L, Gong A, Nan W, Ding P, Fu Y. Considerations and discussions on the clear definition and definite scope of brain-computer interfaces. Front Neurosci 2024; 18:1449208. [PMID: 39161655 PMCID: PMC11330831 DOI: 10.3389/fnins.2024.1449208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2024] [Accepted: 07/22/2024] [Indexed: 08/21/2024] Open
Abstract
Brain-computer interface (BCI) is a revolutionizing human-computer interaction with potential applications in both medical and non-medical fields, emerging as a cutting-edge and trending research direction. Increasing numbers of groups are engaging in BCI research and development. However, in recent years, there has been some confusion regarding BCI, including misleading and hyped propaganda about BCI, and even non-BCI technologies being labeled as BCI. Therefore, a clear definition and a definite scope for BCI are thoroughly considered and discussed in the paper, based on the existing definitions of BCI, including the six key or essential components of BCI. In the review, different from previous definitions of BCI, BCI paradigms and neural coding are explicitly included in the clear definition of BCI provided, and the BCI user (the brain) is clearly identified as a key component of the BCI system. Different people may have different viewpoints on the definition and scope of BCI, as well as some related issues, which are discussed in the article. This review argues that a clear definition and definite scope of BCI will benefit future research and commercial applications. It is hoped that this review will reduce some of the confusion surrounding BCI and promote sustainable development in this field.
Collapse
Affiliation(s)
- Yanxiao Chen
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
| | - Fan Wang
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
| | - Tianwen Li
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
- Faculty of Science, Kunming University of Science and Technology, Kunming, China
| | - Lei Zhao
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
- Faculty of Science, Kunming University of Science and Technology, Kunming, China
| | - Anmin Gong
- School of Information Engineering, Chinese People’s Armed Police Force Engineering University, Xi’an, China
| | - Wenya Nan
- School of Psychology, Shanghai Normal University, Shanghai, China
| | - Peng Ding
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
| | - Yunfa Fu
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
| |
Collapse
|
2
|
Ajrawi SA, Rao R, Sarkar M. A hierarchical recursive feature elimination algorithm to develop brain computer interface application of user behavior for statistical reasoning and decision making. J Neurosci Methods 2024; 408:110161. [PMID: 38718901 DOI: 10.1016/j.jneumeth.2024.110161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Revised: 04/16/2024] [Accepted: 05/02/2024] [Indexed: 05/25/2024]
Abstract
BACKGROUND With the aid of a brain computer interface (BCI), users can communicate and receive signals wirelessly or over wired connections to operate smart devices. A BCI classifier's architecture is quite difficult since numerous elements should be combined. These elements are made up of brain signals, which also include high levels of weak sounds that could provide reliable participant recordings of daily activities. We must use computer vision techniques to create a model in order to control those information. The high dimension and volume of signals present the classification classifier with its primary obstacles. NEW METHOD Due to this, we extracted and classified the brain activity in this study, and we also presented a novel hierarchical recursive feature elimination method that we refer to as HRFE embracing noisy additions. HRFE makes a variety of categorization suggestions to eliminate bias in classifying BCI systems of different types. We put the HRFE to the test on two BCI signal data sets-specifically, dataset I and BCI contests III-using shallow and deep convolution network classification techniques. Just a grid of assets is used to create electrocorticography (ECoG) signals on the contralateral (right) motor cortex, and these signals are recorded in the BCI contests III database. RESULTS Using ECoG signals, we choose the top 20 features that have the biggest effects on distortion and classification selection. COMPARISON WITH EXISTING METHODS The simulation findings show that HRFE has a significant computer vision enhancement when compared to comparable feature selection methods in the literature, particularly for ECoG signal, which achieves about 93% reliability.
Collapse
Affiliation(s)
- Shams Al Ajrawi
- Department of Electrical and Computer Engineering, University of California, San Diego, Alliant International univercity, San Diego, CA, USA; CSML, Alliant International University, San Diego, San Diego, CA, USA; Department of Electrical and Computer Engineering, San Diego State University, San Diego, CA, USA.
| | - Ramesh Rao
- Department of Electrical and Computer Engineering, University of California, San Diego, Alliant International univercity, San Diego, CA, USA.
| | - Mahasweta Sarkar
- Department of Electrical and Computer Engineering, San Diego State University, San Diego, CA, USA.
| |
Collapse
|
3
|
Pfeffer MA, Ling SSH, Wong JKW. Exploring the frontier: Transformer-based models in EEG signal analysis for brain-computer interfaces. Comput Biol Med 2024; 178:108705. [PMID: 38865781 DOI: 10.1016/j.compbiomed.2024.108705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2023] [Revised: 05/01/2024] [Accepted: 06/01/2024] [Indexed: 06/14/2024]
Abstract
This review systematically explores the application of transformer-based models in EEG signal processing and brain-computer interface (BCI) development, with a distinct focus on ensuring methodological rigour and adhering to empirical validations within the existing literature. By examining various transformer architectures, such as the Temporal Spatial Transformer Network (TSTN) and EEG Conformer, this review delineates their capabilities in mitigating challenges intrinsic to EEG data, such as noise and artifacts, and their subsequent implications on decoding and classification accuracies across disparate mental tasks. The analytical scope extends to a meticulous examination of attention mechanisms within transformer models, delineating their role in illuminating critical temporal and spatial EEG features and facilitating interpretability in model decision-making processes. The discourse additionally encapsulates emerging works that substantiate the efficacy of transformer models in noise reduction of EEG signals and diversifying applications beyond the conventional motor imagery paradigm. Furthermore, this review elucidates evident gaps and propounds exploratory avenues in the applications of pre-trained transformers in EEG analysis and the potential expansion into real-time and multi-task BCI applications. Collectively, this review distils extant knowledge, navigates through the empirical findings, and puts forward a structured synthesis, thereby serving as a conduit for informed future research endeavours in transformer-enhanced, EEG-based BCI systems.
Collapse
Affiliation(s)
- Maximilian Achim Pfeffer
- Faculty of Engineering and Information Technology, University of Technology Sydney, CB11 81-113, Broadway, Ultimo, 2007, New South Wales, Australia.
| | - Steve Sai Ho Ling
- Faculty of Engineering and Information Technology, University of Technology Sydney, CB11 81-113, Broadway, Ultimo, 2007, New South Wales, Australia.
| | - Johnny Kwok Wai Wong
- Faculty of Design, Architecture and Building, University of Technology Sydney, 15 Broadway, Ultimo, 2007, New South Wales, Australia.
| |
Collapse
|
4
|
Li G, Jang D, Shin Y, Qiang Y, Qi Y, Wang S, Fang H. Cracking modes and force dynamics in the insertion of neural probes into hydrogel brain phantom. J Neural Eng 2024; 21:046009. [PMID: 38885673 PMCID: PMC11225066 DOI: 10.1088/1741-2552/ad5937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Revised: 04/23/2024] [Accepted: 06/17/2024] [Indexed: 06/20/2024]
Abstract
Objective. The insertion of penetrating neural probes into the brain is crucial for advancing neuroscience, yet it involves various inherent risks. Prototype probes are typically inserted into hydrogel-based brain phantoms and the mechanical responses are analyzed in order to inform the insertion mechanics duringin vivoimplantation. However, the underlying mechanism of the insertion dynamics of neural probes in hydrogel brain phantoms, particularly the phenomenon of cracking, remains insufficiently understood. This knowledge gap leads to misinterpretations and discrepancies when comparing results obtained from phantom studies to those observed under thein vivoconditions. This study aims to elucidate the impact of probe sharpness and dimensions on the cracking mechanisms and insertion dynamics characterized during the insertion of probes in hydrogel phantoms.Approach. The insertion of dummy probes with different shank shapes defined by the tip angle, width, and thickness is systematically studied. The insertion-induced cracks in the transparent hydrogel were accentuated by an immiscible dye, tracked byin situimaging, and the corresponding insertion force was recorded. Three-dimensional finite element analysis models were developed to obtain the contact stress between the probe tip and the phantom.Main results. The findings reveal a dual pattern: for sharp, slender probes, the insertion forces remain consistently low during the insertion process, owing to continuously propagating straight cracks that align with the insertion direction. In contrast, blunt, thick probes induce large forces that increase rapidly with escalating insertion depth, mainly due to the formation of branched crack with a conical cracking surface, and the subsequent internal compression. This interpretation challenges the traditional understanding that neglects the difference in the cracking modes and regards increased frictional force as the sole factor contributing to higher insertion forces. The critical probe sharpness factors separating straight and branched cracking is identified experimentally, and a preliminary explanation of the transition between the two cracking modes is derived from three-dimensional finite element analysis.Significance. This study presents, for the first time, the mechanism underlying two distinct cracking modes during the insertion of neural probes into hydrogel brain phantoms. The correlations between the cracking modes and the insertion force dynamics, as well as the effects of the probe sharpness were established, offering insights into the design of neural probes via phantom studies and informing future investigations into cracking phenomena in brain tissue during probe implantations.
Collapse
Affiliation(s)
- Gen Li
- Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, United States of America
| | - Dongyeol Jang
- Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, United States of America
| | - Yieljae Shin
- Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, United States of America
| | - Yi Qiang
- Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, United States of America
| | - Yongli Qi
- Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, United States of America
| | - Shuodao Wang
- School of Mechanical & Aerospace Engineering, Oklahoma State University, Stillwater, OK 74078, United States of America
| | - Hui Fang
- Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, United States of America
| |
Collapse
|
5
|
Hore A, Bandyopadhyay S, Chakrabarti S. Persistent spiking activity in neuromorphic circuits incorporating post-inhibitory rebound excitation. J Neural Eng 2024; 21:036048. [PMID: 38861961 DOI: 10.1088/1741-2552/ad56c8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Accepted: 06/11/2024] [Indexed: 06/13/2024]
Abstract
Objective. This study introduces a novel approach for integrating the post-inhibitory rebound excitation (PIRE) phenomenon into a neuronal circuit. Excitatory and inhibitory synapses are designed to establish a connection between two hardware neurons, effectively forming a network. The model demonstrates the occurrence of PIRE under strong inhibitory input. Emphasizing the significance of incorporating PIRE in neuromorphic circuits, the study showcases generation of persistent activity within cyclic and recurrent spiking neuronal networks.Approach. The neuronal and synaptic circuits are designed and simulated in Cadence Virtuoso using TSMC 180 nm technology. The operating mechanism of the PIRE phenomenon integrated into a hardware neuron is discussed. The proposed circuit encompasses several parameters for effectively controlling multiple electrophysiological features of a neuron.Main results. The neuronal circuit has been tuned to match the response of a biological neuron. The efficiency of this circuit is evaluated by computing the average power dissipation and energy consumption per spike through simulation. The sustained firing of neural spikes is observed till 1.7 s using the two neuronal networks.Significance. Persistent activity has significant implications for various cognitive functions such as working memory, decision-making, and attention. Therefore, hardware implementation of these functions will require our PIRE-integrated model. Energy-efficient neuromorphic systems are useful in many artificial intelligence applications, including human-machine interaction, IoT devices, autonomous systems, and brain-computer interfaces.
Collapse
|
6
|
Wu H, Wang R, Ma Y, Liang X, Liu C, Yu D, An N, Ning X. Decoding N400m Evoked Component: A Tutorial on Multivariate Pattern Analysis for OP-MEG Data. Bioengineering (Basel) 2024; 11:609. [PMID: 38927845 PMCID: PMC11200846 DOI: 10.3390/bioengineering11060609] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2024] [Revised: 05/26/2024] [Accepted: 06/11/2024] [Indexed: 06/28/2024] Open
Abstract
Multivariate pattern analysis (MVPA) has played an extensive role in interpreting brain activity, which has been applied in studies with modalities such as functional Magnetic Resonance Imaging (fMRI), Magnetoencephalography (MEG) and Electroencephalography (EEG). The advent of wearable MEG systems based on optically pumped magnetometers (OPMs), i.e., OP-MEG, has broadened the application of bio-magnetism in the realm of neuroscience. Nonetheless, it also raises challenges in temporal decoding analysis due to the unique attributes of OP-MEG itself. The efficacy of decoding performance utilizing multimodal fusion, such as MEG-EEG, also remains to be elucidated. In this regard, we investigated the impact of several factors, such as processing methods, models and modalities, on the decoding outcomes of OP-MEG. Our findings indicate that the number of averaged trials, dimensionality reduction (DR) methods, and the number of cross-validation folds significantly affect the decoding performance of OP-MEG data. Additionally, decoding results vary across modalities and fusion strategy. In contrast, decoder type, resampling frequency, and sliding window length exert marginal effects. Furthermore, we introduced mutual information (MI) to investigate how information loss due to OP-MEG data processing affect decoding accuracy. Our study offers insights for linear decoding research using OP-MEG and expand its application in the fields of cognitive neuroscience.
Collapse
Affiliation(s)
- Huanqi Wu
- Key Laboratory of Ultra-Weak Magnetic Field Measurement Technology, Ministry of Education, School of Instrumentation and Optoelectronic Engineering, Beihang University, 37 Xueyuan Rd., Haidian Dist., Beijing 100083, China; (H.W.); (R.W.); (Y.M.); (X.L.); (C.L.)
- Hangzhou Institute of National Extremely-Weak Magnetic Field Infrastructure, 465 Binan Rd., Binjiang Dist., Hangzhou 310000, China
| | - Ruonan Wang
- Key Laboratory of Ultra-Weak Magnetic Field Measurement Technology, Ministry of Education, School of Instrumentation and Optoelectronic Engineering, Beihang University, 37 Xueyuan Rd., Haidian Dist., Beijing 100083, China; (H.W.); (R.W.); (Y.M.); (X.L.); (C.L.)
- Hangzhou Institute of National Extremely-Weak Magnetic Field Infrastructure, 465 Binan Rd., Binjiang Dist., Hangzhou 310000, China
| | - Yuyu Ma
- Key Laboratory of Ultra-Weak Magnetic Field Measurement Technology, Ministry of Education, School of Instrumentation and Optoelectronic Engineering, Beihang University, 37 Xueyuan Rd., Haidian Dist., Beijing 100083, China; (H.W.); (R.W.); (Y.M.); (X.L.); (C.L.)
- Hangzhou Institute of National Extremely-Weak Magnetic Field Infrastructure, 465 Binan Rd., Binjiang Dist., Hangzhou 310000, China
| | - Xiaoyu Liang
- Key Laboratory of Ultra-Weak Magnetic Field Measurement Technology, Ministry of Education, School of Instrumentation and Optoelectronic Engineering, Beihang University, 37 Xueyuan Rd., Haidian Dist., Beijing 100083, China; (H.W.); (R.W.); (Y.M.); (X.L.); (C.L.)
- Hangzhou Institute of National Extremely-Weak Magnetic Field Infrastructure, 465 Binan Rd., Binjiang Dist., Hangzhou 310000, China
| | - Changzeng Liu
- Key Laboratory of Ultra-Weak Magnetic Field Measurement Technology, Ministry of Education, School of Instrumentation and Optoelectronic Engineering, Beihang University, 37 Xueyuan Rd., Haidian Dist., Beijing 100083, China; (H.W.); (R.W.); (Y.M.); (X.L.); (C.L.)
- Hangzhou Institute of National Extremely-Weak Magnetic Field Infrastructure, 465 Binan Rd., Binjiang Dist., Hangzhou 310000, China
| | - Dexin Yu
- Shandong Key Laboratory for Magnetic Field-Free Medicine and Functional Imaging, Institute of Magnetic Field-Free Medicine and Functional Imaging, Shandong University, 27 South Shanda Rd., Licheng Dist., Jinan 250100, China;
| | - Nan An
- Hangzhou Institute of National Extremely-Weak Magnetic Field Infrastructure, 465 Binan Rd., Binjiang Dist., Hangzhou 310000, China
| | - Xiaolin Ning
- Key Laboratory of Ultra-Weak Magnetic Field Measurement Technology, Ministry of Education, School of Instrumentation and Optoelectronic Engineering, Beihang University, 37 Xueyuan Rd., Haidian Dist., Beijing 100083, China; (H.W.); (R.W.); (Y.M.); (X.L.); (C.L.)
- Hangzhou Institute of National Extremely-Weak Magnetic Field Infrastructure, 465 Binan Rd., Binjiang Dist., Hangzhou 310000, China
- Shandong Key Laboratory for Magnetic Field-Free Medicine and Functional Imaging, Institute of Magnetic Field-Free Medicine and Functional Imaging, Shandong University, 27 South Shanda Rd., Licheng Dist., Jinan 250100, China;
- Hefei National Laboratory, Gaoxin Dist., Hefei 230093, China
| |
Collapse
|
7
|
Liu D, Cui J, Pan Z, Zhang H, Cao J, Kong W. Machine to brain: facial expression recognition using brain machine generative adversarial networks. Cogn Neurodyn 2024; 18:863-875. [PMID: 38826642 PMCID: PMC11143176 DOI: 10.1007/s11571-023-09946-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2022] [Revised: 12/29/2022] [Accepted: 02/10/2023] [Indexed: 02/24/2023] Open
Abstract
The human brain can effectively perform Facial Expression Recognition (FER) with a few samples by utilizing its cognitive ability. However, unlike the human brain, even the well-trained deep neural network is data-dependent and lacks cognitive ability. To tackle this challenge, this paper proposes a novel framework, Brain Machine Generative Adversarial Networks (BM-GAN), which utilizes the concept of brain's cognitive ability to guide a Convolutional Neural Network to generate LIKE-electroencephalograph (EEG) features. More specifically, we firstly obtain EEG signals triggered from facial emotion images, then we adopt BM-GAN to carry out the mutual generation of image visual features and EEG cognitive features. BM-GAN intends to use the cognitive knowledge learnt from EEG signals to instruct the model to perceive LIKE-EEG features. Thereby, BM-GAN has a superior performance for FER like the human brain. The proposed model consists of VisualNet, EEGNet, and BM-GAN. More specifically, VisualNet can obtain image visual features from facial emotion images and EEGNet can obtain EEG cognitive features from EEG signals. Subsequently, the BM-GAN completes the mutual generation of image visual features and EEG cognitive features. Finally, the predicted LIKE-EEG features of test images are used for FER. After learning, without the participation of the EEG signals, an average classification accuracy of 96.6 % is obtained on Chinese Facial Affective Picture System dataset using LIKE-EEG features for FER. Experiments demonstrate that the proposed method can produce an excellent performance for FER.
Collapse
Affiliation(s)
- Dongjun Liu
- School of Computer Science, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang China
| | - Jin Cui
- School of Computer Science, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang China
| | - Zeyu Pan
- School of Computer Science, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang China
| | - Hangkui Zhang
- School of Computer Science, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang China
| | - Jianting Cao
- Graduate School of Engineering, Saitama Institute of Technology, Saitama, 369-0293 Japan
| | - Wanzeng Kong
- School of Computer Science, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou, 310018 Zhejiang China
| |
Collapse
|
8
|
Sadras N, Pesaran B, Shanechi MM. Event detection and classification from multimodal time series with application to neural data. J Neural Eng 2024; 21:026049. [PMID: 38513289 DOI: 10.1088/1741-2552/ad3678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Accepted: 03/21/2024] [Indexed: 03/23/2024]
Abstract
The detection of events in time-series data is a common signal-processing problem. When the data can be modeled as a known template signal with an unknown delay in Gaussian noise, detection of the template signal can be done with a traditional matched filter. However, in many applications, the event of interest is represented in multimodal data consisting of both Gaussian and point-process time series. Neuroscience experiments, for example, can simultaneously record multimodal neural signals such as local field potentials (LFPs), which can be modeled as Gaussian, and neuronal spikes, which can be modeled as point processes. Currently, no method exists for event detection from such multimodal data, and as such our objective in this work is to develop a method to meet this need. Here we address this challenge by developing the multimodal event detector (MED) algorithm which simultaneously estimates event times and classes. To do this, we write a multimodal likelihood function for Gaussian and point-process observations and derive the associated maximum likelihood estimator of simultaneous event times and classes. We additionally introduce a cross-modal scaling parameter to account for model mismatch in real datasets. We validate this method in extensive simulations as well as in a neural spike-LFP dataset recorded during an eye-movement task, where the events of interest are eye movements with unknown times and directions. We show that the MED can successfully detect eye movement onset and classify eye movement direction. Further, the MED successfully combines information across data modalities, with multimodal performance exceeding unimodal performance. This method can facilitate applications such as the discovery of latent events in multimodal neural population activity and the development of brain-computer interfaces for naturalistic settings without constrained tasks or prior knowledge of event times.
Collapse
Affiliation(s)
- Nitin Sadras
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Bijan Pesaran
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Maryam M Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Thomas Lord Department of Computer Science, Alfred E. Mann Department of Biomedical Engineering, and the Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, United States of America
| |
Collapse
|
9
|
Kim H, Won K, Ahn M, Jun SC. Comparison of recognition methods for an asynchronous (un-cued) BCI system: an investigation with 40-class SSVEP dataset. Biomed Eng Lett 2024; 14:617-630. [PMID: 38645586 PMCID: PMC11026332 DOI: 10.1007/s13534-024-00357-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 01/16/2024] [Accepted: 01/24/2024] [Indexed: 04/23/2024] Open
Abstract
Steady-state visual evoked potential (SSVEP)-based brain-computer Interface (BCI) has demonstrated the potential to manage multi-command targets to achieve high-speed communication. Recent studies on multi-class SSVEP-based BCI have focused on synchronous systems, which rely on predefined time and task indicators; thus, these systems that use passive approaches may be less suitable for practical applications. Asynchronous systems recognize the user's intention (whether or not the user is willing to use systems) from brain activity; then, after recognizing the user's willingness, they begin to operate by switching swiftly for real-time control. Consequently, various methodologies have been proposed to capture the user's intention. However, in-depth investigation of recognition methods in asynchronous BCI system is lacking. Thus, in this work, three recognition methods (power spectral density analysis, canonical correlation analysis (CCA), and support vector machine (SVM)) used widely in asynchronous SSVEP BCI systems were explored to compare their performance. Further, we categorized asynchronous systems into two approaches (1-stage and 2-stage) based upon the recognition process's design, and compared their performance. To do so, a 40-class SSVEP dataset collected from 40 subjects was introduced. Finally, we found that the CCA-based method in the 2-stage approach demonstrated statistically significantly higher performance with a sensitivity of 97.62 ± 02.06%, specificity of 76.50 ± 23.50%, and accuracy of 75.59 ± 10.09%. Thus, it is expected that the 2-stage approach together with CCA-based recognition and FB-CCA classification have good potential to be implemented in practical asynchronous SSVEP BCI systems.
Collapse
Affiliation(s)
- Heegyu Kim
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Bukgu, Gwangju, 61005 Korea
| | - Kyungho Won
- Hybrid Team, Inria, Univ Rennes, IRISA, CNRS, F35000 Rennes, France
| | - Minkyu Ahn
- School of Computer Science and Electrical Engineering, Handong Global University, Bukgu, Pohang, 37554 Korea
| | - Sung Chan Jun
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Bukgu, Gwangju, 61005 Korea
- School of Artificial Intelligence, Gwangju Institute of Science and Technology, Bukgu, Gwangju, 61005 Korea
| |
Collapse
|
10
|
Soldado-Magraner J, Antonietti A, French J, Higgins N, Young MJ, Larrivee D, Monteleone R. Applying the IEEE BRAIN neuroethics framework to intra-cortical brain-computer interfaces. J Neural Eng 2024; 21:022001. [PMID: 38537269 DOI: 10.1088/1741-2552/ad3852] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Accepted: 03/27/2024] [Indexed: 04/05/2024]
Abstract
Objective. Brain-computer interfaces (BCIs) are neuroprosthetic devices that allow for direct interaction between brains and machines. These types of neurotechnologies have recently experienced a strong drive in research and development, given, in part, that they promise to restore motor and communication abilities in individuals experiencing severe paralysis. While a rich literature analyzes the ethical, legal, and sociocultural implications (ELSCI) of these novel neurotechnologies, engineers, clinicians and BCI practitioners often do not have enough exposure to these topics.Approach. Here, we present the IEEE Neuroethics Framework, an international, multiyear, iterative initiative aimed at developing a robust, accessible set of considerations for diverse stakeholders.Main results. Using the framework, we provide practical examples of ELSCI considerations for BCI neurotechnologies. We focus on invasive technologies, and in particular, devices that are implanted intra-cortically for medical research applications.Significance. We demonstrate the utility of our framework in exposing a wide range of implications across different intra-cortical BCI technology modalities and conclude with recommendations on how to utilize this knowledge in the development and application of ethical guidelines for BCI neurotechnologies.
Collapse
Affiliation(s)
- Joana Soldado-Magraner
- Department of Electrical and Computer Engineering and the Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, United States of America
| | - Alberto Antonietti
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milano 20131, Italy
| | - Jennifer French
- Neurotech Network, St. Petersburg, FL 33733, United States of America
| | - Nathan Higgins
- School of Psychological Sciences, Monash University, Clayton, VIC 3800, Australia
| | - Michael J Young
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, United States of America
| | - Denis Larrivee
- Mind and Brain Institute, University of Navarra Medical School, Pamplona, Navarra 31008, Spain
- Loyola University, Chicago, IL 60611, United States of America
| | - Rebecca Monteleone
- Disability Studies Program, University of Toledo, Toledo, OH 43606, United States of America
| |
Collapse
|
11
|
Jiao Y, Zheng Q, Qiao D, Lang X, Xie L, Pan Y. EEG rhythm separation and time-frequency analysis of fast multivariate empirical mode decomposition for motor imagery BCI. BIOLOGICAL CYBERNETICS 2024; 118:21-37. [PMID: 38472417 DOI: 10.1007/s00422-024-00984-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 02/11/2024] [Indexed: 03/14/2024]
Abstract
Motor imagery electroencephalogram (EEG) is widely employed in brain-computer interface (BCI) systems. As a time-frequency analysis method for nonlinear and non-stationary signals, multivariate empirical mode decomposition (MEMD) and its noise-assisted version (NA-MEMD) has been widely used in the preprocessing step of BCI systems for separating EEG rhythms corresponding to specific brain activities. However, when applied to multichannel EEG signals, MEMD or NA-MEMD often demonstrate low robustness to noise and high computational complexity. To address these issues, we have explored the advantages of our recently proposed fast multivariate empirical mode decomposition (FMEMD) and its noise-assisted version (NA-FMEMD) for analyzing motor imagery data. We emphasize that FMEMD enables a more accurate estimation of EEG frequency information and exhibits a more noise-robust decomposition performance with improved computational efficiency. Comparative analysis with MEMD on simulation data and real-world EEG validates the above assertions. The joint average frequency measure is employed to automatically select intrinsic mode functions that correspond to specific frequency bands. Thus, FMEMD-based classification architecture is proposed. Using FMEMD as a preprocessing algorithm instead of MEMD can improve the classification accuracy by 2.3% on the BCI Competition IV dataset. On the Physiobank Motor/Mental Imagery dataset and BCI Competition IV Dataset 2a, FMEMD-based architecture also attained a comparable performance to complex algorithms. The results indicate that FMEMD proficiently extracts feature information from small benchmark datasets while mitigating dimensionality constraints resulting from computational complexity. Hence, FMEMD or NA-FMEMD can be a powerful time-frequency preprocessing method for BCI.
Collapse
Affiliation(s)
- Yang Jiao
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518026, China
- University of Nottingham Ningbo China, Ningbo, 315100, China
| | - Qian Zheng
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518026, China.
| | - Dan Qiao
- State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, 310027, China
| | - Xun Lang
- Department of Electronic Engineering, Information School, Yunnan University, Kunming, 650091, China
| | - Lei Xie
- State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, 310027, China
| | - Yi Pan
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518026, China.
| |
Collapse
|
12
|
Juan JV, Martínez R, Iáñez E, Ortiz M, Tornero J, Azorín JM. Exploring EEG-based motor imagery decoding: a dual approach using spatial features and spectro-spatial Deep Learning model IFNet. Front Neuroinform 2024; 18:1345425. [PMID: 38486923 PMCID: PMC10937463 DOI: 10.3389/fninf.2024.1345425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Accepted: 02/05/2024] [Indexed: 03/17/2024] Open
Abstract
Introduction In recent years, the decoding of motor imagery (MI) from electroencephalography (EEG) signals has become a focus of research for brain-machine interfaces (BMIs) and neurorehabilitation. However, EEG signals present challenges due to their non-stationarity and the substantial presence of noise commonly found in recordings, making it difficult to design highly effective decoding algorithms. These algorithms are vital for controlling devices in neurorehabilitation tasks, as they activate the patient's motor cortex and contribute to their recovery. Methods This study proposes a novel approach for decoding MI during pedalling tasks using EEG signals. A widespread approach is based on feature extraction using Common Spatial Patterns (CSP) followed by a linear discriminant analysis (LDA) as a classifier. The first approach covered in this work aims to investigate the efficacy of a task-discriminative feature extraction method based on CSP filter and LDA classifier. Additionally, the second alternative hypothesis explores the potential of a spectro-spatial Convolutional Neural Network (CNN) to further enhance the performance of the first approach. The proposed CNN architecture combines a preprocessing pipeline based on filter banks in the frequency domain with a convolutional neural network for spectro-temporal and spectro-spatial feature extraction. Results and discussion To evaluate the approaches and their advantages and disadvantages, EEG data has been recorded from several able-bodied users while pedalling in a cycle ergometer in order to train motor imagery decoding models. The results show levels of accuracy up to 80% in some cases. The CNN approach shows greater accuracy despite higher instability.
Collapse
Affiliation(s)
- Javier V. Juan
- Brain-Machine Interface Systems Lab, Universidad Miguel Hernández de Elche, Elche, Spain
- Center for Clinical Neuroscience HLM, Hospital Los Madroños, Brunete, Spain
| | - Rubén Martínez
- Center for Clinical Neuroscience HLM, Hospital Los Madroños, Brunete, Spain
- Universidad Autónoma de Madrid, Madrid, Spain
- INNTEGRA, Hospital Los Madroños, Brunete, Spain
| | - Eduardo Iáñez
- Brain-Machine Interface Systems Lab, Universidad Miguel Hernández de Elche, Elche, Spain
- Instituto de Investigación en Ingeniería de Elche-I3E, Universidad Miguel Hernández de Elche, Elche, Spain
| | - Mario Ortiz
- Brain-Machine Interface Systems Lab, Universidad Miguel Hernández de Elche, Elche, Spain
- Instituto de Investigación en Ingeniería de Elche-I3E, Universidad Miguel Hernández de Elche, Elche, Spain
| | - Jesús Tornero
- Center for Clinical Neuroscience HLM, Hospital Los Madroños, Brunete, Spain
- INNTEGRA, Hospital Los Madroños, Brunete, Spain
| | - José M. Azorín
- Brain-Machine Interface Systems Lab, Universidad Miguel Hernández de Elche, Elche, Spain
- Instituto de Investigación en Ingeniería de Elche-I3E, Universidad Miguel Hernández de Elche, Elche, Spain
- ValGRAI: Valencian Graduated School and Research Network of Artificial Intelligence, Valencia, Spain
| |
Collapse
|
13
|
Wang Z, Wang X, Shu G, Yin M, Huang S, Yin M. Power-to-Noise Optimization in the Design of Neural Recording Amplifier Based on Current Scaling, Source Degeneration Resistor, and Current Reuse. BIOSENSORS 2024; 14:111. [PMID: 38392030 PMCID: PMC10887131 DOI: 10.3390/bios14020111] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 01/31/2024] [Accepted: 02/13/2024] [Indexed: 02/24/2024]
Abstract
This article presents the design of a low-power, low-noise neural signal amplifier for neural recording. The structure reduces the current consumption of the amplifier through current scaling technology and lowers the input-referred noise of the amplifier by combining a source degeneration resistor and current reuse technologies. The amplifier was fabricated using a 0.18 μm CMOS MS RF G process. The results show the front-end amplifier exhibits a measured mid-band gain of 40 dB/46 dB and a bandwidth ranging from 0.54 Hz to 6.1 kHz; the amplifier's input-referred noise was measured to be 3.1 μVrms, consuming a current of 3.8 μA at a supply voltage of 1.8 V, with a Noise Efficiency Factor (NEF) of 2.97. The single amplifier's active silicon area is 0.082 mm2.
Collapse
Affiliation(s)
- Zhen Wang
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haikou 570100, China; (Z.W.); (X.W.); (G.S.); (M.Y.); (S.H.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haikou 570100, China
| | - Xiao Wang
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haikou 570100, China; (Z.W.); (X.W.); (G.S.); (M.Y.); (S.H.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haikou 570100, China
| | - Guijun Shu
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haikou 570100, China; (Z.W.); (X.W.); (G.S.); (M.Y.); (S.H.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haikou 570100, China
| | - Meng Yin
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haikou 570100, China; (Z.W.); (X.W.); (G.S.); (M.Y.); (S.H.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haikou 570100, China
| | - Shoushuang Huang
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haikou 570100, China; (Z.W.); (X.W.); (G.S.); (M.Y.); (S.H.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haikou 570100, China
| | - Ming Yin
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haikou 570100, China; (Z.W.); (X.W.); (G.S.); (M.Y.); (S.H.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haikou 570100, China
| |
Collapse
|
14
|
Shokri M, Gogliettino AR, Hottowy P, Sher A, Litke AM, Chichilnisky EJ, Pequito S, Muratore D. Spike sorting in the presence of stimulation artifacts: a dynamical control systems approach. J Neural Eng 2024; 21:016022. [PMID: 38271715 PMCID: PMC10853761 DOI: 10.1088/1741-2552/ad228f] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Revised: 11/08/2023] [Accepted: 01/25/2024] [Indexed: 01/27/2024]
Abstract
Objective. Bi-directional electronic neural interfaces, capable of both electrical recording and stimulation, communicate with the nervous system to permit precise calibration of electrical inputs by capturing the evoked neural responses. However, one significant challenge is that stimulation artifacts often mask the actual neural signals. To address this issue, we introduce a novel approach that employs dynamical control systems to detect and decipher electrically evoked neural activity despite the presence of electrical artifacts.Approach. Our proposed method leverages the unique spatiotemporal patterns of neural activity and electrical artifacts to distinguish and identify individual neural spikes. We designed distinctive dynamical models for both the stimulation artifact and each neuron observed during spontaneous neural activity. We can estimate which neurons were active by analyzing the recorded voltage responses across multiple electrodes post-stimulation. This technique also allows us to exclude signals from electrodes heavily affected by stimulation artifacts, such as the stimulating electrode itself, yet still accurately differentiate between evoked spikes and electrical artifacts.Main results. We applied our method to high-density multi-electrode recordings from the primate retina in anex vivosetup, using a grid of 512 electrodes. Through repeated electrical stimulations at varying amplitudes, we were able to construct activation curves for each neuron. The curves obtained with our method closely resembled those derived from manual spike sorting. Additionally, the stimulation thresholds we estimated strongly agreed with those determined through manual analysis, demonstrating high reliability (R2=0.951for human 1 andR2=0.944for human 2).Significance. Our method can effectively separate evoked neural spikes from stimulation artifacts by exploiting the distinct spatiotemporal propagation patterns captured by a dense, large-scale multi-electrode array. This technique holds promise for future applications in real-time closed-loop stimulation systems and for managing multi-channel stimulation strategies.
Collapse
Affiliation(s)
- Mohammad Shokri
- Delft Center for Systems and Control, Delft University of Technology, Delft 2628 CN, The Netherlands
| | - Alex R Gogliettino
- Neurosciences PhD Program, Stanford University, Stanford, CA 94305, United States of America
- Hansen Experimental Physics Laboratory, Stanford University, Stanford, CA 94305, United States of America
| | - Paweł Hottowy
- Faculty of Physics and Applied Computer Science, AGH University of Krakow, Krakow, Poland
| | - Alexander Sher
- Santa Cruz Institute for Particle Physics, University of California, Santa Cruz, CA, United States of America
| | - Alan M Litke
- Santa Cruz Institute for Particle Physics, University of California, Santa Cruz, CA, United States of America
| | - E J Chichilnisky
- Departments of Neurosurgery and Ophthalmology, Stanford University, Stanford, CA 94305, United States of America
| | - Sérgio Pequito
- Division of Systems and Control, Department of Information Technology, Uppsala University, 751 05 Uppsala, Sweden
| | - Dante Muratore
- Microelectronics Department, Delft University of Technology, Delft 2628 CN, The Netherlands
| |
Collapse
|
15
|
Pan H, Fu Y, Zhang Q, Zhang J, Qin X. The decoder design and performance comparative analysis for closed-loop brain-machine interface system. Cogn Neurodyn 2024; 18:147-164. [PMID: 39170600 PMCID: PMC11333431 DOI: 10.1007/s11571-022-09919-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 12/05/2022] [Indexed: 12/24/2022] Open
Abstract
Brain-machine interface (BMI) can convert electroencephalography signals (EEGs) into the control instructions of external devices, and the key of control performance is the accuracy and efficiency of decoder. However, the performance of different decoders obtaining control instructions from complex and variable EEG signals is very different and irregular in the different neural information transfer model. Aiming at this problem, the off-line and on-line performance of eight decoders based on the improved single-joint information transmission (SJIT) model is compared and analyzed in this paper, which can provide a theoretical guidance for decoder design. Firstly, in order to avoid the different types of neural activities in the decoding process on the decoder performance, eight decoders based on the improved SJIT model are designed. And then the off-line decoding performance of these decoders is tested and compared. Secondly, a closed-loop BMI system which combining by the designed decoder and the random forest encoder based on the improved SJIT model is constructed. Finally, based on the constructed closed-loop BMI system, the on-line decoding performance of decoders is compared and analyzed. The results show that the LSTM-based decoder has better on-line decoding performance than others in the improved SJIT model.
Collapse
Affiliation(s)
- Hongguang Pan
- College of Electrical and Control Engineering, Xi’an University of Science and Technology, Xi’an, 710054 Shaanxi China
- Xi’an Key Laboratory of Electrical Equipment Condition Monit oring and Power Supply Security, Xi’an, 710054 China
- Key Laboratory of Industrial Internet of Things & Networked Control, Ministry of Education,on Monit oring and Power Supply Security, Chongqing, 400065 China
| | - Yunpeng Fu
- College of Electrical and Control Engineering, Xi’an University of Science and Technology, Xi’an, 710054 Shaanxi China
- Xi’an Key Laboratory of Electrical Equipment Condition Monit oring and Power Supply Security, Xi’an, 710054 China
| | - Qi Zhang
- AVIC Xi’an Aviation Brake Technology Cl., Ltd, Xi’an, 710061 Shaanxi China
| | - Jingyuan Zhang
- College of Electrical and Control Engineering, Xi’an University of Science and Technology, Xi’an, 710054 Shaanxi China
- Xi’an Key Laboratory of Electrical Equipment Condition Monit oring and Power Supply Security, Xi’an, 710054 China
| | - Xuebin Qin
- College of Electrical and Control Engineering, Xi’an University of Science and Technology, Xi’an, 710054 Shaanxi China
- Xi’an Key Laboratory of Electrical Equipment Condition Monit oring and Power Supply Security, Xi’an, 710054 China
| |
Collapse
|
16
|
Oh J, Nam KW, Kim WJ, Kang BH, Park SH. Flexible Dry Electrode Based on a Wrinkled Surface That Uses Carbon Nanotube/Polymer Composites for Recording Electroencephalograms. MATERIALS (BASEL, SWITZERLAND) 2024; 17:668. [PMID: 38591516 PMCID: PMC10856397 DOI: 10.3390/ma17030668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 10/09/2023] [Accepted: 10/18/2023] [Indexed: 04/10/2024]
Abstract
Electroencephalography (EEG) captures minute electrical signals emanating from the brain. These signals are vulnerable to interference from external noise and dynamic artifacts; hence, accurately recording such signals is challenging. Although dry electrodes are convenient, their signals are of limited quality; consequently, wet electrodes are predominantly used in EEG. Therefore, developing dry electrodes for accurately and stably recording EEG signals is crucial. In this study, we developed flexible dry electrodes using polydimethylsiloxane (PDMS)/carbon-nanotube (CNT) composites with isotropically wrinkled surfaces that effectively combine the advantages of wet and dry electrodes. Adjusting the PDMS crosslinker ratio led to good adhesion, resulting in a highly adhesive CNT/PDMS composite with a low Young's modulus that exhibited excellent electrical and mechanical properties owing to its ability to conformally contact skin. The isotropically wrinkled surface also effectively controls dynamic artifacts during EEG signal detection and ensures accurate signal analysis. The results of this study demonstrate that dry electrodes based on flexible CNT/PDMS composites and corrugated structures can outperform wet electrodes. The introduction of such electrodes is expected to enable the accurate analysis and monitoring of EEG signals in various scenarios, including clinical trials.
Collapse
Affiliation(s)
| | | | | | | | - Sung-Hoon Park
- Department of Mechanical Engineering, Soongsil University, 369 Sangdo-ro, Dongjak-gu, Seoul 06978, Republic of Korea; (J.O.); (K.-W.N.); (W.-J.K.); (B.-H.K.)
| |
Collapse
|
17
|
Qin C, Yuan Q, Liu M, Zhuang L, Xu L, Wang P. Biohybrid tongue based on hypothalamic neuronal network-on-a-chip for real-time blood glucose sensing and assessment. Biosens Bioelectron 2024; 244:115784. [PMID: 37939416 DOI: 10.1016/j.bios.2023.115784] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Revised: 10/14/2023] [Accepted: 10/22/2023] [Indexed: 11/10/2023]
Abstract
The expression of sweet receptors in the hypothalamus has been implicated in energy homeostasis control and the pathogenesis of obesity and diabetes. However, the exact mechanism by which hypothalamic glucose-sensing neurons function remains unclear. Conventional detection methods, such as fiber photometry, optogenetics, brain-machine interfaces, patch clamp and calcium imaging, pose limitations for real-time glucose perception due to their complexity, cytotoxicity and so on. Therefore, this study proposes a biohybrid tongue based on hypothalamic neuronal network (HNN)-on-a-chip coupling with microelectrode array (MEA) for real-time glucose perception. Hypothalamic neuronal cultures were cultivated on a two-dimensional "brain-on-chip" device, enabling the formation of neuronal networks and electrophysiological signal detection. Additionally, we investigated the endogenous expression of sweet taste receptors (T1R2/T1R3) in hypothalamic neuronal cells, providing the basis for the biohybrid tongue based on HNN-on-a-chip's sweetness detection capabilities. The spike signal response to sucrose and glucose stimulation was detected, and concentration-dependent responses were explored with glucose concentrations ranging from 0.01 mM to 8 mM. MEAs allow for real-time recordings, enabling the observation of dynamic changes in neuronal responses to glucose fluctuations over time. The biohybrid tongue based on HNN-on-a-chip can measure various parameters, including spike frequency and amplitude, providing insights into neuronal firing patterns and excitability. Moreover, hypothalamic glucoregulatory neurons that sense and respond to changes in blood glucose was identified, including glucose-excited neurons (GE-Neurons) and glucose-inhibited neurons (GI-Neurons). The detection range for GE-Neurons spans from 0.4 to 6 mM, while GI-Neurons demonstrate sensitivity within the range of 1-8 mM. And the glucose detection limit was firmly established at 0.01 mM. Through non-linear regression analysis, the IC50 for GI-Neurons' spike firing was determined to be 4.18 mM. In conclusion, the biohybrid tongue based on HNN-on-a-chip offers a valuable in vitro tool for studying hypothalamic neurons, elucidating glucose sensing mechanisms, and understanding hypothalamic neuronal function.
Collapse
Affiliation(s)
- Chunlian Qin
- College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, 310058, China; ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China
| | - Qunchen Yuan
- Biosensor National Special Laboratory, Key Laboratory for Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou, 310027, China; Innovation Center for Smart Medical Technologies & Devices, Binjiang Institute of Zhejiang University, Hangzhou, Zhejiang, 310053, China
| | - Mengxue Liu
- Biosensor National Special Laboratory, Key Laboratory for Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou, 310027, China
| | - Liujing Zhuang
- Biosensor National Special Laboratory, Key Laboratory for Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou, 310027, China
| | - Lizhou Xu
- College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, 310058, China; ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China.
| | - Ping Wang
- Biosensor National Special Laboratory, Key Laboratory for Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou, 310027, China; Innovation Center for Smart Medical Technologies & Devices, Binjiang Institute of Zhejiang University, Hangzhou, Zhejiang, 310053, China.
| |
Collapse
|
18
|
Xiao X, Wang L, Xu M, Wang K, Jung TP, Ming D. A data expansion technique based on training and testing sample to boost the detection of SSVEPs for brain-computer interfaces. J Neural Eng 2023; 20:066017. [PMID: 37683663 DOI: 10.1088/1741-2552/acf7f6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2023] [Accepted: 09/08/2023] [Indexed: 09/10/2023]
Abstract
Objective.Currently, steady-state visual evoked potentials (SSVEPs)-based brain-computer interfaces (BCIs) have achieved the highest interaction accuracy and speed among all BCI paradigms. However, its decoding efficacy depends deeply on the number of training samples, and the system performance would have a dramatic drop when the training dataset decreased to a small size. To date, no study has been reported to incorporate the unsupervised learning information from testing trails into the construction of supervised classification model, which is a potential way to mitigate the overfitting effect of limited samples.Approach.This study proposed a novel method for SSVEPs detection, i.e. cyclic shift trials (CSTs), which could combine unsupervised learning information from test trials and supervised learning information from train trials. Furthermore, since SSVEPs are time-locked and phase-locked to the onset of specific flashes, CST could also expand training samples on the basis of its regularity and periodicity. In order to verify the effectiveness of CST, we designed an online SSVEP-BCI system, and tested this system combined CST with two common classification algorithms, i.e. extended canonical correlation analysis and ensemble task-related component analysis.Main results.CST could significantly enhance the signal to noise ratios of SSVEPs and improve the performance of systems especially for the condition of few training samples and short stimulus time. The online information transfer rate could reach up to 236.19 bits min-1using 36 s calibration time of only one training sample for each category.Significance.The proposed CST method can take full advantages of supervised learning information from training samples and unsupervised learning information of testing samples. Furthermore, it is a data expansion technique, which can enhance the SSVEP characteristics and reduce dependence on sample size. Above all, CST is a promising method to improve the performance of SSVEP-based BCI without any additional experimental burden.
Collapse
Affiliation(s)
- Xiaolin Xiao
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, People's Republic of China
- College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, People's Republic of China
- Haihe Laboratory of Brain-computer Interaction and Human-machine Integation, Tianjin 300392, People's Republic of China
| | - Lijie Wang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, People's Republic of China
- Research Center for Augmented Intelligence, Zhejiang Lab, Hangzhou, People's Republic of China
| | - Minpeng Xu
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, People's Republic of China
- College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, People's Republic of China
- Haihe Laboratory of Brain-computer Interaction and Human-machine Integation, Tianjin 300392, People's Republic of China
| | - Kun Wang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, People's Republic of China
- Haihe Laboratory of Brain-computer Interaction and Human-machine Integation, Tianjin 300392, People's Republic of China
| | - Tzyy-Ping Jung
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, People's Republic of China
- The Swartz Centre for Computational Neuroscience, University of California, San Diego, CA, United States of America
| | - Dong Ming
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, People's Republic of China
- College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, People's Republic of China
- Haihe Laboratory of Brain-computer Interaction and Human-machine Integation, Tianjin 300392, People's Republic of China
| |
Collapse
|
19
|
Canny E, Vansteensel MJ, van der Salm SMA, Müller-Putz GR, Berezutskaya J. Boosting brain-computer interfaces with functional electrical stimulation: potential applications in people with locked-in syndrome. J Neuroeng Rehabil 2023; 20:157. [PMID: 37980536 PMCID: PMC10656959 DOI: 10.1186/s12984-023-01272-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Accepted: 10/23/2023] [Indexed: 11/20/2023] Open
Abstract
Individuals with a locked-in state live with severe whole-body paralysis that limits their ability to communicate with family and loved ones. Recent advances in brain-computer interface (BCI) technology have presented a potential alternative for these people to communicate by detecting neural activity associated with attempted hand or speech movements and translating the decoded intended movements to a control signal for a computer. A technique that could potentially enrich the communication capacity of BCIs is functional electrical stimulation (FES) of paralyzed limbs and face to restore body and facial movements of paralyzed individuals, allowing to add body language and facial expression to communication BCI utterances. Here, we review the current state of the art of existing BCI and FES work in people with paralysis of body and face and propose that a combined BCI-FES approach, which has already proved successful in several applications in stroke and spinal cord injury, can provide a novel promising mode of communication for locked-in individuals.
Collapse
Affiliation(s)
- Evan Canny
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Mariska J Vansteensel
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Sandra M A van der Salm
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Laboratory of Brain-Computer Interfaces, Graz University of Technology, Graz, Austria
| | - Julia Berezutskaya
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands.
| |
Collapse
|
20
|
Wu CM, Chen YJ, Chen SC, Zheng SF. Creating an AI-Enhanced Morse Code Translation System Based on Images for People with Severe Disabilities. Bioengineering (Basel) 2023; 10:1281. [PMID: 38002405 PMCID: PMC10669406 DOI: 10.3390/bioengineering10111281] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Revised: 10/24/2023] [Accepted: 10/30/2023] [Indexed: 11/26/2023] Open
Abstract
(1) Background: Patients with severe physical impairments (spinal cord injury, cerebral palsy, amyotrophic lateral sclerosis) often have limited mobility due to physical limitations, and may even be bedridden all day long, losing the ability to take care of themselves. In more severe cases, the ability to speak may even be lost, making even basic communication very difficult. (2) Methods: This research will design a set of image-assistive communication equipment based on artificial intelligence to solve communication problems of daily needs. Using artificial intelligence for facial positioning, and facial-motion-recognition-generated Morse code, and then translating it into readable characters or commands, it allows users to control computer software by themselves and communicate through wireless networks or a Bluetooth protocol to control environment peripherals. (3) Results: In this study, 23 human-typed data sets were subjected to recognition using fuzzy algorithms. The average recognition rates for expert-generated data and data input by individuals with disabilities were 99.83% and 98.6%, respectively. (4) Conclusions: Through this system, users can express their thoughts and needs through their facial movements, thereby improving their quality of life and having an independent living space. Moreover, the system can be used without touching external switches, greatly improving convenience and safety.
Collapse
Affiliation(s)
- Chung-Min Wu
- Department of Intelligent Automation Engineering, National Chin-Yi University of Technology, Taichung 411030, Taiwan;
| | - Yeou-Jiunn Chen
- Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan 710301, Taiwan;
| | - Shih-Chung Chen
- Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan 710301, Taiwan;
| | - Sheng-Feng Zheng
- Department of Intelligent Robotics Engineering, Kun Shan University, Tainan 710303, Taiwan;
| |
Collapse
|
21
|
Lun X, Zhang Y, Zhu M, Lian Y, Hou Y. A Combined Virtual Electrode-Based ESA and CNN Method for MI-EEG Signal Feature Extraction and Classification. SENSORS (BASEL, SWITZERLAND) 2023; 23:8893. [PMID: 37960592 PMCID: PMC10649179 DOI: 10.3390/s23218893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Revised: 10/27/2023] [Accepted: 10/30/2023] [Indexed: 11/15/2023]
Abstract
A Brain-Computer Interface (BCI) is a medium for communication between the human brain and computers, which does not rely on other human neural tissues, but only decodes Electroencephalography (EEG) signals and converts them into commands to control external devices. Motor Imagery (MI) is an important BCI paradigm that generates a spontaneous EEG signal without external stimulation by imagining limb movements to strengthen the brain's compensatory function, and it has a promising future in the field of computer-aided diagnosis and rehabilitation technology for brain diseases. However, there are a series of technical difficulties in the research of motor imagery-based brain-computer interface (MI-BCI) systems, such as: large individual differences in subjects and poor performance of the cross-subject classification model; a low signal-to-noise ratio of EEG signals and poor classification accuracy; and the poor online performance of the MI-BCI system. To address the above problems, this paper proposed a combined virtual electrode-based EEG Source Analysis (ESA) and Convolutional Neural Network (CNN) method for MI-EEG signal feature extraction and classification. The outcomes reveal that the online MI-BCI system developed based on this method can improve the decoding ability of multi-task MI-EEG after training, it can learn generalized features from multiple subjects in cross-subject experiments and has some adaptability to the individual differences of new subjects, and it can decode the EEG intent online and realize the brain control function of the intelligent cart, which provides a new idea for the research of an online MI-BCI system.
Collapse
Affiliation(s)
| | | | | | | | - Yimin Hou
- School of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (X.L.); (Y.Z.); (M.Z.); (Y.L.)
| |
Collapse
|
22
|
Perna A, Angotzi GN, Berdondini L, Ribeiro JF. Advancing the interfacing performances of chronically implantable neural probes in the era of CMOS neuroelectronics. Front Neurosci 2023; 17:1275908. [PMID: 38027514 PMCID: PMC10644322 DOI: 10.3389/fnins.2023.1275908] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 10/10/2023] [Indexed: 12/01/2023] Open
Abstract
Tissue penetrating microelectrode neural probes can record electrophysiological brain signals at resolutions down to single neurons, making them invaluable tools for neuroscience research and Brain-Computer-Interfaces (BCIs). The known gradual decrease of their electrical interfacing performances in chronic settings, however, remains a major challenge. A key factor leading to such decay is Foreign Body Reaction (FBR), which is the cascade of biological responses that occurs in the brain in the presence of a tissue damaging artificial device. Interestingly, the recent adoption of Complementary Metal Oxide Semiconductor (CMOS) technology to realize implantable neural probes capable of monitoring hundreds to thousands of neurons simultaneously, may open new opportunities to face the FBR challenge. Indeed, this shift from passive Micro Electro-Mechanical Systems (MEMS) to active CMOS neural probe technologies creates important, yet unexplored, opportunities to tune probe features such as the mechanical properties of the probe, its layout, size, and surface physicochemical properties, to minimize tissue damage and consequently FBR. Here, we will first review relevant literature on FBR to provide a better understanding of the processes and sources underlying this tissue response. Methods to assess FBR will be described, including conventional approaches based on the imaging of biomarkers, and more recent transcriptomics technologies. Then, we will consider emerging opportunities offered by the features of CMOS probes. Finally, we will describe a prototypical neural probe that may meet the needs for advancing clinical BCIs, and we propose axial insertion force as a potential metric to assess the influence of probe features on acute tissue damage and to control the implantation procedure to minimize iatrogenic injury and subsequent FBR.
Collapse
Affiliation(s)
- Alberto Perna
- Microtechnology for Neuroelectronics Lab, Fondazione Istituto Italiano di Tecnologia, Neuroscience and Brain Technologies, Genova, Italy
- The Open University Affiliated Research Centre at Istituto Italiano di Tecnologia (ARC@IIT), Istituto Italiano di Tecnologia, Genova, Italy
| | - Gian Nicola Angotzi
- Microtechnology for Neuroelectronics Lab, Fondazione Istituto Italiano di Tecnologia, Neuroscience and Brain Technologies, Genova, Italy
| | - Luca Berdondini
- Microtechnology for Neuroelectronics Lab, Fondazione Istituto Italiano di Tecnologia, Neuroscience and Brain Technologies, Genova, Italy
| | - João Filipe Ribeiro
- Microtechnology for Neuroelectronics Lab, Fondazione Istituto Italiano di Tecnologia, Neuroscience and Brain Technologies, Genova, Italy
| |
Collapse
|
23
|
Zhang X, Chen S, Wang Y. Kernel Reinforcement Learning-Assisted Adaptive Decoder Facilitates Stable and Continuous Brain Control Tasks. IEEE Trans Neural Syst Rehabil Eng 2023; 31:4125-4134. [PMID: 37792657 DOI: 10.1109/tnsre.2023.3321756] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/06/2023]
Abstract
Brain-Machine Interfaces (BMIs) assist paralyzed people to brain control (BC) the neuro-prosthesis continuously moving in space. During the BC process, the subject imagines the movement of the real limb and adapts the brain activity according to the sensory feedback. The neural adaptation in the closed-loop control results in complex and changing brain signals. Simultaneously, the decoder interprets the time-varying functional mapping between neural activity and continuous trajectory. It is crucial and challenging to accurately and adaptively track the mapping to help the subject accomplish the BC task with a stable performance. Existing Kalman Filter (KF) based decoders achieve continuous trajectory control by linearly interpreting neural firing observations into self-evolving prosthetic states. However, the linear neural-state mapping might not accurately reflect the movement intention of the subject. In this paper, we propose a novel method that allows subjects to achieve continuous brain control efficiently and stably. The proposed method incorporates a kernel reinforcement learning method into a state-observation model to decode the nonlinearly neural observation into a continuous trajectory state. The state transition function ensures the continuity of the prosthetic state. The kernel reinforcement learning allows the quick adaptation of the nonlinear neural-movement mapping during the BC process. The proposed method is tested in an online brain control reaching task for rats. Compared with KF, our method achieved more successful trials, faster response time, shorter inter-trial time, and remained stable over days. These results demonstrate that the proposed method is an efficient tool to assist subjects in brain control tasks.
Collapse
|
24
|
Coughlin B, Muñoz W, Kfir Y, Young MJ, Meszéna D, Jamali M, Caprara I, Hardstone R, Khanna A, Mustroph ML, Trautmann EM, Windolf C, Varol E, Soper DJ, Stavisky SD, Welkenhuysen M, Dutta B, Shenoy KV, Hochberg LR, Mark Richardson R, Williams ZM, Cash SS, Paulk AC. Modified Neuropixels probes for recording human neurophysiology in the operating room. Nat Protoc 2023; 18:2927-2953. [PMID: 37697108 DOI: 10.1038/s41596-023-00871-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 06/08/2023] [Indexed: 09/13/2023]
Abstract
Neuropixels are silicon-based electrophysiology-recording probes with high channel count and recording-site density. These probes offer a turnkey platform for measuring neural activity with single-cell resolution and at a scale that is beyond the capabilities of current clinically approved devices. Our team demonstrated the first-in-human use of these probes during resection surgery for epilepsy or tumors and deep brain stimulation electrode placement in patients with Parkinson's disease. Here, we provide a better understanding of the capabilities and challenges of using Neuropixels as a research tool to study human neurophysiology, with the hope that this information may inform future efforts toward regulatory approval of Neuropixels probes as research devices. In perioperative procedures, the major concerns are the initial sterility of the device, maintaining a sterile field during surgery, having multiple referencing and grounding schemes available to de-noise recordings (if necessary), protecting the silicon probe from accidental contact before insertion and obtaining high-quality action potential and local field potential recordings. The research team ensures that the device is fully operational while coordinating with the surgical team to remove sources of electrical noise that could otherwise substantially affect the signals recorded by the sensitive hardware. Prior preparation using the equipment and training in human clinical research and working in operating rooms maximize effective communication within and between the teams, ensuring high recording quality and minimizing the time added to the surgery. The perioperative procedure requires ~4 h, and the entire protocol requires multiple weeks.
Collapse
Affiliation(s)
- Brian Coughlin
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - William Muñoz
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Yoav Kfir
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Michael J Young
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - Domokos Meszéna
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - Mohsen Jamali
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Irene Caprara
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Richard Hardstone
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - Arjun Khanna
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Martina L Mustroph
- Department of Neurosurgery, Harvard Medical School and Brigham & Women's Hospital, Boston, MA, USA
| | - Eric M Trautmann
- Department of Neuroscience, Columbia University Medical Center, New York, NY, USA
- Zuckerman Institute, Columbia University, New York, NY, USA
- Grossman Center for the Statistics of Mind, Columbia University Medical Center, New York, NY, USA
| | - Charlie Windolf
- Department of Statistics, Zuckerman Institute, Columbia University, New York, NY, USA
| | - Erdem Varol
- Department of Statistics, Zuckerman Institute, Columbia University, New York, NY, USA
- Department of Computer Science and Engineering, Zuckerman Institute, Columbia University, New York, NY, USA
| | - Dan J Soper
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - Sergey D Stavisky
- Department of Neurological Surgery, University of California Davis, Davis, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| | | | | | - Krishna V Shenoy
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
| | - Leigh R Hochberg
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
- VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI, USA
- School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA
| | - R Mark Richardson
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Ziv M Williams
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA.
| | - Sydney S Cash
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA.
- Department of Neurology, Harvard Medical School, Boston, MA, USA.
| | - Angelique C Paulk
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA.
- Department of Neurology, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
25
|
Zotey V, Andhale A, Shegekar T, Juganavar A. Adaptive Neuroplasticity in Brain Injury Recovery: Strategies and Insights. Cureus 2023; 15:e45873. [PMID: 37885532 PMCID: PMC10598326 DOI: 10.7759/cureus.45873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Accepted: 09/24/2023] [Indexed: 10/28/2023] Open
Abstract
This review addresses the relationship between neuroplasticity and recovery from brain damage. Neuroplasticity's ability to adapt becomes crucial since brain injuries frequently result in severe impairments. We begin by describing the fundamentals of neuroplasticity and how it relates to rehabilitation. Examining different forms of brain injuries and their neurological effects highlights the complex difficulties in rehabilitation. By revealing cellular processes, we shed light on synaptic adaptability following damage. Our study of synaptic plasticity digs into axonal sprouting, dendritic remodeling, and the balance of long-term potentiation. These processes depict neural resilience amid change. Then, after damage, we investigate immediate and slow neuroplastic alterations, separating reorganizations that are adaptive from those that are maladaptive. As we go on to rehabilitation, we evaluate techniques that use neuroplasticity's potential. These methods take advantage of the brain's plasticity for healing, from virtual reality and brain-computer interfaces to constraint-induced movement therapy. Ethics and individualized neurorehabilitation are explored. We scrutinize the promise of combination therapy and the difficulties in putting new knowledge into clinical practice. In conclusion, this analysis highlights neuroplasticity's critical role in brain injury recovery, providing sophisticated approaches to improve life after damage.
Collapse
Affiliation(s)
- Vaishnavi Zotey
- Medicine, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education and Research, Wardha, IND
| | - Amol Andhale
- Medicine, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education and Research, Wardha, IND
| | - Tejas Shegekar
- Medicine, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education and Research, Wardha, IND
| | - Anup Juganavar
- Medicine, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education and Research, Wardha, IND
| |
Collapse
|
26
|
Liu D, Dai W, Zhang H, Jin X, Cao J, Kong W. Brain-Machine Coupled Learning Method for Facial Emotion Recognition. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2023; 45:10703-10717. [PMID: 37030724 DOI: 10.1109/tpami.2023.3257846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Neural network models of machine learning have shown promising prospects for visual tasks, such as facial emotion recognition (FER). However, the generalization of the model trained from a dataset with a few samples is limited. Unlike the machine, the human brain can effectively realize the required information from a few samples to complete the visual tasks. To learn the generalization ability of the brain, in this article, we propose a novel brain-machine coupled learning method for facial emotion recognition to let the neural network learn the visual knowledge of the machine and cognitive knowledge of the brain simultaneously. The proposed method utilizes visual images and electroencephalogram (EEG) signals to couple training the models in the visual and cognitive domains. Each domain model consists of two types of interactive channels, common and private. Since the EEG signals can reflect brain activity, the cognitive process of the brain is decoded by a model following reverse engineering. Decoding the EEG signals induced by the facial emotion images, the common channel in the visual domain can approach the cognitive process in the cognitive domain. Moreover, the knowledge specific to each domain is found in each private channel using an adversarial strategy. After learning, without the participation of the EEG signals, only the concatenation of both channels in the visual domain is used to classify facial emotion images based on the visual knowledge of the machine and the cognitive knowledge learned from the brain. Experiments demonstrate that the proposed method can produce excellent performance on several public datasets. Further experiments show that the proposed method trained from the EEG signals has good generalization ability on new datasets and can be applied to other network models, illustrating the potential for practical applications.
Collapse
|
27
|
Ziai Y, Zargarian SS, Rinoldi C, Nakielski P, Sola A, Lanzi M, Truong YB, Pierini F. Conducting polymer-based nanostructured materials for brain-machine interfaces. WILEY INTERDISCIPLINARY REVIEWS. NANOMEDICINE AND NANOBIOTECHNOLOGY 2023; 15:e1895. [PMID: 37141863 DOI: 10.1002/wnan.1895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Revised: 03/14/2023] [Accepted: 04/05/2023] [Indexed: 05/06/2023]
Abstract
As scientists discovered that raw neurological signals could translate into bioelectric information, brain-machine interfaces (BMI) for experimental and clinical studies have experienced massive growth. Developing suitable materials for bioelectronic devices to be used for real-time recording and data digitalizing has three important necessitates which should be covered. Biocompatibility, electrical conductivity, and having mechanical properties similar to soft brain tissue to decrease mechanical mismatch should be adopted for all materials. In this review, inorganic nanoparticles and intrinsically conducting polymers are discussed to impart electrical conductivity to systems, where soft materials such as hydrogels can offer reliable mechanical properties and a biocompatible substrate. Interpenetrating hydrogel networks offer more mechanical stability and provide a path for incorporating polymers with desired properties into one strong network. Promising fabrication methods, like electrospinning and additive manufacturing, allow scientists to customize designs for each application and reach the maximum potential for the system. In the near future, it is desired to fabricate biohybrid conducting polymer-based interfaces loaded with cells, giving the opportunity for simultaneous stimulation and regeneration. Developing multi-modal BMIs, Using artificial intelligence and machine learning to design advanced materials are among the future goals for this field. This article is categorized under: Therapeutic Approaches and Drug Discovery > Nanomedicine for Neurological Disease.
Collapse
Affiliation(s)
- Yasamin Ziai
- Department of Biosystems and Soft Matter, Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
| | - Seyed Shahrooz Zargarian
- Department of Biosystems and Soft Matter, Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
| | - Chiara Rinoldi
- Department of Biosystems and Soft Matter, Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
| | - Paweł Nakielski
- Department of Biosystems and Soft Matter, Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
| | - Antonella Sola
- Commonwealth Scientific and Industrial Research Organisation (CSIRO), Manufacturing Business Unit, Clayton, Victoria, Australia
| | - Massimiliano Lanzi
- Department of Industrial Chemistry "Toso Montanari", University of Bologna, Bologna, Italy
| | - Yen Bach Truong
- Commonwealth Scientific and Industrial Research Organisation (CSIRO), Manufacturing Business Unit, Clayton, Victoria, Australia
| | - Filippo Pierini
- Department of Biosystems and Soft Matter, Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
| |
Collapse
|
28
|
Albanese GA, Zenzeri J, De Santis D. The Effect of Feedback Modality When Learning a Novel Wrist Sensorimotor Transformation Through a Body-Machine Interface. IEEE Int Conf Rehabil Robot 2023; 2023:1-6. [PMID: 37941291 DOI: 10.1109/icorr58425.2023.10304784] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2023]
Abstract
Body-Machine Interfaces (BoMIs) are promising assistive and rehabilitative tools for helping individuals with impaired motor abilities regain independence. When operating a BoMI, the user has to learn a novel sensorimotor transformation between the movement of certain body parts and the output of the device. In this study, we investigated how different feedback modalities impacted learning to operate a BoMI. Forty-seven able-bodied participants learned to control the velocity of a 1D cursor using the 3D rotation of their dominant wrist to reach as many targets as possible in a given amount of time. The map was designed to maximize cursor speed for movements around a predefined axis of wrist rotation. We compared the user's performance and control efficiency under three feedback modalities: i) visual feedback of the cursor position, ii) proprioceptive feedback of the cursor position delivered by a wrist manipulandum, iii) both i) and ii). We found that visual feedback led to a greater number of targets reached than proprioceptive feedback alone. Conversely, proprioceptive feedback yielded greater alignment between the axis of rotation of the wrist and the optimal axis represented by the map. These results suggest that proprioceptive feedback may be preferable over visual feedback when information about intrinsic task components, i.e. joint configurations, is of interest as in rehabilitative interventions aiming to promote more effective learning strategies.
Collapse
|
29
|
Nara S, Raza H, Carreiras M, Molinaro N. Decoding numeracy and literacy in the human brain: insights from MEG and MVPA. Sci Rep 2023; 13:10979. [PMID: 37414784 PMCID: PMC10326015 DOI: 10.1038/s41598-023-37113-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Accepted: 06/15/2023] [Indexed: 07/08/2023] Open
Abstract
Numbers and letters are the fundamental building blocks of our everyday social interactions. Previous studies have focused on determining the cortical pathways shaped by numeracy and literacy in the human brain, partially supporting the hypothesis of distinct perceptual neural circuits involved in the visual processing of the two categories. In this study, we aim to investigate the temporal dynamics for number and letter processing. We present magnetoencephalography (MEG) data from two experiments (N = 25 each). In the first experiment, single numbers, letters, and their respective false fonts (false numbers and false letters) were presented, whereas, in the second experiment, numbers, letters, and their respective false fonts were presented as a string of characters. We used multivariate pattern analysis techniques (time-resolved decoding and temporal generalization), testing the strong hypothesis that the neural correlates supporting letter and number processing can be logistically classified as categorically separate. Our results show a very early dissociation (~ 100 ms) between numbers, and letters when compared to false fonts. Number processing can be dissociated with similar accuracy when presented as isolated items or strings of characters, while letter processing shows dissociable classification accuracy for single items compared to strings. These findings reinforce the evidence indicating that early visual processing can be differently shaped by the experience with numbers and letters; this dissociation is stronger for strings compared to single items, thus showing that combinatorial mechanisms for numbers and letters could be categorically distinguished and influence early visual processing.
Collapse
Affiliation(s)
- Sanjeev Nara
- Department of Mathematics and Computer Science, Physics, Geography, Mathematical Institute, Justus-Liebig-Universität Gießen, Gießen, Germany.
- Basque Center on Cognition, Brain and Language, Donostia/San Sebastian, Spain.
| | - Haider Raza
- School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK
| | - Manuel Carreiras
- Basque Center on Cognition, Brain and Language, Donostia/San Sebastian, Spain
- Ikerbasque, Basque Foundation for Science, Bilbao, Spain
| | - Nicola Molinaro
- Basque Center on Cognition, Brain and Language, Donostia/San Sebastian, Spain
- Ikerbasque, Basque Foundation for Science, Bilbao, Spain
| |
Collapse
|
30
|
Li Y, Chen B, Wang G, Yoshimura N, Koike Y. Partial maximum correntropy regression for robust electrocorticography decoding. Front Neurosci 2023; 17:1213035. [PMID: 37457015 PMCID: PMC10347400 DOI: 10.3389/fnins.2023.1213035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Accepted: 06/15/2023] [Indexed: 07/18/2023] Open
Abstract
The Partial Least Square Regression (PLSR) method has shown admirable competence for predicting continuous variables from inter-correlated electrocorticography signals in the brain-computer interface. However, PLSR is essentially formulated with the least square criterion, thus, being considerably prone to the performance deterioration caused by the brain recording noises. To address this problem, this study aims to propose a new robust variant for PLSR. To this end, the maximum correntropy criterion (MCC) is utilized to propose a new robust implementation of PLSR, called Partial Maximum Correntropy Regression (PMCR). The half-quadratic optimization is utilized to calculate the robust projectors for the dimensionality reduction, and the regression coefficients are optimized by a fixed-point optimization method. The proposed PMCR is evaluated with a synthetic example and a public electrocorticography dataset under three performance indicators. For the synthetic example, PMCR realized better prediction results compared with the other existing methods. PMCR could also abstract valid information with a limited number of decomposition factors in a noisy regression scenario. For the electrocorticography dataset, PMCR achieved superior decoding performance in most cases, and also realized the minimal neurophysiological pattern deterioration with the interference of the noises. The experimental results demonstrate that, the proposed PMCR could outperform the existing methods in a noisy, inter-correlated, and high-dimensional decoding task. PMCR could alleviate the performance degradation caused by the adverse noises and ameliorate the electrocorticography decoding robustness for the brain-computer interface.
Collapse
Affiliation(s)
- Yuanhao Li
- Institute of Innovative Research, Tokyo Institute of Technology, Yokohama, Japan
| | - Badong Chen
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
| | - Gang Wang
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, Xi'an Jiaotong University, Xi'an, China
| | - Natsue Yoshimura
- School of Computing, Tokyo Institute of Technology, Yokohama, Japan
| | - Yasuharu Koike
- Institute of Innovative Research, Tokyo Institute of Technology, Yokohama, Japan
| |
Collapse
|
31
|
Sen O, Sheehan AM, Raman PR, Khara KS, Khalifa A, Chatterjee B. Machine-Learning Methods for Speech and Handwriting Detection Using Neural Signals: A Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:5575. [PMID: 37420741 DOI: 10.3390/s23125575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Revised: 06/09/2023] [Accepted: 06/12/2023] [Indexed: 07/09/2023]
Abstract
Brain-Computer Interfaces (BCIs) have become increasingly popular in recent years due to their potential applications in diverse fields, ranging from the medical sector (people with motor and/or communication disabilities), cognitive training, gaming, and Augmented Reality/Virtual Reality (AR/VR), among other areas. BCI which can decode and recognize neural signals involved in speech and handwriting has the potential to greatly assist individuals with severe motor impairments in their communication and interaction needs. Innovative and cutting-edge advancements in this field have the potential to develop a highly accessible and interactive communication platform for these people. The purpose of this review paper is to analyze the existing research on handwriting and speech recognition from neural signals. So that the new researchers who are interested in this field can gain thorough knowledge in this research area. The current research on neural signal-based recognition of handwriting and speech has been categorized into two main types: invasive and non-invasive studies. We have examined the latest papers on converting speech-activity-based neural signals and handwriting-activity-based neural signals into text data. The methods of extracting data from the brain have also been discussed in this review. Additionally, this review includes a brief summary of the datasets, preprocessing techniques, and methods used in these studies, which were published between 2014 and 2022. This review aims to provide a comprehensive summary of the methodologies used in the current literature on neural signal-based recognition of handwriting and speech. In essence, this article is intended to serve as a valuable resource for future researchers who wish to investigate neural signal-based machine-learning methods in their work.
Collapse
Affiliation(s)
- Ovishake Sen
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | - Anna M Sheehan
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | - Pranay R Raman
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | - Kabir S Khara
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | - Adam Khalifa
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | | |
Collapse
|
32
|
Xiao X, Gao R, Zhou X, Yi W, Xu F, Wang K, Xu M, Ming D. A novel visual brain-computer interfaces paradigm based on evoked related potentials evoked by weak and small number of stimuli. Front Neurosci 2023; 17:1178283. [PMID: 37342465 PMCID: PMC10278229 DOI: 10.3389/fnins.2023.1178283] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Accepted: 03/20/2023] [Indexed: 06/23/2023] Open
Abstract
Introduction Traditional visual Brain-Computer Interfaces (v-BCIs) usually use large-size stimuli to attract more attention from users and then elicit more distinct and robust EEG responses, which would cause visual fatigue and limit the length of use of the system. On the contrary, small-size stimuli always need multiple and repeated stimulus to code more instructions and increase separability among each code. These common v-BCIs paradigms can cause problems such as redundant coding, long calibration time, and visual fatigue. Methods To address these problems, this study presented a novel v-BCI paradigm using weak and small number of stimuli, and realized a nine-instruction v-BCI system that controlled by only three tiny stimuli. Each of these stimuli were located between instructions, occupied area with eccentricities subtended 0.4°, and flashed in the row-column paradigm. The weak stimuli around each instruction would evoke specific evoked related potentials (ERPs), and a template-matching method based on discriminative spatial pattern (DSP) was employed to recognize these ERPs containing the intention of users. Nine subjects participated in the offline and online experiments using this novel paradigm. Results The average accuracy of the offline experiment was 93.46% and the online average information transfer rate (ITR) was 120.95 bits/min. Notably, the highest online ITR achieved 177.5 bits/min. Discussion These results demonstrate the feasibility of using a weak and small number of stimuli to implement a friendly v-BCI. Furthermore, the proposed novel paradigm achieved higher ITR than traditional ones using ERPs as the controlled signal, which showed its superior performance and may have great potential of being widely used in various fields.
Collapse
Affiliation(s)
- Xiaolin Xiao
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Runyuan Gao
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
| | - Xiaoyu Zhou
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
| | - Weibo Yi
- Beijing Institute of Mechanical Equipment, Beijing, China
| | - Fangzhou Xu
- International School for Optoelectronic Engineering, Qilu University of Technology, Shandong Academy of Sciences, Jinan, China
| | - Kun Wang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Minpeng Xu
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
- International School for Optoelectronic Engineering, Qilu University of Technology, Shandong Academy of Sciences, Jinan, China
| | - Dong Ming
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| |
Collapse
|
33
|
Abdeladim L, Shin H, Jagadisan UK, Ogando MB, Adesnik H. Probing inter-areal computations with a cellular resolution two-photon holographic mesoscope. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.03.02.530875. [PMID: 37090604 PMCID: PMC10120651 DOI: 10.1101/2023.03.02.530875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
Brain computation depends on intricately connected yet highly distributed neural networks. Due to the absence of the requisite technologies, causally testing fundamental hypotheses on the nature of inter-areal processing have remained largely out-of-each. Here we developed the first two photon holographic mesoscope, a system capable of simultaneously reading and writing neural activity patterns with single cell resolution across large regions of the brain. We demonstrate the precise photo-activation of spatial and temporal sequences of neurons in one brain area while reading out the downstream effect in several other regions. Investigators can use this new platform to understand feed-forward and feed-back processing in distributed neural circuits with single cell precision for the first time.
Collapse
|
34
|
Farina D, Vujaklija I, Brånemark R, Bull AMJ, Dietl H, Graimann B, Hargrove LJ, Hoffmann KP, Huang HH, Ingvarsson T, Janusson HB, Kristjánsson K, Kuiken T, Micera S, Stieglitz T, Sturma A, Tyler D, Weir RFF, Aszmann OC. Toward higher-performance bionic limbs for wider clinical use. Nat Biomed Eng 2023; 7:473-485. [PMID: 34059810 DOI: 10.1038/s41551-021-00732-x] [Citation(s) in RCA: 87] [Impact Index Per Article: 87.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2019] [Accepted: 04/01/2021] [Indexed: 12/19/2022]
Abstract
Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.
Collapse
Affiliation(s)
- Dario Farina
- Department of Bioengineering, Imperial College London, London, UK.
| | - Ivan Vujaklija
- Department of Electrical Engineering and Automation, Aalto University, Espoo, Finland
| | - Rickard Brånemark
- Center for Extreme Bionics, Biomechatronics Group, MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Anthony M J Bull
- Department of Bioengineering, Imperial College London, London, UK
| | - Hans Dietl
- Ottobock Products SE & Co. KGaA, Vienna, Austria
| | | | - Levi J Hargrove
- Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL, USA
- Department of Physical Medicine & Rehabilitation, Northwestern University, Chicago, IL, USA
- Department of Biomedical Engineering, Northwestern University, Chicago, IL, USA
| | - Klaus-Peter Hoffmann
- Department of Medical Engineering & Neuroprosthetics, Fraunhofer-Institut für Biomedizinische Technik, Sulzbach, Germany
| | - He Helen Huang
- NCSU/UNC Joint Department of Biomedical Engineering, North Carolina State University, Raleigh, NC, USA
- University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Thorvaldur Ingvarsson
- Department of Research and Development, Össur Iceland, Reykjavík, Iceland
- Faculty of Medicine, University of Iceland, Reykjavík, Iceland
| | - Hilmar Bragi Janusson
- School of Engineering and Natural Sciences, University of Iceland, Reykjavík, Iceland
| | | | - Todd Kuiken
- Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL, USA
- Department of Physical Medicine & Rehabilitation, Northwestern University, Chicago, IL, USA
- Department of Biomedical Engineering, Northwestern University, Chicago, IL, USA
| | - Silvestro Micera
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pontedera, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pontedera, Italy
- Bertarelli Foundation Chair in Translational NeuroEngineering, Center for Neuroprosthetics and Institute of Bioengineering, School of Engineering, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland
| | - Thomas Stieglitz
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering-IMTEK, BrainLinks-BrainTools Center and Bernstein Center Freiburg, University of Freiburg, Freiburg, Germany
| | - Agnes Sturma
- Department of Bioengineering, Imperial College London, London, UK
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic and Reconstructive Surgery, Medical University of Vienna, Vienna, Austria
| | - Dustin Tyler
- Case School of Engineering, Case Western Reserve University, Cleveland, OH, USA
- Louis Stokes Veterans Affairs Medical Centre, Cleveland, OH, USA
| | - Richard F Ff Weir
- Biomechatronics Development Laboratory, Bioengineering Department, University of Colorado Denver and VA Eastern Colorado Healthcare System, Aurora, CO, USA
| | - Oskar C Aszmann
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic and Reconstructive Surgery, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
35
|
Jeakle EN, Abbott JR, Usoro JO, Wu Y, Haghighi P, Radhakrishna R, Sturgill BS, Nakajima S, Thai TTD, Pancrazio JJ, Cogan SF, Hernandez-Reynoso AG. Chronic Stability of Local Field Potentials Using Amorphous Silicon Carbide Microelectrode Arrays Implanted in the Rat Motor Cortex. MICROMACHINES 2023; 14:680. [PMID: 36985087 PMCID: PMC10054633 DOI: 10.3390/mi14030680] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 03/15/2023] [Accepted: 03/17/2023] [Indexed: 06/18/2023]
Abstract
Implantable microelectrode arrays (MEAs) enable the recording of electrical activity of cortical neurons, allowing the development of brain-machine interfaces. However, MEAs show reduced recording capabilities under chronic conditions, prompting the development of novel MEAs that can improve long-term performance. Conventional planar, silicon-based devices and ultra-thin amorphous silicon carbide (a-SiC) MEAs were implanted in the motor cortex of female Sprague-Dawley rats, and weekly anesthetized recordings were made for 16 weeks after implantation. The spectral density and bandpower between 1 and 500 Hz of recordings were compared over the implantation period for both device types. Initially, the bandpower of the a-SiC devices and standard MEAs was comparable. However, the standard MEAs showed a consistent decline in both bandpower and power spectral density throughout the 16 weeks post-implantation, whereas the a-SiC MEAs showed substantially more stable performance. These differences in bandpower and spectral density between standard and a-SiC MEAs were statistically significant from week 6 post-implantation until the end of the study at 16 weeks. These results support the use of ultra-thin a-SiC MEAs to develop chronic, reliable brain-machine interfaces.
Collapse
Affiliation(s)
- Eleanor N. Jeakle
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Justin R. Abbott
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Joshua O. Usoro
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Yupeng Wu
- Department of Materials Science and Engineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Pegah Haghighi
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Rahul Radhakrishna
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Brandon S. Sturgill
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Shido Nakajima
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Teresa T. D. Thai
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Joseph J. Pancrazio
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Stuart F. Cogan
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| | - Ana G. Hernandez-Reynoso
- Department of Bioengineering, The University of Texas at Dallas, 800 W. Campbell Road, Richardson, TX 75080-3021, USA
| |
Collapse
|
36
|
Chiarion G, Sparacino L, Antonacci Y, Faes L, Mesin L. Connectivity Analysis in EEG Data: A Tutorial Review of the State of the Art and Emerging Trends. Bioengineering (Basel) 2023; 10:bioengineering10030372. [PMID: 36978763 PMCID: PMC10044923 DOI: 10.3390/bioengineering10030372] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 03/10/2023] [Accepted: 03/14/2023] [Indexed: 03/30/2023] Open
Abstract
Understanding how different areas of the human brain communicate with each other is a crucial issue in neuroscience. The concepts of structural, functional and effective connectivity have been widely exploited to describe the human connectome, consisting of brain networks, their structural connections and functional interactions. Despite high-spatial-resolution imaging techniques such as functional magnetic resonance imaging (fMRI) being widely used to map this complex network of multiple interactions, electroencephalographic (EEG) recordings claim high temporal resolution and are thus perfectly suitable to describe either spatially distributed and temporally dynamic patterns of neural activation and connectivity. In this work, we provide a technical account and a categorization of the most-used data-driven approaches to assess brain-functional connectivity, intended as the study of the statistical dependencies between the recorded EEG signals. Different pairwise and multivariate, as well as directed and non-directed connectivity metrics are discussed with a pros-cons approach, in the time, frequency, and information-theoretic domains. The establishment of conceptual and mathematical relationships between metrics from these three frameworks, and the discussion of novel methodological approaches, will allow the reader to go deep into the problem of inferring functional connectivity in complex networks. Furthermore, emerging trends for the description of extended forms of connectivity (e.g., high-order interactions) are also discussed, along with graph-theory tools exploring the topological properties of the network of connections provided by the proposed metrics. Applications to EEG data are reviewed. In addition, the importance of source localization, and the impacts of signal acquisition and pre-processing techniques (e.g., filtering, source localization, and artifact rejection) on the connectivity estimates are recognized and discussed. By going through this review, the reader could delve deeply into the entire process of EEG pre-processing and analysis for the study of brain functional connectivity and learning, thereby exploiting novel methodologies and approaches to the problem of inferring connectivity within complex networks.
Collapse
Affiliation(s)
- Giovanni Chiarion
- Mathematical Biology and Physiology, Department Electronics and Telecommunications, Politecnico di Torino, 10129 Turin, Italy
| | - Laura Sparacino
- Department of Engineering, University of Palermo, 90128 Palermo, Italy
| | - Yuri Antonacci
- Department of Engineering, University of Palermo, 90128 Palermo, Italy
| | - Luca Faes
- Department of Engineering, University of Palermo, 90128 Palermo, Italy
| | - Luca Mesin
- Mathematical Biology and Physiology, Department Electronics and Telecommunications, Politecnico di Torino, 10129 Turin, Italy
| |
Collapse
|
37
|
Asahina T, Shimba K, Kotani K, Jimbo Y. Improving the accuracy of decoding monkey brain-machine interface data by estimating the state of unobserved cell assemblies. J Neurosci Methods 2023; 385:109764. [PMID: 36476748 DOI: 10.1016/j.jneumeth.2022.109764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Revised: 11/27/2022] [Accepted: 12/02/2022] [Indexed: 12/12/2022]
Abstract
BACKGROUND The brain-machine interface is a technology that has been used for improving the quality of life of individuals with physical disabilities and also healthy individuals. It is important to improve the methods used for decoding the brain-machine interface data as the accuracy and speed of movements achieved using the existing technology are not comparable to the normal body. COMPARISON WITH THE EXISTING METHOD Decoding of brain-machine interface data using the proposed method resulted in improved decoding accuracy compared to the existing method. CONCLUSIONS The results demonstrated the usefulness of cell assembly state estimation method for decoding the brain-machine interface data. NEW METHOD We incorporated a novel method of estimating cell assembly states using spike trains with the existing decoding method that used only firing rate data. Synaptic connectivity pattern was used as feature values in addition to firing rate. Publicly available monkey brain-machine interface datasets were used in the study. RESULTS As long as the decoding was successful, the root mean square error of the proposed method was significantly smaller than the existing method. Artificial neural netowork-based decoding method resulted in more stable decoding, and also improved the decoding accuracy due to incorporation of synaptic connectivity pattern.
Collapse
Affiliation(s)
- Takahiro Asahina
- School of Engineering, The University of Tokyo, Tokyo, Japan; Japan Society for the Promotion of Science, Japan.
| | - Kenta Shimba
- School of Engineering, The University of Tokyo, Tokyo, Japan
| | - Kiyoshi Kotani
- Research Center for Advanced Science and Technology, The University of Tokyo, Tokyo, Japan
| | - Yasuhiko Jimbo
- School of Engineering, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
38
|
Parallel genetic algorithm based common spatial patterns selection on time–frequency decomposed EEG signals for motor imagery brain-computer interface. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104397] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
39
|
Fang K, Mei H, Tang Y, Wang W, Wang H, Wang Z, Dai Z. Grade-control outdoor turning flight of robo-pigeon with quantitative stimulus parameters. Front Neurorobot 2023; 17:1143601. [PMID: 37139263 PMCID: PMC10149694 DOI: 10.3389/fnbot.2023.1143601] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 03/29/2023] [Indexed: 05/05/2023] Open
Abstract
Introduction The robo-pigeon using homing pigeons as a motion carrier has great potential in search and rescue operations due to its superior weight-bearing capacity and sustained flight capabilities. However, before deploying such robo-pigeons, it is necessary to establish a safe, stable, and long-term effective neuro-electrical stimulation interface and quantify the motion responses to various stimuli. Methods In this study, we investigated the effects of stimulation variables such as stimulation frequency (SF), stimulation duration (SD), and inter-stimulus interval (ISI) on the turning flight control of robo-pigeons outdoors, and evaluated the efficiency and accuracy of turning flight behavior accordingly. Results The results showed that the turning angle can be significantly controlled by appropriately increasing SF and SD. Increasing ISI can significantly control the turning radius of robotic pigeons. The success rate of turning flight control decreases significantly when the stimulation parameters exceed SF > 100 Hz or SD > 5 s. Thus, the robo-pigeon's turning angle from 15 to 55° and turning radius from 25 to 135 m could be controlled in a graded manner by selecting varying stimulus variables. Discussion These findings can be used to optimize the stimulation strategy of robo-pigeons to achieve precise control of their turning flight behavior outdoors. The results also suggest that robo-pigeons have potential for use in search and rescue operations where precise control of flight behavior is required.
Collapse
Affiliation(s)
- Ke Fang
- Institute of Bio-Inspired Structure and Surface Engineering, College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
| | - Hao Mei
- Institute of Bio-Inspired Structure and Surface Engineering, College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
| | - Yezhong Tang
- Institute of Bio-Inspired Structure and Surface Engineering, College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
- Chengdu Institute of Biology, Chinese Academy of Sciences, Chengdu, Sichuan, China
| | - Wenbo Wang
- Institute of Bio-Inspired Structure and Surface Engineering, College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
| | - Hao Wang
- Institute of Bio-Inspired Structure and Surface Engineering, College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
- Hao Wang
| | - Zhouyi Wang
- Institute of Bio-Inspired Structure and Surface Engineering, College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
- *Correspondence: Zhouyi Wang
| | - Zhendong Dai
- Institute of Bio-Inspired Structure and Surface Engineering, College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
- Zhendong Dai
| |
Collapse
|
40
|
Ziafati A, Maleki A. Genetic algorithm based ensemble system using MLR and MsetCCA methods for SSVEP frequency recognition. Med Eng Phys 2023; 111:103945. [PMID: 36792239 DOI: 10.1016/j.medengphy.2022.103945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2021] [Revised: 11/20/2022] [Accepted: 12/19/2022] [Indexed: 12/24/2022]
Abstract
BCI systems provide a direct communication channel between the human and the machine using brain signals. Among the various methods of steady-state visual evoked potential (SSVEP) stimulation frequency detection, multiple linear regression (MLR), and multiset canonical correlation analysis (MsetCCA) methods have achieved high accurate results in recent studies. The purpose of this study is to utilize both approaches and benefit from them using a genetic algorithm (GA). This algorithm leads to high-performance optimization due to its large number of regulatory parameters. Signal analysis was performed for the windows with 0.5 to 4 s duration length and with 0.5-second incremental steps. In this paper, we were able to achieve 100% accuracy of recognition for 2-second time-windows using the genetic algorithm to optimally ensemble SSVEP stimulation frequency detection methods. The accuracy of the proposed system indicates a significant improvement in detection compared to either MLR or MsetCCA alone and indicates that the ensemble system is correctly optimized using the genetic algorithm. Genetic algorithm is one of the most widely used algorithms because of its high regulatory parameters leading to its high flexibility. The improvement in detection of the proposed system is due to the use of the strengths of both two methods, and the optimal choice of the system response to visual stimuli.
Collapse
Affiliation(s)
- Amir Ziafati
- Biomedical Engineering Department, Semnan University, Semnan, Iran
| | - Ali Maleki
- Biomedical Engineering Department, Semnan University, Semnan, Iran.
| |
Collapse
|
41
|
Comparative study of attention-related features on attention monitoring systems with a single EEG channel. J Neurosci Methods 2022; 382:109711. [PMID: 36126733 DOI: 10.1016/j.jneumeth.2022.109711] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Revised: 08/21/2022] [Accepted: 09/16/2022] [Indexed: 01/05/2023]
Abstract
The easy-to-use attention monitoring systems usually detect the participant's attentional status via processing electroencephalogram (EEG) data recorded from a single FPz channel. But due to the influence of noises and artifacts, the attention-monitoring performance needs to be further improved to suit different individuals and devices. This paper compared the attention-related features extracted using four state-of-the-art methods including delta/beta1 (D/B1), α + β + δ + θ + R, entropy and optimized complex network (OCN). The classification performance was evaluated using receiver operating characteristic (ROC) curves and area under the ROC curves (AUC) on two EEG data acquisition devices, i.e., a BrainAmp device with high precision and a Sichiray device with low cost, respectively. Considering the varied performance on different individuals and devices, this paper proposed a novel Mutual information-based feature fusion (MIFF) method, selecting the optimal combinations of the attention-related features for classification, to enhance the attention detection performance. The experimental results showed that the proposed MIFF method outperformed the state-of-the-art methods regardless of data length on both devices. Especially, the proposed method with data length of 2.5 s achieved an average AUC of 0.8505 on the low-cost Sichiray device, which is 56.08 % higher than that of D/B1, 27.28 % higher than that of α + β + δ + θ + R, 17.42 % higher than that of entropy, and 15.48 % higher than that of OCN.
Collapse
|
42
|
Tong J, Wei X, Dong E, Sun Z, Du S, Duan F. Hybrid mental tasks based human computer interface via integration of pronunciation and motor imagery. J Neural Eng 2022; 19. [PMID: 36228578 DOI: 10.1088/1741-2552/ac9a01] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Accepted: 10/13/2022] [Indexed: 12/24/2022]
Abstract
Objective.Among the existing active brain-computer interfaces (BCI), the motor imagination (MI) is widely used. To operate the MI BCI effectively, subjects need to carry out trainings on corresponding imagining tasks. Here, we studied how to reduce the discomfort and fatigue of active BCI imaginary tasks and the inability to concentrate on them while improving the accuracy.Approach.This paper proposes a hybrid BCI composed of MI and pronunciation imagination (PI). The electroencephalogram signals of ten subjects are recognized by the adaptive Riemannian distance classification and the improved frequency selective filter-bank Common Spatial Pattern recognition.Main results.The results show that under the new paradigm with the combination of MI and PI, the recognition accuracy is higher than the MI alone. The highest recognition rate of the proposed hybrid system can reach more than 90%. Furthermore, through the subjects' scoring results of the operation difficulty, it is concluded that the designed hybrid paradigm is more operable than the traditional BCI paradigm.Significance.The separable tasks in the active BCI are limited and the accuracy needs to be improved. The new hybrid paradigm proposed by us improves the accuracy and operability of the active BCI system, providing a new possibility for the research direction of the active BCI.
Collapse
Affiliation(s)
- Jigang Tong
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, TianjinUniversity of Technology, Tianjin 300384, People's Republic of China
| | - Xiaoying Wei
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, TianjinUniversity of Technology, Tianjin 300384, People's Republic of China
| | - Enzeng Dong
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, TianjinUniversity of Technology, Tianjin 300384, People's Republic of China
| | - Zhe Sun
- Computational Engineering Applications Unit, Head Office for Information Systems and Cybersecurity, RIKEN, Saitama, Japan
| | - Shengzhi Du
- Department of Electrical Engineering, Tshwane University of Technology, Pretoria 0001, South Africa
| | - Feng Duan
- College of Artificial Intelligence, Nankai University, Tianjin, People's Republic of China
| |
Collapse
|
43
|
Wang G, Cerf M. Brain-Computer Interface using neural network and temporal-spectral features. Front Neuroinform 2022; 16:952474. [PMID: 36277476 PMCID: PMC9580359 DOI: 10.3389/fninf.2022.952474] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Accepted: 08/24/2022] [Indexed: 11/24/2022] Open
Abstract
Brain-Computer Interfaces (BCIs) are increasingly useful for control. Such BCIs can be used to assist individuals who lost mobility or control over their limbs, for recreational purposes such as gaming or semi-autonomous driving, or as an interface toward man-machine integration. Thus far, the performance of algorithms used for thought decoding has been limited. We show that by extracting temporal and spectral features from electroencephalography (EEG) signals and, following, using deep learning neural network to classify those features, one can significantly improve the performance of BCIs in predicting which motor action was imagined by a subject. Our movement prediction algorithm uses Sequential Backward Selection technique to jointly choose temporal and spectral features and a radial basis function neural network for the classification. The method shows an average performance increase of 3.50% compared to state-of-the-art benchmark algorithms. Using two popular public datasets our algorithm reaches 90.08% accuracy (compared to an average benchmark of 79.99%) on the first dataset and 88.74% (average benchmark: 82.01%) on the second dataset. Given the high variability within- and across-subjects in EEG-based action decoding, we suggest that using features from multiple modalities along with neural network classification protocol is likely to increase the performance of BCIs across various tasks.
Collapse
Affiliation(s)
- Gan Wang
- School of Mechanical and Electrical Engineering, Soochow University, Suchow, China
| | - Moran Cerf
- Interdepartmental Neuroscience Program, Northwestern University, Evanston, IL, United States
- *Correspondence: Moran Cerf,
| |
Collapse
|
44
|
Dong E, Zhang H, Zhu L, Du S, Tong J. A multi-modal brain-computer interface based on threshold discrimination and its application in wheelchair control. Cogn Neurodyn 2022; 16:1123-1133. [PMID: 36237403 PMCID: PMC9508306 DOI: 10.1007/s11571-021-09779-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Revised: 12/02/2021] [Accepted: 12/23/2021] [Indexed: 11/25/2022] Open
Abstract
In this study, we propose a novel multi-modal brain-computer interface (BCI) system based on the threshold discrimination, which is proposed for the first time to distinguish between SSVEP and MI potentials. The system combines these two heterogeneous signals to increase the number of control commands and improve the performance of asynchronous control of external devices. In this research, an electric wheelchair is controlled as an example. The user can continuously control the wheelchair to turn left/right through motion imagination (MI) by imagining left/right-hand movement and generate another 6 commands for the wheelchair control by focusing on the SSVEP stimulation panel. Ten subjects participated in a MI training session and eight of them completed a mobile obstacle-avoidance experiment in a complex environment requesting high control accuracy for successful manipulation. Comparing with the single-modal BCI-controlled wheelchair system, the results demonstrate that the proposed multi-modal method is effective by providing more satisfactory control accuracy, and show the potential of BCI-controlled systems to be applied in complex daily tasks.
Collapse
Affiliation(s)
- Enzeng Dong
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, Tianjin University of Technology, Tianjin, 300384 China
| | - Haoran Zhang
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, Tianjin University of Technology, Tianjin, 300384 China
| | - Lin Zhu
- China North Industries Group 210 Research Institute, Beijing, China
| | - Shengzhi Du
- Department of Electrical Engineering, Tshwane University of Technology, Pretoria, 0001 South Africa
| | - Jigang Tong
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, Tianjin University of Technology, Tianjin, 300384 China
| |
Collapse
|
45
|
Song M, Jeong H, Kim J, Jang SH, Kim J. An EEG-based asynchronous MI-BCI system to reduce false positives with a small number of channels for neurorehabilitation: A pilot study. Front Neurorobot 2022; 16:971547. [PMID: 36172602 PMCID: PMC9510756 DOI: 10.3389/fnbot.2022.971547] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Accepted: 08/08/2022] [Indexed: 11/22/2022] Open
Abstract
Many studies have used motor imagery-based brain–computer interface (MI-BCI) systems for stroke rehabilitation to induce brain plasticity. However, they mainly focused on detecting motor imagery but did not consider the effect of false positive (FP) detection. The FP could be a threat to patients with stroke as it can induce wrong-directed brain plasticity that would result in adverse effects. In this study, we proposed a rehabilitative MI-BCI system that focuses on rejecting the FP. To this end, we first identified numerous electroencephalogram (EEG) signals as the causes of the FP, and based on the characteristics of the signals, we designed a novel two-phase classifier using a small number of EEG channels, including the source of the FP. Through experiments with eight healthy participants and nine patients with stroke, our proposed MI-BCI system showed 71.76% selectivity and 13.70% FP rate by using only four EEG channels in the patient group with stroke. Moreover, our system can compensate for day-to-day variations for prolonged session intervals by recalibration. The results suggest that our proposed system, a practical approach for the clinical setting, could improve the therapeutic effect of MI-BCI by reducing the adverse effect of the FP.
Collapse
Affiliation(s)
- Minsu Song
- Department of Medical Device, Korea Institute of Machinery and Materials, Daegu, South Korea
| | - Hojun Jeong
- School of Mechanical Engineering, Sungkyunkwan University, Gyeonggi-do, South Korea
| | - Jongbum Kim
- Department of Robotics Engineering, Daegu Gyeongbuk Institute of Science and Technology, Daegu, South Korea
| | - Sung-Ho Jang
- Department of Physical Medicine and Rehabilitation, College of Medicine, Yeungnam University, Daegu, South Korea
| | - Jonghyun Kim
- School of Mechanical Engineering, Sungkyunkwan University, Gyeonggi-do, South Korea
- *Correspondence: Jonghyun Kim
| |
Collapse
|
46
|
Higashi T, Maeda K, Ogawa T, Haseyama M. Brain Decoding of Multiple Subjects for Estimating Visual Information Based on a Probabilistic Generative Model. SENSORS (BASEL, SWITZERLAND) 2022; 22:6148. [PMID: 36015909 PMCID: PMC9416613 DOI: 10.3390/s22166148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 08/10/2022] [Accepted: 08/14/2022] [Indexed: 06/15/2023]
Abstract
Brain decoding is a process of decoding human cognitive contents from brain activities. However, improving the accuracy of brain decoding remains difficult due to the unique characteristics of the brain, such as the small sample size and high dimensionality of brain activities. Therefore, this paper proposes a method that effectively uses multi-subject brain activities to improve brain decoding accuracy. Specifically, we distinguish between the shared information common to multi-subject brain activities and the individual information based on each subject's brain activities, and both types of information are used to decode human visual cognition. Both types of information are extracted as features belonging to a latent space using a probabilistic generative model. In the experiment, an publicly available dataset and five subjects were used, and the estimation accuracy was validated on the basis of a confidence score ranging from 0 to 1, and a large value indicates superiority. The proposed method achieved a confidence score of 0.867 for the best subject and an average of 0.813 for the five subjects, which was the best compared to other methods. The experimental results show that the proposed method can accurately decode visual cognition compared with other existing methods in which the shared information is not distinguished from the individual information.
Collapse
Affiliation(s)
- Takaaki Higashi
- Graduate School of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan
| | - Keisuke Maeda
- Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan
| | - Takahiro Ogawa
- Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan
| | - Miki Haseyama
- Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan
| |
Collapse
|
47
|
Ziafati A, Maleki A. Boosting the Evoked Response of Brain to Enhance the Reference Signals of CCA Method. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2107-2115. [PMID: 35914031 DOI: 10.1109/tnsre.2022.3192413] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Brain-computer interface (BCI) systems can be used to communicate and express desires from people with severe nervous system damage. Among BCI systems based on evoked responses, steady state visual evoked potential (SSVEP) responses are the most widely used. Canonical correlation analysis (CCA)-based methods have been widely used in SSVEP-based online BCIs due to their low computation and high speed, and many methods have been introduced to improve the results. In this research, a method for constructing reference signals used in CCA based on the amplified evoked response of brain is introduced. In the proposed method, after removing the latency in the training signals, to construct reference signals, multilayer perceptron neural networks of the fitting type are used instead of the usual sine/cosine signals. The results show the success of this method in boosting the evoked responses of brain. The detection accuracy in 100-second time windows was 100%, and the information transfer rate in the same period was 240 bits per minute. Making reference signals similar to the recorded electroencephalogram allowed us to make more similarities in the CCA between the signals under consideration, and the reference signals, and to dramatically improve the results.
Collapse
|
48
|
Kasahara K, DaSalla CS, Honda M, Hanakawa T. Basal ganglia-cortical connectivity underlies self-regulation of brain oscillations in humans. Commun Biol 2022; 5:712. [PMID: 35842523 PMCID: PMC9288463 DOI: 10.1038/s42003-022-03665-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Accepted: 06/30/2022] [Indexed: 11/09/2022] Open
Abstract
Brain-computer interfaces provide an artificial link by which the brain can directly interact with the environment. To achieve fine brain-computer interface control, participants must modulate the patterns of the cortical oscillations generated from the motor and somatosensory cortices. However, it remains unclear how humans regulate cortical oscillations, the controllability of which substantially varies across individuals. Here, we performed simultaneous electroencephalography (to assess brain-computer interface control) and functional magnetic resonance imaging (to measure brain activity) in healthy participants. Self-regulation of cortical oscillations induced activity in the basal ganglia-cortical network and the neurofeedback control network. Successful self-regulation correlated with striatal activity in the basal ganglia-cortical network, through which patterns of cortical oscillations were likely modulated. Moreover, basal ganglia-cortical network and neurofeedback control network connectivity correlated with strong and weak self-regulation, respectively. The findings indicate that the basal ganglia-cortical network is important for self-regulation, the understanding of which should help advance brain-computer interface technology. Simultaneous fMRI-EEG in 26 healthy participants indicate that the basal ganglia cortical network and the neurofeedback control network play different roles in self-regulation, providing further insight into the neural correlates for brain-machine interface control and feedback.
Collapse
Affiliation(s)
- Kazumi Kasahara
- Department of Advanced Neuroimaging, Integrative Brain Imaging Center, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan.,Department of Functional Brain Research, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan.,Human Informatics and Interaction Research Institute, National Institute of Advanced Industrial Science and Technology, Ibaraki, 305-8566, Japan
| | - Charles S DaSalla
- Department of Advanced Neuroimaging, Integrative Brain Imaging Center, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan.,Department of Functional Brain Research, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan
| | - Manabu Honda
- Department of Functional Brain Research, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan
| | - Takashi Hanakawa
- Department of Advanced Neuroimaging, Integrative Brain Imaging Center, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan. .,Department of Functional Brain Research, National Center of Neurology and Psychiatry, Tokyo, 187-8551, Japan. .,Integrated Neuroanatomy and Neuroimaging, Kyoto University Graduate School of Medicine, Kyoto, 606-8501, Japan.
| |
Collapse
|
49
|
Song Z, Zhang X, Wang Y. Cluster Kernel Reinforcement Learning-based Kalman Filter for Three-Lever Discrimination Task in Brain-Machine Interface. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:690-693. [PMID: 36086404 DOI: 10.1109/embc48229.2022.9871669] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Brain-Machine Interface (BMI) translates paralyzed people's neural activity into control commands of the prosthesis so that their lost motor functions could be restored. The neural activities represent brain states that change continuously over time which brings the challenge to the online decoder. Reinforcement Learning (RL) has the advantage to construct the dynamic neural-kinematic mapping during the interaction. However, existing RL decoders output discrete actions as a classification problem and cannot provide continuous estimation. Previous work has combined Kalman Filter (KF) with RL for BMI, which achieves a continuous motor state estimation. However, this method adopts a neural network structure, which might get stuck in local optimum and cannot provide an efficient online update for the neural-kinematic mapping. In this paper, we propose a Cluster Kernel Reinforcement Learning-based Kalman Filter (CKRL-based KF) to avoid the local optimum problem for online neural-kinematic updating. The neural patterns are projected into Reproducing Kernel Hilbert Space (RKHS), which builds a universal approximation to guarantee the global optimum. We compare our proposed algorithm with the existing method on rat data collected during a brain control three-lever discrimination task. Our preliminary results show that the proposed method has a higher trial accuracy with lower variance across data segments, which shows its potential to improve the performance for online BMI control. Clinical Relevance- This paper provides a more stable decoding method for adaptive and continuous neural decoding. It is promising for clinical applications in BMI.
Collapse
|
50
|
Bodda S, Diwakar S. Exploring EEG spectral and temporal dynamics underlying a hand grasp movement. PLoS One 2022; 17:e0270366. [PMID: 35737671 PMCID: PMC9223346 DOI: 10.1371/journal.pone.0270366] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2021] [Accepted: 06/08/2022] [Indexed: 11/28/2022] Open
Abstract
For brain-computer interfaces, resolving the differences between pre-movement and movement requires decoding neural ensemble activity in the motor cortex's functional regions and behavioural patterns. Here, we explored the underlying neural activity and mechanisms concerning a grasped motor task by recording electroencephalography (EEG) signals during the execution of hand movements in healthy subjects. The grasped movement included different tasks; reaching the target, grasping the target, lifting the object upwards, and moving the object in the left or right directions. 163 trials of EEG data were acquired from 30 healthy participants who performed the grasped movement tasks. Rhythmic EEG activity was analysed during the premovement (alert task) condition and compared against grasped movement tasks while the arm was moved towards the left or right directions. The short positive to negative deflection that initiated around -0.5ms as a wave before the onset of movement cue can be used as a potential biomarker to differentiate movement initiation and movement. A rebound increment of 14% of beta oscillations and 26% gamma oscillations in the central regions was observed and could be used to distinguish pre-movement and grasped movement tasks. Comparing movement initiation to grasp showed a decrease of 10% in beta oscillations and 13% in gamma oscillations, and there was a rebound increment 4% beta and 3% gamma from grasp to grasped movement. We also investigated the combination MRCPs and spectral estimates of α, β, and γ oscillations as features for machine learning classifiers that could categorize movement conditions. Support vector machines with 3rd order polynomial kernel yielded 70% accuracy. Pruning the ranked features to 5 leaf nodes reduced the error rate by 16%. For decoding grasped movement and in the context of BCI applications, this study identifies potential biomarkers, including the spatio-temporal characteristics of MRCPs, spectral information, and choice of classifiers for optimally distinguishing initiation and grasped movement.
Collapse
Affiliation(s)
- Sandeep Bodda
- Amrita Mind Brain Center, Amrita Vishwa Vidyapeetham, Kollam, Kerala, India
| | - Shyam Diwakar
- Amrita Mind Brain Center, Amrita Vishwa Vidyapeetham, Kollam, Kerala, India
- Department of Electronics and Communication Engineering, School of Engineering, Amrita Vishwa Vidyapeetham, Kollam, Kerala, India
| |
Collapse
|