1
|
Meng L, Hu X. Unsupervised neural decoding for concurrent and continuous multi-finger force prediction. Comput Biol Med 2024; 173:108384. [PMID: 38554657 DOI: 10.1016/j.compbiomed.2024.108384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2023] [Revised: 02/27/2024] [Accepted: 03/24/2024] [Indexed: 04/02/2024]
Abstract
Reliable prediction of multi-finger forces is crucial for neural-machine interfaces. Various neural decoding methods have progressed substantially for accurate motor output predictions. However, most neural decoding methods are performed in a supervised manner, i.e., the finger forces are needed for model training, which may not be suitable in certain contexts, especially in scenarios involving individuals with an arm amputation. To address this issue, we developed an unsupervised neural decoding approach to predict multi-finger forces using spinal motoneuron firing information. We acquired high-density surface electromyogram (sEMG) signals of the finger extensor muscle when subjects performed single-finger and multi-finger tasks of isometric extensions. We first extracted motor units (MUs) from sEMG signals of the single-finger tasks. Because of inevitable finger muscle co-activation, MUs controlling the non-targeted fingers can also be recruited. To ensure an accurate finger force prediction, these MUs need to be teased out. To this end, we clustered the decomposed MUs based on inter-MU distances measured by the dynamic time warping technique, and we then labeled the MUs using the mean firing rate or the firing rate phase amplitude. We merged the clustered MUs related to the same target finger and assigned weights based on the consistency of the MUs being retained. As a result, compared with the supervised neural decoding approach and the conventional sEMG amplitude approach, our new approach can achieve a higher R2 (0.77 ± 0.036 vs. 0.71 ± 0.11 vs. 0.61 ± 0.09) and a lower root mean square error (5.16 ± 0.58 %MVC vs. 5.88 ± 1.34 %MVC vs. 7.56 ± 1.60 %MVC). Our findings can pave the way for the development of accurate and robust neural-machine interfaces, which can significantly enhance the experience during human-robotic hand interactions in diverse contexts.
Collapse
Affiliation(s)
- Long Meng
- Department of Mechanical Engineering, Pennsylvania State University-University Park, PA, USA
| | - Xiaogang Hu
- Department of Mechanical Engineering, Pennsylvania State University-University Park, PA, USA; Department of Kinesiology, Pennsylvania State University-University Park, PA, USA; Department of Physical Medicine & Rehabilitation, Pennsylvania State Hershey College of Medicine, PA, USA; Huck Institutes of the Life Sciences, Pennsylvania State University-University Park, PA, USA; Center for Neural Engineering, Pennsylvania State University-University Park, PA, USA.
| |
Collapse
|
2
|
Qiao N, Ma L, Zhang Y, Wang L. Update on Nonhuman Primate Models of Brain Disease and Related Research Tools. Biomedicines 2023; 11:2516. [PMID: 37760957 PMCID: PMC10525665 DOI: 10.3390/biomedicines11092516] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Revised: 08/19/2023] [Accepted: 09/06/2023] [Indexed: 09/29/2023] Open
Abstract
The aging of the population is an increasingly serious issue, and many age-related illnesses are on the rise. These illnesses pose a serious threat to the health and safety of elderly individuals and create a serious economic and social burden. Despite substantial research into the pathogenesis of these diseases, their etiology and pathogenesis remain unclear. In recent decades, rodent models have been used in attempts to elucidate these disorders, but such models fail to simulate the full range of symptoms. Nonhuman primates (NHPs) are the most ideal neuroscientific models for studying the human brain and are more functionally similar to humans because of their high genetic similarities and phenotypic characteristics in comparison with humans. Here, we review the literature examining typical NHP brain disease models, focusing on NHP models of common diseases such as dementia, Parkinson's disease, and epilepsy. We also explore the application of electroencephalography (EEG), magnetic resonance imaging (MRI), and optogenetic study methods on NHPs and neural circuits associated with cognitive impairment.
Collapse
Affiliation(s)
- Nan Qiao
- School of Life Sciences, Hebei University, 180 Wusi Dong Lu, Baoding 071002, China;
- Beijing Institute of Radiation Medicine, 27 Taiping Road, Beijing 100850, China;
| | - Lizhen Ma
- Beijing Institute of Radiation Medicine, 27 Taiping Road, Beijing 100850, China;
| | - Yi Zhang
- School of Life Sciences, Hebei University, 180 Wusi Dong Lu, Baoding 071002, China;
- Beijing Institute of Radiation Medicine, 27 Taiping Road, Beijing 100850, China;
| | - Lifeng Wang
- School of Life Sciences, Hebei University, 180 Wusi Dong Lu, Baoding 071002, China;
- Beijing Institute of Radiation Medicine, 27 Taiping Road, Beijing 100850, China;
| |
Collapse
|
3
|
Wu X, Li G, Jiang S, Wellington S, Liu S, Wu Z, Metcalfe B, Chen L, Zhang D. Decoding Continuous Kinetic Information of Grasp from Stereo-electroencephalographic (SEEG) Recordings. J Neural Eng 2022; 19. [PMID: 35395645 DOI: 10.1088/1741-2552/ac65b1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2021] [Accepted: 04/08/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Brain-computer interfaces (BCIs) have the potential to bypass damaged neural pathways and restore functionality lost due to injury or disease. Approaches to decoding kinematic information are well documented; however, the decoding of kinetic information has received less attention. Additionally, the possibility of using stereo-electroencephalography (SEEG) for kinetic decoding during hand grasping tasks is still largely unknown. Thus, the objective of this paper is to demonstrate kinetic parameter decoding using SEEG in patients performing a grasping task with two different force levels under two different ascending rates. APPROACH Temporal-spectral representations were studied to investigate frequency modulation under different force tasks. Then, force amplitude was decoded from SEEG recordings using multiple decoders, including a linear model, a partial least squares (PLS) model, an unscented Kalman filter, and three deep learning models (shallow convolutional neural network, deep convolutional neural network and the proposed CNN+RNN neural network). MAIN RESULTS The current study showed that: 1) for some channel, both low-frequency modulation (event-related desynchronization (ERD)) and high-frequency modulation (event-related synchronization (ERS)) were sustained during prolonged force holding periods; 2) continuously changing grasp force can be decoded from the SEEG signals; 3) the novel CNN+RNN deep learning model achieved the best decoding performance, with the predicted force magnitude closely aligned to the ground truth under different force amplitudes and changing rates. SIGNIFICANCE This work verified the possibility of decoding continuously changing grasp force using SEEG recordings. The result presented in this study demonstrated the potential of SEEG recordings for future BCI application.
Collapse
Affiliation(s)
- Xiaolong Wu
- Electric, Electronic and Engineering, University of Bath, Pulteney Court PD42.2,Pulteney Road,BA2 4HL, Bath, Bath, Somerset, BA2 4HL, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Guangye Li
- Shanghai Jiao Tong University, Shanghai JiaoTong university, Shanghai, China, Shanghai, 200240, CHINA
| | - Shize Jiang
- Fudan University Huashan Hospital Department of Neurosurgery, Fudan University Huanshan hospital, Shanghai, Shanghai, 201906, CHINA
| | - Scott Wellington
- University of Bath, University of Bath, Bath, UK, Bath, Bath and North East Somer, BA2 7AY, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Shengjie Liu
- Shanghai Jiao Tong University, Shanghai jIaotong University, Shanghai, China, Shanghai, 200240, CHINA
| | - Zehan Wu
- Huashan Hospital Fudan University, Huashan hospital, Shanghai, China, Shanghai, Shanghai, 200040, CHINA
| | - Benjamin Metcalfe
- University of Bath, University of Bath, UK, Bath, Bath and North East Somer, BA2 7AY, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Liang Chen
- Fudan University Huashan Hospital Department of Neurosurgery, Fudan University Huashan Hospital, Shanghai, China, Shanghai, Shanghai, 201906, CHINA
| | - Dingguo Zhang
- University of Bath, University of Bath, UK, Bath, Somerset, BA2 4HL, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| |
Collapse
|
4
|
Choi H, Lim S, Min K, Ahn KH, Lee KM, Jang DP. Non-human primate epidural ECoG analysis using explainable deep learning technology. J Neural Eng 2021; 18. [PMID: 34695809 DOI: 10.1088/1741-2552/ac3314] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2021] [Accepted: 10/25/2021] [Indexed: 11/12/2022]
Abstract
Objective.With the development in the field of neural networks,explainable AI(XAI), is being studied to ensure that artificial intelligence models can be explained. There are some attempts to apply neural networks to neuroscientific studies to explain neurophysiological information with high machine learning performances. However, most of those studies have simply visualized features extracted from XAI and seem to lack an active neuroscientific interpretation of those features. In this study, we have tried to actively explain the high-dimensional learning features contained in the neurophysiological information extracted from XAI, compared with the previously reported neuroscientific results.Approach. We designed a deep neural network classifier using 3D information (3D DNN) and a 3D class activation map (3D CAM) to visualize high-dimensional classification features. We used those tools to classify monkey electrocorticogram (ECoG) data obtained from the unimanual and bimanual movement experiment.Main results. The 3D DNN showed better classification accuracy than other machine learning techniques, such as 2D DNN. Unexpectedly, the activation weight in the 3D CAM analysis was high in the ipsilateral motor and somatosensory cortex regions, whereas the gamma-band power was activated in the contralateral areas during unimanual movement, which suggests that the brain signal acquired from the motor cortex contains information about both contralateral movement and ipsilateral movement. Moreover, the hand-movement classification system used critical temporal information at movement onset and offset when classifying bimanual movements.Significance.As far as we know, this is the first study to use high-dimensional neurophysiological information (spatial, spectral, and temporal) with the deep learning method, reconstruct those features, and explain how the neural network works. We expect that our methods can be widely applied and used in neuroscience and electrophysiology research from the point of view of the explainability of XAI as well as its performance.
Collapse
Affiliation(s)
- Hoseok Choi
- Department of Neurology, University of California, San Francisco, CA, United States of America.,Department of Biomedical Engineering, Hanyang University, Seoul, Republic of Korea
| | - Seokbeen Lim
- Department of Biomedical Engineering, Hanyang University, Seoul, Republic of Korea
| | - Kyeongran Min
- Department of Biomedical Engineering, Hanyang University, Seoul, Republic of Korea.,Samsung SDS Artificial Intelligence Research Center, Seoul, Republic of Korea
| | - Kyoung-Ha Ahn
- Department of Neurology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Kyoung-Min Lee
- Department of Neurology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Dong Pyo Jang
- Department of Biomedical Engineering, Hanyang University, Seoul, Republic of Korea
| |
Collapse
|