1
|
Qin Y, Li B, Wang W, Shi X, Wang H, Wang X. ETCNet: An EEG-based motor imagery classification model combining efficient channel attention and temporal convolutional network. Brain Res 2024; 1823:148673. [PMID: 37956749 DOI: 10.1016/j.brainres.2023.148673] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 08/16/2023] [Accepted: 11/07/2023] [Indexed: 11/15/2023]
Abstract
Brain-computer interface (BCI) enables the control of external devices using signals from the brain, offering immense potential in assisting individuals with neuromuscular disabilities. Among the different paradigms of BCI systems, the motor imagery (MI) based electroencephalogram (EEG) signal is widely recognized as exceptionally promising. Deep learning (DL) has found extensive applications in the processing of MI signals, wherein convolutional neural networks (CNN) have demonstrated superior performance compared to conventional machine learning (ML) approaches. Nevertheless, challenges related to subject independence and subject dependence persist, while the inherent low signal-to-noise ratio of EEG signals remains a critical aspect that demands attention. Accurately deciphering intentions from EEG signals continues to present a formidable challenge. This paper introduces an advanced end-to-end network that effectively combines the efficient channel attention (ECA) and temporal convolutional network (TCN) components for the classification of motor imagination signals. We incorporated an ECA module prior to feature extraction in order to enhance the extraction of channel-specific features. A compact convolutional network model uses for feature extraction in the middle part. Finally, the time characteristic information is obtained by using TCN. The results show that our network is a lightweight network that is characterized by few parameters and fast speed. Our network achieves an average accuracy of 80.71% on the BCI Competition IV-2a dataset.
Collapse
Affiliation(s)
- Yuxin Qin
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, China; Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, China
| | - Baojiang Li
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, China; Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, China.
| | - Wenlong Wang
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, China; Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, China
| | - Xingbin Shi
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, China; Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, China
| | - Haiyan Wang
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, China; Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, China
| | - Xichao Wang
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, China; Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, China
| |
Collapse
|
2
|
Yang T, Zhang P, Xing L, Hu J, Feng R, Zhong J, Li W, Zhang Y, Zhu Q, Yang Y, Gao F, Qian Z. Insights into brain perceptions of the different taste qualities and hedonic valence of food via scalp electroencephalogram. Food Res Int 2023; 173:113311. [PMID: 37803622 DOI: 10.1016/j.foodres.2023.113311] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 07/03/2023] [Accepted: 07/21/2023] [Indexed: 10/08/2023]
Abstract
Investigating brain activity is essential for exploring taste-experience related cues. The paper aimed to explore implicit (unconscious) emotional or physiological responses related to taste experiences using scalp electroencephalogram (EEG). We performed implicit measures of tastants of differing perceptual types (bitter, salty, sour and sweet) and intensities (low, medium, and high). The results showed that subjects were partially sensitive to different sensory intensities, i.e., for high intensities, taste stimuli could induce activation of different rhythm signals in the brain, with α and θ bands possibly being more sensitive to different taste types. Furthermore, the neural representations and corresponding sensory qualities (e.g., "sweet: pleasant" or "bitter: unpleasant") of different tastes could be discriminated at 250-1,500 ms after stimulus onset, and different tastes exhibited distinct temporal dynamic differences. Source localization indicated that different taste types activate brain areas associated with emotional eating, reward processing, and motivated tendencies, etc. Overall, our findings reveal a larger sophisticated taste map that accounted for the diversity of taste types in the human brain and assesses the emotion, reward, and motivated behavior represented by different tastes. This study provided basic insights and a perceptual foundation for the relationship between taste experience-related decisions and the prediction of brain activity.
Collapse
Affiliation(s)
- Tianyi Yang
- Department of Biomedical Engineering, Key Laboratory of Multi-modal Brain-Computer Precision Drive Ministry of Industry and Information Technology, Key Laboratory of Digital Medical Equipment and Technology of Jiangsu Province, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, PR China
| | - Peng Zhang
- Department of Biomedical Engineering, Key Laboratory of Multi-modal Brain-Computer Precision Drive Ministry of Industry and Information Technology, Key Laboratory of Digital Medical Equipment and Technology of Jiangsu Province, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, PR China
| | - Lidong Xing
- Department of Biomedical Engineering, Key Laboratory of Multi-modal Brain-Computer Precision Drive Ministry of Industry and Information Technology, Key Laboratory of Digital Medical Equipment and Technology of Jiangsu Province, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, PR China
| | - Jin Hu
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University, National Center for Neurological Disorders, National Key Laboratory of Medical Neurobiology, Institutes of Brain Science, Shanghai Key Lab. of Brain Function and Regeneration, Institute of Neurosurgery, Shanghai 200040, PR China
| | - Rui Feng
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University, National Center for Neurological Disorders, National Key Laboratory of Medical Neurobiology, Institutes of Brain Science, Shanghai Key Lab. of Brain Function and Regeneration, Institute of Neurosurgery, Shanghai 200040, PR China
| | - Junjie Zhong
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University, National Center for Neurological Disorders, National Key Laboratory of Medical Neurobiology, Institutes of Brain Science, Shanghai Key Lab. of Brain Function and Regeneration, Institute of Neurosurgery, Shanghai 200040, PR China
| | - Weitao Li
- Department of Biomedical Engineering, Key Laboratory of Multi-modal Brain-Computer Precision Drive Ministry of Industry and Information Technology, Key Laboratory of Digital Medical Equipment and Technology of Jiangsu Province, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, PR China
| | - Yizhi Zhang
- Department of Biomedical Engineering, Key Laboratory of Multi-modal Brain-Computer Precision Drive Ministry of Industry and Information Technology, Key Laboratory of Digital Medical Equipment and Technology of Jiangsu Province, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, PR China
| | - Qiaoqiao Zhu
- Department of Biomedical Engineering, Key Laboratory of Multi-modal Brain-Computer Precision Drive Ministry of Industry and Information Technology, Key Laboratory of Digital Medical Equipment and Technology of Jiangsu Province, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, PR China
| | - Yamin Yang
- Department of Biomedical Engineering, Key Laboratory of Multi-modal Brain-Computer Precision Drive Ministry of Industry and Information Technology, Key Laboratory of Digital Medical Equipment and Technology of Jiangsu Province, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, PR China
| | - Fan Gao
- Department of Biomedical Engineering, Key Laboratory of Multi-modal Brain-Computer Precision Drive Ministry of Industry and Information Technology, Key Laboratory of Digital Medical Equipment and Technology of Jiangsu Province, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, PR China.
| | - Zhiyu Qian
- Department of Biomedical Engineering, Key Laboratory of Multi-modal Brain-Computer Precision Drive Ministry of Industry and Information Technology, Key Laboratory of Digital Medical Equipment and Technology of Jiangsu Province, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, PR China.
| |
Collapse
|
3
|
Du Q, Luo J, Cheng Q, Wang Y, Guo S. Vibrotactile enhancement in hand rehabilitation has a reinforcing effect on sensorimotor brain activities. Front Neurosci 2022; 16:935827. [PMID: 36267238 PMCID: PMC9577243 DOI: 10.3389/fnins.2022.935827] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Accepted: 08/31/2022] [Indexed: 11/13/2022] Open
Abstract
Objective Stroke patients often suffer from hand dysfunction or loss of tactile perception, which in turn interferes with hand rehabilitation. Tactile-enhanced multi-sensory feedback rehabilitation is an approach worth considering, but its effectiveness has not been well studied. By using functional near-infrared spectroscopy (fNIRS) to analyze the causal activity patterns in the sensorimotor cortex, the present study aims to investigate the cortical hemodynamic effects of hand rehabilitation training when tactile stimulation is applied, and to provide a basis for rehabilitation program development. Methods A vibrotactile enhanced pneumatically actuated hand rehabilitation device was tested on the less-preferred hand of 14 healthy right-handed subjects. The training tasks consisted of move hand and observe video (MO), move hand and vibration stimulation (MV), move hand, observe video, and vibration stimulation (MOV), and a contrast resting task. Region of interest (ROI), a laterality index (LI), and causal brain network analysis methods were used to explore the brain’s cortical blood flow response to a multi-sensory feedback rehabilitation task from multiple perspectives. Results (1) A more pronounced contralateral activation in the right-brain region occurred under the MOV stimulation. Rehabilitation tasks containing vibrotactile enhancement (MV and MOV) had significantly more oxyhemoglobin than the MO task at 5 s after the task starts, indicating faster contralateral activation in sensorimotor brain regions. (2) Five significant lateralized channel connections were generated under the MV and MOV tasks (p < 0.05), one significant lateralized channel connection was generated by the MO task, and the Rest were not, showing that MV and MOV caused stronger lateralization activation. (3) We investigated all thresholds of granger causality (GC) resulting in consistent relative numbers of effect connections. MV elicited stronger causal interactions between the left and right cerebral hemispheres, and at the GC threshold of 0.4, there were 13 causal network connection pairs for MV, 7 for MO, and 9 for MOV. Conclusion Vibrotactile cutaneous stimulation as a tactile enhancement can produce a stronger stimulation of the brain’s sensorimotor brain areas, promoting the establishment of neural pathways, and causing a richer effect between the left and right cerebral hemispheres. The combination of kinesthetic, vibrotactile, and visual stimulation can achieve a more prominent training efficiency from the perspective of functional cerebral hemodynamics.
Collapse
Affiliation(s)
- Qiang Du
- Academy for Engineering and Technology, Fudan University, Shanghai, China
- Shanghai Engineering Research Center of AI and Robotics, Shanghai, China
- Engineering Research Center of AI and Robotics, Ministry of Education, Shanghai, China
| | - Jingjing Luo
- Academy for Engineering and Technology, Fudan University, Shanghai, China
- Shanghai Engineering Research Center of AI and Robotics, Shanghai, China
- Engineering Research Center of AI and Robotics, Ministry of Education, Shanghai, China
- Jihua Laboratory, Foshan, China
- *Correspondence: Jingjing Luo,
| | - Qiying Cheng
- Academy for Engineering and Technology, Fudan University, Shanghai, China
- Shanghai Engineering Research Center of AI and Robotics, Shanghai, China
- Engineering Research Center of AI and Robotics, Ministry of Education, Shanghai, China
| | - Youhao Wang
- Academy for Engineering and Technology, Fudan University, Shanghai, China
- Shanghai Engineering Research Center of AI and Robotics, Shanghai, China
- Engineering Research Center of AI and Robotics, Ministry of Education, Shanghai, China
| | - Shijie Guo
- Academy for Engineering and Technology, Fudan University, Shanghai, China
- Shanghai Engineering Research Center of AI and Robotics, Shanghai, China
- Engineering Research Center of AI and Robotics, Ministry of Education, Shanghai, China
- Department of the State Key Laboratory of Reliability and Intelligence of Electrical Equipment and the Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, Hebei University of Technology, Tianjin, China
- Shijie Guo,
| |
Collapse
|
4
|
Xu F, Wang Y, Li H, Yu X, Wang C, Liu M, Jiang L, Feng C, Li J, Wang D, Yan Z, Zhang Y, Leng J. Time-Varying Effective Connectivity for Describing the Dynamic Brain Networks of Post-stroke Rehabilitation. Front Aging Neurosci 2022; 14:911513. [PMID: 35686023 PMCID: PMC9171495 DOI: 10.3389/fnagi.2022.911513] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2022] [Accepted: 04/19/2022] [Indexed: 11/13/2022] Open
Abstract
Hemiplegia is a common motor dysfunction caused by a stroke. However, the dynamic network mechanism of brain processing information in post-stroke hemiplegic patients has not been revealed when performing motor imagery (MI) tasks. We acquire electroencephalography (EEG) data from healthy subjects and post-stroke hemiplegic patients and use the Fugl-Meyer assessment (FMA) to assess the degree of motor function damage in stroke patients. Time-varying MI networks are constructed using the adaptive directed transfer function (ADTF) method to explore the dynamic network mechanism of MI in post-stroke hemiplegic patients. Finally, correlation analysis has been conducted to study potential relationships between global efficiency and FMA scores. The performance of our proposed method has shown that the brain network pattern of stroke patients does not significantly change from laterality to bilateral symmetry when performing MI recognition. The main change is that the contralateral motor areas of the brain damage and the effective connection between the frontal lobe and the non-motor areas are enhanced, to compensate for motor dysfunction in stroke patients. We also find that there is a correlation between FMA scores and global efficiency. These findings help us better understand the dynamic brain network of patients with post-stroke when processing MI information. The network properties may provide a reliable biomarker for the objective evaluation of the functional rehabilitation diagnosis of stroke patients.
Collapse
Affiliation(s)
- Fangzhou Xu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
- *Correspondence: Fangzhou Xu
| | - Yuandong Wang
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
- School of Electrical Engineering and Automation, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Han Li
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
- School of Electrical Engineering and Automation, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Xin Yu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
- School of Electrical Engineering and Automation, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Chongfeng Wang
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
- School of Electrical Engineering and Automation, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Ming Liu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
- School of Electrical Engineering and Automation, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Lin Jiang
- The Clinical Hospital of Chengdu Brain Science Institute, Ministry of Education Key Lab for Neuroinformation, University of Electronic Science and Technology of China, Chengdu, China
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Chao Feng
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Jianfei Li
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Dezheng Wang
- The Department of Physical Medicine and Rehabilitation, Qilu Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Zhiguo Yan
- School of Electrical Engineering and Automation, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
- Zhiguo Yan
| | - Yang Zhang
- The Department of Physical Medicine and Rehabilitation, Qilu Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China
- Yang Zhang
| | - Jiancai Leng
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
- Jiancai Leng
| |
Collapse
|
5
|
Pitsik EN, Frolov NS, Shusharina N, Hramov AE. Age-Related Changes in Functional Connectivity during the Sensorimotor Integration Detected by Artificial Neural Network. SENSORS 2022; 22:s22072537. [PMID: 35408153 PMCID: PMC9003057 DOI: 10.3390/s22072537] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Revised: 03/21/2022] [Accepted: 03/24/2022] [Indexed: 02/01/2023]
Abstract
Large-scale functional connectivity is an important indicator of the brain’s normal functioning. The abnormalities in the connectivity pattern can be used as a diagnostic tool to detect various neurological disorders. The present paper describes the functional connectivity assessment based on artificial intelligence to reveal age-related changes in neural response in a simple motor execution task. Twenty subjects of two age groups performed repetitive motor tasks on command, while the whole-scalp EEG was recorded. We applied the model based on the feed-forward multilayer perceptron to detect functional relationships between five groups of sensors located over the frontal, parietal, left, right, and middle motor cortex. Functional dependence was evaluated with the predicted and original time series coefficient of determination. Then, we applied statistical analysis to highlight the significant features of the functional connectivity network assessed by our model. Our findings revealed the connectivity pattern is consistent with modern ideas of the healthy aging effect on neural activation. Elderly adults demonstrate a pronounced activation of the whole-brain theta-band network and decreased activation of frontal–parietal and motor areas of the mu-band. Between-subject analysis revealed a strengthening of inter-areal task-relevant links in elderly adults. These findings can be interpreted as an increased cognitive demand in elderly adults to perform simple motor tasks with the dominant hand, induced by age-related working memory decline.
Collapse
Affiliation(s)
- Elena N. Pitsik
- Baltic Center for Artificial Intelligence and Neurotechnology, Immanuel Kant Baltic Federal University, Kaliningrad 236041, Russia; (E.N.P.); (N.S.F.); (N.S.)
- Neuroscience and Cognitive Technology Laboratory, Innopolis University, Kazan 420500, Russia
| | - Nikita S. Frolov
- Baltic Center for Artificial Intelligence and Neurotechnology, Immanuel Kant Baltic Federal University, Kaliningrad 236041, Russia; (E.N.P.); (N.S.F.); (N.S.)
- Neuroscience and Cognitive Technology Laboratory, Innopolis University, Kazan 420500, Russia
| | - Natalia Shusharina
- Baltic Center for Artificial Intelligence and Neurotechnology, Immanuel Kant Baltic Federal University, Kaliningrad 236041, Russia; (E.N.P.); (N.S.F.); (N.S.)
| | - Alexander E. Hramov
- Baltic Center for Artificial Intelligence and Neurotechnology, Immanuel Kant Baltic Federal University, Kaliningrad 236041, Russia; (E.N.P.); (N.S.F.); (N.S.)
- Neuroscience and Cognitive Technology Laboratory, Innopolis University, Kazan 420500, Russia
- Correspondence:
| |
Collapse
|