1
|
Li D, Xie L, Wang Z, Yang H. Brain Emotion Perception Inspired EEG Emotion Recognition With Deep Reinforcement Learning. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:12979-12992. [PMID: 37126638 DOI: 10.1109/tnnls.2023.3265730] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
Inspired by the well-known Papez circuit theory and neuroscience knowledge of reinforcement learning, a double dueling deep Q network (DQN) is built incorporating the electroencephalogram (EEG) signals of the frontal lobe as prior information, which is named frontal lobe double dueling DQN (FLD3QN). The framework of FLD3QN is constructed in accord with the brain emotion mechanism which takes the frontal lobe and the thalamus as the core, in which the part of the Papez circuit is simulated by the bifrontal lobe residual convolution neural network (BiFRCNN). Moreover, a step penalty factor is designed to constrain the number of mistakes of the agent. The ablation studies results on the public EEG emotion dataset DEAP verified the important roles of the frontal lobe and the Papez circuit in modeling the procedure of learning rewards during the perception of emotions, with a great increase in the average accuracies by 25.24% and 23.31% in valence and arousal dimensions.
Collapse
|
2
|
Hu L, Tan C, Xu J, Qiao R, Hu Y, Tian Y. Decoding emotion with phase-amplitude fusion features of EEG functional connectivity network. Neural Netw 2024; 172:106148. [PMID: 38309138 DOI: 10.1016/j.neunet.2024.106148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 12/20/2023] [Accepted: 01/23/2024] [Indexed: 02/05/2024]
Abstract
Decoding emotional neural representations from the electroencephalographic (EEG)-based functional connectivity network (FCN) is of great scientific importance for uncovering emotional cognition mechanisms and developing harmonious human-computer interactions. However, existing methods mainly rely on phase-based FCN measures (e.g., phase locking value [PLV]) to capture dynamic interactions between brain oscillations in emotional states, which fail to reflect the energy fluctuation of cortical oscillations over time. In this study, we initially examined the efficacy of amplitude-based functional networks (e.g., amplitude envelope correlation [AEC]) in representing emotional states. Subsequently, we proposed an efficient phase-amplitude fusion framework (PAF) to fuse PLV and AEC and used common spatial pattern (CSP) to extract fused spatial topological features from PAF for multi-class emotion recognition. We conducted extensive experiments on the DEAP and MAHNOB-HCI datasets. The results showed that: (1) AEC-derived discriminative spatial network topological features possess the ability to characterize emotional states, and the differential network patterns of AEC reflect dynamic interactions in brain regions associated with emotional cognition. (2) The proposed fusion features outperformed other state-of-the-art methods in terms of classification accuracy for both datasets. Moreover, the spatial filter learned from PAF is separable and interpretable, enabling a description of affective activation patterns from both phase and amplitude perspectives.
Collapse
Affiliation(s)
- Liangliang Hu
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China; West China Institute of Children's Brain and Cognition, Chongqing University of Education, Chongqing 400065, China.
| | - Congming Tan
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
| | - Jiayang Xu
- School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
| | - Rui Qiao
- School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
| | - Yilin Hu
- School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
| | - Yin Tian
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China; School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing 400065, China; Institute for Advanced Sciences, Chongqing University of Posts and Telecommunications, Chongqing 400065, China; Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongqing 400064, China.
| |
Collapse
|
3
|
Jeong JH, Cho JH, Lee YE, Lee SH, Shin GH, Kweon YS, Millán JDR, Müller KR, Lee SW. 2020 International brain-computer interface competition: A review. Front Hum Neurosci 2022; 16:898300. [PMID: 35937679 PMCID: PMC9354666 DOI: 10.3389/fnhum.2022.898300] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Accepted: 07/01/2022] [Indexed: 11/16/2022] Open
Abstract
The brain-computer interface (BCI) has been investigated as a form of communication tool between the brain and external devices. BCIs have been extended beyond communication and control over the years. The 2020 international BCI competition aimed to provide high-quality neuroscientific data for open access that could be used to evaluate the current degree of technical advances in BCI. Although there are a variety of remaining challenges for future BCI advances, we discuss some of more recent application directions: (i) few-shot EEG learning, (ii) micro-sleep detection (iii) imagined speech decoding, (iv) cross-session classification, and (v) EEG(+ear-EEG) detection in an ambulatory environment. Not only did scientists from the BCI field compete, but scholars with a broad variety of backgrounds and nationalities participated in the competition to address these challenges. Each dataset was prepared and separated into three data that were released to the competitors in the form of training and validation sets followed by a test set. Remarkable BCI advances were identified through the 2020 competition and indicated some trends of interest to BCI researchers.
Collapse
Affiliation(s)
- Ji-Hoon Jeong
- School of Computer Science, Chungbuk National University, Cheongju, South Korea
| | - Jeong-Hyun Cho
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Young-Eun Lee
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Seo-Hyun Lee
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Gi-Hwan Shin
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Young-Seok Kweon
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - José del R. Millán
- Department of Electrical and Computer Engineering, University of Texas at Austin, Austin, TX, United States
| | - Klaus-Robert Müller
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
- Machine Learning Group, Department of Computer Science, Berlin Institute of Technology, Berlin, Germany
- Max Planck Institute for Informatics, Saarbrucken, Germany
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
| | - Seong-Whan Lee
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
| |
Collapse
|
4
|
Xie Z, Deng Y, Xie C, Yao Y. Changes of adrenocorticotropic hormone rhythm and cortisol circadian rhythm in patients with depression complicated with anxiety and their effects on the psychological state of patients. Front Psychiatry 2022; 13:1030811. [PMID: 36741558 PMCID: PMC9889930 DOI: 10.3389/fpsyt.2022.1030811] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Accepted: 12/05/2022] [Indexed: 01/19/2023] Open
Abstract
Objective: This work was to explore the rhythm of adrenocorticotropic hormone (ACTH) and cortisol in patients with depression and anxiety and their effects on mental state. In this work, with depression complicated with anxiety patients as the A-MDD group (n = 21), and depression without anxiety symptoms as the NA-MDD group (n = 21). Firstly, data features were extracted according to the electroencephalo-graph (EEG) data of different patients, and a DR model was constructed for diagnosis. The Hamilton Depression Scale 24 (HAMD-24) was employed to evaluate the severity, and the ACTH and cortisol levels were detected and compared for patients in the A-MDD group and NA-MDD group. In addition, the psychological status of the patients was assessed using the Toronto Alexithymia Scale (TAS). As a result, the AI-based DR model showed a high recognition accuracy for depression. The HAMD-24 score in the A-MDD group (31.81 ± 5.39 points) was statistically higher than the score in the NA-MDD group (25.25 ± 5.02 points) (P < 0.05). No visible difference was found in ACTH levels of patients in different groups (P > 0.05). The incidence of cortisol rhythm disorder (CRD) in the A-MDD group was much higher (P < 0.05). The differences in TAS scores between the two groups were significantly statistically significant (P < 0.01). In conclusion, the AI-based DR Model achieves a more accurate identification of depression; depression with or without anxiety has different effects on the mental state of patients. CRD may be one of the biological markers of depression combined with anxiety.
Collapse
Affiliation(s)
- Zheng Xie
- Department of Psychological Medicine, Henan Provincial People's Hospital, People's Hospital of Zhengzhou University, Zhengzhou, Henan, China
| | - Yajie Deng
- Department of Psychological Medicine, Henan Provincial People's Hospital, People's Hospital of Zhengzhou University, Zhengzhou, Henan, China
| | - Chunyu Xie
- Henan Center for Disease Control and Prevention, Zhengzhou, Henan, China
| | - Yuanlong Yao
- Medical College of Henan University, Kaifeng, Henan, China
| |
Collapse
|
5
|
Li W, Huan W, Hou B, Tian Y, Zhang Z, Song A. Can Emotion be Transferred? – A Review on Transfer Learning for EEG-Based Emotion Recognition. IEEE Trans Cogn Dev Syst 2021. [DOI: 10.1109/tcds.2021.3098842] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|