1
|
Chen Q, Mao X, Song Y, Wang K. An EEG-based emotion recognition method by fusing multi-frequency-spatial features under multi-frequency bands. J Neurosci Methods 2025; 415:110360. [PMID: 39778774 DOI: 10.1016/j.jneumeth.2025.110360] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2024] [Revised: 12/23/2024] [Accepted: 01/03/2025] [Indexed: 01/11/2025]
Abstract
BACKGROUND Recognition of emotion changes is of great significance to a person's physical and mental health. At present, EEG-based emotion recognition methods are mainly focused on time or frequency domains, but rarely on spatial information. Therefore, the goal of this study is to improve the performance of emotion recognition by integrating frequency and spatial domain information under multi-frequency bands. NEW METHODS Firstly, EEG signals of four frequency bands are extracted, and then three frequency-spatial features of differential entropy (DE) symmetric difference (SD) and symmetric quotient (SQ) are separately calculated. Secondly, according to the distribution of EEG electrodes, a series of brain maps are constructed by three frequency-spatial features for each frequency band. Thirdly, a Multi-Parallel-Input Convolutional Neural Network (MPICNN) uses the constructed brain maps to train and obtain the emotion recognition model. Finally, the subject-dependent experiments are conducted on DEAP and SEED-IV datasets. RESULTS The experimental results of DEAP dataset show that the average accuracy of four-class emotion recognition, namely, high-valence high-arousal, high-valence low-arousal, low-valence high-arousal and low-valence low-arousal, reaches 98.71 %. The results of SEED-IV dataset show the average accuracy of four-class emotion recognition, namely, happy, sad, neutral and fear reaches 92.55 %. COMPARISON WITH EXISTING METHODS This method has a best classification performance compared with the state-of-the-art methods on both four-class emotion recognition datasets. CONCLUSIONS This EEG-based emotion recognition method fused multi-frequency-spatial features under multi-frequency bands, and effectively improved the recognition performance compared with the existing methods.
Collapse
Affiliation(s)
- Qiuyu Chen
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao, China
| | - Xiaoqian Mao
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao, China.
| | - Yuebin Song
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao, China
| | - Kefa Wang
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao, China
| |
Collapse
|
2
|
Vakitbilir N, Islam A, Gomez A, Stein KY, Froese L, Bergmann T, Sainbhi AS, McClarty D, Raj R, Zeiler FA. Multivariate Modelling and Prediction of High-Frequency Sensor-Based Cerebral Physiologic Signals: Narrative Review of Machine Learning Methodologies. SENSORS (BASEL, SWITZERLAND) 2024; 24:8148. [PMID: 39771880 PMCID: PMC11679405 DOI: 10.3390/s24248148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/14/2024] [Revised: 12/09/2024] [Accepted: 12/18/2024] [Indexed: 01/11/2025]
Abstract
Monitoring cerebral oxygenation and metabolism, using a combination of invasive and non-invasive sensors, is vital due to frequent disruptions in hemodynamic regulation across various diseases. These sensors generate continuous high-frequency data streams, including intracranial pressure (ICP) and cerebral perfusion pressure (CPP), providing real-time insights into cerebral function. Analyzing these signals is crucial for understanding complex brain processes, identifying subtle patterns, and detecting anomalies. Computational models play an essential role in linking sensor-derived signals to the underlying physiological state of the brain. Multivariate machine learning models have proven particularly effective in this domain, capturing intricate relationships among multiple variables simultaneously and enabling the accurate modeling of cerebral physiologic signals. These models facilitate the development of advanced diagnostic and prognostic tools, promote patient-specific interventions, and improve therapeutic outcomes. Additionally, machine learning models offer great flexibility, allowing different models to be combined synergistically to address complex challenges in sensor-based data analysis. Ensemble learning techniques, which aggregate predictions from diverse models, further enhance predictive accuracy and robustness. This review explores the use of multivariate machine learning models in cerebral physiology as a whole, with an emphasis on sensor-derived signals related to hemodynamics, cerebral oxygenation, metabolism, and other modalities such as electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) where applicable. It will detail the operational principles, mathematical foundations, and clinical implications of these models, providing a deeper understanding of their significance in monitoring cerebral function.
Collapse
Affiliation(s)
- Nuray Vakitbilir
- Department of Biomedical Engineering, Price Faculty of Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada; (A.I.); (K.Y.S.); (A.S.S.); (F.A.Z.)
| | - Abrar Islam
- Department of Biomedical Engineering, Price Faculty of Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada; (A.I.); (K.Y.S.); (A.S.S.); (F.A.Z.)
| | - Alwyn Gomez
- Section of Neurosurgery, Department of Surgery, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB R3A 1R9, Canada
- Department of Human Anatomy and Cell Science, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB R3E 0J9, Canada
| | - Kevin Y. Stein
- Department of Biomedical Engineering, Price Faculty of Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada; (A.I.); (K.Y.S.); (A.S.S.); (F.A.Z.)
| | - Logan Froese
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden;
| | - Tobias Bergmann
- Undergraduate Engineering, Price Faculty of Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada;
| | - Amanjyot Singh Sainbhi
- Department of Biomedical Engineering, Price Faculty of Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada; (A.I.); (K.Y.S.); (A.S.S.); (F.A.Z.)
| | - Davis McClarty
- Undergraduate Medicine, College of Medicine, Rady Faculty of Health Sciences, Winnipeg, MB R3E 3P5, Canada;
| | - Rahul Raj
- Department of Neurosurgery, University of Helsinki, 00100 Helsinki, Finland;
| | - Frederick A. Zeiler
- Department of Biomedical Engineering, Price Faculty of Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada; (A.I.); (K.Y.S.); (A.S.S.); (F.A.Z.)
- Section of Neurosurgery, Department of Surgery, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB R3A 1R9, Canada
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden;
- Pan Am Clinic Foundation, Winnipeg, MB R3M 3E4, Canada
| |
Collapse
|
3
|
Qian R, Xiong X, Zhou J, Yu H, Sha K. CSA-SA-CRTNN: A Dual-Stream Adaptive Convolutional Cyclic Hybrid Network Combining Attention Mechanisms for EEG Emotion Recognition. Brain Sci 2024; 14:817. [PMID: 39199509 PMCID: PMC11353053 DOI: 10.3390/brainsci14080817] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2024] [Revised: 08/11/2024] [Accepted: 08/14/2024] [Indexed: 09/01/2024] Open
Abstract
In recent years, EEG-based emotion recognition technology has made progress, but there are still problems of low model efficiency and loss of emotional information, and there is still room for improvement in recognition accuracy. To fully utilize EEG's emotional information and improve recognition accuracy while reducing computational costs, this paper proposes a Convolutional-Recurrent Hybrid Network with a dual-stream adaptive approach and an attention mechanism (CSA-SA-CRTNN). Firstly, the model utilizes a CSAM module to assign corresponding weights to EEG channels. Then, an adaptive dual-stream convolutional-recurrent network (SA-CRNN and MHSA-CRNN) is applied to extract local spatial-temporal features. After that, the extracted local features are concatenated and fed into a temporal convolutional network with a multi-head self-attention mechanism (MHSA-TCN) to capture global information. Finally, the extracted EEG information is used for emotion classification. We conducted binary and ternary classification experiments on the DEAP dataset, achieving 99.26% and 99.15% accuracy for arousal and valence in binary classification and 97.69% and 98.05% in ternary classification, and on the SEED dataset, we achieved an accuracy of 98.63%, surpassing relevant algorithms. Additionally, the model's efficiency is significantly higher than other models, achieving better accuracy with lower resource consumption.
Collapse
Affiliation(s)
| | | | - Jianhua Zhou
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China; (R.Q.); (X.X.); (H.Y.); (K.S.)
| | | | | |
Collapse
|
4
|
Yu H, Xiong X, Zhou J, Qian R, Sha K. CATM: A Multi-Feature-Based Cross-Scale Attentional Convolutional EEG Emotion Recognition Model. SENSORS (BASEL, SWITZERLAND) 2024; 24:4837. [PMID: 39123882 PMCID: PMC11314657 DOI: 10.3390/s24154837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/24/2024] [Revised: 07/21/2024] [Accepted: 07/23/2024] [Indexed: 08/12/2024]
Abstract
Aiming at the problem that existing emotion recognition methods fail to make full use of the information in the time, frequency, and spatial domains in the EEG signals, which leads to the low accuracy of EEG emotion classification, this paper proposes a multi-feature, multi-frequency band-based cross-scale attention convolutional model (CATM). The model is mainly composed of a cross-scale attention module, a frequency-space attention module, a feature transition module, a temporal feature extraction module, and a depth classification module. First, the cross-scale attentional convolution module extracts spatial features at different scales for the preprocessed EEG signals; then, the frequency-space attention module assigns higher weights to important channels and spatial locations; next, the temporal feature extraction module extracts temporal features of the EEG signals; and, finally, the depth classification module categorizes the EEG signals into emotions. We evaluated the proposed method on the DEAP dataset with accuracies of 99.70% and 99.74% in the valence and arousal binary classification experiments, respectively; the accuracy in the valence-arousal four-classification experiment was 97.27%. In addition, considering the application of fewer channels, we also conducted 5-channel experiments, and the binary classification accuracies of valence and arousal were 97.96% and 98.11%, respectively. The valence-arousal four-classification accuracy was 92.86%. The experimental results show that the method proposed in this paper exhibits better results compared to other recent methods, and also achieves better results in few-channel experiments.
Collapse
Affiliation(s)
| | | | - Jianhua Zhou
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China; (H.Y.); (X.X.); (R.Q.); (K.S.)
| | | | | |
Collapse
|
5
|
Mizrahi D, Laufer I, Zuckerman I. Comparative analysis of ROCKET-driven and classic EEG features in predicting attachment styles. BMC Psychol 2024; 12:87. [PMID: 38388958 PMCID: PMC10882770 DOI: 10.1186/s40359-024-01576-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2023] [Accepted: 02/05/2024] [Indexed: 02/24/2024] Open
Abstract
Predicting attachment styles using AI algorithms remains relatively unexplored in scientific literature. This study addresses this gap by employing EEG data to evaluate the effectiveness of ROCKET-driven features versus classic features, both analyzed using the XGBoost machine learning algorithm, for classifying 'secure' or 'insecure' attachment styles.Participants, fourth-year engineering students aged 20-35, first completed the ECR-R questionnaire. A subset then underwent EEG sessions while performing the Arrow Flanker Task, receiving success or failure feedback for each trial.Our findings reveal the effectiveness of both feature sets. The dataset with ROCKET-derived features demonstrated an 88.41% True Positive Rate (TPR) in classifying 'insecure' attachment styles, compared to the classic features dataset, which achieved a notable TPR as well. Visual representations further support ROCKET-derived features' proficiency in identifying insecure attachment tendencies, while the classic features exhibited limitations in classification accuracy. Although the ROCKET-derived features exhibited higher TPR, the classic features also presented a substantial predictive ability.In conclusion, this study advances the integration of AI in psychological assessments, emphasizing the significance of feature selection for specific datasets and applications. While both feature sets effectively classified EEG-based attachment styles, the ROCKET-derived features demonstrated a superior performance across multiple metrics, making them the preferred choice for this study.
Collapse
Affiliation(s)
- Dor Mizrahi
- Department of Industrial Engineering and Management, Ariel University, Ariel, Israel
| | - Ilan Laufer
- Department of Industrial Engineering and Management, Ariel University, Ariel, Israel
| | - Inon Zuckerman
- Department of Industrial Engineering and Management, Ariel University, Ariel, Israel
| |
Collapse
|