1
|
Hosseini MSK, Firoozabadi SM, Badie K, Azadfallah P. Personality-Based Emotion Recognition Using EEG Signals with a CNN-LSTM Network. Brain Sci 2023; 13:947. [PMID: 37371425 DOI: 10.3390/brainsci13060947] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 05/27/2023] [Accepted: 05/30/2023] [Indexed: 06/29/2023] Open
Abstract
The accurate detection of emotions has significant implications in healthcare, psychology, and human-computer interaction. Integrating personality information into emotion recognition can enhance its utility in various applications. The present study introduces a novel deep learning approach to emotion recognition, which utilizes electroencephalography (EEG) signals and the Big Five personality traits. The study recruited 60 participants and recorded their EEG data while they viewed unique sequence stimuli designed to effectively capture the dynamic nature of human emotions and personality traits. A pre-trained convolutional neural network (CNN) was used to extract emotion-related features from the raw EEG data. Additionally, a long short-term memory (LSTM) network was used to extract features related to the Big Five personality traits. The network was able to accurately predict personality traits from EEG data. The extracted features were subsequently used in a novel network to predict emotional states within the arousal and valence dimensions. The experimental results showed that the proposed classifier outperformed common classifiers, with a high accuracy of 93.97%. The findings suggest that incorporating personality traits as features in the designed network, for emotion recognition, leads to higher accuracy, highlighting the significance of examining these traits in the analysis of emotions.
Collapse
Affiliation(s)
| | - Seyed Mohammad Firoozabadi
- Department of Medical Physics, Faculty of Medicine, Tarbiat Modares University, Tehran 14117-13116, Iran
| | - Kambiz Badie
- Content & E-Services Research Group, IT Research Faculty, ICT Research Institute, Tehran 14399-55471, Iran
| | - Parviz Azadfallah
- Department of Psychology, Faculty of Humanities, Tarbiat Modares University, Tehran 14117-13116, Iran
| |
Collapse
|
2
|
Awan AW, Usman SM, Khalid S, Anwar A, Alroobaea R, Hussain S, Almotiri J, Ullah SS, Akram MU. An Ensemble Learning Method for Emotion Charting Using Multimodal Physiological Signals. SENSORS (BASEL, SWITZERLAND) 2022; 22:9480. [PMID: 36502183 PMCID: PMC9739519 DOI: 10.3390/s22239480] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 11/24/2022] [Accepted: 11/28/2022] [Indexed: 06/17/2023]
Abstract
Emotion charting using multimodal signals has gained great demand for stroke-affected patients, for psychiatrists while examining patients, and for neuromarketing applications. Multimodal signals for emotion charting include electrocardiogram (ECG) signals, electroencephalogram (EEG) signals, and galvanic skin response (GSR) signals. EEG, ECG, and GSR are also known as physiological signals, which can be used for identification of human emotions. Due to the unbiased nature of physiological signals, this field has become a great motivation in recent research as physiological signals are generated autonomously from human central nervous system. Researchers have developed multiple methods for the classification of these signals for emotion detection. However, due to the non-linear nature of these signals and the inclusion of noise, while recording, accurate classification of physiological signals is a challenge for emotion charting. Valence and arousal are two important states for emotion detection; therefore, this paper presents a novel ensemble learning method based on deep learning for the classification of four different emotional states including high valence and high arousal (HVHA), low valence and low arousal (LVLA), high valence and low arousal (HVLA) and low valence high arousal (LVHA). In the proposed method, multimodal signals (EEG, ECG, and GSR) are preprocessed using bandpass filtering and independent components analysis (ICA) for noise removal in EEG signals followed by discrete wavelet transform for time domain to frequency domain conversion. Discrete wavelet transform results in spectrograms of the physiological signal and then features are extracted using stacked autoencoders from those spectrograms. A feature vector is obtained from the bottleneck layer of the autoencoder and is fed to three classifiers SVM (support vector machine), RF (random forest), and LSTM (long short-term memory) followed by majority voting as ensemble classification. The proposed system is trained and tested on the AMIGOS dataset with k-fold cross-validation. The proposed system obtained the highest accuracy of 94.5% and shows improved results of the proposed method compared with other state-of-the-art methods.
Collapse
Affiliation(s)
- Amna Waheed Awan
- Department of Computer Engineering, Bahria University, Islamabad 44000, Pakistan
| | - Syed Muhammad Usman
- Department of Creative Technologies, Faculty of Computing and AI, Air University, Islamabad 44000, Pakistan
| | - Shehzad Khalid
- Department of Computer Engineering, Bahria University, Islamabad 44000, Pakistan
| | - Aamir Anwar
- School of Computing and Engineering, The University of West London, London W5 5RF, UK
| | - Roobaea Alroobaea
- Department of Computer Science, College of Computers and Information Technology, Taif University, P.O. Box 11099, Taif 21944, Saudi Arabia
| | - Saddam Hussain
- School of Digital Science, Universiti Brunei Darussalam, Jalan Tungku Link, Gadong BE1410, Brunei
| | - Jasem Almotiri
- Department of Computer Science, College of Computers and Information Technology, Taif University, P.O. Box 11099, Taif 21944, Saudi Arabia
| | - Syed Sajid Ullah
- Department of Information and Communication Technology, University of Agder (UiA), N-4898 Grimstad, Norway
- Department of Electrical and Computer Engineering, Villanova University, Villanova, PA 19085, USA
| | - Muhammad Usman Akram
- College of Eletrical and Mechanical Engineering (E & ME), National University of Science and Technology (NUST), Islamabad 44000, Pakistan
| |
Collapse
|
3
|
Kiani M, Andreu-Perez J, Hagras H, Rigato S, Filippetti ML. Towards Understanding Human Functional Brain Development With Explainable Artificial Intelligence: Challenges and Perspectives. IEEE COMPUT INTELL M 2022. [DOI: 10.1109/mci.2021.3129956] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
4
|
Rogala J, Dreszer J, Malinowska U, Waligóra M, Pluta A, Antonova I, Wróbel A. Stronger connectivity and higher extraversion protect against stress-related deterioration of cognitive functions. Sci Rep 2021; 11:17452. [PMID: 34465808 PMCID: PMC8408208 DOI: 10.1038/s41598-021-96718-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Accepted: 08/13/2021] [Indexed: 11/09/2022] Open
Abstract
Here we attempted to define the relationship between: EEG activity, personality and coping during lockdown. We were in a unique situation since the COVID-19 outbreak interrupted our independent longitudinal study. We already collected a significant amount of data before lockdown. During lockdown, a subgroup of participants willingly continued their engagement in the study. These circumstances provided us with an opportunity to examine the relationship between personality/cognition and brain rhythms in individuals who continued their engagement during lockdown compared to control data collected well before pandemic. The testing consisted of a one-time assessment of personality dimensions and two sessions of EEG recording and deductive reasoning task. Participants were divided into groups based on the time they completed the second session: before or during the COVID-19 outbreak ‘Pre-pandemic Controls’ and ‘Pandemics’, respectively. The Pandemics were characterized by a higher extraversion and stronger connectivity, compared to Pre-pandemic Controls. Furthermore, the Pandemics improved their cognitive performance under long-term stress as compared to the Pre-Pandemic Controls matched for personality traits to the Pandemics. The Pandemics were also characterized by increased EEG connectivity during lockdown. We posit that stronger EEG connectivity and higher extraversion could act as a defense mechanism against stress-related deterioration of cognitive functions.
Collapse
Affiliation(s)
- Jacek Rogala
- Bioimaging Research Center, World Hearing Center, Institute of Physiology and Pathology of Hearing, Warsaw, Poland. .,The Center for Systemic Risk Analysis, Faculty of "Artes Liberales", University of Warsaw, Warsaw, Poland.
| | - Joanna Dreszer
- The Center for Systemic Risk Analysis, Faculty of "Artes Liberales", University of Warsaw, Warsaw, Poland.,Institute of Psychology, Faculty of Philosophy and Social Sciences, Nicolaus Copernicus University in Toruń, Toruń, Poland
| | - Urszula Malinowska
- Instytut Biologii Doświadczalnej Im. Marcelego Nenckiego, Warsaw, Poland
| | - Marek Waligóra
- Instytut Biologii Doświadczalnej Im. Marcelego Nenckiego, Warsaw, Poland
| | - Agnieszka Pluta
- Faculty of Psychology, The University of Warsaw, Warsaw, Poland
| | - Ingrida Antonova
- Instytut Biologii Doświadczalnej Im. Marcelego Nenckiego, Warsaw, Poland
| | - Andrzej Wróbel
- The Center for Systemic Risk Analysis, Faculty of "Artes Liberales", University of Warsaw, Warsaw, Poland.,Instytut Biologii Doświadczalnej Im. Marcelego Nenckiego, Warsaw, Poland.,Institute of Philosophy, Faculty of Epistemology, University of Warsaw, Warsaw, Poland
| |
Collapse
|
5
|
Adeluyi O, Risco-Castillo MA, Liz Crespo M, Cicuttin A, Lee JA. A Computerized Bioinspired Methodology for Lightweight and Reliable Neural Telemetry. SENSORS (BASEL, SWITZERLAND) 2020; 20:s20226461. [PMID: 33198191 PMCID: PMC7696551 DOI: 10.3390/s20226461] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Revised: 11/07/2020] [Accepted: 11/09/2020] [Indexed: 06/11/2023]
Abstract
Personalized health monitoring of neural signals usually results in a very large dataset, the processing and transmission of which require considerable energy, storage, and processing time. We present bioinspired electroceptive compressive sensing (BeCoS) as an approach for minimizing these penalties. It is a lightweight and reliable approach for the compression and transmission of neural signals inspired by active electroceptive sensing used by weakly electric fish. It uses a signature signal and a sensed pseudo-sparse differential signal to transmit and reconstruct the signals remotely. We have used EEG datasets to compare BeCoS with the block sparse Bayesian learning-bound optimization (BSBL-BO) technique-A popular compressive sensing technique used for low-energy wireless telemonitoring of EEG signals. We achieved average coherence, latency, compression ratio, and estimated per-epoch power values that were 35.38%, 62.85%, 53.26%, and 13 mW better than BSBL-BO, respectively, while structural similarity was only 6.295% worse. However, the original and reconstructed signals remain visually similar. BeCoS senses the signals as a derivative of a predefined signature signal resulting in a pseudo-sparse signal that significantly improves the efficiency of the monitoring process. The results show that BeCoS is a promising approach for the health monitoring of neural signals.
Collapse
Affiliation(s)
- Olufemi Adeluyi
- Ministry of Communications and Digital Economy, Federal Secretariat, Abuja 900001, Nigeria;
| | - Miguel A. Risco-Castillo
- Engineering Physics, Department of Science, National University of Engineering, Av. Tupac Amaru 210, Cercado de Lima 15333, Peru;
| | - María Liz Crespo
- Multidisciplinary Lab, International Centre for Theoretical Physics, Via Beirut 31, 34100 Trieste, Italy; (M.L.C.); (A.C.)
| | - Andres Cicuttin
- Multidisciplinary Lab, International Centre for Theoretical Physics, Via Beirut 31, 34100 Trieste, Italy; (M.L.C.); (A.C.)
| | - Jeong-A Lee
- Department of Computer Engineering, Chosun University, 309 Pilmun-Daero, Dong-Gu, Gwangju 61452, Korea
| |
Collapse
|
6
|
García-Monge A, Rodríguez-Navarro H, González-Calvo G, Bores-García D. Brain Activity during Different Throwing Games: EEG Exploratory Study. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:E6796. [PMID: 32957731 PMCID: PMC7559334 DOI: 10.3390/ijerph17186796] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/25/2020] [Revised: 08/16/2020] [Accepted: 09/15/2020] [Indexed: 11/16/2022]
Abstract
The purpose of this study is to explore the differences in brain activity in various types of throwing games by making encephalographic records. Three conditions of throwing games were compared looking for significant differences (simple throwing, throwing to a goal, and simultaneous throwing with another player). After signal processing, power spectral densities were compared through variance analysis (p ≤ 0.001). Significant differences were found especially in high-beta oscillations (22-30 Hz). "Goal" and "Simultaneous" throwing conditions show significantly higher values than those shown for throws without opponent. This can be explained by the higher demand for motor control and the higher arousal in competition situations. On the other hand, the high-beta records of the "Goal" condition are significantly higher than those of the "Simultaneous" throwing, which could be understood from the association of the beta waves with decision-making processes. These results support the difference in brain activity during similar games. This has several implications: opening up a path to study the effects of each specific game on brain activity and calling into question the transfer of research findings on animal play to all types of human play.
Collapse
Affiliation(s)
- Alfonso García-Monge
- Department of Didactics of Musical, Artistic and Body Expression, Faculty of Education of Valladolid, University of Valladolid, 47011 Valladolid, Spain;
| | - Henar Rodríguez-Navarro
- Department of Pedagogy, Faculty of Education of Valladolid, University of Valladolid, 47011 Valladolid, Spain;
| | - Gustavo González-Calvo
- Department of Didactics of Musical, Artistic and Body Expression, Faculty of Education of Palencia, University of Valladolid, 34004 Palencia, Spain;
| | - Daniel Bores-García
- Department of Physical Therapy, Occupational Therapy, Rehabilitation and Physical Medicine, Faculty of Health Sciences, Rey Juan Carlos University, Alcorcón, 28922 Madrid, Spain
| |
Collapse
|