1
|
Xu F, Pan D, Zheng H, Ouyang Y, Jia Z, Zeng H. EESCN: A novel spiking neural network method for EEG-based emotion recognition. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 243:107927. [PMID: 38000320 DOI: 10.1016/j.cmpb.2023.107927] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Revised: 10/16/2023] [Accepted: 11/13/2023] [Indexed: 11/26/2023]
Abstract
BACKGROUND AND OBJECTIVE Although existing artificial neural networks have achieved good results in electroencephalograph (EEG) emotion recognition, further improvements are needed in terms of bio-interpretability and robustness. In this research, we aim to develop a highly efficient and high-performance method for emotion recognition based on EEG. METHODS We propose an Emo-EEGSpikeConvNet (EESCN), a novel emotion recognition method based on spiking neural network (SNN). It consists of a neuromorphic data generation module and a NeuroSpiking framework. The neuromorphic data generation module converts EEG data into 2D frame format as input to the NeuroSpiking framework, while the NeuroSpiking framework is used to extract spatio-temporal features of EEG for classification. RESULTS EESCN achieves high emotion recognition accuracies on DEAP and SEED-IV datasets, ranging from 94.56% to 94.81% on DEAP and a mean accuracy of 79.65% on SEED-IV. Compared to existing SNN methods, EESCN significantly improves EEG emotion recognition performance. In addition, it also has the advantages of faster running speed and less memory footprint. CONCLUSIONS EESCN has shown excellent performance and efficiency in EEG-based emotion recognition with potential for practical applications requiring portability and resource constraints.
Collapse
Affiliation(s)
- FeiFan Xu
- Hangzhou Dianzi University, School of Computer Science and Technology, HangZhou, ZheJiang, China.
| | - Deng Pan
- Hangzhou Dianzi University, School of Computer Science and Technology, HangZhou, ZheJiang, China.
| | - Haohao Zheng
- Hangzhou Dianzi University, School of Computer Science and Technology, HangZhou, ZheJiang, China.
| | - Yu Ouyang
- Hangzhou Dianzi University, School of Computer Science and Technology, HangZhou, ZheJiang, China.
| | - Zhe Jia
- Hangzhou Dianzi University, School of Computer Science and Technology, HangZhou, ZheJiang, China.
| | - Hong Zeng
- Hangzhou Dianzi University, School of Computer Science and Technology, HangZhou, ZheJiang, China; Key Laboratory of Brain Machine Collaborative of Zhejiang Province, HangZhou, ZheJiang, China.
| |
Collapse
|
2
|
Aung ST, Hassan M, Brady M, Mannan ZI, Azam S, Karim A, Zaman S, Wongsawat Y. Entropy-Based Emotion Recognition from Multichannel EEG Signals Using Artificial Neural Network. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:6000989. [PMID: 36275950 PMCID: PMC9584707 DOI: 10.1155/2022/6000989] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Accepted: 09/22/2022] [Indexed: 11/17/2022]
Abstract
Humans experience a variety of emotions throughout the course of their daily lives, including happiness, sadness, and rage. As a result, an effective emotion identification system is essential for electroencephalography (EEG) data to accurately reflect emotion in real-time. Although recent studies on this problem can provide acceptable performance measures, it is still not adequate for the implementation of a complete emotion recognition system. In this research work, we propose a new approach for an emotion recognition system, using multichannel EEG calculation with our developed entropy known as multivariate multiscale modified-distribution entropy (MM-mDistEn) which is combined with a model based on an artificial neural network (ANN) to attain a better outcome over existing methods. The proposed system has been tested with two different datasets and achieved better accuracy than existing methods. For the GAMEEMO dataset, we achieved an average accuracy ± standard deviation of 95.73% ± 0.67 for valence and 96.78% ± 0.25 for arousal. Moreover, the average accuracy percentage for the DEAP dataset reached 92.57% ± 1.51 in valence and 80.23% ± 1.83 in arousal.
Collapse
Affiliation(s)
- Si Thu Aung
- Department of Biomedical Engineering, Faculty of Engineering, Mahidol University, Salaya, Thailand
| | - Mehedi Hassan
- Computer Science and Engineering, North Western University, Khulna, Bangladesh
| | - Mark Brady
- Asia Pacific College of Business and Law, Charles Darwin University, Casuarina, NT, Australia
| | - Zubaer Ibna Mannan
- Department of Smart Computing, Kyungdong University, Global Campus, Goseong-Gun, Republic of Korea
| | - Sami Azam
- College of Engineering IT and Environment, Charles Darwin University, Casuarina, NT, Australia
| | - Asif Karim
- College of Engineering IT and Environment, Charles Darwin University, Casuarina, NT, Australia
| | - Sadika Zaman
- Computer Science and Engineering, North Western University, Khulna, Bangladesh
| | - Yodchanan Wongsawat
- Department of Biomedical Engineering, Faculty of Engineering, Mahidol University, Salaya, Thailand
| |
Collapse
|
3
|
Maciejewska K, Froelich W. Hierarchical Classification of Event-Related Potentials for the Recognition of Gender Differences in the Attention Task. ENTROPY (BASEL, SWITZERLAND) 2021; 23:1547. [PMID: 34828245 PMCID: PMC8617798 DOI: 10.3390/e23111547] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/17/2021] [Revised: 11/15/2021] [Accepted: 11/17/2021] [Indexed: 11/16/2022]
Abstract
Research on the functioning of human cognition has been a crucial problem studied for years. Electroencephalography (EEG) classification methods may serve as a precious tool for understanding the temporal dynamics of human brain activity, and the purpose of such an approach is to increase the statistical power of the differences between conditions that are too weak to be detected using standard EEG methods. Following that line of research, in this paper, we focus on recognizing gender differences in the functioning of the human brain in the attention task. For that purpose, we gathered, analyzed, and finally classified event-related potentials (ERPs). We propose a hierarchical approach, in which the electrophysiological signal preprocessing is combined with the classification method, enriched with a segmentation step, which creates a full line of electrophysiological signal classification during an attention task. This approach allowed us to detect differences between men and women in the P3 waveform, an ERP component related to attention, which were not observed using standard ERP analysis. The results provide evidence for the high effectiveness of the proposed method, which outperformed a traditional statistical analysis approach. This is a step towards understanding neuronal differences between men's and women's brains during cognition, aiming to reduce the misdiagnosis and adverse side effects in underrepresented women groups in health and biomedical research.
Collapse
Affiliation(s)
- Karina Maciejewska
- Institute of Biomedical Engineering, Faculty of Science and Technology, University of Silesia in Katowice, 75 Pulku Piechoty 1a Street, 41-500 Chorzow, Poland
| | - Wojciech Froelich
- Institute of Computer Science, Faculty of Science and Technology, University of Silesia in Katowice, Bedzinska 39 Street, 41-205 Sosnowiec, Poland;
| |
Collapse
|
4
|
PrimePatNet87: Prime pattern and tunable q-factor wavelet transform techniques for automated accurate EEG emotion recognition. Comput Biol Med 2021; 138:104867. [PMID: 34543892 DOI: 10.1016/j.compbiomed.2021.104867] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Revised: 09/08/2021] [Accepted: 09/08/2021] [Indexed: 11/24/2022]
Abstract
Nowadays, many deep models have been presented to recognize emotions using electroencephalogram (EEG) signals. These deep models are computationally intensive, it takes a longer time to train the model. Also, it is difficult to achieve high classification performance using for emotion classification using machine learning techniques. To overcome these limitations, we present a hand-crafted conventional EEG emotion classification network. In this work, we have used novel prime pattern and tunable q-factor wavelet transform (TQWT) techniques to develop an automated model to classify human emotions. Our proposed cognitive model comprises feature extraction, feature selection, and classification steps. We have used TQWT on the EEG signals to obtain the sub-bands. The prime pattern and statistical feature generator are employed on the generated sub-bands and original signal to generate 798 features. 399 (half of them) out of 798 features are selected using minimum redundancy maximum relevance (mRMR) selector, and misclassification rates of each signal are evaluated using support vector machine (SVM) classifier. The proposed network generated 87 feature vectors hence, this model is named PrimePatNet87. In the last step of the feature generation, the best 20 feature vectors which are selected based on the calculated misclassification rates, are concatenated. The generated feature vector is subjected to the feature selection and the most significant 1000 features are selected using the mRMR selector. These selected features are then classified using an SVM classifier. In the last phase, iterative majority voting has been used to generate a general result. We have used three publicly available datasets, namely DEAP, DREAMER, and GAMEEMO, to develop our proposed model. Our presented PrimePatNet87 model reached over 99% classification accuracy on whole datasets with leave one subject out (LOSO) validation. Our results demonstrate that the developed prime pattern network is accurate and ready for real-world applications.
Collapse
|
5
|
Li P, Yin C, Li M, Li H, Yang B. A dry electroencephalogram electrode for applications in steady-state visual evoked potential-based brain-computer interface systems. Biosens Bioelectron 2021; 187:113326. [PMID: 34004544 DOI: 10.1016/j.bios.2021.113326] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2021] [Revised: 05/05/2021] [Accepted: 05/06/2021] [Indexed: 02/02/2023]
Abstract
High-efficiency electroencephalogram (EEG) dry electrodes are a key component of brain-computer interface (BCI) technology because of their direct contact with the scalp. In this study, a semi-flexible polydopamine (PDA)/Pt-TiO2 electrode is prepared for the dry-contact acquisition of EEG signals. The PDA biofilm adheres strongly to the scalp and maintains a dynamic balance of water and ions. The Pt nanoparticles and TiO2 nanotube array together result in fast electron transfer. Therefore, the interface impedance between the dry PDA/Pt-TiO2 electrode and scalp is as low as 19.63-24.53 kΩ. The spontaneous EEG signal collected simultaneously using the dry PDA/Pt-TiO2 and wet Ag/AgCl electrodes had a correlation coefficient of up to 99.9%. In a steady-state visual evoked potential (SSVEP)-based BCI system, the dry electrode was used to collect EEG feedback signals for stimulations at 27 different frequencies in the range of 7-19.25 Hz. For these feedback signals, O1, Oz, and O2 channels in the occipital area exhibited high signal-to-noise ratios of 11.3, 11.8, and 11 dB, respectively. A volunteer wore an EEG headband with three PDA/Pt-TiO2 dry electrodes and successfully controlled the robotic arm of the SSVEP-BCI system in the untrained mode. The dry PDA/Pt-TiO2 electrode-based EEG cap is comfortable to wear, the identification signals of the SSVEP paradigm are accurate, and it is suitable for controlling external devices including a keyboard in the SSVEP-BCI system.
Collapse
Affiliation(s)
- Phenghai Li
- Tianjin Key Laboratory of Film Electronic and Communication Devices, Engineering Research Center of Optoelectronic Devices & Communication Technology (Ministry of Education), School of Electrical and Electronic Engineering, Tianjin University of Technology, Tianjin, 300384, PR China
| | - Can Yin
- Tianjin Key Laboratory of Film Electronic and Communication Devices, Engineering Research Center of Optoelectronic Devices & Communication Technology (Ministry of Education), School of Electrical and Electronic Engineering, Tianjin University of Technology, Tianjin, 300384, PR China
| | - Mingji Li
- Tianjin Key Laboratory of Film Electronic and Communication Devices, Engineering Research Center of Optoelectronic Devices & Communication Technology (Ministry of Education), School of Electrical and Electronic Engineering, Tianjin University of Technology, Tianjin, 300384, PR China.
| | - Hongji Li
- Tianjin Key Laboratory of Organic Solar Cells and Photochemical Conversion, Tianjin Key Laboratory of Drug Targeting and Bioimaging, School of Chemistry and Chemical Engineering, Tianjin University of Technology, Tianjin, 300384, PR China.
| | - Baohe Yang
- Tianjin Key Laboratory of Film Electronic and Communication Devices, Engineering Research Center of Optoelectronic Devices & Communication Technology (Ministry of Education), School of Electrical and Electronic Engineering, Tianjin University of Technology, Tianjin, 300384, PR China
| |
Collapse
|
6
|
Vukelić M, Lingelbach K, Pollmann K, Peissner M. Oscillatory EEG Signatures of Affective Processes during Interaction with Adaptive Computer Systems. Brain Sci 2020; 11:35. [PMID: 33396330 PMCID: PMC7824422 DOI: 10.3390/brainsci11010035] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2020] [Revised: 12/16/2020] [Accepted: 12/24/2020] [Indexed: 11/23/2022] Open
Abstract
Affect monitoring is being discussed as a novel strategy to make adaptive systems more user-oriented. Basic knowledge about oscillatory processes and functional connectivity underlying affect during naturalistic human-computer interactions (HCI) is, however, scarce. This study assessed local oscillatory power entrainment and distributed functional connectivity in a close-to-naturalistic HCI-paradigm. Sixteen participants interacted with a simulated assistance system which deliberately evoked positive (supporting goal-achievement) and negative (impeding goal-achievement) affective reactions. Electroencephalography (EEG) was used to examine the reactivity of the cortical system during the interaction by studying both event-related (de-)synchronization (ERD/ERS) and event-related functional coupling of cortical networks towards system-initiated assistance. Significantly higher α-band and β-band ERD in centro-parietal and parieto-occipital regions and β-band ERD in bi-lateral fronto-central regions were observed during impeding system behavior. Supportive system behavior activated significantly higher γ-band ERS in bi-hemispheric parietal-occipital regions. This was accompanied by functional coupling of remote β-band and γ-band activity in the medial frontal, left fronto-central and parietal regions, respectively. Our findings identify oscillatory signatures of positive and negative affective processes as reactions to system-initiated assistance. The findings contribute to the development of EEG-based neuroadaptive assistance loops by suggesting a non-obtrusive method for monitoring affect in HCI.
Collapse
Affiliation(s)
- Mathias Vukelić
- Fraunhofer Institute for Industrial Engineering IAO, 70569 Stuttgart, Germany; (K.P.); (M.P.)
| | - Katharina Lingelbach
- Institute of Human Factors and Technology Management IAT, University of Stuttgart, 70569 Stuttgart, Germany;
- Department of Psychology, University of Oldenburg, 26129 Oldenburg, Germany
| | - Kathrin Pollmann
- Fraunhofer Institute for Industrial Engineering IAO, 70569 Stuttgart, Germany; (K.P.); (M.P.)
| | - Matthias Peissner
- Fraunhofer Institute for Industrial Engineering IAO, 70569 Stuttgart, Germany; (K.P.); (M.P.)
| |
Collapse
|