1
|
Li C, Li P, Zhang Y, Li N, Si Y, Li F, Cao Z, Chen H, Chen B, Yao D, Xu P. Effective Emotion Recognition by Learning Discriminative Graph Topologies in EEG Brain Networks. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:10258-10272. [PMID: 37022389 DOI: 10.1109/tnnls.2023.3238519] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Multichannel electroencephalogram (EEG) is an array signal that represents brain neural networks and can be applied to characterize information propagation patterns for different emotional states. To reveal these inherent spatial graph features and increase the stability of emotion recognition, we propose an effective emotion recognition model that performs multicategory emotion recognition with multiple emotion-related spatial network topology patterns (MESNPs) by learning discriminative graph topologies in EEG brain networks. To evaluate the performance of our proposed MESNP model, we conducted single-subject and multisubject four-class classification experiments on two public datasets, MAHNOB-HCI and DEAP. Compared with existing feature extraction methods, the MESNP model significantly enhances the multiclass emotional classification performance in the single-subject and multisubject conditions. To evaluate the online version of the proposed MESNP model, we designed an online emotion monitoring system. We recruited 14 participants to conduct the online emotion decoding experiments. The average online experimental accuracy of the 14 participants was 84.56%, indicating that our model can be applied in affective brain-computer interface (aBCI) systems. The offline and online experimental results demonstrate that the proposed MESNP model effectively captures discriminative graph topology patterns and significantly improves emotion classification performance. Moreover, the proposed MESNP model provides a new scheme for extracting features from strongly coupled array signals.
Collapse
|
2
|
Naghibi M, Farrokhi A, Faizi M. Small Urban Green Spaces: Insights into Perception, Preference, and Psychological Well-being in a Densely Populated Areas of Tehran, Iran. ENVIRONMENTAL HEALTH INSIGHTS 2024; 18:11786302241248314. [PMID: 38756542 PMCID: PMC11097736 DOI: 10.1177/11786302241248314] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Accepted: 04/03/2024] [Indexed: 05/18/2024]
Abstract
In metropolitan areas worldwide, abandoned properties are prevalent, prompting a need for small urban green spaces (SUGS) to meet the growing demand. Understanding residents' preferences and perceptions of transformed spaces is vital for effective urban design. This study delves into residents' preferences and perceptions regarding the transformation of such spaces into SUGS and their impact on psychological well-being. By examining how these preferences and perceived health benefits shape the value of transformed spaces, the research aims to inform effective urban design strategies. The participants underwent visual stimulation, with psychological reactions recorded through Electroencephalogram (EEG) readings and assessed via Questionnaire. Machine learning techniques analyzed EEG sub-band data, achieving an average accuracy of 92.8% when comparing leftover and designed spaces. Results revealed that different types of transformed spaces provoke distinct physiological and preference responses. Specifically, viewing SUGS was associated with significant changes in gamma wave power, suggesting a correlation between enhanced gamma activity and increased feelings of empathy. Moreover, participants also reported enhanced comfort, relaxation, and overall mood, and a strong preference for SUGS over untransformed spaces, emphasizing the value placed on these areas for their health benefits. This research highlights the positive impact of even SUGS on mental health, using EEG data to assess emotional states triggered by urban spaces. The study concludes with a call for further research to investigate the long-term benefits of SUGS on well-being, alongside an exploration of the gamma band as a neural marker for emotional restoration in urban green spaces. This research highlights the crucial role of urban design in fostering psychological well-being through the strategic development of green spaces, suggesting a paradigm shift toward more inclusive, health-promoting urban environments.
Collapse
Affiliation(s)
- Maryam Naghibi
- Urbanism Department, Faculty of Architecture and the Built Environment, Delft University of Technology, Delft, The Netherlands
| | - Ashkan Farrokhi
- School of Cognitive Sciences, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran
| | - Mohsen Faizi
- School of Architecture and Environmental Design, Iran University of Science and Technology, Tehran, Iran
| |
Collapse
|
3
|
Hu W, Bian G, Huang L, Pi Y, Zhang X, Zhang X, de Albuquerque VHC, Wu W. Constructing Bodily Emotion Maps Based on High-Density Body Surface Potentials for Psychophysiological Computing. IEEE J Biomed Health Inform 2024; 28:2500-2511. [PMID: 38051611 DOI: 10.1109/jbhi.2023.3339382] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
Emotion is a complex physiological and psychological activity, accompanied by subjective physiological sensations and objective physiological changes. The body sensation map describes the changes in body sensation associated with emotion in a topographic manner, but it relies on subjective evaluations from participants. Physiological signals are a more reliable measure of emotion, but most research focuses on the central nervous system, neglecting the importance of the peripheral nervous system. In this study, a body surface potential mapping (BSPM) system was constructed, and an experiment was designed to induce emotions and obtain high-density body surface potential information under negative and non-negative emotions. Then, by constructing and analyzing the functional connectivity network of BSPs, the high-density electrophysiological characteristics are obtained and visualized as bodily emotion maps. The results showed that the functional connectivity network of BSPs under negative emotions had denser connections, and emotion maps based on local clustering coefficient (LCC) are consistent with BSMs under negative emotions. in addition, our features can classify negative and non-negative emotions with the highest classification accuracy of 80.77%. In conclusion, this study constructs an emotion map based on high-density BSPs, which offers a novel approach to psychophysiological computing.
Collapse
|
4
|
Strauss DJ, Francis AL, Vibell J, Corona-Strauss FI. The role of attention in immersion: The two-competitor model. Brain Res Bull 2024; 210:110923. [PMID: 38462137 DOI: 10.1016/j.brainresbull.2024.110923] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Revised: 11/19/2023] [Accepted: 03/06/2024] [Indexed: 03/12/2024]
Abstract
Currently, we face an exponentially increasing interest in immersion, especially sensory-driven immersion, mainly due to the rapid development of ideas and business models centered around a digital virtual universe as well as the increasing availability of affordable immersive technologies for education, communication, and entertainment. However, a clear definition of 'immersion', in terms of established neurocognitive concepts and measurable properties, remains elusive, slowing research on the human side of immersive interfaces. To address this problem, we propose a conceptual, taxonomic model of attention in immersion. We argue (a) modeling immersion theoretically as well as studying immersion experimentally requires a detailed characterization of the role of attention in immersion, even though (b) attention, while necessary, cannot be a sufficient condition for defining immersion. Our broader goal is to characterize immersion in terms that will be compatible with established psychophysiolgical measures that could then in principle be used for the assessment and eventually the optimization of an immersive experience. We start from the perspective that immersion requires the projection of attention to an induced reality, and build on accepted taxonomies of different modes of attention for the development of our two-competitor model. The two-competitor model allows for a quantitative implementation and has an easy graphical interpretation. It helps to highlight the important link between different modes of attention and affect in studying immersion.
Collapse
Affiliation(s)
- Daniel J Strauss
- Systems Neuroscience & Neurotechnology Unit, Faculty of Medicine, Saarland University & School of Engineering, htw saar, Homburg/Saar, Germany.
| | - Alexander L Francis
- Speech Perception & Cognitive Effort Lab, Dept. of Speech, Language & Hearing Sciences, Purdue University, West Lafayette, IN, USA
| | - Jonas Vibell
- Brain & Behavior Lab, Dept. of Psychology, University of Hawai'i at Manoa, Honololulu, HI, USA
| | - Farah I Corona-Strauss
- Systems Neuroscience & Neurotechnology Unit, Faculty of Medicine, Saarland University & School of Engineering, htw saar, Homburg/Saar, Germany
| |
Collapse
|
5
|
Hu L, Tan C, Xu J, Qiao R, Hu Y, Tian Y. Decoding emotion with phase-amplitude fusion features of EEG functional connectivity network. Neural Netw 2024; 172:106148. [PMID: 38309138 DOI: 10.1016/j.neunet.2024.106148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 12/20/2023] [Accepted: 01/23/2024] [Indexed: 02/05/2024]
Abstract
Decoding emotional neural representations from the electroencephalographic (EEG)-based functional connectivity network (FCN) is of great scientific importance for uncovering emotional cognition mechanisms and developing harmonious human-computer interactions. However, existing methods mainly rely on phase-based FCN measures (e.g., phase locking value [PLV]) to capture dynamic interactions between brain oscillations in emotional states, which fail to reflect the energy fluctuation of cortical oscillations over time. In this study, we initially examined the efficacy of amplitude-based functional networks (e.g., amplitude envelope correlation [AEC]) in representing emotional states. Subsequently, we proposed an efficient phase-amplitude fusion framework (PAF) to fuse PLV and AEC and used common spatial pattern (CSP) to extract fused spatial topological features from PAF for multi-class emotion recognition. We conducted extensive experiments on the DEAP and MAHNOB-HCI datasets. The results showed that: (1) AEC-derived discriminative spatial network topological features possess the ability to characterize emotional states, and the differential network patterns of AEC reflect dynamic interactions in brain regions associated with emotional cognition. (2) The proposed fusion features outperformed other state-of-the-art methods in terms of classification accuracy for both datasets. Moreover, the spatial filter learned from PAF is separable and interpretable, enabling a description of affective activation patterns from both phase and amplitude perspectives.
Collapse
Affiliation(s)
- Liangliang Hu
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China; West China Institute of Children's Brain and Cognition, Chongqing University of Education, Chongqing 400065, China.
| | - Congming Tan
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
| | - Jiayang Xu
- School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
| | - Rui Qiao
- School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
| | - Yilin Hu
- School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
| | - Yin Tian
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China; School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing 400065, China; Institute for Advanced Sciences, Chongqing University of Posts and Telecommunications, Chongqing 400065, China; Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongqing 400064, China.
| |
Collapse
|
6
|
Bai Y, Yu M, Li Y. Dynamic Neural Patterns of Human Emotions in Virtual Reality: Insights from EEG Microstate Analysis. Brain Sci 2024; 14:113. [PMID: 38391688 PMCID: PMC10886836 DOI: 10.3390/brainsci14020113] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Revised: 01/10/2024] [Accepted: 01/22/2024] [Indexed: 02/24/2024] Open
Abstract
Emotions play a crucial role in human life and affect mental health. Understanding the neural patterns associated with emotions is essential. Previous studies carried out some exploration of the neural features of emotions, but most have designed experiments in two-dimensional (2D) environments, which differs from real-life scenarios. To create a more real environment, this study investigated emotion-related brain activity using electroencephalography (EEG) microstate analysis in a virtual reality (VR) environment. We recruited 42 healthy volunteers to participate in our study. We explored the dynamic features of different emotions, and four characteristic microstates were analyzed. In the alpha band, microstate A exhibited a higher occurrence in both negative and positive emotions than in neutral emotions. Microstate C exhibited a prolonged duration of negative emotions compared to positive emotions, and a higher occurrence was observed in both microstates C and D during positive emotions. Notably, a unique transition pair was observed between microstates B and C during positive emotions, whereas a unique transition pair was observed between microstates A and D during negative emotions. This study emphasizes the potential of integrating virtual reality (VR) and EEG to facilitate experimental design. Furthermore, this study enhances our comprehension of neural activities during various emotional states.
Collapse
Affiliation(s)
- Yicai Bai
- School of Life Sciences, Shanghai University, Shanghai 200444, China
| | - Minchang Yu
- School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China
| | - Yingjie Li
- School of Life Sciences, Shanghai University, Shanghai 200444, China
- College of International Education, Shanghai University, Shanghai 200444, China
| |
Collapse
|
7
|
Wang S, Luo Z, Zhao S, Zhang Q, Liu G, Wu D, Yin E, Chen C. Classification of EEG Signals Based on Sparrow Search Algorithm-Deep Belief Network for Brain-Computer Interface. Bioengineering (Basel) 2023; 11:30. [PMID: 38247907 PMCID: PMC10813095 DOI: 10.3390/bioengineering11010030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Revised: 12/15/2023] [Accepted: 12/22/2023] [Indexed: 01/23/2024] Open
Abstract
In brain-computer interface (BCI) systems, challenges are presented by the recognition of motor imagery (MI) brain signals. Established recognition approaches have achieved favorable performance from patterns like SSVEP, AEP, and P300, whereas the classification methods for MI need to be improved. Hence, seeking a classification method that exhibits high accuracy and robustness for application in MI-BCI systems is essential. In this study, the Sparrow search algorithm (SSA)-optimized Deep Belief Network (DBN), called SSA-DBN, is designed to recognize the EEG features extracted by the Empirical Mode Decomposition (EMD). The performance of the DBN is enhanced by the optimized hyper-parameters obtained through the SSA. Our method's efficacy was tested on three datasets: two public and one private. Results indicate a relatively high accuracy rate, outperforming three baseline methods. Specifically, on the private dataset, our approach achieved an accuracy of 87.83%, marking a significant 10.38% improvement over the standard DBN algorithm. For the BCI IV 2a dataset, we recorded an accuracy of 86.14%, surpassing the DBN algorithm by 9.33%. In the SMR-BCI dataset, our method attained a classification accuracy of 87.21%, which is 5.57% higher than that of the conventional DBN algorithm. This study demonstrates enhanced classification capabilities in MI-BCI, potentially contributing to advancements in the field of BCI.
Collapse
Affiliation(s)
- Shuai Wang
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| | - Zhiguo Luo
- Defense Innovation Institute, Academy of Military Sciences (AMS), Beijing 100071, China; (Z.L.); (S.Z.)
| | - Shaokai Zhao
- Defense Innovation Institute, Academy of Military Sciences (AMS), Beijing 100071, China; (Z.L.); (S.Z.)
| | - Qilong Zhang
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| | - Guangrong Liu
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| | - Dongyue Wu
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| | - Erwei Yin
- Defense Innovation Institute, Academy of Military Sciences (AMS), Beijing 100071, China; (Z.L.); (S.Z.)
| | - Chao Chen
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| |
Collapse
|
8
|
Zhang X, Liu L, Li Y, Wang K, Zheng G, Zhang Y, Cheng J, Wen B. Altered local spontaneous brain activity pattern in children with right-eye amblyopia of varying degrees: evidence from fMRI. Neuroradiology 2023; 65:1757-1766. [PMID: 37749259 DOI: 10.1007/s00234-023-03221-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2022] [Accepted: 09/06/2023] [Indexed: 09/27/2023]
Abstract
PURPOSE To investigate the abnormal changes of local brain activity in children with right-eye amblyopia of varying degrees. METHODS Data of resting-state functional magnetic resonance imaging were collected from 16 children with severe amblyopia, 17 children with mild to moderate amblyopia, and 15 children with normal binocular vision. Local brain activity was analyzed using the amplitude of low-frequency fluctuations (ALFF) and regional homogeneity (ReHo). RESULTS There were extensive ALFF differences among the three groups in 10 brain regions. There were extensive differences in ReHo among the three groups in 11 brain regions. The ALFF and ReHo of the right orbital part of the middle frontal gyrus displayed a significantly positive correlation with the best-corrected visual acuity of the right eye, respectively. The ALFF value and ReHo value of the right orbital part of the middle frontal gyrus followed the pattern of normal control < mild to moderate amblyopia < severe amblyopia. CONCLUSION This study demonstrated that there were changes in specific patterns of ALFF and ReHo in children with right-eye amblyopia of different degrees in brain regions performing visual sensorimotor and attentional control functions.
Collapse
Affiliation(s)
- Xiaopan Zhang
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, 450052, China
- Henan Key Laboratory of Magnetic Resonance and Brain Function, Zhengzhou, 450052, China
| | - Liang Liu
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, 450052, China
- Henan Key Laboratory of Magnetic Resonance and Brain Function, Zhengzhou, 450052, China
| | - Yadong Li
- Department of Ophthalmology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, 450052, China
| | - Kejia Wang
- Department of Ophthalmology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, 450052, China
| | - Guangying Zheng
- Department of Ophthalmology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, 450052, China
| | - Yong Zhang
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, 450052, China
- Henan Key Laboratory of Magnetic Resonance and Brain Function, Zhengzhou, 450052, China
| | - Jingliang Cheng
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, 450052, China
| | - Baohong Wen
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, 450052, China.
| |
Collapse
|
9
|
Yu M, Bai Y, Li Y. Emo-regulator: An emotion-regulation training system fusing virtual reality and EEG-based neurofeedback. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083545 DOI: 10.1109/embc40787.2023.10340975] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Good emotion-regulation ability (ERA) is a vital sign of psychological health; conversely, emotion dysregulation may lead to mental or neurological disorders, such as anxiety disorders and depression. This study developed an emotion-regulation training system, Emo-regulator, fusing virtual reality (VR) and EEG-based neurofeedback to enhance subjects' ability to down-regulate negative emotions. Emo-regulator first elicited negative emotions in subjects through VR scenarios and then asked them to regulate emotions using cognitive reappraisal to change the emotional responses elicited by the VRs. Meanwhile, EEG signals from the subjects were collected and analyzed in real time by machine learning to predict the emotional states of the subjects (negative or positive). Emo-regulator changed the VR scenarios according to the prediction results and completed the feedback. Eight subjects used Emo-regulator for two weeks, and the results showed it could help the subjects improve their emotion regulation, and its usability is above average.Clinical Relevance-Emo-regulator can help subjects improve their ability to down-regulate negative emotions and increase the frequency of cognitive reappraisal use during emotion regulation.
Collapse
|
10
|
Gao X, Zhang S, Liu K, Tan Z, Zhao G, Han Y, Cheng Y, Li C, Li F, Tian Y, Li P. An Adaptive Joint CCA-ICA Method for Ocular Artifact Removal and its Application to Emotion Classification. J Neurosci Methods 2023; 390:109841. [PMID: 36948359 DOI: 10.1016/j.jneumeth.2023.109841] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2023] [Revised: 03/10/2023] [Accepted: 03/19/2023] [Indexed: 03/24/2023]
Abstract
BACKGROUND The quality of Electroencephalogram (EEG) signals is critical for revealing the neural mechanism of emotions. However, ocular artifacts decreased the signal to noise ratio (SNR) and covered the inherent cognitive component of EEGs, which pose a great challenge in neuroscience research. NEW METHOD We proposed a novel unsupervised learning algorithm to adaptively remove the ocular artifacts by combining canonical correlation analysis (CCA), independent component analysis (ICA), higher-order statistics, empirical mode decomposition (EMD), and wavelet denoising techniques. Specifically, the combination of CCA and ICA aimed to improve the quality of source separation, while the higher-order statistics further located the source of ocular artifacts. Subsequently, these noised sources were further corrected by EMD and wavelet denoising to improve SNR of EEG signals. RESULTS We evaluated the performance of our proposed method with simulation studies and real EEG applications. The results of simulation study showed our proposed method could significantly improve the quality of signals under almost all noise conditions compared to four state-of-art methods. Consistently, the experiments of real EEG applications showed that the proposed methods could efficiently restrict the components of ocular artifacts and preserve the inherent information of cognition processing to improve the reliability of related analysis such as power spectral density (PSD) and emotion recognition. COMPARISON WITH EXISTING METHODS Our proposed model outperforms the comparative methods in EEG recovery, which further improve the application performance such as PSD analysis and emotion recognition. CONCLUSIONS The superior performance of our proposed method suggests that it is promising for removing ocular artifacts from EEG signals, which offers an efficient EEG preprocessing technology for the development of brain computer interface such as emotion recognition.
Collapse
Affiliation(s)
- Xiaohui Gao
- the School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongqing 400064, China; Institute for Advanced Sciences, Chongqing University of Posts and Communications
| | - Shilai Zhang
- the Clinical Hospital of Chengdu Brain Science In-stitute, MOE Key Lab for Neuroinformation and School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Ke Liu
- the Chongqing University of Posts and Telecommunications Chongqing Key Laboratory of Computational Intelligence Chongqing, 400065, China
| | - Ziqin Tan
- the School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongqing 400064, China; Institute for Advanced Sciences, Chongqing University of Posts and Communications
| | - Guanyi Zhao
- the School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongqing 400064, China; Institute for Advanced Sciences, Chongqing University of Posts and Communications
| | - Yumeng Han
- the School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongqing 400064, China; Institute for Advanced Sciences, Chongqing University of Posts and Communications
| | - Yue Cheng
- the Key Laboratory of Intelligent Analysis and Decision on Complex Systems, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China
| | - Cunbo Li
- the Clinical Hospital of Chengdu Brain Science In-stitute, MOE Key Lab for Neuroinformation and School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Fali Li
- the Clinical Hospital of Chengdu Brain Science In-stitute, MOE Key Lab for Neuroinformation and School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China.
| | - Yin Tian
- the School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongqing 400064, China; Institute for Advanced Sciences, Chongqing University of Posts and Communications.
| | - Peiyang Li
- the School of Bioinformatics, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongqing 400064, China; Institute for Advanced Sciences, Chongqing University of Posts and Communications.
| |
Collapse
|
11
|
Gao X, Huang W, Liu Y, Zhang Y, Zhang J, Li C, Chelangat Bore J, Wang Z, Si Y, Tian Y, Li P. A novel robust Student’s t-based Granger causality for EEG based brain network analysis. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
12
|
Zhang Y, Yan G, Chang W, Huang W, Yuan Y. EEG-based multi-frequency band functional connectivity analysis and the application of spatio-temporal features in emotion recognition. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
13
|
Recognising situation awareness associated with different workloads using EEG and eye-tracking features in air traffic control tasks. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.110179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|
14
|
Xie Z, Pan J, Li S, Ren J, Qian S, Ye Y, Bao W. Musical Emotions Recognition Using Entropy Features and Channel Optimization Based on EEG. ENTROPY (BASEL, SWITZERLAND) 2022; 24:1735. [PMID: 36554139 PMCID: PMC9777832 DOI: 10.3390/e24121735] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/08/2022] [Revised: 11/15/2022] [Accepted: 11/22/2022] [Indexed: 06/17/2023]
Abstract
The dynamic of music is an important factor to arouse emotional experience, but current research mainly uses short-term artificial stimulus materials, which cannot effectively awaken complex emotions and reflect their dynamic brain response. In this paper, we used three long-term stimulus materials with many dynamic emotions inside: the "Waltz No. 2" containing pleasure and excitement, the "No. 14 Couplets" containing excitement, briskness, and nervousness, and the first movement of "Symphony No. 5 in C minor" containing passion, relaxation, cheerfulness, and nervousness. Approximate entropy (ApEn) and sample entropy (SampEn) were applied to extract the non-linear features of electroencephalogram (EEG) signals under long-term dynamic stimulation, and the K-Nearest Neighbor (KNN) method was used to recognize emotions. Further, a supervised feature vector dimensionality reduction method was proposed. Firstly, the optimal channel set for each subject was obtained by using a particle swarm optimization (PSO) algorithm, and then the number of times to select each channel in the optimal channel set of all subjects was counted. If the number was greater than or equal to the threshold, it was a common channel suitable for all subjects. The recognition results based on the optimal channel set demonstrated that each accuracy of two categories of emotions based on "Waltz No. 2" and three categories of emotions based on "No. 14 Couplets" was generally above 80%, respectively, and the recognition accuracy of four categories based on the first movement of "Symphony No. 5 in C minor" was about 70%. The recognition accuracy based on the common channel set was about 10% lower than that based on the optimal channel set, but not much different from that based on the whole channel set. This result suggested that the common channel could basically reflect the universal features of the whole subjects while realizing feature dimension reduction. The common channels were mainly distributed in the frontal lobe, central region, parietal lobe, occipital lobe, and temporal lobe. The channel number distributed in the frontal lobe was greater than the ones in other regions, indicating that the frontal lobe was the main emotional response region. Brain region topographic map based on the common channel set showed that there were differences in entropy intensity between different brain regions of the same emotion and the same brain region of different emotions. The number of times to select each channel in the optimal channel set of all 30 subjects showed that the principal component channels representing five brain regions were Fp1/F3 in the frontal lobe, CP5 in the central region, Pz in the parietal lobe, O2 in the occipital lobe, and T8 in the temporal lobe, respectively.
Collapse
Affiliation(s)
- Zun Xie
- Department of Arts and Design, Anhui University of Technology, Ma’anshan 243002, China
| | - Jianwei Pan
- Department of Arts and Design, Anhui University of Technology, Ma’anshan 243002, China
| | - Songjie Li
- Department of Management Science and Engineering, Anhui University of Technology, Ma’anshan 243002, China
| | - Jing Ren
- Department of Management Science and Engineering, Anhui University of Technology, Ma’anshan 243002, China
| | - Shao Qian
- Department of Management Science and Engineering, Anhui University of Technology, Ma’anshan 243002, China
| | - Ye Ye
- Department of Mechanical Engineering, Anhui University of Technology, Ma’anshan 243002, China
| | - Wei Bao
- Department of Management Science and Engineering, Anhui University of Technology, Ma’anshan 243002, China
| |
Collapse
|
15
|
Use of Differential Entropy for Automated Emotion Recognition in a Virtual Reality Environment with EEG Signals. Diagnostics (Basel) 2022; 12:diagnostics12102508. [PMID: 36292197 PMCID: PMC9601226 DOI: 10.3390/diagnostics12102508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 10/13/2022] [Accepted: 10/14/2022] [Indexed: 11/20/2022] Open
Abstract
Emotion recognition is one of the most important issues in human–computer interaction (HCI), neuroscience, and psychology fields. It is generally accepted that emotion recognition with neural data such as electroencephalography (EEG) signals, functional magnetic resonance imaging (fMRI), and near-infrared spectroscopy (NIRS) is better than other emotion detection methods such as speech, mimics, body language, facial expressions, etc., in terms of reliability and accuracy. In particular, EEG signals are bioelectrical signals that are frequently used because of the many advantages they offer in the field of emotion recognition. This study proposes an improved approach for EEG-based emotion recognition on a publicly available newly published dataset, VREED. Differential entropy (DE) features were extracted from four wavebands (theta 4–8 Hz, alpha 8–13 Hz, beta 13–30 Hz, and gamma 30–49 Hz) to classify two emotional states (positive/negative). Five classifiers, namely Support Vector Machine (SVM), k-Nearest Neighbor (kNN), Naïve Bayesian (NB), Decision Tree (DT), and Logistic Regression (LR) were employed with DE features for the automated classification of two emotional states. In this work, we obtained the best average accuracy of 76.22% ± 2.06 with the SVM classifier in the classification of two states. Moreover, we observed from the results that the highest average accuracy score was produced with the gamma band, as previously reported in studies in EEG-based emotion recognition.
Collapse
|
16
|
Daşdemir Y. Cognitive investigation on the effect of augmented reality-based reading on emotion classification performance: A new dataset. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103942] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
17
|
EEG-Based Identification of Emotional Neural State Evoked by Virtual Environment Interaction. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19042158. [PMID: 35206341 PMCID: PMC8872045 DOI: 10.3390/ijerph19042158] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Revised: 02/11/2022] [Accepted: 02/11/2022] [Indexed: 11/17/2022]
Abstract
Classifying emotional states is critical for brain–computer interfaces and psychology-related domains. In previous studies, researchers have tried to identify emotions using neural data such as electroencephalography (EEG) signals or brain functional magnetic resonance imaging (fMRI). In this study, we propose a machine learning framework for emotion state classification using EEG signals in virtual reality (VR) environments. To arouse emotional neural states in brain signals, we provided three VR stimuli scenarios to 15 participants. Fifty-four features were extracted from the collected EEG signals under each scenario. To find the optimal classification in our research design, three machine learning algorithms (XGBoost classifier, support vector classifier, and logistic regression) were applied. Additionally, various class conditions were used in machine learning classifiers to validate the performance of our framework. To evaluate the classification performance, we utilized five evaluation metrics (precision, recall, f1-score, accuracy, and AUROC). Among the three classifiers, the XGBoost classifiers showed the best performance under all experimental conditions. Furthermore, the usability of features, including differential asymmetry and frequency band pass categories, were checked from the feature importance of XGBoost classifiers. We expect that our framework can be applied widely not only to psychological research but also to mental health-related issues.
Collapse
|