1
|
Farhadi Sedehi J, Jafarnia Dabanloo N, Maghooli K, Sheikhani A. Multimodal insights into granger causality connectivity: Integrating physiological signals and gated eye-tracking data for emotion recognition using convolutional neural network. Heliyon 2024; 10:e36411. [PMID: 39253213 PMCID: PMC11381760 DOI: 10.1016/j.heliyon.2024.e36411] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2024] [Revised: 08/14/2024] [Accepted: 08/14/2024] [Indexed: 09/11/2024] Open
Abstract
This study introduces a groundbreaking method to enhance the accuracy and reliability of emotion recognition systems by combining electrocardiogram (ECG) with electroencephalogram (EEG) data, using an eye-tracking gated strategy. Initially, we propose a technique to filter out irrelevant portions of emotional data by employing pupil diameter metrics from eye-tracking data. Subsequently, we introduce an innovative approach for estimating effective connectivity to capture the dynamic interaction between the brain and the heart during emotional states of happiness and sadness. Granger causality (GC) is estimated and utilized to optimize input for a highly effective pre-trained convolutional neural network (CNN), specifically ResNet-18. To assess this methodology, we employed EEG and ECG data from the publicly available MAHNOB-HCI database, using a 5-fold cross-validation approach. Our method achieved an impressive average accuracy and area under the curve (AUC) of 91.00 % and 0.97, respectively, for GC-EEG-ECG images processed with ResNet-18. Comparative analysis with state-of-the-art studies clearly shows that augmenting ECG with EEG and refining data with an eye-tracking strategy significantly enhances emotion recognition performance across various emotions.
Collapse
Affiliation(s)
- Javid Farhadi Sedehi
- Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Nader Jafarnia Dabanloo
- Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Keivan Maghooli
- Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Ali Sheikhani
- Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| |
Collapse
|
2
|
Patel P, Balasubramanian S, Annavarapu RN. Cross subject emotion identification from multichannel EEG sub-bands using Tsallis entropy feature and KNN classifier. Brain Inform 2024; 11:7. [PMID: 38441825 PMCID: PMC11358557 DOI: 10.1186/s40708-024-00220-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Accepted: 02/05/2024] [Indexed: 08/29/2024] Open
Abstract
Human emotion recognition remains a challenging and prominent issue, situated at the convergence of diverse fields, such as brain-computer interfaces, neuroscience, and psychology. This study utilizes an EEG data set for investigating human emotion, presenting novel findings and a refined approach for EEG-based emotion detection. Tsallis entropy features, computed for q values of 2, 3, and 4, are extracted from signal bands, including theta-θ (4-7 Hz), alpha-α (8-15 Hz), beta-β (16-31 Hz), gamma-γ (32-55 Hz), and the overall frequency range (0-75 Hz). These Tsallis entropy features are employed to train and test a KNN classifier, aiming for accurate identification of two emotional states: positive and negative. In this study, the best average accuracy of 79% and an F-score of 0.81 were achieved in the gamma frequency range for the Tsallis parameter q = 3. In addition, the highest accuracy and F-score of 84% and 0.87 were observed. Notably, superior performance was noted in the anterior and left hemispheres compared to the posterior and right hemispheres in the context of emotion studies. The findings show that the proposed method exhibits enhanced performance, making it a highly competitive alternative to existing techniques. Furthermore, we identify and discuss the shortcomings of the proposed approach, offering valuable insights into potential avenues for improvements.
Collapse
Affiliation(s)
- Pragati Patel
- Department of Physics, Pondicherry University, Puducherry, 605014, India
| | | | | |
Collapse
|
3
|
Rowe EG, Garrido MI, Tsuchiya N. Feedforward connectivity patterns from visual areas to the front of the brain contain information about sensory stimuli regardless of awareness or report. Cortex 2024; 172:284-300. [PMID: 38142179 DOI: 10.1016/j.cortex.2023.11.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 10/11/2023] [Accepted: 11/21/2023] [Indexed: 12/25/2023]
Abstract
Current theories of consciousness can be categorized to some extent by their predictions about the putative role of the prefrontal cortex (PFC) in conscious perception. One family of the theories proposes that the PFC is necessary for conscious perception. The other postulates that the PFC is not necessary and that other areas (e.g., posterior cortical areas) are more important for conscious perception. No-report paradigms could potentially arbitrate the debate as they disentangle task reporting from conscious perception. While previous no-report paradigms tend to point to a reduction in PFC activity, they have not examined the critical role of the PFC in "monitoring" or "reading out" the patterns of activity in the sensory cortex to generate conscious perception. To address this, we reanalysed electroencephalography (EEG) data from a no-report inattentional blindness paradigm (Shafto & Pitts, 2015). We examined the role of feedforward input patterns to the PFC from sensory cortices. We employed nonparametric spectral Granger causality and quantified the amount of information that reflected the contents of consciousness using multivariate classifiers. Unexpectedly, regardless of whether the stimulus was consciously seen or not, we found that information relating to the current sensory stimulus was present in the pattern of inputs from visual areas to the PFC. In light of these findings, we suggest various theories of consciousness need to be revised to accommodate the fact that the contents of consciousness are decodable from the input patterns from posterior sensory regions to the PFC, regardless of awareness (or report).
Collapse
Affiliation(s)
- Elise G Rowe
- School of Psychological Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Clayton, Victoria, Australia; Turner Institute for Brain and Mental Health, Monash University, Clayton, Victoria, Australia.
| | - Marta I Garrido
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Victoria, Australia; ARC Centre of Excellence for Integrative Brain Function, Victoria, Australia
| | - Naotsugu Tsuchiya
- School of Psychological Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Clayton, Victoria, Australia; Turner Institute for Brain and Mental Health, Monash University, Clayton, Victoria, Australia; Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Suita, Osaka, Japan; Department of Qualia Structure, ATR Computational Neuroscience Laboratories, Seika-cho, Soraku-gun, Kyoto, Japan; ARC Centre of Excellence for Integrative Brain Function, Victoria, Australia
| |
Collapse
|
4
|
Zhang H, Zhou QQ, Chen H, Hu XQ, Li WG, Bai Y, Han JX, Wang Y, Liang ZH, Chen D, Cong FY, Yan JQ, Li XL. The applied principles of EEG analysis methods in neuroscience and clinical neurology. Mil Med Res 2023; 10:67. [PMID: 38115158 PMCID: PMC10729551 DOI: 10.1186/s40779-023-00502-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/25/2023] [Accepted: 11/23/2023] [Indexed: 12/21/2023] Open
Abstract
Electroencephalography (EEG) is a non-invasive measurement method for brain activity. Due to its safety, high resolution, and hypersensitivity to dynamic changes in brain neural signals, EEG has aroused much interest in scientific research and medical fields. This article reviews the types of EEG signals, multiple EEG signal analysis methods, and the application of relevant methods in the neuroscience field and for diagnosing neurological diseases. First, three types of EEG signals, including time-invariant EEG, accurate event-related EEG, and random event-related EEG, are introduced. Second, five main directions for the methods of EEG analysis, including power spectrum analysis, time-frequency analysis, connectivity analysis, source localization methods, and machine learning methods, are described in the main section, along with different sub-methods and effect evaluations for solving the same problem. Finally, the application scenarios of different EEG analysis methods are emphasized, and the advantages and disadvantages of similar methods are distinguished. This article is expected to assist researchers in selecting suitable EEG analysis methods based on their research objectives, provide references for subsequent research, and summarize current issues and prospects for the future.
Collapse
Affiliation(s)
- Hao Zhang
- School of Systems Science, Beijing Normal University, Beijing, 100875, China
| | - Qing-Qi Zhou
- College of Electrical and Control Engineering, North China University of Technology, Beijing, 100041, China
| | - He Chen
- School of Automation Science and Engineering, South China University of Technology, Guangzhou, 510641, China
| | - Xiao-Qing Hu
- Department of Psychology, the State Key Laboratory of Brain and Cognitive Sciences, The University of Hong Kong, Hong Kong SAR, 999077, China
- HKU-Shenzhen Institute of Research and Innovation, Shenzhen, 518057, Guangdong, China
| | - Wei-Guang Li
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong SAR, 999077, China
| | - Yang Bai
- Department of Rehabilitation Medicine, the First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
- Rehabilitation Medicine Clinical Research Center of Jiangxi Province, Nanchang, 330006, China
| | - Jun-Xia Han
- Beijing Key Laboratory of Learning and Cognition, School of Psychology, Capital Normal University, Beijing, 100048, China
| | - Yao Wang
- School of Communication Science, Beijing Language and Culture University, Beijing, 100083, China
| | - Zhen-Hu Liang
- Institute of Electrical Engineering, Yanshan University, Qinhuangdao, 066004, Hebei, China.
| | - Dan Chen
- School of Computer Science, Wuhan University, Wuhan, 430072, China.
| | - Feng-Yu Cong
- School of Biomedical Engineering, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, 116081, Liaoning, China.
| | - Jia-Qing Yan
- College of Electrical and Control Engineering, North China University of Technology, Beijing, 100041, China.
| | - Xiao-Li Li
- School of Automation Science and Engineering, South China University of Technology, Guangzhou, 510641, China.
- Guangdong Artificial Intelligence and Digital Economy Laboratory (Guangzhou), Guangzhou, 510335, China.
| |
Collapse
|
5
|
Akhand MAH, Maria MA, Kamal MAS, Murase K. Improved EEG-based emotion recognition through information enhancement in connectivity feature map. Sci Rep 2023; 13:13804. [PMID: 37612354 PMCID: PMC10447430 DOI: 10.1038/s41598-023-40786-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Accepted: 08/16/2023] [Indexed: 08/25/2023] Open
Abstract
Electroencephalography (EEG), despite its inherited complexity, is a preferable brain signal for automatic human emotion recognition (ER), which is a challenging machine learning task with emerging applications. In any automatic ER, machine learning (ML) models classify emotions using the extracted features from the EEG signals, and therefore, such feature extraction is a crucial part of ER process. Recently, EEG channel connectivity features have been widely used in ER, where Pearson correlation coefficient (PCC), mutual information (MI), phase-locking value (PLV), and transfer entropy (TE) are well-known methods for connectivity feature map (CFM) construction. CFMs are typically formed in a two-dimensional configuration using the signals from two EEG channels, and such two-dimensional CFMs are usually symmetric and hold redundant information. This study proposes the construction of a more informative CFM that can lead to better ER. Specifically, the proposed innovative technique intelligently combines CFMs' measures of two different individual methods, and its outcomes are more informative as a fused CFM. Such CFM fusion does not incur additional computational costs in training the ML model. In this study, fused CFMs are constructed by combining every pair of methods from PCC, PLV, MI, and TE; and the resulting fused CFMs PCC + PLV, PCC + MI, PCC + TE, PLV + MI, PLV + TE, and MI + TE are used to classify emotion by convolutional neural network. Rigorous experiments on the DEAP benchmark EEG dataset show that the proposed CFMs deliver better ER performances than CFM with a single connectivity method (e.g., PCC). At a glance, PLV + MI-based ER is shown to be the most promising one as it outperforms the other methods.
Collapse
Affiliation(s)
- M A H Akhand
- Department of Computer Science and Engineering, Khulna University of Engineering & Technology, Khulna, 9203, Bangladesh.
| | - Mahfuza Akter Maria
- Department of Computer Science and Engineering, Khulna University of Engineering & Technology, Khulna, 9203, Bangladesh
| | - Md Abdus Samad Kamal
- Graduate School of Science and Technology, Gunma University, Kiryu, 376-8515, Japan
| | - Kazuyuki Murase
- Department of Information Technology, International Professional University of Technology in Osaka, 3-3-1 Umeda, Kita-ku, Osaka, 530-0001, Japan
| |
Collapse
|
6
|
Alskafi FA, Khandoker AH, Marzbanrad F, Jelinek HF. EEG-based Emotion Recognition Using Sub-Band Time-Delay Correlations. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083727 DOI: 10.1109/embc40787.2023.10340014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Emotion recognition is a challenging task with many potential applications in psychology, psychiatry, and human-computer interaction (HCI). The use of time-delay information in the controlled time-delay stability (cTDS) algorithm can help to capture the temporal dynamics of EEG signals, including sub-band information and bi-directional coupling that can aid in emotion recognition and identification of specific connectivity patterns between brain rhythms. Incorporating EEG frequency bands can be used to design better emotion recognition systems. This paper evaluates the cTDS algorithm for binary classification tasks of arousal and valence using EEG sub-band signals. This method achieved a high accuracy of 91.1% for arousal and 91.7% for valence based on one electrode recording site at Fp1. The cTDS algorithm is a promising approach to analyzing brain network interactions. It can be particularly applicable to arousal and valence classification tasks, especially within a complex, multimodal feature space associated with understanding psychiatric disorders and HCI applications.
Collapse
|
7
|
Vempati R, Sharma LD. EEG rhythm based emotion recognition using multivariate decomposition and ensemble machine learning classifier. J Neurosci Methods 2023; 393:109879. [PMID: 37182604 DOI: 10.1016/j.jneumeth.2023.109879] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 04/26/2023] [Accepted: 05/08/2023] [Indexed: 05/16/2023]
Abstract
Recently, electroencephalogram (EEG) signals have shown great potential to recognize human emotions. The goal of effective computing is to assist computers in understanding various types of emotions via human-computer interaction (HCI). Multichannel EEG signals are used to measure the electrical activity of the brain in space and time. Automated emotion recognition using multichannel EEG signals is an interesting area of cognitive neuroscience and affective computing research. This research proposes EEG multichannel rhythmic features and ensemble machine learning (EML) classifiers with leave-one-subject-out cross-validation (LOSOCV) for automatic emotion classification from multichannel EEG recordings. Multivariate fast iterative filtering (MvFIF) is used to assess the EEG rhythm sequences. EEG rhythms delta(δ), theta(θ), alpha(α), beta(β), and gamma(γ) are separated based on the mean frequency of the EEG rhythm sequence. Three Hjorth parameters and nine entropy features were extracted from multichannel EEG rhythms. Extracted features are selected using the minimum redundancy maximum relevance (mRMR) approach. The experimental design was performed on two emotional datasets (GAMEEMO and DREAMER). The validation showed that gamma rhythm multichannel features with EML-based subspace K-nearest neighbor (SS KNN) were as high as 93.5%-99.8%, achieving high classification accuracy. The comparisons of δ, θ, α, β, and γ rhythms with EML, support vector machine (SVM), and artificial neural network (ANN) were performed. we also analyzed multi-class emotions (HVHA, HVLA, LVHA, LVLA) with an ensemble-based bagging tree on gamma rhythm. It provides a novel solution for multichannel rhythm-specific features in EEG data analysis.
Collapse
Affiliation(s)
| | - Lakhan Dev Sharma
- School of Electronics Engineering VIT-AP University, Andhra Pradesh, 522237, India.
| |
Collapse
|
8
|
Lam WS, Lam WH, Jaaman SH, Lee PF. Bibliometric Analysis of Granger Causality Studies. ENTROPY (BASEL, SWITZERLAND) 2023; 25:e25040632. [PMID: 37190420 PMCID: PMC10137504 DOI: 10.3390/e25040632] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/25/2023] [Revised: 03/26/2023] [Accepted: 04/03/2023] [Indexed: 05/17/2023]
Abstract
Granger causality provides a framework that uses predictability to identify causation between time series variables. This is important to policymakers for effective policy management and recommendations. Granger causality is recognized as the primary advance on the causation problem. The objective of this paper is to conduct a bibliometric analysis of Granger causality publications indexed in the Web of Science database. Harzing's Publish or Perish and VOSviewer were used for performance analysis and science mapping. The first paper indexed was published in 1981 and there has been an upward trend in the annual publication of Granger causality studies which are shifting towards the areas of environmental science, energy, and economics. Most of the publications are articles and proceeding papers under the areas of business economics, environmental science ecology, and neurosciences/neurology. China has the highest number of publications while the United States has the highest number of citations. England has the highest citation impact. This paper also constructed country co-authorship, co-analysis of cited references, cited sources, and cited authors, keyword co-occurrence, and keyword overlay visualization maps.
Collapse
Affiliation(s)
- Weng Siew Lam
- Department of Physical and Mathematical Science, Faculty of Science, Kampar Campus, Universiti Tunku Abdul Rahman, Jalan Universiti, Bandar Barat, Kampar 31900, Perak, Malaysia
| | - Weng Hoe Lam
- Department of Physical and Mathematical Science, Faculty of Science, Kampar Campus, Universiti Tunku Abdul Rahman, Jalan Universiti, Bandar Barat, Kampar 31900, Perak, Malaysia
| | - Saiful Hafizah Jaaman
- Department of Mathematical Sciences, Faculty of Science and Technology, Universiti Kebangsaan Malaysia (UKM), Bangi 43600, Selangor, Malaysia
| | - Pei Fun Lee
- Department of Physical and Mathematical Science, Faculty of Science, Kampar Campus, Universiti Tunku Abdul Rahman, Jalan Universiti, Bandar Barat, Kampar 31900, Perak, Malaysia
| |
Collapse
|
9
|
Cui D, Xuan H, Liu J, Gu G, Li X. Emotion Recognition on EEG Signal Using ResNeXt Attention 2D-3D Convolution Neural Networks. Neural Process Lett 2022. [DOI: 10.1007/s11063-022-11120-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
10
|
Zhang J, Zhang X, Chen G, Zhao Q. Granger-Causality-Based Multi-Frequency Band EEG Graph Feature Extraction and Fusion for Emotion Recognition. Brain Sci 2022; 12:1649. [PMID: 36552109 PMCID: PMC9776073 DOI: 10.3390/brainsci12121649] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 11/08/2022] [Accepted: 11/25/2022] [Indexed: 12/05/2022] Open
Abstract
Graph convolutional neural networks (GCN) have attracted much attention in the task of electroencephalogram (EEG) emotion recognition. However, most features of current GCNs do not take full advantage of the causal connection between the EEG signals in different frequency bands during the process of constructing the adjacency matrix. Based on the causal connectivity between the EEG channels obtained by Granger causality (GC) analysis, this paper proposes a multi-frequency band EEG graph feature extraction and fusion method for EEG emotion recognition. First, the original GC matrices between the EEG signals at each frequency band are calculated via GC analysis, and then they are adaptively converted to asymmetric binary GC matrices through an optimal threshold. Then, a kind of novel GC-based GCN feature (GC-GCN) is constructed by using differential entropy features and the binary GC matrices as the node values and adjacency matrices, respectively. Finally, on the basis of the GC-GCN features, a new multi-frequency band feature fusion method (GC-F-GCN) is proposed, which integrates the graph information of the EEG signals at different frequency bands for the same node. The experimental results demonstrate that the proposed GC-F-GCN method achieves better recognition performance than the state-of-the-art GCN methods, for which average accuracies of 97.91%, 98.46%, and 98.15% were achieved for the arousal, valence, and arousal-valence classifications, respectively.
Collapse
Affiliation(s)
| | - Xueying Zhang
- College of Information and Computer, Taiyuan University of Technology, Taiyuan 030024, China
| | | | | |
Collapse
|
11
|
Tang C, Gao T, Li Y, Chen B. EEG channel selection based on sequential backward floating search for motor imagery classification. Front Neurosci 2022; 16:1045851. [PMID: 36340754 PMCID: PMC9633952 DOI: 10.3389/fnins.2022.1045851] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 10/05/2022] [Indexed: 11/13/2022] Open
Abstract
Brain-computer interfaces (BCIs) based on motor imagery (MI) utilizing multi-channel electroencephalogram (EEG) data are commonly used to improve motor function of people with motor disabilities. EEG channel selection can enhance MI classification accuracy by selecting informative channels, accordingly reducing redundant information. The sequential backward floating search (SBFS) approach has been considered as one of the best feature selection methods. In this paper, SBFS is first implemented to select the optimal EEG channels in MI-BCI. Further, to reduce the time complexity of SBFS, the modified SBFS is proposed and applied to left and right hand MI tasks. In the modified SBFS, based on the map of EEG channels at the scalp, the symmetrical channels are selected as channel pairs and acceleration is thus realized by removing or adding multiple channels in each iteration. Extensive experiments were conducted on four public BCI datasets. Experimental results show that the SBFS achieves significantly higher classification accuracy (p < 0.001) than using all channels and conventional MI channels (i.e., C3, C4, and Cz). Moreover, the proposed method outperforms the state-of-the-art selection methods.
Collapse
Affiliation(s)
- Chao Tang
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
| | - Tianyi Gao
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
| | - Yuanhao Li
- Institute of Innovative Research, Tokyo Institute of Technology, Yokohama, Japan
| | - Badong Chen
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
- *Correspondence: Badong Chen
| |
Collapse
|
12
|
Soleimani B, Das P, Dushyanthi Karunathilake IM, Kuchinsky SE, Simon JZ, Babadi B. NLGC: Network localized Granger causality with application to MEG directional functional connectivity analysis. Neuroimage 2022; 260:119496. [PMID: 35870697 PMCID: PMC9435442 DOI: 10.1016/j.neuroimage.2022.119496] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Revised: 06/21/2022] [Accepted: 07/19/2022] [Indexed: 11/25/2022] Open
Abstract
Identifying the directed connectivity that underlie networked activity between different cortical areas is critical for understanding the neural mechanisms behind sensory processing. Granger causality (GC) is widely used for this purpose in functional magnetic resonance imaging analysis, but there the temporal resolution is low, making it difficult to capture the millisecond-scale interactions underlying sensory processing. Magnetoencephalography (MEG) has millisecond resolution, but only provides low-dimensional sensor-level linear mixtures of neural sources, which makes GC inference challenging. Conventional methods proceed in two stages: First, cortical sources are estimated from MEG using a source localization technique, followed by GC inference among the estimated sources. However, the spatiotemporal biases in estimating sources propagate into the subsequent GC analysis stage, may result in both false alarms and missing true GC links. Here, we introduce the Network Localized Granger Causality (NLGC) inference paradigm, which models the source dynamics as latent sparse multivariate autoregressive processes and estimates their parameters directly from the MEG measurements, integrated with source localization, and employs the resulting parameter estimates to produce a precise statistical characterization of the detected GC links. We offer several theoretical and algorithmic innovations within NLGC and further examine its utility via comprehensive simulations and application to MEG data from an auditory task involving tone processing from both younger and older participants. Our simulation studies reveal that NLGC is markedly robust with respect to model mismatch, network size, and low signal-to-noise ratio, whereas the conventional two-stage methods result in high false alarms and mis-detections. We also demonstrate the advantages of NLGC in revealing the cortical network-level characterization of neural activity during tone processing and resting state by delineating task- and age-related connectivity changes.
Collapse
Affiliation(s)
- Behrad Soleimani
- Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA; Institute for Systems Research, University of Maryland, College Park, MD, USA.
| | - Proloy Das
- Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, MA, USA.
| | - I M Dushyanthi Karunathilake
- Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA; Institute for Systems Research, University of Maryland, College Park, MD, USA.
| | - Stefanie E Kuchinsky
- Audiology and Speech Pathology Center, Walter Reed National Military Medical Center, Bethesda, MD, USA.
| | - Jonathan Z Simon
- Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA; Institute for Systems Research, University of Maryland, College Park, MD, USA; Department of Biology, University of Maryland College Park, MD, USA.
| | - Behtash Babadi
- Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA; Institute for Systems Research, University of Maryland, College Park, MD, USA.
| |
Collapse
|
13
|
Abdulrahman A, Baykara M, Alakus TB. A Novel Approach for Emotion Recognition Based on EEG Signal Using Deep Learning. APPLIED SCIENCES 2022; 12:10028. [DOI: 10.3390/app121910028] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
Abstract
Emotion can be defined as a voluntary or involuntary reaction to external factors. People express their emotions through actions, such as words, sounds, facial expressions, and body language. However, emotions expressed in such actions are sometimes manipulated by people and real feelings cannot be conveyed clearly. Therefore, understanding and analyzing emotions is essential. Recently, emotion analysis studies based on EEG signals appear to be in the foreground, due to the more reliable data collected. In this study, emotion analysis based on EEG signals was performed and a deep learning model was proposed. The study consists of four stages. In the first stage, EEG data were obtained from the GAMEEMO dataset. In the second stage, EEG signals were transformed with both VMD (variation mode decomposition) and EMD (empirical mode decomposition), and a total of 14 (nine from EMD, five from VMD) IMFs were obtained from each signal. In the third stage, statistical features were obtained from IMFs and maximum value, minimum value, and average values were used for this. In the last stage, both binary-class and multi-class classifications were made. The proposed deep learning model is compared with kNN (k nearest neighbor), SVM (support vector machines), and RF (random forest). At the end of the study, an accuracy of 70.89% in binary-class classification and 90.33% in multi-class classification was obtained with the proposed deep learning method.
Collapse
|
14
|
Zhang J, Zhang X, Chen G, Huang L, Sun Y. EEG emotion recognition based on cross-frequency granger causality feature extraction and fusion in the left and right hemispheres. Front Neurosci 2022; 16:974673. [PMID: 36161187 PMCID: PMC9491730 DOI: 10.3389/fnins.2022.974673] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 08/17/2022] [Indexed: 11/13/2022] Open
Abstract
EEG emotion recognition based on Granger causality (GC) brain networks mainly focus on the EEG signal from the same-frequency bands, however, there are still some causality relationships between EEG signals in the cross-frequency bands. Considering the functional asymmetric of the left and right hemispheres to emotional response, this paper proposes an EEG emotion recognition scheme based on cross-frequency GC feature extraction and fusion in the left and right hemispheres. Firstly, we calculate the GC relationship of EEG signals according to the frequencies and hemispheres, and mainly focus on the causality of the cross-frequency EEG signals in left and right hemispheres. Then, to remove the redundant connections of the GC brain network, an adaptive two-stage decorrelation feature extraction scheme is proposed under the condition of maintaining the best emotion recognition performance. Finally, a multi-GC feature fusion scheme is designed to balance the recognition accuracy and feature number of each GC feature, which comprehensively considers the influence of the recognition accuracy and computational complexity. Experimental results on the DEAP emotion dataset show that the proposed scheme can achieve an average accuracy of 84.91% for four classifications, which improved the classification accuracy by up to 8.43% compared with that of the traditional same-frequency band GC features.
Collapse
|
15
|
Li R, Yang D, Fang F, Hong KS, Reiss AL, Zhang Y. Concurrent fNIRS and EEG for Brain Function Investigation: A Systematic, Methodology-Focused Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22155865. [PMID: 35957421 PMCID: PMC9371171 DOI: 10.3390/s22155865] [Citation(s) in RCA: 31] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Revised: 07/27/2022] [Accepted: 07/30/2022] [Indexed: 05/29/2023]
Abstract
Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) stand as state-of-the-art techniques for non-invasive functional neuroimaging. On a unimodal basis, EEG has poor spatial resolution while presenting high temporal resolution. In contrast, fNIRS offers better spatial resolution, though it is constrained by its poor temporal resolution. One important merit shared by the EEG and fNIRS is that both modalities have favorable portability and could be integrated into a compatible experimental setup, providing a compelling ground for the development of a multimodal fNIRS-EEG integration analysis approach. Despite a growing number of studies using concurrent fNIRS-EEG designs reported in recent years, the methodological reference of past studies remains unclear. To fill this knowledge gap, this review critically summarizes the status of analysis methods currently used in concurrent fNIRS-EEG studies, providing an up-to-date overview and guideline for future projects to conduct concurrent fNIRS-EEG studies. A literature search was conducted using PubMed and Web of Science through 31 August 2021. After screening and qualification assessment, 92 studies involving concurrent fNIRS-EEG data recordings and analyses were included in the final methodological review. Specifically, three methodological categories of concurrent fNIRS-EEG data analyses, including EEG-informed fNIRS analyses, fNIRS-informed EEG analyses, and parallel fNIRS-EEG analyses, were identified and explained with detailed description. Finally, we highlighted current challenges and potential directions in concurrent fNIRS-EEG data analyses in future research.
Collapse
Affiliation(s)
- Rihui Li
- Center for Interdisciplinary Brain Sciences Research, Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94305, USA
- Department of Biomedical Engineering, University of Houston, Houston, TX 77004, USA
| | - Dalin Yang
- School of Mechanical Engineering, Pusan National University, Pusan 43241, Korea
- Mallinckrodt Institute of Radiology, Washington University School of Medicine in St. Louis, 4515 McKinley Avenue, St. Louis, MO 63110, USA
| | - Feng Fang
- Department of Biomedical Engineering, University of Houston, Houston, TX 77004, USA
| | - Keum-Shik Hong
- School of Mechanical Engineering, Pusan National University, Pusan 43241, Korea
| | - Allan L. Reiss
- Center for Interdisciplinary Brain Sciences Research, Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Yingchun Zhang
- Department of Biomedical Engineering, University of Houston, Houston, TX 77004, USA
| |
Collapse
|
16
|
Bagherzadeh S, Shahabi MS, Shalbaf A. Detection of schizophrenia using hybrid of deep learning and brain effective connectivity image from electroencephalogram signal. Comput Biol Med 2022; 146:105570. [DOI: 10.1016/j.compbiomed.2022.105570] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2022] [Revised: 04/14/2022] [Accepted: 04/25/2022] [Indexed: 02/06/2023]
|
17
|
García-Martínez B, Fernández-Caballero A, Martínez-Rodrigo A, Alcaraz R, Novais P. Evaluation of Brain Functional Connectivity from Electroencephalographic Signals Under Different Emotional States. Int J Neural Syst 2022; 32:2250026. [PMID: 35469551 DOI: 10.1142/s0129065722500265] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The identification of the emotional states corresponding to the four quadrants of the valence/arousal space has been widely analyzed in the scientific literature by means of multiple techniques. Nevertheless, most of these methods were based on the assessment of each brain region separately, without considering the possible interactions among different areas. In order to study these interconnections, this study computes for the first time the functional connectivity metric called cross-sample entropy for the analysis of the brain synchronization in four groups of emotions from electroencephalographic signals. Outcomes reported a strong synchronization in the interconnections among central, parietal and occipital areas, while the interactions between left frontal and temporal structures with the rest of brain regions presented the lowest coordination. These differences were statistically significant for the four groups of emotions. All emotions were simultaneously classified with a 95.43% of accuracy, overcoming the results reported in previous studies. Moreover, the differences between high and low levels of valence and arousal, taking into account the state of the counterpart dimension, also provided notable findings about the degree of synchronization in the brain within different emotional conditions and the possible implications of these outcomes from a psychophysiological point of view.
Collapse
Affiliation(s)
- Beatriz García-Martínez
- Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingenieros Industriales, Universidad de Castilla-La Mancha, 02071 Albacete, Spain.,Instituto de Investigación en Informática de Albacete, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
| | - Antonio Fernández-Caballero
- Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingenieros Industriales, Universidad de Castilla-La Mancha, 02071 Albacete, Spain.,Instituto de Investigación en Informática de Albacete, Universidad de Castilla-La Mancha, 02071 Albacete, Spain.,CIBERSAM (Biomedical Research Networking Centre in Mental Health), Madrid, Spain
| | - Arturo Martínez-Rodrigo
- Research Group in Electronic, Biomedical and Telecommunication Engineering, Facultad de Comunicación, Universidad de, Castilla-La Mancha, 16071 Cuenca, Spain.,Instituto de Tecnologías Audiovisuales de, Castilla-La Mancha, Universidad de Castilla-La, Mancha, 16071 Cuenca, Spain
| | - Raúl Alcaraz
- Research Group in Electronic, Biomedical and Telecommunication Engineering, Escuela Politécnica de Cuenca, Universidad, de Castilla-La Mancha, 16071 Cuenca, Spain
| | - Paulo Novais
- Algoritmi Center, Department of Informatics, Universidade do Minho, 4800-058 Guimaräes, Portugal
| |
Collapse
|
18
|
Rosoł M, Młyńczak M, Cybulski G. Granger causality test with nonlinear neural-network-based methods: Python package and simulation study. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 216:106669. [PMID: 35151111 DOI: 10.1016/j.cmpb.2022.106669] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Revised: 01/25/2022] [Accepted: 01/26/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND AND OBJECTIVE Causality defined by Granger in 1969 is a widely used concept, particularly in neuroscience and economics. As there is an increasing interest in nonlinear causality research, a Python package with a neural-network-based causality analysis approach was created. It allows performing causality tests using neural networks based on Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), or Multilayer Perceptron (MLP). The aim of this paper is to present the nonlinear method for causality analysis and the created Python package. METHODS The created functions with the autoregressive (AR) and Generalized Radial Basis Functions (GRBF) neural network models were tested on simulated signals in two cases: with nonlinear dependency and with absence of causality from Y to X signal. The train-test split (70/30) was used. Errors obtained on the test set were compared using the Wilcoxon signed-rank test to determine the presence of the causality. For the chosen model, the proposed method of study the change of causality over time was presented. RESULTS In the case when X was a polynomial of Y, nonlinear methods were able to detect the causality, while the AR model did not manage to indicate it. The best results (in terms of the prediction accuracy) were obtained for the MLP for the lag of 150 (MSE equal to 0.011, compared to 0.041 and 0.036 for AR and GRBF, respectively). When there was no causality between the signals, none of the proposed and AR models did indicate false causality, while it was detected by GRBF models in one case. Only the proposed models gave the expected results in each of the tested scenarios. CONCLUSIONS The proposed method appeared to be superior to the compared methods. They were able to detect non-linear causality, make accurate forecasting and not indicate false causality. The created package enables easy usage of neural networks to study the causal relationship between signals. The neural-networks-based approach is a suitable method that allows the detection of a nonlinear causal relationship, which cannot be detected by the classical Granger method. Unlike other similar tools, the package allows for the study of changes in causality over time.
Collapse
Affiliation(s)
- Maciej Rosoł
- Faculty of Mechatronics, Institute of Metrology and Biomedical Engineering, Warsaw University of Technology, Warsaw, Poland.
| | - Marcel Młyńczak
- Faculty of Mechatronics, Institute of Metrology and Biomedical Engineering, Warsaw University of Technology, Warsaw, Poland
| | - Gerard Cybulski
- Faculty of Mechatronics, Institute of Metrology and Biomedical Engineering, Warsaw University of Technology, Warsaw, Poland
| |
Collapse
|
19
|
Liu H, Gao Y, Huang W, Li R, Houston M, Benoit JS, Roh J, Zhang Y. Inter-muscular coherence and functional coordination in the human upper extremity after stroke. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2022; 19:4506-4525. [PMID: 35430825 DOI: 10.3934/mbe.2022208] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Muscle coordination and motor function of stroke patients are weakened by stroke-related motor impairments. Our earlier studies have determined alterations in inter-muscular coordination patterns (muscle synergies). However, the functional connectivity of these synergistically paired or unpaired muscles is still unclear in stroke patients. The goal of this study is to quantify the alterations of inter-muscular coherence (IMC) among upper extremity muscles that have been shown to be synergistically or non-synergistically activated in stroke survivors. In a three-dimensional isometric force matching task, surface EMG signals are collected from 6 age-matched, neurologically intact healthy subjects and 10 stroke patients, while the target force space is divided into 8 subspaces. According to the results of muscle synergy identification with non-negative matrix factorization algorithm, muscle pairs are classified as synergistic and non-synergistic. In both control and stroke groups, IMC is then calculated for all available muscle pairs. The results show that synergistic muscle pairs have higher coherence in both groups. Furthermore, anterior and middle deltoids, identified as synergistic muscles in both groups, exhibited significantly weaker IMC at alpha band in stroke patients. The anterior and posterior deltoids, identified as synergistic muscles only in stroke patients, revealed significantly higher IMC in stroke group at low gamma band. On the contrary, anterior deltoid and pectoralis major, identified as synergistic muscles in control group only, revealed significantly higher IMC in control group in alpha band. The results of muscle synergy and IMC analyses provide congruent and complementary information for investigating the mechanism that underlies post-stroke motor recovery.
Collapse
Affiliation(s)
- Hongming Liu
- Zhuoyue Honors College, Hangzhou Dianzi University, Hangzhou 310018, China
- College of Automation, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Yunyuan Gao
- College of Automation, Hangzhou Dianzi University, Hangzhou 310018, China
- Key labortory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 311247, China
| | - Wei Huang
- College of Automation, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Rihui Li
- Department of Biomedical Engineering, University of Houston, Houston 75835, United States
| | - Michael Houston
- Department of Biomedical Engineering, University of Houston, Houston 75835, United States
| | - Julia S Benoit
- Texas Institute for Measurement Evaluation and Statistics, University of Houston, Houston 75835, United States
| | - Jinsook Roh
- Department of Biomedical Engineering, University of Houston, Houston 75835, United States
| | - Yingchun Zhang
- Department of Biomedical Engineering, University of Houston, Houston 75835, United States
| |
Collapse
|
20
|
Li Z, Wang X, Shen W, Yang S, Zhao DY, Hu J, Wang D, Liu J, Xin H, Zhang Y, Li P, Zhang B, Cai H, Liang Y, Li X. Objective Recognition of Tinnitus Location Using Electroencephalography Connectivity Features. Front Neurosci 2022; 15:784721. [PMID: 35058742 PMCID: PMC8764239 DOI: 10.3389/fnins.2021.784721] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Accepted: 11/16/2021] [Indexed: 11/26/2022] Open
Abstract
Purpose: Tinnitus is a common but obscure auditory disease to be studied. This study will determine whether the connectivity features in electroencephalography (EEG) signals can be used as the biomarkers for an efficient and fast diagnosis method for chronic tinnitus. Methods: In this study, the resting-state EEG signals of tinnitus patients with different tinnitus locations were recorded. Four connectivity features [including the Phase-locking value (PLV), Phase lag index (PLI), Pearson correlation coefficient (PCC), and Transfer entropy (TE)] and two time-frequency domain features in the EEG signals were extracted, and four machine learning algorithms, included two support vector machine models (SVM), a multi-layer perception network (MLP) and a convolutional neural network (CNN), were used based on the selected features to classify different possible tinnitus sources. Results: Classification accuracy was highest when the SVM algorithm or the MLP algorithm was applied to the PCC feature sets, achieving final average classification accuracies of 99.42 or 99.1%, respectively. And based on the PLV feature, the classification result was also particularly good. And MLP ran the fastest, with an average computing time of only 4.2 s, which was more suitable than other methods when a real-time diagnosis was required. Conclusion: Connectivity features of the resting-state EEG signals could characterize the differentiation of tinnitus location. The connectivity features (PCC and PLV) were more suitable as the biomarkers for the objective diagnosing of tinnitus. And the results were helpful for clinicians in the initial diagnosis of tinnitus.
Collapse
Affiliation(s)
| | - Xinzui Wang
- Jihua Laboratory, Foshan, China.,Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China
| | - Weidong Shen
- Department of Otolaryngology, Head and Neck Surgery, Chinese PLA General Hospital, Institute of Otolaryngology, Beijing, China
| | - Shiming Yang
- Department of Otolaryngology, Head and Neck Surgery, Chinese PLA General Hospital, Institute of Otolaryngology, Beijing, China
| | | | - Jimin Hu
- Jiangsu Testing and Inspection Institute for Medical Devices, Nanjing, China
| | - Dawei Wang
- Jiangsu Testing and Inspection Institute for Medical Devices, Nanjing, China
| | | | | | | | | | | | | | | | | |
Collapse
|
21
|
Tuncer T, Dogan S, Baygin M, Rajendra Acharya U. Tetromino pattern based accurate EEG emotion classification model. Artif Intell Med 2022; 123:102210. [PMID: 34998511 DOI: 10.1016/j.artmed.2021.102210] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2021] [Revised: 10/31/2021] [Accepted: 11/01/2021] [Indexed: 11/17/2022]
Abstract
Nowadays, emotion recognition using electroencephalogram (EEG) signals is becoming a hot research topic. The aim of this paper is to classify emotions of EEG signals using a novel game-based feature generation function with high accuracy. Hence, a multileveled handcrafted feature generation automated emotion classification model using EEG signals is presented. A novel textural features generation method inspired by the Tetris game called Tetromino is proposed in this work. The Tetris game is one of the famous games worldwide, which uses various characters in the game. First, the EEG signals are subjected to discrete wavelet transform (DWT) to create various decomposition levels. Then, novel features are generated from the decomposed DWT sub-bands using the Tetromino method. Next, the maximum relevance minimum redundancy (mRMR) features selection method is utilized to select the most discriminative features, and the selected features are classified using support vector machine classifier. Finally, each channel's results (validation predictions) are obtained, and the mode function-based voting method is used to obtain the general results. We have validated our developed model using three databases (DREAMER, GAMEEMO, and DEAP). We have attained 100% accuracies using DREAMER and GAMEEMO datasets. Furthermore, over 99% of classification accuracy is achieved for DEAP dataset. Thus, our developed emotion detection model has yielded the best classification accuracy rate compared to the state-of-the-art techniques and is ready to be tested for clinical application after validating with more diverse datasets. Our results show the success of the presented Tetromino pattern-based EEG signal classification model validated using three public emotional EEG datasets.
Collapse
Affiliation(s)
- Turker Tuncer
- Department of Digital Forensics Engineering, College of Technology, Firat University, Elazig, Turkey.
| | - Sengul Dogan
- Department of Digital Forensics Engineering, College of Technology, Firat University, Elazig, Turkey
| | - Mehmet Baygin
- Department of Computer Engineering, Faculty of Engineering, Ardahan University, Ardahan, Turkey
| | - U Rajendra Acharya
- Ngee Ann Polytechnic, Department of Electronics and Computer Engineering, 599489, Singapore; Department of Biomedical Engineering, School of Science and Technology, SUSS University, Singapore; Department of Biomedical Informatics and Medical Engineering, Asia University, Taichung, Taiwan
| |
Collapse
|
22
|
Liu H, Zhang Y, Li Y, Kong X. Review on Emotion Recognition Based on Electroencephalography. Front Comput Neurosci 2021; 15:758212. [PMID: 34658828 PMCID: PMC8518715 DOI: 10.3389/fncom.2021.758212] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Accepted: 08/31/2021] [Indexed: 11/13/2022] Open
Abstract
Emotions are closely related to human behavior, family, and society. Changes in emotions can cause differences in electroencephalography (EEG) signals, which show different emotional states and are not easy to disguise. EEG-based emotion recognition has been widely used in human-computer interaction, medical diagnosis, military, and other fields. In this paper, we describe the common steps of an emotion recognition algorithm based on EEG from data acquisition, preprocessing, feature extraction, feature selection to classifier. Then, we review the existing EEG-based emotional recognition methods, as well as assess their classification effect. This paper will help researchers quickly understand the basic theory of emotion recognition and provide references for the future development of EEG. Moreover, emotion is an important representation of safety psychology.
Collapse
Affiliation(s)
- Haoran Liu
- The Boiler and Pressure Vessel Safety Inspection Institute of Henan Province, Zhengzhou, China
| | - Ying Zhang
- Patent Examination Cooperation (Henan) Center of the Patent Office, CNIPA, Zhengzhou, China
| | - Yujun Li
- The Boiler and Pressure Vessel Safety Inspection Institute of Henan Province, Zhengzhou, China
| | - Xiangyi Kong
- The Boiler and Pressure Vessel Safety Inspection Institute of Henan Province, Zhengzhou, China
| |
Collapse
|
23
|
Ren Z, Li R, Chen B, Zhang H, Ma Y, Wang C, Lin Y, Zhang Y. EEG-Based Driving Fatigue Detection Using a Two-Level Learning Hierarchy Radial Basis Function. Front Neurorobot 2021; 15:618408. [PMID: 33643018 PMCID: PMC7905350 DOI: 10.3389/fnbot.2021.618408] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2020] [Accepted: 01/05/2021] [Indexed: 11/13/2022] Open
Abstract
Electroencephalography (EEG)-based driving fatigue detection has gained increasing attention recently due to the non-invasive, low-cost, and potable nature of the EEG technology, but it is still challenging to extract informative features from noisy EEG signals for driving fatigue detection. Radial basis function (RBF) neural network has drawn lots of attention as a promising classifier due to its linear-in-the-parameters network structure, strong non-linear approximation ability, and desired generalization property. The RBF network performance heavily relies on network parameters such as the number of the hidden nodes, number of the center vectors, width, and output weights. However, global optimization methods that directly optimize all the network parameters often result in high evaluation cost and slow convergence. To enhance the accuracy and efficiency of EEG-based driving fatigue detection model, this study aims to develop a two-level learning hierarchy RBF network (RBF-TLLH) which allows for global optimization of the key network parameters. Experimental EEG data were collected, at both fatigue and alert states, from six healthy participants in a simulated driving environment. Principal component analysis was first utilized to extract features from EEG signals, and the proposed RBF-TLLH was then employed for driving status (fatigue vs. alert) classification. The results demonstrated that the proposed RBF-TLLH approach achieved a better classification performance (mean accuracy: 92.71%; area under the receiver operating curve: 0.9199) compared to other widely used artificial neural networks. Moreover, only three core parameters need to be determined using the training datasets in the proposed RBF-TLLH classifier, which increases its reliability and applicability. The findings demonstrate that the proposed RBF-TLLH approach can be used as a promising framework for reliable EEG-based driving fatigue detection.
Collapse
Affiliation(s)
- Ziwu Ren
- Robotics and Microsystems Center, Soochow University, Suzhou, China
| | - Rihui Li
- Department of Biomedical Engineering, University of Houston, Houston, TX, United States
| | - Bin Chen
- College of Automation, Intelligent Control & Robotics Institute, Hangzhou Dianzi University, Hangzhou, China
| | - Hongmiao Zhang
- Robotics and Microsystems Center, Soochow University, Suzhou, China
| | - Yuliang Ma
- College of Automation, Intelligent Control & Robotics Institute, Hangzhou Dianzi University, Hangzhou, China
| | - Chushan Wang
- Guangdong Provincial Work Injury Rehabilitation Hospital, Guangzhou, China
| | - Ying Lin
- Department of Industrial Engineering, University of Houston, Houston, TX, United States
| | - Yingchun Zhang
- Department of Biomedical Engineering, University of Houston, Houston, TX, United States
| |
Collapse
|