Effective Emotion Recognition by Learning Discriminative Graph Topologies in EEG Brain Networks

EEG-based deception detection using weighted dual perspective visibility graph analysis

Deception detection is a critical aspect across various domains. Integrating advanced signal processing techniques, particularly in neuroscientific studies, has opened new avenues for exploring deception at a deeper level. This study uses electroencephalogram (EEG) signals from a balanced cohort of 22 participants, consisting of both males and females, aged between 22 and 29, engaged in a visual task for instructed deception. We propose a novel approach in the realm of deception detection utilizing the Weighted Dual Perspective Visibility Graph (WDPVG) method to decode instructed deception by converting average epochs from each EEG channel into a complex network. Six graph-based features, including average and deviation of strength, weighted clustering coefficient, weighted clustering coefficient entropy, average weighted shortest path length, and modularity, are extracted, comprehensively representing the underlying brain dynamics associated with deception. Subsequently, these features are employed for classification using three distinct algorithms: K Nearest Neighbors (KNN), Support Vector Machine (SVM), and Decision Tree (DT). Experimental results reveal promising accuracy rates for KNN (66.64%), SVM (86.25%), and DT (82.46%). Furthermore, the features distributions of EEG networks are analyzed across different brain lobes, comparing truth-telling and lying modes. These analyses reveal the frontal and parietal lobes' potential in distinguishing between truth and deception, highlighting their active role during deceptive behavior. The findings demonstrate the WDPVG method's effectiveness in decoding deception from EEG signals, offering insights into the neural basis of deceptive behavior. This research could enhance the understanding of neuroscience and deception detection, providing a framework for future real-world applications.

Emotion Recognition Based on a EEG-fNIRS Hybrid Brain Network in the Source Space

Background/Objectives: Studies have shown that emotion recognition based on electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) multimodal physiological signals exhibits superior performance compared to that of unimodal approaches. Nonetheless, there remains a paucity of in-depth investigations analyzing the inherent relationship between EEG and fNIRS and constructing brain networks to improve the performance of emotion recognition. Methods: In this study, we introduce an innovative method to construct hybrid brain networks in the source space based on simultaneous EEG-fNIRS signals for emotion recognition. Specifically, we perform source localization on EEG signals to derive the EEG source signals. Subsequently, causal brain networks are established in the source space by analyzing the Granger causality between the EEG source signals, while coupled brain networks in the source space are formed by assessing the coupling strength between the EEG source signals and the fNIRS signals. The resultant causal brain networks and coupled brain networks are integrated to create hybrid brain networks in the source space, which serve as features for emotion recognition. Results: The effectiveness of our proposed method is validated on multiple emotion datasets. The experimental results indicate that the recognition performance of our approach significantly surpasses that of the baseline method. Conclusions: This work offers a novel perspective on the fusion of EEG and fNIRS signals in an emotion-evoked experimental paradigm and provides a feasible solution for enhancing emotion recognition performance.

Granger Causal Inference Based on Dual Laplacian Distribution and Its Application to MI-BCI Classification

Granger causality-based effective brain connectivity provides a powerful tool to probe the neural mechanism for information processing and the potential features for brain computer interfaces. However, in real applications, traditional Granger causality is prone to the influence of outliers, such as inevitable ocular artifacts, resulting in unreasonable brain linkages and the failure to decipher inherent cognition states. In this work, motivated by constructing the sparse causality brain networks under the strong physiological outlier noise conditions, we proposed a dual Laplacian Granger causality analysis (DLap-GCA) by imposing Laplacian distributions on both model parameters and residuals. In essence, the first Laplacian assumption on residuals will resist the influence of outliers in electroencephalogram (EEG) on causality inference, and the second Laplacian assumption on model parameters will sparsely characterize the intrinsic interactions among multiple brain regions. Through simulation study, we quantitatively verified its effectiveness in suppressing the influence of complex outliers, the stable capacity for model estimation, and sparse network inference. The application to motor-imagery (MI) EEG further reveals that our method can effectively capture the inherent hemispheric lateralization of MI tasks with sparse patterns even under strong noise conditions. The MI classification based on the network features derived from the proposed approach shows higher accuracy than other existing traditional approaches, which is attributed to the discriminative network structures being captured in a timely manner by DLap-GCA even under the single-trial online condition. Basically, these results consistently show its robustness to the influence of complex outliers and the capability of characterizing representative brain networks for cognition information processing, which has the potential to offer reliable network structures for both cognitive studies and future brain-computer interface (BCI) realization.

Toward Explainable Affective Computing: A Review

Affective computing has an unprecedented potential to change the way humans interact with technology. While the last decades have witnessed vast progress in the field, multimodal affective computing systems are generally black box by design. As affective systems start to be deployed in real-world scenarios, such as education or healthcare, a shift of focus toward improved transparency and interpretability is needed. In this context, how do we explain the output of affective computing models? and how to do so without limiting predictive performance? In this article, we review affective computing work from an explainable AI (XAI) perspective, collecting and synthesizing relevant papers into three major XAI approaches: premodel (applied before training), in-model (applied during training), and postmodel (applied after training). We present and discuss the most fundamental challenges in the field, namely, how to relate explanations back to multimodal and time-dependent data, how to integrate context and inductive biases into explanations using mechanisms such as attention, generative modeling, or graph-based methods, and how to capture intramodal and cross-modal interactions in post hoc explanations. While explainable affective computing is still nascent, existing methods are promising, contributing not only toward improved transparency but, in many cases, surpassing state-of-the-art results. Based on these findings, we explore directions for future research and discuss the importance of data-driven XAI and explanation goals, and explainee needs definition, as well as causability or the extent to which a given method leads to human understanding.

Unsupervised heterogeneous domain adaptation for EEG classification

Objective.Domain adaptation has been recognized as a potent solution to the challenge of limited training data for electroencephalography (EEG) classification tasks. Existing studies primarily focus on homogeneous environments, however, the heterogeneous properties of EEG data arising from device diversity cannot be overlooked. This motivates the development of heterogeneous domain adaptation methods that can fully exploit the knowledge from an auxiliary heterogeneous domain for EEG classification.Approach.In this article, we propose a novel model named informative representation fusion (IRF) to tackle the problem of unsupervised heterogeneous domain adaptation in the context of EEG data. In IRF, we consider different perspectives of data, i.e. independent identically distributed (iid) and non-iid, to learn different representations. Specifically, from the non-iid perspective, IRF models high-order correlations among data by hypergraphs and develops hypergraph encoders to obtain data representations of each domain. From the non-iid perspective, by applying multi-layer perceptron networks to the source and target domain data, we achieve another type of representation for both domains. Subsequently, an attention mechanism is used to fuse these two types of representations to yield informative features. To learn transferable representations, the maximum mean discrepancy is utilized to align the distributions of the source and target domains based on the fused features.Main results.Experimental results on several real-world datasets demonstrate the effectiveness of the proposed model.Significance.This article handles an EEG classification situation where the source and target EEG data lie in different spaces, and what's more, under an unsupervised learning setting. This situation is practical in the real world but barely studied in the literature. The proposed model achieves high classification accuracy, and this study is important for the commercial applications of EEG-based BCIs.

An improved empirical mode decomposition method with ensemble classifiers for analysis of multichannel EEG in BCI emotion recognition

Emotion recognition using EEG is a difficult study because the signals' unstable behavior, which is brought on by the brain's complex neuronal activity, makes it difficult to extract the underlying patterns inside it. Therefore, to analyse the signal more efficiently, in this article, a hybrid model based on IEMD-KW-Ens (Improved Empirical Mode Decomposition-Kruskal Wallis-Ensemble classifiers) technique is used. Here IEMD based technique is proposed to interpret EEG signals by adding an improved sifting stopping criterion with median filter to get the optimal decomposed EEG signals for further processing. A mixture of time, frequency and non-linear distinct features are extracted for constructing the feature vector. Afterward, we conducted feature selection using KW test to remove the insignificant ones from the feature set. Later the classification of emotions in three-dimensional model is performed in two categories i.e. machine learning based RUSBoosted trees and deep learning based convolutional neural network (CNN) for DEAP and DREAMER datasets and the outcomes are evaluated for valence, arousal, and dominance classes. The findings demonstrate that the hybrid model can successfully classify emotions in multichannel EEG signals. The decomposition approach is also instructive for improving the model's utility in emotional computing.

Frequency-specific segregation and integration of human cerebral cortex: An intrinsic functional atlas

The cognitive and behavioral functions of the human brain are supported by its frequency multiplexing mechanism. However, there is limited understanding of the dynamics of the functional network topology. This study aims to investigate the frequency-specific topology of the functional human brain using 7T rs-fMRI data. Frequency-specific parcellations were first performed, revealing frequency-dependent dynamics within the frontoparietal control, parietal memory, and visual networks. An intrinsic functional atlas containing 456 parcels was proposed and validated using stereo-EEG. Graph theory analysis suggested that, in addition to the task-positive vs. task-negative organization observed in static networks, there was a cognitive control system additionally from a frequency perspective. The reproducibility and plausibility of the identified hub sets were confirmed through 3T fMRI analysis, and their artificial removal had distinct effects on network topology. These results indicate a more intricate and subtle dynamics of the functional human brain and emphasize the significance of accurate topography.

Micro-Expression Recognition Based on Nodal Efficiency in the EEG Functional Networks

Micro-expression recognition based on ima- ges has made some progress, yet limitations persist. For instance, image-based recognition of micro-expressions is affected by factors such as ambient light, changes in head posture, and facial occlusion. The high temporal resolution of electroencephalogram (EEG) technology can record brain activity associated with micro-expressions and identify them objectively from a neurophysiological standpoint. Accordingly, this study introduces a novel method for recognizing micro-expressions using node efficiency features of brain networks derived from EEG signals. We designed a real-time Supervision and Emotional Expression Suppression (SEES) experimental paradigm to collect video and EEG data reflecting micro- and macro-expression states from 70 participants experiencing positive emotions. By constructing functional brain networks based on graph theory, we analyzed the network efficiencies at both macro- and micro-levels. The participants exhibited lower connection density, global efficiency, and nodal efficiency in the alpha, beta, and gamma networks during micro-expressions compared to macro-expressions. We then selected the optimal subset of nodal efficiency features using a random forest algorithm and applied them to various classifiers, including Support Vector Machine (SVM), Gradient-Boosted Decision Tree (GBDT), Logistic Regression (LR), Random Forest (RF), and eXtreme Gradient Boosting (XGBoost). These classifiers achieved promising accuracy in micro-expression recognition, with SVM exhibiting the highest accuracy of 92.6% when 15 channels were selected. This study provides a new neuroscientific indicator for recognizing micro-expressions based on EEG signals, thereby broadening the potential applications for micro-expression recognition.

Differentiating Between Alzheimer's Disease and Frontotemporal Dementia Based on the Resting-State Multilayer EEG Network

Frontotemporal dementia (FTD) is frequently misdiagnosed as Alzheimer's disease (AD) due to similar clinical symptoms. In this study, we constructed frequency-based multilayer resting-state electroencephalogram (EEG) networks and extracted representative network features to improve the differentiation between AD and FTD. When compared with healthy controls (HC), AD showed primarily stronger delta-alpha cross-couplings and weaker theta-sigma cross-couplings. Notably, when comparing the AD and FTD groups, we found that the AD exhibited stronger delta-alpha and delta-beta connectivity than the FTD. Thereafter, by extracting the representative network features and then applying these features in the classification between AD and FTD, an accuracy of 81.1% was achieved. Finally, a multivariable linear regressive model was built, based on the differential topologies, and then adopted to predict the scores of the Mini-Mental State Examination (MMSE) scale. Accordingly, the predicted and actual measured scores were indeed significantly correlated with each other ( r = 0.274, p = 0.036). These findings consistently suggest that frequency-based multilayer resting-state networks can be utilized for classifying AD and FTD and have potential applications for clinical diagnosis.