Responses of functional brain networks while watching 2D and 3D videos: An EEG study

Differential Brain Activation for Four Emotions in VR-2D and VR-3D Modes

Similar to traditional imaging, virtual reality (VR) imagery encompasses nonstereoscopic (VR-2D) and stereoscopic (VR-3D) modes. Currently, Russell's emotional model has been extensively studied in traditional 2D and VR-3D modes, but there is limited comparative research between VR-2D and VR-3D modes. In this study, we investigate whether Russell's emotional model exhibits stronger brain activation states in VR-3D mode compared to VR-2D mode. By designing an experiment covering four emotional categories (high arousal-high pleasure (HAHV), high arousal-low pleasure (HALV), low arousal-low pleasure (LALV), and low arousal-high pleasure (LAHV)), EEG signals were collected from 30 healthy undergraduate and graduate students while watching videos in both VR modes. Initially, power spectral density (PSD) computations revealed distinct brain activation patterns in different emotional states across the two modes, with VR-3D videos inducing significantly higher brainwave energy, primarily in the frontal, temporal, and occipital regions. Subsequently, Differential entropy (DE) feature sets, selected via a dual ten-fold cross-validation Support Vector Machine (SVM) classifier, demonstrate satisfactory classification accuracy, particularly superior in the VR-3D mode. The paper subsequently presents a deep learning-based EEG emotion recognition framework, adeptly utilizing the frequency, spatial, and temporal information of EEG data to improve recognition accuracy. The contribution of each individual feature to the prediction probabilities is discussed through machine-learning interpretability based on Shapley values. The study reveals notable differences in brain activation states for identical emotions between the two modes, with VR-3D mode showing more pronounced activation.

Responses of functional brain networks in micro-expressions: An EEG study

Micro-expressions (MEs) can reflect an individual’s subjective emotions and true mental state, and they are widely used in the fields of mental health, justice, law enforcement, intelligence, and security. However, one of the major challenges of working with MEs is that their neural mechanism is not entirely understood. To the best of our knowledge, the present study is the first to use electroencephalography (EEG) to investigate the reorganizations of functional brain networks involved in MEs. We aimed to reveal the underlying neural mechanisms that can provide electrophysiological indicators for ME recognition. A real-time supervision and emotional expression suppression experimental paradigm was designed to collect video and EEG data of MEs and no expressions (NEs) of 70 participants expressing positive emotions. Based on the graph theory, we analyzed the efficiency of functional brain network at the scalp level on both macro and micro scales. The results revealed that in the presence of MEs compared with NEs, the participants exhibited higher global efficiency and nodal efficiency in the frontal, occipital, and temporal regions. Additionally, using the random forest algorithm to select a subset of functional connectivity features as input, the support vector machine classifier achieved a classification accuracy for MEs and NEs of 0.81, with an area under the curve of 0.85. This finding demonstrates the possibility of using EEG to recognize MEs, with a wide range of application scenarios, such as persons wearing face masks or patients with expression disorders.

Frontal-occipital network alterations while viewing 2D & 3D movies: a source-level EEG and graph theory approach

Many researchers have measured the differences in electroencephalography (EEG) while viewing 2D and 3D movies to uncover the neuromechanism underlying distinct viewing experiences. Using whole-brain network analyses of scalp EEG, our previous study reported that beta and gamma bands presented higher global efficiencies while viewing 3D movies. However, scalp EEG is influenced by volume conduction, not allowing inference from a neuroanatomy perspective; thus, source reconstruction techniques are recommended. This paper is the first to measure the differences in the frontal-occipital networks in EEG source space during 2D and 3D movie viewing. EEG recordings from 40 subjects were performed during 2D and 3D movie viewing. We constructed frontal-occipital networks of alpha, beta, and gamma bands in EEG source space and analyzed network efficiencies. We found that the beta band exhibited higher global efficiency in 3D movie viewing than in 2D movie viewing; however, the alpha global efficiency was not statistically significant. In addition, a support vector machine (SVM) classifier, taking functional connectivities as classification features, was built to identify whether the frontal-occipital networks contain patterns that could distinguish 2D and 3D movie viewing. Using the 6 most important functional connectivity features of the beta band, we obtained the best accuracy of 0.933. Our findings shed light on uncovering the neuromechanism underlying distinct experiences while viewing 2D and 3D movies.

Where Is My Mind (Looking at)? A Study of the EEG–Visual Attention Relationship

Visual attention estimation is an active field of research at the crossroads of different disciplines: computer vision, deep learning, and medicine. One of the most common approaches to estimate a saliency map representing attention is based on the observed images. In this paper, we show that visual attention can be retrieved from EEG acquisition. The results are comparable to traditional predictions from observed images, which is of great interest. Image-based saliency estimation being participant independent, the estimation from EEG could take into account the subject specificity. For this purpose, a set of signals has been recorded, and different models have been developed to study the relationship between visual attention and brain activity. The results are encouraging and comparable with other approaches estimating attention with other modalities. Being able to predict a visual saliency map from EEG could help in research studying the relationship between brain activity and visual attention. It could also help in various applications: vigilance assessment during driving, neuromarketing, and also in the help for the diagnosis and treatment of visual attention-related diseases. For the sake of reproducibility, the codes and dataset considered in this paper have been made publicly available to promote research in the field.