Impact of Affective Multimedia Content on the Electroencephalogram and Facial Expressions

A Survey on Physiological Signal-Based Emotion Recognition

Physiological signals are the most reliable form of signals for emotion recognition, as they cannot be controlled deliberately by the subject. Existing review papers on emotion recognition based on physiological signals surveyed only the regular steps involved in the workflow of emotion recognition such as pre-processing, feature extraction, and classification. While these are important steps, such steps are required for any signal processing application. Emotion recognition poses its own set of challenges that are very important to address for a robust system. Thus, to bridge the gap in the existing literature, in this paper, we review the effect of inter-subject data variance on emotion recognition, important data annotation techniques for emotion recognition and their comparison, data pre-processing techniques for each physiological signal, data splitting techniques for improving the generalization of emotion recognition models and different multimodal fusion techniques and their comparison. Finally, we discuss key challenges and future directions in this field.

Affective Video Tagging Framework using Human Attention Modelling through EEG Signals

The explosion of multimedia content over the past years is not surprising, thus their efficient management and analysis methods are always in demand. The affectiveness of any multimedia content deals with analyzing human perception and cognition while watching it. Human attention is also one of the important parameters, as it describes the engagement and interestingness of the user while watching that content. Considering this aspect, a video tagging framework is proposed in which the EEG signals of participants are used to analyze human perception while watching videos. A rigorous analysis has been performed on different scalp locations and frequency rhythms of brain signals to formulate significant features corresponding to affective and interesting video content. The analysis presented in this paper shows that the extracted human attention-based features are generating promising results with the accuracy of 93.2% using SVM-based classification model which supports the applicability of the model for various BCI-based applications for automatic classification of multimedia content.

Impact of Visual Design Elements and Principles in Human Electroencephalogram Brain Activity Assessed with Spectral Methods and Convolutional Neural Networks

The visual design elements and principles (VDEPs) can trigger behavioural changes and emotions in the viewer, but their effects on brain activity are not clearly understood. In this paper, we explore the relationships between brain activity and colour (cold/warm), light (dark/bright), movement (fast/slow), and balance (symmetrical/asymmetrical) VDEPs. We used the public DEAP dataset with the electroencephalogram signals of 32 participants recorded while watching music videos. The characteristic VDEPs for each second of the videos were manually tagged for by a team of two visual communication experts. Results show that variations in the light/value, rhythm/movement, and balance in the music video sequences produce a statistically significant effect over the mean absolute power of the Delta, Theta, Alpha, Beta, and Gamma EEG bands (p < 0.05). Furthermore, we trained a Convolutional Neural Network that successfully predicts the VDEP of a video fragment solely by the EEG signal of the viewer with an accuracy ranging from 0.7447 for Colour VDEP to 0.9685 for Movement VDEP. Our work shows evidence that VDEPs affect brain activity in a variety of distinguishable ways and that a deep learning classifier can infer visual VDEP properties of the videos from EEG activity.

A dry electroencephalogram electrode for applications in steady-state visual evoked potential-based brain-computer interface systems

High-efficiency electroencephalogram (EEG) dry electrodes are a key component of brain-computer interface (BCI) technology because of their direct contact with the scalp. In this study, a semi-flexible polydopamine (PDA)/Pt-TiO₂ electrode is prepared for the dry-contact acquisition of EEG signals. The PDA biofilm adheres strongly to the scalp and maintains a dynamic balance of water and ions. The Pt nanoparticles and TiO₂ nanotube array together result in fast electron transfer. Therefore, the interface impedance between the dry PDA/Pt-TiO₂ electrode and scalp is as low as 19.63-24.53 kΩ. The spontaneous EEG signal collected simultaneously using the dry PDA/Pt-TiO₂ and wet Ag/AgCl electrodes had a correlation coefficient of up to 99.9%. In a steady-state visual evoked potential (SSVEP)-based BCI system, the dry electrode was used to collect EEG feedback signals for stimulations at 27 different frequencies in the range of 7-19.25 Hz. For these feedback signals, O1, Oz, and O2 channels in the occipital area exhibited high signal-to-noise ratios of 11.3, 11.8, and 11 dB, respectively. A volunteer wore an EEG headband with three PDA/Pt-TiO₂ dry electrodes and successfully controlled the robotic arm of the SSVEP-BCI system in the untrained mode. The dry PDA/Pt-TiO₂ electrode-based EEG cap is comfortable to wear, the identification signals of the SSVEP paradigm are accurate, and it is suitable for controlling external devices including a keyboard in the SSVEP-BCI system.