Affective Man-Machine Interface: Unveiling Human Emotions through Biosignals

Acting Emotions: a comprehensive dataset of elicited emotions

Emotions encompass physiological systems that can be assessed through biosignals like electromyography and electrocardiography. Prior investigations in emotion recognition have primarily focused on general population samples, overlooking the specific context of theatre actors who possess exceptional abilities in conveying emotions to an audience, namely acting emotions. We conducted a study involving 11 professional actors to collect physiological data for acting emotions to investigate the correlation between biosignals and emotion expression. Our contribution is the DECEiVeR (DatasEt aCting Emotions Valence aRousal) dataset, a comprehensive collection of various physiological recordings meticulously curated to facilitate the recognition of a set of five emotions. Moreover, we conduct a preliminary analysis on modeling the recognition of acting emotions from raw, low- and mid-level temporal and spectral data and the reliability of physiological data across time. Our dataset aims to leverage a deeper understanding of the intricate interplay between biosignals and emotional expression. It provides valuable insights into acting emotion recognition and affective computing by exposing the degree to which biosignals capture emotions elicited from inner stimuli.

AVDOS-VR: Affective Video Database with Physiological Signals and Continuous Ratings Collected Remotely in VR

Investigating emotions relies on pre-validated stimuli to evaluate induced responses through subjective self-ratings and physiological changes. The creation of precise affect models necessitates extensive datasets. While datasets related to pictures, words, and sounds are abundant, those associated with videos are comparatively scarce. To overcome this challenge, we present the first virtual reality (VR) database with continuous self-ratings and physiological measures, including facial EMG. Videos were rated online using a head-mounted VR device (HMD) with attached emteqPRO mask and a cinema VR environment in remote home and laboratory settings with minimal setup requirements. This led to an affective video database with continuous valence and arousal self-rating measures and physiological responses (PPG, facial-EMG (7x), IMU). The AVDOS-VR database includes data from 37 participants who watched 30 randomly ordered videos (10 positive, neutral, and negative). Each 30-second video was assessed with two-minute relaxation between categories. Validation results suggest that remote data collection is ecologically valid, providing an effective strategy for future affective study designs. All data can be accessed via: www.gnacek.com/affective-video-database-online-study .

A dataset of continuous affect annotations and physiological signals for emotion analysis

From a computational viewpoint, emotions continue to be intriguingly hard to understand. In research, a direct and real-time inspection in realistic settings is not possible. Discrete, indirect, post-hoc recordings are therefore the norm. As a result, proper emotion assessment remains a problematic issue. The Continuously Annotated Signals of Emotion (CASE) dataset provides a solution as it focusses on real-time continuous annotation of emotions, as experienced by the participants, while watching various videos. For this purpose, a novel, intuitive joystick-based annotation interface was developed, that allowed for simultaneous reporting of valence and arousal, that are instead often annotated independently. In parallel, eight high quality, synchronized physiological recordings (1000 Hz, 16-bit ADC) were obtained from ECG, BVP, EMG (3x), GSR (or EDA), respiration and skin temperature sensors. The dataset consists of the physiological and annotation data from 30 participants, 15 male and 15 female, who watched several validated video-stimuli. The validity of the emotion induction, as exemplified by the annotation and physiological data, is also presented.

Quantification of vascular function changes under different emotion states: A pilot study

Recent studies have indicated that physiological parameters change with different emotion states. This study aimed to quantify the changes of vascular function at different emotion and sub-emotion states. Twenty young subjects were studied with their finger photoplethysmographic (PPG) pulses recorded at three distinct emotion states: natural (1 minute), happiness and sadness (10 minutes for each). Within the period of happiness and sadness emotion states, two sub-emotion states (calmness and outburst) were identified with the synchronously recorded videos. Reflection index (RI) and stiffness index (SI), two widely used indices of vascular function, were derived from the PPG pulses to quantify their differences between three emotion states, as well as between two sub-emotion states. The results showed that, when compared with the natural emotion, RI and SI decreased in both happiness and sadness emotions. The decreases in RI were significant for both happiness and sadness emotions (both P< 0.01), but the decreases in SI was only significant for sadness emotion (P< 0.01). Moreover, for comparing happiness and sadness emotions, there was significant difference in RI (P< 0.01), but not in SI (P= 0.9). In addition, significant larger RI values were observed with the outburst sub-emotion in comparison with the calmness one for both happiness and sadness emotions (both P< 0.01) whereas significant larger SI values were observed with the outburst sub-emotion only in sadness emotion (P< 0.05). Moreover, gender factor hardly influence the RI and SI results for all three emotion measurements. This pilot study confirmed that vascular function changes with diffenrt emotion states could be quantified by the simple PPG measurement.

Frequency Band Analysis of Electrocardiogram (ECG) Signals for Human Emotional State Classification Using Discrete Wavelet Transform (DWT)

[Purpose] Intelligent emotion assessment systems have been highly successful in a variety of applications, such as e-learning, psychology, and psycho-physiology. This study aimed to assess five different human emotions (happiness, disgust, fear, sadness, and neutral) using heart rate variability (HRV) signals derived from an electrocardiogram (ECG). [Subjects] Twenty healthy university students (10 males and 10 females) with a mean age of 23 years participated in this experiment. [Methods] All five emotions were induced by audio-visual stimuli (video clips). ECG signals were acquired using 3 electrodes and were preprocessed using a Butterworth 3rd order filter to remove noise and baseline wander. The Pan-Tompkins algorithm was used to derive the HRV signals from ECG. Discrete wavelet transform (DWT) was used to extract statistical features from the HRV signals using four wavelet functions: Daubechies6 (db6), Daubechies7 (db7), Symmlet8 (sym8), and Coiflet5 (coif5). The k-nearest neighbor (KNN) and linear discriminant analysis (LDA) were used to map the statistical features into corresponding emotions. [Results] KNN provided the maximum average emotion classification rate compared to LDA for five emotions (sadness − 50.28%; happiness − 79.03%; fear − 77.78%; disgust − 88.69%; and neutral − 78.34%). [Conclusion] The results of this study indicate that HRV may be a reliable indicator of changes in the emotional state of subjects and provides an approach to the development of a real-time emotion assessment system with a higher reliability than other systems.