1
|
Yang P, Liu N, Liu X, Shu Y, Ji W, Ren Z, Sheng J, Yu M, Yi R, Zhang D, Liu YJ. A Multimodal Dataset for Mixed Emotion Recognition. Sci Data 2024; 11:847. [PMID: 39103399 DOI: 10.1038/s41597-024-03676-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 07/23/2024] [Indexed: 08/07/2024] Open
Abstract
Mixed emotions have attracted increasing interest recently, but existing datasets rarely focus on mixed emotion recognition from multimodal signals, hindering the affective computing of mixed emotions. On this basis, we present a multimodal dataset with four kinds of signals recorded while watching mixed and non-mixed emotion videos. To ensure effective emotion induction, we first implemented a rule-based video filtering step to select the videos that could elicit stronger positive, negative, and mixed emotions. Then, an experiment with 80 participants was conducted, in which the data of EEG, GSR, PPG, and frontal face videos were recorded while they watched the selected video clips. We also recorded the subjective emotional rating on PANAS, VAD, and amusement-disgust dimensions. In total, the dataset consists of multimodal signal data and self-assessment data from 73 participants. We also present technical validations for emotion induction and mixed emotion classification from physiological signals and face videos. The average accuracy of the 3-class classification (i.e., positive, negative, and mixed) can reach 80.96% when using SVM and features from all modalities, which indicates the possibility of identifying mixed emotional states.
Collapse
Affiliation(s)
- Pei Yang
- Tsinghua University, Department of Computer Science and Technology, Beijing, 100084, China
| | - Niqi Liu
- Tsinghua University, Department of Computer Science and Technology, Beijing, 100084, China
| | - Xinge Liu
- Tsinghua University, Department of Computer Science and Technology, Beijing, 100084, China
| | - Yezhi Shu
- Tsinghua University, Department of Computer Science and Technology, Beijing, 100084, China
| | - Wenqi Ji
- Tsinghua University, Department of Computer Science and Technology, Beijing, 100084, China
| | - Ziqi Ren
- Tsinghua University, Department of Computer Science and Technology, Beijing, 100084, China
| | - Jenny Sheng
- Tsinghua University, Department of Computer Science and Technology, Beijing, 100084, China
| | - Minjing Yu
- Tianjin University, College of Intelligence and Computing, Tianjin, 300350, China
| | - Ran Yi
- Shanghai Jiao Tong University, Department of Computer Science and Engineering, Shanghai, 200240, China
| | - Dan Zhang
- Tsinghua University, Department of Psychology, Beijing, 100084, China
| | - Yong-Jin Liu
- Tsinghua University, Department of Computer Science and Technology, Beijing, 100084, China.
| |
Collapse
|
2
|
Yang Y, Gong S, Cao Y, Qiu Y, Xu X, Wang Y. Linking teacher support to achievement emotion profile: the mediating role of basic psychological need satisfaction. Front Psychol 2024; 15:1352337. [PMID: 39149704 PMCID: PMC11325168 DOI: 10.3389/fpsyg.2024.1352337] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 07/11/2024] [Indexed: 08/17/2024] Open
Abstract
The current study used a person-centered approach to explore the co-occurrence of college students' achievement emotions. It also examined the impact of teacher support on achievement emotion profiles and the mediating effect of need satisfaction. A total of 866 college students participated in the survey. A robust three-step latent profile analysis was employed to analyze the data. Four profiles of achievement emotions were identified: moderate mixed emotions, the blends of high positive emotions, the blends of moderate positive emotions, and high mixed emotions. Higher perceived teacher support was associated with a greater likelihood of being classified into the blends of moderate positive emotion profile or the blends of high positive emotion profile. Moreover, basic psychological need satisfaction mediated the relationship between teacher support and the four emotion profiles. Our findings contribute to a more comprehensive understanding of the role of teacher support in shaping achievement emotion profiles, helping to broaden the application of self-determination theory to explain the mechanism by which external support influences emotion profiles.
Collapse
Affiliation(s)
- Yang Yang
- Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan, China
- Mental Health Education Center, Huanghuai University, Zhumadian, China
| | - Shaoying Gong
- Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan, China
| | - Yang Cao
- School of Health Humanities, Hubei Shizhen Laboratory, Hubei University of Chinese Medicine, Wuhan, China
| | - Yin Qiu
- Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan, China
- Wuhan College, Wuhan, China
| | - Xizheng Xu
- Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan, China
- Hunan Police Academy, Changsha, China
| | - Yanqing Wang
- Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan, China
| |
Collapse
|
3
|
Boecker L, Petrowsky HM, Loschelder DD, Lange J. The interplay of social rank perceptions of Trump and Biden and emotions following the U.S. presidential election 2020. Cogn Emot 2024:1-19. [PMID: 38922603 DOI: 10.1080/02699931.2024.2356713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Accepted: 04/13/2024] [Indexed: 06/27/2024]
Abstract
The outcome of the 2020 U.S. election between Trump and Biden evoked strong emotions. In U.S. American (Study 1; N = 405) and German (Study 2; N = 123) samples, we investigated how observers' group membership (i.e. political orientation) and the social rank attainment of both candidates (i.e. dominance vs. prestige) predicted emotional reactions. Trump was generally perceived as more dominant, and Biden as more prestigious. However, perceptions of social rank attainment differed depending on the observers' political orientation, either matching or not matching with the leaders (i.e. Republicans and Democrats, respectively). The candidate who did not share the participants' political orientation was perceived as less prestigious and more dominant and elicited stronger contrastive emotions (i.e. schadenfreude, malicious envy) and weaker assimilative emotions (i.e. happy-for-ness, sympathy, anger), and vice versa. Crucially, dominance and prestige perceptions explained variance in the emotional reactions of more conservative and more liberal participants. Prestige positively predicted assimilative emotions and dominance contrastive emotions. Our work advances theorising by providing evidence that dominance and prestige perceptions contribute to the elicitation of various emotions. Furthermore, it suggests that prestige and dominance are not fixed characteristics of liberal and conservative leaders but depend on the observers' group membership.
Collapse
Affiliation(s)
- Lea Boecker
- Department of Economic Psychology, Social Psychology & Experimental Methods, University of Lunenburg, Lunenburg, Germany
| | - Hannes M Petrowsky
- Department of Economic Psychology, Social Psychology & Experimental Methods, University of Lunenburg, Lunenburg, Germany
| | - David D Loschelder
- Department of Economic Psychology, Social Psychology & Experimental Methods, University of Lunenburg, Lunenburg, Germany
| | - Jens Lange
- Department of Differential Psychology and Psychological Assessment, University of Hamburg, Hamburg, Germany
| |
Collapse
|
4
|
Vaccaro AG, Wu H, Iyer R, Shakthivel S, Christie NC, Damasio A, Kaplan J. Neural patterns associated with mixed valence feelings differ in consistency and predictability throughout the brain. Cereb Cortex 2024; 34:bhae122. [PMID: 38566509 DOI: 10.1093/cercor/bhae122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Revised: 03/07/2024] [Accepted: 03/08/2024] [Indexed: 04/04/2024] Open
Abstract
Mixed feelings, the simultaneous presence of feelings with positive and negative valence, remain an understudied topic. They pose a specific set of challenges due to individual variation, and their investigation requires analtyic approaches focusing on individually self-reported states. We used functional magnetic resonance imaging (fMRI) to scan 27 subjects watching an animated short film chosen to induce bittersweet mixed feelings. The same subjects labeled when they had experienced positive, negative, and mixed feelings. Using hidden-Markov models, we found that various brain regions could predict the onsets of new feeling states as determined by self-report. The ability of the models to identify these transitions suggests that these states may exhibit unique and consistent neural signatures. We next used the subjects' self-reports to evaluate the spatiotemporal consistency of neural patterns for positive, negative, and mixed states. The insula had unique and consistent neural signatures for univalent states, but not for mixed valence states. The anterior cingulate and ventral medial prefrontal cortex had consistent neural signatures for both univalent and mixed states. This study is the first to demonstrate that subjectively reported changes in feelings induced by naturalistic stimuli can be predicted from fMRI and the first to show direct evidence for a neurally consistent representation of mixed feelings.
Collapse
Affiliation(s)
- Anthony G Vaccaro
- Department of Psychology, Brain and Creativity Institute, University of Southern California, 3620 McClintock Avenue, Los Angeles, CA 90089, United States
| | - Helen Wu
- Department of Psychology, Brain and Creativity Institute, University of Southern California, 3620 McClintock Avenue, Los Angeles, CA 90089, United States
| | - Rishab Iyer
- Department of Psychology, Brain and Creativity Institute, University of Southern California, 3620 McClintock Avenue, Los Angeles, CA 90089, United States
| | - Shruti Shakthivel
- Department of Psychology, Brain and Creativity Institute, University of Southern California, 3620 McClintock Avenue, Los Angeles, CA 90089, United States
| | - Nina C Christie
- Department of Psychology, Brain and Creativity Institute, University of Southern California, 3620 McClintock Avenue, Los Angeles, CA 90089, United States
| | - Antonio Damasio
- Department of Psychology, Brain and Creativity Institute, University of Southern California, 3620 McClintock Avenue, Los Angeles, CA 90089, United States
| | - Jonas Kaplan
- Department of Psychology, Brain and Creativity Institute, University of Southern California, 3620 McClintock Avenue, Los Angeles, CA 90089, United States
| |
Collapse
|
5
|
Socada JL, Söderholm JJ, Rosenström T, Lahti J, Ekelund J, Isometsä ET. Affect dimensions and variability during major depressive episodes: Ecological momentary assessment of unipolar, bipolar, and borderline patients and healthy controls. J Psychiatr Res 2024; 170:408-416. [PMID: 38218014 DOI: 10.1016/j.jpsychires.2024.01.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Revised: 12/21/2023] [Accepted: 01/05/2024] [Indexed: 01/15/2024]
Abstract
Differentiating major depressive episodes (MDEs) of major depressive disorder (MDD), bipolar disorder (MDE/BD) and the MDEs comorbid with borderline personality disorder (MDE/BPD) is crucial for appropriate treatment, and knowledge of phenomenological differences may aid this. However, studies comparing affect experiences of these three patient groups and healthy subjects are scarce. In our study, participants (N = 114), including patients with MDD (n = 34), MDE/BD (n = 27), and MDE/BPD (n = 24), and healthy controls (HC, n = 29) responded to ecological momentary assessment (EMA) with ten circumplex model affect items ten times daily for seven days (7709 recordings). Explorative factor analysis resulted in two affect dimensions. The positive dimension included active, excited, cheerful (high arousal), and content (low arousal) affects, and the negative dimension irritated, angry, and nervous (high arousal) affects. Relative to HC, patients reported 3.5-fold negative affects (mean MDD 1.36 (SD 0.92), MDE/BD 1.43 (0.76), MDE/BPD 1.81 (0.95) vs. HC 0.44 (0.49) (p < 0.01)) but 0.5-fold positive affects (2.01 (0.90), 1.95 (0.89), 2.24 (1.03), vs. 3.2 (0.95), respectively (p < 0.01)). We used multilevel modelling. Negative-affect within-individual stability was lowest in MDE/BPD and highest in MDD. Negative affect predicted concurrent positive affect more in MDE/BPD than in MDD. Moderate size of subcohorts and no inpatients were limitations. Despite apparently similar MDEs, affective experiences may differ between BPD, BD, and MDD patients. Clinical subgroups of patients with depression may vary in affective instability and concurrent presence of negative and positive affects during depression.
Collapse
Affiliation(s)
- J Lumikukka Socada
- Department of Psychiatry, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
| | - John J Söderholm
- Department of Psychiatry, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
| | - Tom Rosenström
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
| | - Jari Lahti
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland; Folkhälsan Research Centre, Helsinki, Finland
| | - Jesper Ekelund
- Department of Psychiatry, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
| | - Erkki T Isometsä
- Department of Psychiatry, University of Helsinki and Helsinki University Hospital, Helsinki, Finland.
| |
Collapse
|
6
|
Oh VYS. Direct versus indirect measures of mixed emotions in predictive models: a comparison of predictive validity, multicollinearity, and the influence of confounding variables. Front Psychol 2023; 14:1231845. [PMID: 37671105 PMCID: PMC10475543 DOI: 10.3389/fpsyg.2023.1231845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Accepted: 08/09/2023] [Indexed: 09/07/2023] Open
Abstract
Mixed emotions have been assessed using both direct measures that utilize self-report questionnaires as well as indirect measures that are computed from scores of positive and negative emotions. This study provides a pre-registered methodological examination on the use of direct and indirect measures of mixed emotions in predictive models. Two samples (N = 749) were collected, and path analyses were performed to compare direct measures and indirect measures in predicting psychological conflict, receptivity, and well-being, controlling for demographics, positive emotions, and negative emotions. We also tested whether trait dialecticism, need for cognition, social desirability, or acquiescence could account for these associations. In both samples, results suggest that indirect measures may be more susceptible to multicollinearity when controlling for positive and negative emotions. Specifically, variance inflation factors (VIF) were consistently higher for indirect measures calculated using the minimum index (MIN; VIFSample-1 = 3.53; VIFSample-2 = 9.46) than direct measures (VIFSample-1 = 2.52; VIFSample-2 = 1.68). Direct measures remained consistently associated with increased conflict and reduced coherence upon controlling for positive and negative emotions, while indirect measures remained consistently associated only with increased conflict. We found little evidence that response biases explained associations between direct measures or indirect measures with each of the outcomes. Specifically, associations between mixed emotions with psychological conflict, receptivity, and well-being largely remained unchanged in models that controlled for trait dialecticism, need for cognition, social desirability, or acquiescence. Implications and recommendations based on our findings are discussed.
Collapse
Affiliation(s)
- Vincent Y. S. Oh
- Singapore University of Social Sciences, School of Humanities and Behavioural Sciences, Singapore, Singapore
| |
Collapse
|