1
|
Du B, Jia S, Zhou X, Zhang M, He W. The priming effect of emotional words on body expressions: Two ERP studies. Int J Psychophysiol 2024; 202:112370. [PMID: 38802049 DOI: 10.1016/j.ijpsycho.2024.112370] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 05/21/2024] [Accepted: 05/21/2024] [Indexed: 05/29/2024]
Abstract
The impact of emotional words on the recognition of body expression and the underlying neurodynamic mechanisms remain poorly understood. This study used a classic supraliminal priming paradigm and event related potential (ERP) to investigate the effect of emotion-label words (experiment 1) and emotional verbs (experiment 2) on the recognition of body expressions. The behavioral results revealed that individuals exhibited a higher accuracy in recognizing happy expressions when presented with a happy-label word condition, in contrast to neutral expressions. Furthermore, it was observed that the accuracy of recognizing happy body expressions was reduced when preceded by angry verb priming, compared to happy and neutral priming conditions. Conversely, the accuracy of recognizing angry body expressions was higher in response to angry verb priming than happy and neutral primings. The ERP results showed that, in the recognition of happy body expressions, the P300 amplitude elicited by angry-label words was more positive, while a congruent verb-expression condition elicited more positive P300 amplitude than an incongruent condition in the left hemisphere and midline. However, in the recognition of angry body expressions, the N400 amplitude elicited by a congruent verb-expression condition was smaller than that elicited by an incongruent condition. These results suggest that both abstract emotion-label words and specific emotional verbs influence the recognition of body expressions. In addition, integrating happy semantic context and body expression might occur at the P300 stage, whereas integrating angry semantic context and body expression might occur at the N400 stage. These findings present novel evidence regarding the criticality of emotional context in the recognition of emotions.
Collapse
Affiliation(s)
- Bixuan Du
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian 116029, China; Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian 116029, China; Hangzhou Fuyang Chunjiang Central Elementary School, Hangzhou 311421, China
| | - Shuxin Jia
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian 116029, China; Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian 116029, China
| | - Xing Zhou
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian 116029, China; Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian 116029, China
| | - Mingming Zhang
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian 116029, China; Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian 116029, China.
| | - Weiqi He
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian 116029, China; Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian 116029, China.
| |
Collapse
|
2
|
Zhang M, Cai Z, Pan L, Hong F, Guo X, Yang L, Liu Z. MotionDiffuse: Text-Driven Human Motion Generation With Diffusion Model. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024; 46:4115-4128. [PMID: 38285589 DOI: 10.1109/tpami.2024.3355414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/31/2024]
Abstract
Human motion modeling is important for many modern graphics applications, which typically require professional skills. In order to remove the skill barriers for laymen, recent motion generation methods can directly generate human motions conditioned on natural languages. However, it remains challenging to achieve diverse and fine-grained motion generation with various text inputs. To address this problem, we propose MotionDiffuse, one of the first diffusion model-based text-driven motion generation frameworks, which demonstrates several desired properties over existing methods. 1) Probabilistic Mapping. Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected. 2) Realistic Synthesis. MotionDiffuse excels at modeling complicated data distribution and generating vivid motion sequences. 3) Multi-Level Manipulation. MotionDiffuse responds to fine-grained instructions on body parts, and arbitrary-length motion synthesis with time-varied text prompts. Our experiments show MotionDiffuse outperforms existing SoTA methods by convincing margins on text-driven motion generation and action-conditioned motion generation. A qualitative analysis further demonstrates MotionDiffuse's controllability for comprehensive motion generation.
Collapse
|
3
|
Aly L, Godinho L, Bota P, Bernardes G, da Silva HP. Acting Emotions: a comprehensive dataset of elicited emotions. Sci Data 2024; 11:147. [PMID: 38296997 PMCID: PMC10831041 DOI: 10.1038/s41597-024-02957-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Accepted: 01/12/2024] [Indexed: 02/02/2024] Open
Abstract
Emotions encompass physiological systems that can be assessed through biosignals like electromyography and electrocardiography. Prior investigations in emotion recognition have primarily focused on general population samples, overlooking the specific context of theatre actors who possess exceptional abilities in conveying emotions to an audience, namely acting emotions. We conducted a study involving 11 professional actors to collect physiological data for acting emotions to investigate the correlation between biosignals and emotion expression. Our contribution is the DECEiVeR (DatasEt aCting Emotions Valence aRousal) dataset, a comprehensive collection of various physiological recordings meticulously curated to facilitate the recognition of a set of five emotions. Moreover, we conduct a preliminary analysis on modeling the recognition of acting emotions from raw, low- and mid-level temporal and spectral data and the reliability of physiological data across time. Our dataset aims to leverage a deeper understanding of the intricate interplay between biosignals and emotional expression. It provides valuable insights into acting emotion recognition and affective computing by exposing the degree to which biosignals capture emotions elicited from inner stimuli.
Collapse
Affiliation(s)
- Luís Aly
- Faculty of Engineering, Department of Informatics Engineering, University of Porto, Porto, 4200-465, Portugal.
- INESC-TEC, Telecommunications and Multimedia, Porto, 4200-465, Portugal.
| | - Leonor Godinho
- Instituto de Telecomunicações, Instituto Superior Técnico, Department of Bioengineering, Lisbon, 1049-001, Portugal
| | - Patricia Bota
- Instituto de Telecomunicações, Instituto Superior Técnico, Department of Bioengineering, Lisbon, 1049-001, Portugal
| | - Gilberto Bernardes
- Faculty of Engineering, Department of Informatics Engineering, University of Porto, Porto, 4200-465, Portugal
- INESC-TEC, Telecommunications and Multimedia, Porto, 4200-465, Portugal
| | - Hugo Plácido da Silva
- Instituto de Telecomunicações, Instituto Superior Técnico, Department of Bioengineering, Lisbon, 1049-001, Portugal
| |
Collapse
|
4
|
Zhang M, Zhou Y, Xu X, Ren Z, Zhang Y, Liu S, Luo W. Multi-view emotional expressions dataset using 2D pose estimation. Sci Data 2023; 10:649. [PMID: 37739952 PMCID: PMC10516935 DOI: 10.1038/s41597-023-02551-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Accepted: 09/07/2023] [Indexed: 09/24/2023] Open
Abstract
Human body expressions convey emotional shifts and intentions of action and, in some cases, are even more effective than other emotion models. Despite many datasets of body expressions incorporating motion capture available, there is a lack of more widely distributed datasets regarding naturalized body expressions based on the 2D video. In this paper, therefore, we report the multi-view emotional expressions dataset (MEED) using 2D pose estimation. Twenty-two actors presented six emotional (anger, disgust, fear, happiness, sadness, surprise) and neutral body movements from three viewpoints (left, front, right). A total of 4102 videos were captured. The MEED consists of the corresponding pose estimation results (i.e., 397,809 PNG files and 397,809 JSON files). The size of MEED exceeds 150 GB. We believe this dataset will benefit the research in various fields, including affective computing, human-computer interaction, social neuroscience, and psychiatry.
Collapse
Affiliation(s)
- Mingming Zhang
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, Liaoning, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Yanan Zhou
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, Liaoning, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Xinye Xu
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, Liaoning, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Ziwei Ren
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, Liaoning, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Yihan Zhang
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, Liaoning, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Shenglan Liu
- School of Innovation and Entrepreneurship, Dalian University of Technology, Dalian, 116024, Liaoning, China.
- Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, 116024, Liaoning, China.
| | - Wenbo Luo
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, Liaoning, China.
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China.
| |
Collapse
|
5
|
Zhang M, Yu L, Zhang K, Du B, Zhan B, Jia S, Chen S, Han F, Li Y, Liu S, Yi X, Liu S, Luo W. Construction and validation of the Dalian emotional movement open-source set (DEMOS). Behav Res Methods 2023; 55:2353-2366. [PMID: 35931937 DOI: 10.3758/s13428-022-01887-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/24/2022] [Indexed: 11/08/2022]
Abstract
Human body movements are important for emotion recognition and social communication and have received extensive attention from researchers. In this field, emotional biological motion stimuli, as depicted by point-light displays, are widely used. However, the number of stimuli in the existing material library is small, and there is a lack of standardized indicators, which subsequently limits experimental design and conduction. Therefore, based on our prior kinematic dataset, we constructed the Dalian Emotional Movement Open-source Set (DEMOS) using computational modeling. The DEMOS has three views (i.e., frontal 0°, left 45°, and left 90°) and in total comprises 2664 high-quality videos of emotional biological motion, each displaying happiness, sadness, anger, fear, disgust, and neutral. All stimuli were validated in terms of recognition accuracy, emotional intensity, and subjective movement. The objective movement for each expression was also calculated. The DEMOS can be downloaded for free from https://osf.io/83fst/ . To our knowledge, this is the largest multi-view emotional biological motion set based on the whole body. The DEMOS can be applied in many fields, including affective computing, social cognition, and psychiatry.
Collapse
Affiliation(s)
- Mingming Zhang
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Lu Yu
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Keye Zhang
- School of Social and Behavioral Sciences, Nanjing University, Nanjing, 210023, China
| | - Bixuan Du
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Bin Zhan
- State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Shuxin Jia
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Shaohua Chen
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Fengxu Han
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Yiwen Li
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Shuaicheng Liu
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Xi Yi
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China
| | - Shenglan Liu
- School of Innovation and Entrepreneurship, Dalian University of Technology, Dalian, 116024, China.
- Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, 116024, China.
| | - Wenbo Luo
- Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian, 116029, China.
- Key Laboratory of Brain and Cognitive Neuroscience, Liaoning Province, Dalian, 116029, China.
| |
Collapse
|
6
|
Dong A, Wang F, Shuai Z, Zhang K, Qian D, Tian Y. A new kinematic dataset of lower limbs action for balance testing. Sci Data 2023; 10:209. [PMID: 37059747 PMCID: PMC10104813 DOI: 10.1038/s41597-023-02105-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Accepted: 03/25/2023] [Indexed: 04/16/2023] Open
Abstract
Balance is a common performance but nevertheless an essential part of performance analysis investigations in ski. Many skier pay attention to the training of balance ability in training. Inertial Measurement Unit, as a kind of Multiplex-type human motion capture system, is widely used because of its humanized human-computer interaction design, low energy consumption and more freedom provided by the environment. The purpose of this research is to use sensor to establish a kinematics dataset of balance test tasks extracted from skis to help quantify skier' balance ability. Perception Neuron Studio motion capture device is used in present. The dataset contains a total of 20 participants' data (half male) of the motion and sensor data, which is collected at a 100 Hz sampling frequency. To our knowledge, this dataset is the only one that uses a BOSU ball in the balance test. We hope that this dataset will contribute to multiple fields of cross-technology integration in physical training and functional testing, including big-data analysis, sports equipment design and sports biomechanical analysis.
Collapse
Affiliation(s)
- Anqi Dong
- Beijing Sport University, Beijing, China
| | - Fei Wang
- Beijing Sport University, Beijing, China
| | | | | | | | | |
Collapse
|
7
|
Avola D, Cascio M, Cinque L, Fagioli A, Foresti GL. Affective Action and Interaction Recognition by Multi-view Representation Learning from Handcrafted Low-level Skeleton Features. Int J Neural Syst 2022; 32:2250040. [DOI: 10.1142/s012906572250040x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
8
|
Reliability and generalization of gait biometrics using 3D inertial sensor data and 3D optical system trajectories. Sci Rep 2022; 12:8414. [PMID: 35589793 PMCID: PMC9120026 DOI: 10.1038/s41598-022-12452-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2021] [Accepted: 05/04/2022] [Indexed: 11/25/2022] Open
Abstract
Particularities in the individuals’ style of walking have been explored for at least three decades as a biometric trait, empowering the automatic gait recognition field. Whereas gait recognition works usually focus on improving end-to-end performance measures, this work aims at understanding which individuals’ traces are more relevant to improve subjects’ separability. For such, a manifold projection technique and a multi-sensor gait dataset were adopted to investigate the impact of each data source characteristics on this separability. Assessments have shown it is hard to distinguish individuals based only on their walking patterns in a subject-based identification scenario. In this setup, the subjects’ separability is more related to their physical characteristics than their movements related to gait cycles and biomechanical events. However, this study’s results also points to the feasibility of learning identity characteristics from individuals’ walking patterns learned from similarities and differences between subjects in a verification setup. The explorations concluded that periodic components occurring in frequencies between 6 and 10 Hz are more significant for learning these patterns than events and other biomechanical movements related to the gait cycle, as usually explored in the literature.
Collapse
|