The emotion-facial expression link: evidence from human and automatic expression recognition

While it has been taken for granted in the development of several automatic facial expression recognition tools, the question of the coherence between subjective feelings and facial expressions is still a subject of debate. On one hand, the "Basic Emotion View" conceives emotions as genetically hardwired and, therefore, being genuinely displayed through facial expressions. Consequently, emotion recognition is perceiver independent. On the other hand, the constructivist approach conceives emotions as socially constructed, the emotional meaning of a facial expression being inferred by the perceiver. Hence, emotion recognition is perceiver dependent. In order (1) to evaluate the coherence between the subjective feeling of emotions and their spontaneous facial displays, and (2) to compare the recognition of such displays by human perceivers and by an automatic facial expression classifier, 232 videos of expressers recruited to carry out an emotion elicitation task were annotated by 1383 human perceivers as well as by Affdex, an automatic classifier. Results show a weak consistency between self-reported emotional states by expressers and their facial emotional displays. They also show low accuracy both of human perceivers and of the automatic classifier to infer the subjective feeling from the spontaneous facial expressions displayed by expressers. However, the results are more in favor of a perceiver-dependent view. Based on these results, the hypothesis of genetically hardwired emotion genuinely displayed is difficult to support, whereas the idea of emotion and facial expression as being socially constructed appears to be more likely. Accordingly, automatic emotion recognition tools based on facial expressions should be questioned.

The Influence of the Opposite Sex on Hypothetical Aggressive Inclinations

Previous research has supported the hypothesis that the presence of a potential mate influences social behavior; however, there is a paucity of work that has extended this to the study of aggression. Thus, the current research had participants (N = 1,035) view a picture of an attractive or unattractive member of the opposite sex. Participants then imagined themselves in front of the imagined person in a social setting in which they were provoked or not before completing an assessment of aggressive intentions. Results from the 2 (sex of participant) × 2 (attractiveness of the picture) × 2 (provocation or not) analysis of variance showed that males viewing an image of an attractive female had higher aggressive intentions when provoked than males viewing a nonattractive female; however, these effects were not found for female participants.

Does Alcohol Catch the Eye? Investigating Young Adults' Attention to Alcohol Consumption

Many studies on young adults' motivations for drinking overlook the symbolic aspects of alcohol use. However, research indicates that young adults' alcohol consumption is also driven by signaling motivations. Although the interest of a receiver is a necessary prerequisite of a signal, no previous studies have verified whether drinking behavior indeed attracts young adults' attention. Therefore, we conducted two studies. A two-part eye-tracking study ( N1 = 135, N2 = 140) showed that both young men and young women pay special visual attention to male and female drinking behavior. Additionally, a recall experiment ( N = 321) confirmed that observed male and female drinking is better remembered than observed nonsignaling, functional behavior. Moreover, alcoholic beverages also receive special attention, as they were recalled better than other functional products, and also nonalcoholic drinks similar in color and shape. In summary, the experiments clearly showed that male and female drinking behavior can be used as a signal, as both behaviors clearly function as an attention-attracting cue. Additionally, as alcoholic beverages draw more attention than nonalcoholic drinks, this attention is clearly linked to the alcohol element of the drinking behavior.

A User Perception--Based Approach to Create Smiling Embodied Conversational Agents

In order to improve the social capabilities of embodied conversational agents, we propose a computational model to enable agents to automatically select and display appropriate smiling behavior during human--machine interaction. A smile may convey different communicative intentions depending on subtle characteristics of the facial expression and contextual cues. To construct such a model, as a first step, we explore the morphological and dynamic characteristics of different types of smiles (polite, amused, and embarrassed smiles) that an embodied conversational agent may display. The resulting lexicon of smiles is based on a corpus of virtual agents’ smiles directly created by users and analyzed through a machine-learning technique. Moreover, during an interaction, a smiling expression impacts on the observer’s perception of the interpersonal stance of the speaker. As a second step, we propose a probabilistic model to automatically compute the user’s potential perception of the embodied conversational agent’s social stance depending on its smiling behavior and on its physical appearance. This model, based on a corpus of users’ perceptions of smiling and nonsmiling virtual agents, enables a virtual agent to determine the appropriate smiling behavior to adopt given the interpersonal stance it wants to express. An experiment using real human--virtual agent interaction provided some validation of the proposed model.

Mind and creativity: Insights from rasa theory with special focus on sahṛdaya (the appreciative critic)

This article introduces a non-Western approach to mind and creativity, an approach inspired by the notion of sahṛdaya (the appreciative critic) found in the Indian tradition of aesthetics known as rasa. The rasa tradition is preoccupied with a virtual reality governed by the principle of non-duality ( advaita), which we illuminate with another virtual reality—as formulated by the mathematical principle of symmetry in quantum mechanics. Far reaching implications of the principle of symmetry/non-duality for psychology in general, and psychology of creativity in particular, are explored.

The Body That Speaks: Recombining Bodies and Speech Sources in Unscripted Face-to-Face Communication

This article examines advances in research methods that enable experimental substitution of the speaking body in unscripted face-to-face communication. A taxonomy of six hybrid social agents is presented by combining three types of bodies (mechanical, virtual, and human) with either an artificial or human speech source. Our contribution is to introduce and explore the significance of two particular hybrids: (1) the cyranoid method that enables humans to converse face-to-face through the medium of another person's body, and (2) the echoborg method that enables artificial intelligence to converse face-to-face through the medium of a human body. These two methods are distinct in being able to parse the unique influence of the human body when combined with various speech sources. We also introduce a new framework for conceptualizing the body's role in communication, distinguishing three levels: self's perspective on the body, other's perspective on the body, and self's perspective of other's perspective on the body. Within each level the cyranoid and echoborg methodologies make important research questions tractable. By conceptualizing and synthesizing these methods, we outline a novel paradigm of research on the role of the body in unscripted face-to-face communication.

Mating and Memory : Can Mating Cues Enhance Cognitive Performance?

The literature on sexual selection and the social brain hypothesis suggest that human cognition and communication evolved, in part, for the purpose of displaying desirable cognitive abilities to potential mates. An evolutionary approach to social cognition implies that proximate mating motives may lead people to display desirable mental traits. In signaling such traits, one can increase the likelihood of attracting a potential mate. Two experiments demonstrated that exposure to mating cues-highly attractive opposite-sex faces-led people to display enhancements in declarative memory-a process underlying a variety of abilities such as resource acquisition, intelligence, and creativity. Experiment 1 showed that men (but not women) displayed enhanced memory for details of a story that was presented during exposure to highly attractive opposite-sex faces. Experiment 2 demonstrated that heightened displays of declarative memory reflect an enhancement in retrieval rather than in encoding. Findings contribute to the literatures on human mating and cognitive performance and provide novel insight into links between social processes and basic cognition.

Gender Differences in the Perceptions of Genuine and Simulated Laughter and Amused Facial Expressions

This article addresses gender differences in laughter and smiling from an evolutionary perspective. Laughter and smiling can be responses to successful display behavior or signals of affiliation amongst conversational partners—differing social and evolutionary agendas mean there are different motivations when interpreting these signals. Two experiments assess perceptions of genuine and simulated male and female laughter and amusement social signals. Results show male simulation can always be distinguished. Female simulation is more complicated as males seem to distinguish cues of simulation yet judge simulated signals to be genuine. Females judge other female’s genuine signals to have higher levels of simulation. Results highlight the importance of laughter and smiling in human interactions, use of dynamic stimuli, and using multiple methodologies to assess perception.

Expressive body movement responses to music are coherent, consistent, and low dimensional

Embodied music cognition stresses the role of the human body as mediator for the encoding and decoding of musical expression. In this paper, we set up a low dimensional functional model that accounts for 70% of the variability in the expressive body movement responses to music. With the functional principal component analysis, we modeled individual body movements as a linear combination of a group average and a number of eigenfunctions. The group average and the eigenfunctions are common to all subjects and make up what we call the commonalities. An individual performance is then characterized by a set of scores (the individualities), one score per eigenfunction. The model is based on experimental data which finds high levels of coherence/consistency between participants when grouped according to musical education. This shows an ontogenetic effect. Participants without formal musical education focus on the torso for the expression of basic musical structure (tempo). Musically trained participants decode additional structural elements in the music and focus on body parts having more degrees of freedom (such as the hands). Our results confirm earlier studies that different body parts move differently along with the music.