FACSGen 2.0 animation software: generating three-dimensional FACS-valid facial expressions for emotion research

In this article, we present FACSGen 2.0, new animation software for creating static and dynamic three-dimensional facial expressions on the basis of the Facial Action Coding System (FACS). FACSGen permits total control over the action units (AUs), which can be animated at all levels of intensity and applied alone or in combination to an infinite number of faces. In two studies, we tested the validity of the software for the AU appearance defined in the FACS manual and the conveyed emotionality of FACSGen expressions. In Experiment 1, four FACS-certified coders evaluated the complete set of 35 single AUs and 54 AU combinations for AU presence or absence, appearance quality, intensity, and asymmetry. In Experiment 2, lay participants performed a recognition task on emotional expressions created with FACSGen software and rated the similarity of expressions displayed by human and FACSGen faces. Results showed good to excellent classification levels for all AUs by the four FACS coders, suggesting that the AUs are valid exemplars of FACS specifications. Lay participants' recognition rates for nine emotions were high, and comparisons of human and FACSGen expressions were very similar. The findings demonstrate the effectiveness of the software in producing reliable and emotionally valid expressions, and suggest its application in numerous scientific areas, including perception, emotion, and clinical and neuroscience research.

Affect bursts: dynamic patterns of facial expression

Affect bursts consist of spontaneous and short emotional expressions in which facial, vocal, and gestural components are highly synchronized. Although the vocal characteristics have been examined in several recent studies, the facial modality remains largely unexplored. This study investigated the facial correlates of affect bursts that expressed five different emotions: anger, fear, sadness, joy, and relief. Detailed analysis of 59 facial actions with the Facial Action Coding System revealed a reasonable degree of emotion differentiation for individual action units (AUs). However, less convergence was shown for specific AU combinations for a limited number of prototypes. Moreover, expression of facial actions peaked in a cumulative-sequential fashion with significant differences in their sequential appearance between emotions. When testing for the classification of facial expressions within a dimensional approach, facial actions differed significantly as a function of the valence and arousal level of the five emotions, thereby allowing further distinction between joy and relief. The findings cast doubt on the existence of fixed patterns of facial responses for each emotion, resulting in unique facial prototypes. Rather, the results suggest that each emotion can be portrayed by several different expressions that share multiple facial actions.

Advocating a Componential Appraisal Model to Guide Emotion Recognition

Most models of automatic emotion recognition use a discrete perspective and a black-box approach, i.e., they output an emotion label chosen from a limited pool of candidate terms, on the basis of purely statistical methods. Although these models are successful in emotion classification, a number of practical and theoretical drawbacks limit the range of possible applications. In this paper, the authors suggest the adoption of an appraisal perspective in modeling emotion recognition. The authors propose to use appraisals as an intermediate layer between expressive features (input) and emotion labeling (output). The model would then be made of two parts: first, expressive features would be used to estimate appraisals; second, resulting appraisals would be used to predict an emotion label. While the second part of the model has already been the object of several studies, the first is unexplored. The authors argue that this model should be built on the basis of both theoretical predictions and empirical results about the link between specific appraisals and expressive features. For this purpose, the authors suggest to use the component process model of emotion, which includes detailed predictions of efferent effects of appraisals on facial expression, voice, and body movements.

Introducing the Geneva Multimodal expression corpus for experimental research on emotion perception

Research on the perception of emotional expressions in faces and voices is exploding in psychology, the neurosciences, and affective computing. This article provides an overview of some of the major emotion expression (EE) corpora currently available for empirical research and introduces a new, dynamic, multimodal corpus of emotion expressions, the Geneva Multimodal Emotion Portrayals Core Set (GEMEP-CS). The design features of the corpus are outlined and justified, and detailed validation data for the core set selection are presented and discussed. Finally, an associated database with microcoded facial, vocal, and body action elements, as well as observer ratings, is introduced.

Subtly Different Positive Emotions Can Be Distinguished by Their Facial Expressions

Positive emotions are crucial to social relationships and social interaction. Although smiling is a frequently studied facial action, investigations of positive emotional expressions are underrepresented in the literature. This may be partly because of the assumption that all positive emotions share the smile as a common signal but lack specific facial configurations. The present study investigated prototypical expressions of four positive emotions—interest, pride, pleasure, and joy. The Facial Action Coding System was used to microcode facial expression of representative samples of these emotions taken from the Geneva Multimodal Emotion Portrayal corpus. The data showed that the frequency and duration of several action units differed between emotions, indicating that actors did not use the same pattern of expression to encode them. The authors argue that an appraisal perspective is suitable to describe how subtly differentiated positive emotional states differ in their prototypical facial expressions.

Psychophysics of emotion: the QUEST for emotional attention

To investigate the mechanisms involved in automatic processing of facial expressions, we used the QUEST procedure to measure the display durations needed to make a gender decision on emotional faces portraying fearful, happy, or neutral facial expressions. In line with predictions of appraisal theories of emotion, our results showed greater processing priority of emotional stimuli regardless of their valence. Whereas all experimental conditions led to an averaged threshold of about 50 ms, fearful and happy facial expressions led to significantly less variability in the responses than neutral faces. Results suggest that attention may have been automatically drawn by the emotion portrayed by face targets, yielding more informative perceptions and less variable responses. The temporal resolution of the perceptual system (expressed by the thresholds) and the processing priority of the stimuli (expressed by the variability in the responses) may influence subjective and objective measures of awareness, respectively.

Emotions are emergent processes: they require a dynamic computational architecture

Emotion is a cultural and psychobiological adaptation mechanism which allows each individual to react flexibly and dynamically to environmental contingencies. From this claim flows a description of the elements theoretically needed to construct a virtual agent with the ability to display human-like emotions and to respond appropriately to human emotional expression. This article offers a brief survey of the desirable features of emotion theories that make them ideal blueprints for agent models. In particular, the component process model of emotion is described, a theory which postulates emotion-antecedent appraisal on different levels of processing that drive response system patterning predictions. In conclusion, investing seriously in emergent computational modelling of emotion using a nonlinear dynamic systems approach is suggested.

The Case of the Disappearing Intentional Object: Constraints on a Definition of Emotion

Taking our lead from Solomon’s emphasis on the importance of the intentional object of emotion, we review the history of repeated attempts to make this object disappear. We adduce evidence suggesting that in the case of James and Schachter, the intentional object got lost unintentionally. By contrast, modern constructivists (in particular Barrett) seem quite determined to deny the centrality of the intentional object in accounting for the occurrence of emotions. Griffiths, however, downplays the role objects have in emotion noting that these do not qualify as intentional. We argue that these disappearing acts, deliberate or not, generate fruitless debate and add little to the advancement of our understanding of emotion as an adaptive mechanism to cope with events that are relevant to an organism’s life.

Cross-modal Emotional Attention: Emotional Voices Modulate Early Stages of Visual Processing

Emotional attention, the boosting of the processing of emotionally relevant stimuli, has, up to now, mainly been investigated within a sensory modality, for instance, by using emotional pictures to modulate visual attention. In real-life environments, however, humans typically encounter simultaneous input to several different senses, such as vision and audition. As multiple signals entering different channels might originate from a common, emotionally relevant source, the prioritization of emotional stimuli should be able to operate across modalities. In this study, we explored cross-modal emotional attention. Spatially localized utterances with emotional and neutral prosody served as cues for a visually presented target in a cross-modal dot-probe task. Participants were faster to respond to targets that appeared at the spatial location of emotional compared to neutral prosody. Event-related brain potentials revealed emotional modulation of early visual target processing at the level of the P1 component, with neural sources in the striate visual cortex being more active for targets that appeared at the spatial location of emotional compared to neutral prosody. These effects were not found using synthesized control sounds matched for mean fundamental frequency and amplitude envelope. These results show that emotional attention can operate across sensory modalities by boosting early sensory stages of processing, thus facilitating the multimodal assessment of emotionally relevant stimuli in the environment.

Sequential unfolding of novelty and pleasantness appraisals of odors: evidence from facial electromyography and autonomic reactions

We investigated the effects of odors on appraisal processes and consequent emotional responses. The main goal was to test whether an odor is detected as novel or familiar before it is evaluated as pleasant or unpleasant. Participants performed a recognition task in which they were presented with pairs of unpleasant or pleasant odors (sample and target odors). Within a pair, the sample and target were either identical or different to assess participants' novelty detection; unpleasant and pleasant target odors were contrasted to examine participants' appraisal of intrinsic pleasantness. We measured facial expressions using electromyography and physiological reactions using electrocardiogram and electrodermal activity in response to odors. The earliest effects on facial muscles and heart rate occurred in response to novelty detection. Later effects on facial muscles and heart rate were related to pleasantness evaluation. This study is the first to demonstrate the existence of a sequence of appraisal checks for odors eliciting emotional reaction.

Using a Probe Detection Task to Assess the Timing of Intrinsic Pleasantness Appraisals

The occurrence and timing of emotion-antecedent appraisal checks are difficult to assess. We report an attempt to estimate the time window of the intrinsic pleasantness check using a dual-task probe paradigm. In three experiments, participants viewed negative and positive pictures. Their other task was speeded response on a probe superimposed on the pictures with different stimulus onset asynchronies (SOAs). Longer probe-reaction times were observed for negative than for positive pictures. This effect appeared at SOA 300 ms or 350 ms, suggesting that the intrinsic pleasantness appraisal check yields a differential behavioral outcome around 300 ms after stimulus onset, and seems to continue unless attention to picture content is inhibited. This paradigm might be successfully used for the mental chronography of appraisal processes.

Beyond fear: rapid spatial orienting toward positive emotional stimuli

There is much empirical evidence for modulation of attention by negative -- particularly fear-relevant -- emotional stimuli. This modulation is often explained in terms of a fear module. Appraisal theories of emotion posit a more general mechanism, predicting attention capture by stimuli that are relevant for the needs and goals of the organism, regardless of valence. To examine the brain-activation patterns underlying attentional modulation, we recorded event-related potentials from 20 subjects performing a dot-probe task in which the cues were fear-inducing and nurturance-inducing stimuli (i.e., anger faces and baby faces). Highly similar validity modulation was found for the P1 time-locked to target onset, indicating early attentional capture by both positive and negative emotional stimuli. Topographic segmentation analysis and source localization indicate that the same amplification process is involved whether attention orienting is triggered by negative, fear-relevant stimuli or positive, nurturance-relevant stimuli. These results confirm that biological relevance, and not exclusively fear, produces an automatic spatial orienting toward the location of a stimulus.