Brain Networks, Emotion Components, and Appraised Relevance

Modeling emotion processes remains a conceptual and methodological challenge in affective sciences. In responding to the other target articles in this special section on “Emotion and the Brain” and the comments on our article, we address the issue of potentially separate brain networks subserving the functions of the different emotion components. In particular, we discuss the suggested role of component synchronization in producing information integration for the dynamic emergence of a coherent emotion process, as well as the links between incentive salience (“wanting”) and concern-relevance in the elicitation of emotion.

An Appraisal-Driven Componential Approach to the Emotional Brain

This article suggests that methodological and conceptual advancements in affective sciences militate in favor of adopting an appraisal-driven componential approach to further investigate the emotional brain. Here we propose to operationalize this approach by distinguishing five functional networks of the emotional brain: (a) the elicitation network, (b) the expression network, (c) the autonomic reaction network, (d) the action tendency network, and (e) the feeling network, and discuss these networks in the context of the affective neuroscience literature. We also propose that further investigating the “appraising brain” is the royal road to better understand the elicitation network, and may be key to revealing the neural causal mechanisms underlying the emotion process as a whole.

The semantic structure of emotion words across languages is consistent with componential appraisal models of emotion

Appraisal theories of emotion, and particularly the Component Process Model, claim that the different components of the emotion process (action tendencies, physiological reactions, expressions, and feeling experiences) are essentially driven by the results of cognitive appraisals and that the feeling component constitutes a central integration and representation of these processes. Given the complexity of the proposed architecture, comprehensive experimental tests of these predictions are difficult to perform and to date are lacking. Encouraged by the "lexical sedimentation" hypothesis, here we propose an indirect examination of the compatibility of the theoretical assumptions with the semantic structure of a set of major emotion words as measured in a cross-language and cross-cultural study. Specifically, we performed a secondary analysis of the large-scale data set with ratings of affective features covering all components of the emotion process for 24 emotion words in 27 countries, constituting profiles of emotion-specific appraisals, action tendencies, physiological reactions, expressions, and feeling experiences. The results of a series of hierarchical regression analyses to examine the prediction of the theoretical model are highly consistent with the claim that appraisal patterns determine the structure of the response components, which in turn predict central dimensions of the feeling component.

Comment: Comorbidity Between Mental and Somatic Pathologies: Deficits in Emotional Competence as Health Risk Factors

I strongly endorse many of the suggestions made by the authors of the extremely useful reviews in this issue. In particular, the need to identify the complex causal mechanisms underlying the major health risk factors requires urgent attention of the research community. I suggest considering the important role of emotional disturbances as contributors to health risks given the empirically established comorbidity between mental and somatic illness. Better knowledge of these mechanisms is an essential prerequisite to develop tailored personalized prevention and intervention programs, including reliable and valid assessment of deficits.

Evidence of emotion-antecedent appraisal checks in electroencephalography and facial electromyography

In the present study, we applied Machine Learning (ML) methods to identify psychobiological markers of cognitive processes involved in the process of emotion elicitation as postulated by the Component Process Model (CPM). In particular, we focused on the automatic detection of five appraisal checks—novelty, intrinsic pleasantness, goal conduciveness, control, and power—in electroencephalography (EEG) and facial electromyography (EMG) signals. We also evaluated the effects on classification accuracy of averaging the raw physiological signals over different numbers of trials, and whether the use of minimal sets of EEG channels localized over specific scalp regions of interest are sufficient to discriminate between appraisal checks. We demonstrated the effectiveness of our approach on two data sets obtained from previous studies. Our results show that novelty and power appraisal checks can be consistently detected in EEG signals above chance level (binary tasks). For novelty, the best classification performance in terms of accuracy was achieved using features extracted from the whole scalp, and by averaging across 20 individual trials in the same experimental condition (UAR = 83.5 ± 4.2; N = 25). For power, the best performance was obtained by using the signals from four pre-selected EEG channels averaged across all trials available for each participant (UAR = 70.6 ± 5.3; N = 24). Together, our results indicate that accurate classification can be achieved with a relatively small number of trials and channels, but that averaging across a larger number of individual trials is beneficial for the classification for both appraisal checks. We were not able to detect any evidence of the appraisal checks under study in the EMG data. The proposed methodology is a promising tool for the study of the psychophysiological mechanisms underlying emotional episodes, and their application to the development of computerized tools (e.g., Brain-Computer Interface) for the study of cognitive processes involved in emotions.

The nomological network of emotion knowledge and emotion understanding in adults: evidence from two new performance-based tests

Emotion understanding, which can broadly be defined as expertise in the meaning of emotion, is a core component of emotional intelligence and facilitates better intra- and interpersonal outcomes. However, to date only very few standard tests to measure emotion understanding in healthy adults exist. Here, we present two new performance-based tests that were developed and are scored based on componential emotion theory and large-scale cross-cultural empirical findings. These instruments intend to measure facets of emotion understanding that are not included in existing tests. The first test (Geneva EMOtion Knowledge test - Blends; GEMOK-Blends) measures the ability to understand and label emotional experiences of a target person from a description of emotion features covering five emotion components (appraisal, feeling, action tendencies, expression, and physiology) embedded in a written vignette. The second test (GEMOK-Features) measures semantic knowledge about which features from each component are characteristic of emotion episodes described by a specific emotion label. In four studies, we found evidence for the good internal consistency and construct validity of these tests. Both tests were positively correlated with other emotional abilities and cognitive ability and showed meaningful associations with a variety of personality and demographic variables.

The expression of emotion in the singing voice: Acoustic patterns in vocal performance

There has been little research on the acoustic correlates of emotional expression in the singing voice. In this study, two pertinent questions are addressed: How does a singer's emotional interpretation of a musical piece affect acoustic parameters in the sung vocalizations? Are these patterns specific enough to allow statistical discrimination of the intended expressive targets? Eight professional opera singers were asked to sing the musical scale upwards and downwards (using meaningless content) to express different emotions, as if on stage. The studio recordings were acoustically analyzed with a standard set of parameters. The results show robust vocal signatures for the emotions studied. Overall, there is a major contrast between sadness and tenderness on the one hand, and anger, joy, and pride on the other. This is based on low vs high levels on the components of loudness, vocal dynamics, high perturbation variation, and a tendency for high low-frequency energy. This pattern can be explained by the high power and arousal characteristics of the emotions with high levels on these components. A multiple discriminant analysis yields classification accuracy greatly exceeding chance level, confirming the reliability of the acoustic patterns.

The effect of context and audio-visual modality on emotions elicited by a musical performance

In this work, we compared emotions induced by the same performance of Schubert Lieder during a live concert and in a laboratory viewing/listening setting to determine the extent to which laboratory research on affective reactions to music approximates real listening conditions in dedicated performances. We measured emotions experienced by volunteer members of an audience that attended a Lieder recital in a church (Context 1) and emotional reactions to an audio-video-recording of the same performance in a university lecture hall (Context 2). Three groups of participants were exposed to three presentation versions in Context 2: (1) an audio-visual recording, (2) an audio-only recording, and (3) a video-only recording. Participants achieved statistically higher levels of emotional convergence in the live performance than in the laboratory context, and the experience of particular emotions was determined by complex interactions between auditory and visual cues in the performance. This study demonstrates the contribution of the performance setting and the performers' appearance and nonverbal expression to emotion induction by music, encouraging further systematic research into the factors involved.

Effects of achievement contexts on the meaning structure of emotion words

Little is known about the impact of context on the meaning of emotion words. In the present study, we used a semantic profiling instrument (GRID) to investigate features representing five emotion components (appraisal, bodily reaction, expression, action tendencies, and feeling) of 11 emotion words in situational contexts involving success or failure. We compared these to the data from an earlier study in which participants evaluated the typicality of features out of context. Profile analyses identified features for which typicality changed as a function of context for all emotion words, except contentment, with appraisal features being most frequently affected. Those context effects occurred for both hypothesised basic and non-basic emotion words. Moreover, both data sets revealed a four-dimensional structure. The four dimensions were largely similar (valence, power, arousal, and novelty). The results suggest that context may not change the underlying dimensionality but affects facets of the meaning of emotion words.

Emotion Inferences from Vocal Expression Correlate Across Languages and Cultures

Whereas the perception of emotion from facial expression has been extensively studied cross-culturally, little is known about judges’ ability to infer emotion from vocal cues. This article reports the results from a study conducted in nine countries in Europe, the United States, and Asia on vocal emotion portrayals of anger, sadness, fear, joy, and neutral voice as produced by professional German actors. Data show an overall accuracy of 66% across all emotions and countries. Although accuracy was substantially better than chance, there were sizable differences ranging from 74% in Germany to 52% in Indonesia. However, patterns of confusion were very similar across all countries. These data suggest the existence of similar inference rules from vocal expression across cultures. Generally, accuracy decreased with increasing language dissimilarity from German in spite of the use of language-free speech samples. It is concluded that culture- and language-specific paralinguistic patterns may influence the decoding process.

The Role of Injustice in the Elicitation of Differential Emotional Reactions

Data from a large-scale study on emotional experiences in 37 countries are used to examine correlates of emotion-antecedent events being judged as unfair or unjust. This study included 2,921 students who reported situations in which they had experienced joy, anger; fear, sadness, disgust, shame, and guilt and described their situation appraisals and reactions. Anger-producing events were most frequently perceived as very unfair followed by disgust, sadness, fear, guilt, and shame. The results showed strong main effects of the perception of injustice for all negative emotions. Events experienced as unjust were described as more immoral, more obstructive to plans and goals, and having more negative effects on personal relationships. In addition, events regarded as unjust elicited feelings that were longer in duration and more intense. It is concluded that perceived injustice plays a powerful role in the elicitation of many different negative emotions and may serve as a mediating variable in emotion-antecedent appraisal.

Mapping Emotion Terms into Affective Space

Abstract. Recent empirical work on the semantics of emotion terms across many different cultures and languages, using a theoretical componential approach, suggested that four dimensions are needed to parsimoniously describe the semantic space of the emotion domain as reflected in emotion terms ( Fontaine, Scherer, Roesch, & Ellsworth, 2007 ; Fontaine, Scherer, & Soriano, 2013 ). In addition to valence, power, and arousal, a novelty dimension was discovered that mostly differentiated surprise from other emotions. Here, we further explore the existence and nature of the fourth dimension in semantic emotion space using a much larger and much more representative set of emotion terms. A group of 156 participants each rated 10 out of a set of 80 French emotion terms with respect to semantic meaning. The meaning of an emotion term was evaluated with respect to 68 emotion features representing the appraisal, action tendency, bodily reaction, expression, and feeling components of the emotion process. A principal component analysis confirmed the four-dimensional valence, power, arousal, and novelty structure. Moreover, this larger and much more representative set of emotion terms revealed that the novelty dimension not only differentiates surprise terms from other emotion terms, but also identifies substantial variation within the fear and joy emotion families.

The Look of Fear from the Eyes Varies with the Dynamic Sequence of Facial Actions

Abstract. Most research on the ability to interpret expressions from the eyes has utilized static information. This research investigates whether the dynamic sequence of facial actions in the eye region influences the judgments of perceivers. Dynamic fear expressions involving the eye region and eyebrows were created which systematically differed in the sequential occurrence of facial actions. Participants rated the intensity of sequential fear expressions, either in addition to a simultaneous, full-blown expression (Experiment 1) or in combination with different levels of eye gaze (Experiment 2). The results showed that the degree of attributed emotion and the appraisal ratings differed as a function of the sequence of facial expressions of fear, with direct gaze resulting in stronger subjective responses. The findings challenge current notions surrounding the study of static facial displays from the eyes and suggest that emotion perception is a dynamic process shaped by the time course of the facial actions of an expression. Possible implications for the field of affective computing and clinical research are discussed.

Temporal dynamics and potential neural sources of goal conduciveness, control, and power appraisal

A major emotion theory, the Component Process Model, predicts that emotion-antecedent appraisal proceeds sequentially (e.g., goal conduciveness>control>power appraisal). In a gambling task, feedback manipulated information about goal conduciveness (outcome: win, loss), control (perceived high and low control), and power appraisals (choice options to change the outcome). Using mean amplitudes of event-related potentials, we examine the sequential prediction of these appraisal criteria. Additionally, we apply source localization analysis to estimate the neural sources of the evoked components of interest. Early ERPs (230-300 ms) show main effects of goal conduciveness and power but no interaction effects suggesting goal obstructiveness assessment of task-relevant feedback information. Late ERPs (350-600 ms) reveal main effects of all appraisals and interaction effects representing the integration of all appraisal information. Source localization analysis suggests distinct neural sources for these appraisal criteria.

Path Models of Vocal Emotion Communication

We propose to use a comprehensive path model of vocal emotion communication, encompassing encoding, transmission, and decoding processes, to empirically model data sets on emotion expression and recognition. The utility of the approach is demonstrated for two data sets from two different cultures and languages, based on corpora of vocal emotion enactment by professional actors and emotion inference by naïve listeners. Lens model equations, hierarchical regression, and multivariate path analysis are used to compare the relative contributions of objectively measured acoustic cues in the enacted expressions and subjective voice cues as perceived by listeners to the variance in emotion inference from vocal expressions for four emotion families (fear, anger, happiness, and sadness). While the results confirm the central role of arousal in vocal emotion communication, the utility of applying an extended path modeling framework is demonstrated by the identification of unique combinations of distal cues and proximal percepts carrying information about specific emotion families, independent of arousal. The statistical models generated show that more sophisticated acoustic parameters need to be developed to explain the distal underpinnings of subjective voice quality percepts that account for much of the variance in emotion inference, in particular voice instability and roughness. The general approach advocated here, as well as the specific results, open up new research strategies for work in psychology (specifically emotion and social perception research) and engineering and computer science (specifically research and development in the domain of affective computing, particularly on automatic emotion detection and synthetic emotion expression in avatars).

Emotion perception from a componential perspective

The common conceptual understanding of emotion is that they are multi-componential, including subjective feelings, appraisals, psychophysiological activation, action tendencies, and motor expressions. Emotion perception, however, has traditionally been studied in terms of emotion labels, such as "happy", which do not clearly indicate whether one, some, or all emotion components are perceived. We examine whether emotion percepts are multi-componential and extend previous research by using more ecologically valid, dynamic, and multimodal stimuli and an alternative response measure. The results demonstrate that observers can reliably infer multiple types of information (subjective feelings, appraisals, action tendencies, and social messages) from complex emotion expressions. Furthermore, this finding appears to be robust to changes in response items. The results are discussed in light of their implications for research on emotion perception.

Appraisals Generate Specific Configurations of Facial Muscle Movements in a Gambling Task: Evidence for the Component Process Model of Emotion

Scherer’s Component Process Model provides a theoretical framework for research on the production mechanism of emotion and facial emotional expression. The model predicts that appraisal results drive facial expressions, which unfold sequentially and cumulatively over time. In two experiments, we examined facial muscle activity changes (via facial electromyography recordings over the corrugator, cheek, and frontalis regions) in response to events in a gambling task. These events were experimentally manipulated feedback stimuli which presented simultaneous information directly affecting goal conduciveness (gambling outcome: win, loss, or break-even) and power appraisals (Experiment 1 and 2), as well as control appraisal (Experiment 2). We repeatedly found main effects of goal conduciveness (starting ~600 ms), and power appraisals (starting ~800 ms after feedback onset). Control appraisal main effects were inconclusive. Interaction effects of goal conduciveness and power appraisals were obtained in both experiments (Experiment 1: over the corrugator and cheek regions; Experiment 2: over the frontalis region) suggesting amplified goal conduciveness effects when power was high in contrast to invariant goal conduciveness effects when power was low. Also an interaction of goal conduciveness and control appraisals was found over the cheek region, showing differential goal conduciveness effects when control was high and invariant effects when control was low. These interaction effects suggest that the appraisal of having sufficient control or power affects facial responses towards gambling outcomes. The result pattern suggests that corrugator and frontalis regions are primarily related to cognitive operations that process motivational pertinence, whereas the cheek region would be more influenced by coping implications. Our results provide first evidence demonstrating that cognitive-evaluative mechanisms related to goal conduciveness, control, and power appraisals affect facial expressions dynamically over time, immediately after an event is perceived. In addition, our results provide further indications for the chronography of appraisal-driven facial movements and the underlying cognitive processes.

Emotion categories and dimensions in the facial communication of affect: An integrated approach

We investigated the role of facial behavior in emotional communication, using both categorical and dimensional approaches. We used a corpus of enacted emotional expressions (GEMEP) in which professional actors are instructed, with the help of scenarios, to communicate a variety of emotional experiences. The results of Study 1 replicated earlier findings showing that only a minority of facial action units are associated with specific emotional categories. Likewise, facial behavior did not show a specific association with particular emotional dimensions. Study 2 showed that facial behavior plays a significant role both in the detection of emotions and in the judgment of their dimensional aspects, such as valence, arousal, dominance, and unpredictability. In addition, a mediation model revealed that the association between facial behavior and recognition of the signaler's emotional intentions is mediated by perceived emotional dimensions. We conclude that, from a production perspective, facial action units convey neither specific emotions nor specific emotional dimensions, but are associated with several emotions and several dimensions. From the perceiver's perspective, facial behavior facilitated both dimensional and categorical judgments, and the former mediated the effect of facial behavior on recognition accuracy. The classification of emotional expressions into discrete categories may, therefore, rely on the perception of more general dimensions such as valence and arousal and, presumably, the underlying appraisals that are inferred from facial movements.

When and Why Are Emotions Disturbed? Suggestions Based on Theory and Data From Emotion Research

Diagnosing emotion disturbances should be informed by current knowledge about normal emotion processes. I identify four major functions of emotion as well as sources for potential dysfunctions and suggest that emotions should only be diagnosed as pathological when they are clearly dysfunctional, which requires considering eliciting events, realistic person-specific appraisal patterns, and adaptive responses or action tendencies. Evidence from actuarial research on the reported length of naturally occurring emotion episodes (including potential determinants) illustrates appropriateness criteria for the clinical evaluation of emotion duration—an essential element in the DSM-5 symptom catalogue for major depression episodes, especially in bereavement. The need for more actuarial evidence on normal emotion responses and its consideration by the clinical community is highlighted.

Automated Recognition of Emotion Appraisals

Most computer models for the automatic recognition of emotion from nonverbal signals (e.g., facial or vocal expression) have adopted a discrete emotion perspective, i.e., they output a categorical emotion from a limited pool of candidate labels. The discrete perspective suffers from practical and theoretical drawbacks that limit the generalizability of such systems. The authors of this chapter propose instead to adopt an appraisal perspective in modeling emotion recognition, i.e., to infer the subjective cognitive evaluations that underlie both the nonverbal cues and the overall emotion states. In a first step, expressive features would be used to infer appraisals; in a second step, the inferred appraisals would be used to predict an emotion label. The first step is practically unexplored in emotion literature. Such a system would allow to (a) link models of emotion recognition and production, (b) add contextual information to the inference algorithm, and (c) allow detection of subtle emotion states.