Temporal Dynamics of Natural Static Emotional Facial Expressions Decoding: A Study Using Event- and Eye Fixation-Related Potentials

This study aims at examining the precise temporal dynamics of the emotional facial decoding as it unfolds in the brain, according to the emotions displayed. To characterize this processing as it occurs in ecological settings, we focused on unconstrained visual explorations of natural emotional faces (i.e., free eye movements). The General Linear Model (GLM; Smith and Kutas, 2015a,b; Kristensen et al., 2017a) enables such a depiction. It allows deconvolving adjacent overlapping responses of the eye fixation-related potentials (EFRPs) elicited by the subsequent fixations and the event-related potentials (ERPs) elicited at the stimuli onset. Nineteen participants were displayed with spontaneous static facial expressions of emotions (Neutral, Disgust, Surprise, and Happiness) from the DynEmo database (Tcherkassof et al., 2013). Behavioral results on participants' eye movements show that the usual diagnostic features in emotional decoding (eyes for negative facial displays and mouth for positive ones) are consistent with the literature. The impact of emotional category on both the ERPs and the EFRPs elicited by the free exploration of the emotional faces is observed upon the temporal dynamics of the emotional facial expression processing. Regarding the ERP at stimulus onset, there is a significant emotion-dependent modulation of the P2-P3 complex and LPP components' amplitude at the left frontal site for the ERPs computed by averaging. Yet, the GLM reveals the impact of subsequent fixations on the ERPs time-locked on stimulus onset. Results are also in line with the valence hypothesis. The observed differences between the two estimation methods (Average vs. GLM) suggest the predominance of the right hemisphere at the stimulus onset and the implication of the left hemisphere in the processing of the information encoded by subsequent fixations. Concerning the first EFRP, the Lambda response and the P2 component are modulated by the emotion of surprise compared to the neutral emotion, suggesting an impact of high-level factors, in parieto-occipital sites. Moreover, no difference is observed on the second and subsequent EFRP. Taken together, the results stress the significant gain obtained in analyzing the EFRPs using the GLM method and pave the way toward efficient ecological emotional dynamic stimuli analyses.

Ausência de Prejuízo no Reconhecimento de Expressões Faciais entre Indivíduos com Parkinson

RESUMO: Entre os sintomas não motores da doença de Parkinson, dificuldades no reconhecimento de expressões faciais emocionais vêm sendo amplamente discutidas, pois as áreas cerebrais relacionadas a tal habilidade podem estar afetadas na doença. Este estudo investigou, em idosos, o reconhecimento das seis expressões emocionais faciais consideradas universais por meio do instrumento Teste de Percepção Emocional de Faces, em que participantes executam uma tarefa de reconhecimento de expressões emocionais faciais. Participaram 41 indivíduos com idade média de 64,9 anos, 27 homens e 14 mulheres. Não foi observada significativa dificuldade no reconhecimento de nenhuma das expressões emocionais por parte dos parkinsonianos. Tal resultado pode indicar a necessidade do desenvolvimento de instrumentos e técnicas mais adequadas para esse tipo de investigação na população brasileira.

Embodied Resistance to Persuasion in Advertising

From the literature on resistance to persuasion in advertising, much is known about how people can resist advertising by adopting resistance strategies, such as avoidance, counter-arguing, and selective attention (e.g., Fransen et al., 2015b). However, the role of emotion regulation and bodily expression in resisting persuasion is so far underexplored. This is a surprising observation if one considers that at least 40% of advertisements use positive emotions (i.e., happiness) to persuade people to like the ad, brand, and product (Weinberger et al., 1995). In this article we present a framework in which we apply previous knowledge and theories on emotion regulation and embodiment to the process of resistance to persuasion. In doing so, we specifically address the role of facial expression in the course of resistance. The literature and findings from our own research lead us to propose that people can resist persuasion by controlling their facial expression of emotion when exposed to an advertisement. Controlling the expression of emotions elicited by an ad (for example refusing to smile) might be a fruitful way to resist the ad’s persuasive potential. Moreover, we argue that co-viewers can affect embodied resistance to persuasion. Showing the viability of embodied resistance to persuasion is relevant in view of the fact that ads trying to persuade us by addressing our positive emotions are ubiquitous. Embodied resistance might help people to cope with these induced positive emotions in order to resist advertisements and might therefore work as a novel and effective strategy to resist persuasion.

The Human Face as a Dynamic Tool for Social Communication

As a highly social species, humans frequently exchange social information to support almost all facets of life. One of the richest and most powerful tools in social communication is the face, from which observers can quickly and easily make a number of inferences - about identity, gender, sex, age, race, ethnicity, sexual orientation, physical health, attractiveness, emotional state, personality traits, pain or physical pleasure, deception, and even social status. With the advent of the digital economy, increasing globalization and cultural integration, understanding precisely which face information supports social communication and which produces misunderstanding is central to the evolving needs of modern society (for example, in the design of socially interactive digital avatars and companion robots). Doing so is challenging, however, because the face can be thought of as comprising a high-dimensional, dynamic information space, and this impacts cognitive science and neuroimaging, and their broader applications in the digital economy. New opportunities to address this challenge are arising from the development of new methods and technologies, coupled with the emergence of a modern scientific culture that embraces cross-disciplinary approaches. Here, we briefly review one such approach that combines state-of-the-art computer graphics, psychophysics and vision science, cultural psychology and social cognition, and highlight the main knowledge advances it has generated. In the light of current developments, we provide a vision of the future directions in the field of human facial communication within and across cultures.

Simple lines and shapes are associated with, and communicate, distinct emotions

We investigated whether lines and shapes that present face-like features would be associated with emotions. In Experiment 1, participants associated concave, convex, or straight lines with the words happy or sad. Participants found it easiest to associate the concave line with happy and the convex line with sad. In Experiment 2, participants rated (valence, pleasantness, liking, and tension) and categorised (valence and emotion words) two convex and concave lines that were paired with six distinct pairs of eyes. The presence of eyes affected participants' valence ratings and response latencies; more congruent eye-mouth matches produced more consistent ratings and faster reaction times. In Experiment 3, we examined whether dots that resembled eyes would be associated with emotional words. Participants found it easier to match certain sets of dots with specific emotions. These results suggest that facial gestures that are associated with specific emotions can be captured using relatively simple shapes and lines.

Appraisals Generate Specific Configurations of Facial Muscle Movements in a Gambling Task: Evidence for the Component Process Model of Emotion

Scherer’s Component Process Model provides a theoretical framework for research on the production mechanism of emotion and facial emotional expression. The model predicts that appraisal results drive facial expressions, which unfold sequentially and cumulatively over time. In two experiments, we examined facial muscle activity changes (via facial electromyography recordings over the corrugator, cheek, and frontalis regions) in response to events in a gambling task. These events were experimentally manipulated feedback stimuli which presented simultaneous information directly affecting goal conduciveness (gambling outcome: win, loss, or break-even) and power appraisals (Experiment 1 and 2), as well as control appraisal (Experiment 2). We repeatedly found main effects of goal conduciveness (starting ~600 ms), and power appraisals (starting ~800 ms after feedback onset). Control appraisal main effects were inconclusive. Interaction effects of goal conduciveness and power appraisals were obtained in both experiments (Experiment 1: over the corrugator and cheek regions; Experiment 2: over the frontalis region) suggesting amplified goal conduciveness effects when power was high in contrast to invariant goal conduciveness effects when power was low. Also an interaction of goal conduciveness and control appraisals was found over the cheek region, showing differential goal conduciveness effects when control was high and invariant effects when control was low. These interaction effects suggest that the appraisal of having sufficient control or power affects facial responses towards gambling outcomes. The result pattern suggests that corrugator and frontalis regions are primarily related to cognitive operations that process motivational pertinence, whereas the cheek region would be more influenced by coping implications. Our results provide first evidence demonstrating that cognitive-evaluative mechanisms related to goal conduciveness, control, and power appraisals affect facial expressions dynamically over time, immediately after an event is perceived. In addition, our results provide further indications for the chronography of appraisal-driven facial movements and the underlying cognitive processes.

PERVALE-S: a new cognitive task to assess deaf people's ability to perceive basic and social emotions

A poorly understood aspect of deaf people (DP) is how their emotional information is processed. Verbal ability is key to improve emotional knowledge in people. Nevertheless, DP are unable to distinguish intonation, intensity, and the rhythm of language due to lack of hearing. Some DP have acquired both lip-reading abilities and sign language, but others have developed only sign language. PERVALE-S was developed to assess the ability of DP to perceive both social and basic emotions. PERVALE-S presents different sets of visual images of a real deaf person expressing both basic and social emotions, according to the normative standard of emotional expressions in Spanish Sign Language. Emotional expression stimuli were presented at two different levels of intensity (1: low; and 2: high) because DP do not distinguish an object in the same way as hearing people (HP) do. Then, participants had to click on the more suitable emotional expression. PERVALE-S contains video instructions (given by a sign language interpreter) to improve DP's understanding about how to use the software. DP had to watch the videos before answering the items. To test PERVALE-S, a sample of 56 individuals was recruited (18 signers, 8 lip-readers, and 30 HP). Participants also performed a personality test (High School Personality Questionnaire adapted) and a fluid intelligence (Gf) measure (RAPM). Moreover, all deaf participants were rated by four teachers for the deaf.

RESULTS

there were no significant differences between deaf and HP in performance in PERVALE-S. Confusion matrices revealed that embarrassment, envy, and jealousy were worse perceived. Age was just related to social-emotional tasks (but not in basic emotional tasks). Emotional perception ability was related mainly to warmth and consciousness, but negatively related to tension. Meanwhile, Gf was related to only social-emotional tasks. There were no gender differences.

Contextualizing Facial Activity

Drawing on research reviewed in this special section, the present article discusses how various contextual factors impact on production and decoding of emotion-related facial activity. Although emotion-related variables often contribute to activation of prototypical “emotion expressions” and perceivers can often infer emotional meanings from these facial configurations, neither process is invariant or direct. Many facial movements are directed towards or away from events in the shared environment, and their effects depend on these relational orientations. Facial activity is not only a medium for descriptive representation of internal affective states, but also a means of adjusting to, and operating on, external objects, and of influencing other people’s appraisals of those objects.