Rapid extraction of emotional expression: evidence from evoked potential fields during brief presentation of face stimuli

Emotional text messages affect the early processing of emoticons depending on their emotional congruence: evidence from the N170 and EPN event related potentials

Emoticons have been considered pragmatic cues that enhance emotional expressivity during computer-mediated communication. Yet, it is unclear how emoticons are processed in ambiguous text-based communication due to incongruences between the emoticon's emotional valence and its context. In this study, we investigated the electrophysiological correlates of contextual influence on the early emotional processing of emoticons, during an emotional congruence judgment task. Participants were instructed to judge the congruence between a text message expressing an emotional situation (positive or negative), and a subsequent emoticon expressing positive or negative emotions. We analyzed early event-related potentials elicited by emoticons related to face processing (N170) and emotional salience in visual perception processing (Early Posterior Negativity, EPN). Our results show that accuracy and Reaction Times depend on the interaction between the emotional valence of the context and the emoticon. Negative emoticons elicited a larger N170, suggesting that the emotional information of the emoticon is integrated at the early stages of the perceptual process. During emoticon processing, a valence effect was observed with enhanced EPN amplitudes in occipital areas for emoticons representing negative valences. Moreover, we observed a congruence effect in parieto-temporal sites within the same time-window, with larger amplitudes for the congruent condition. We conclude that, similar to face processing, emoticons are processed differently according to their emotional content and the context in which they are embedded. A congruent context might enhance the emotional salience of the emoticon (and therefore, its emotional expression) during the early stages of their processing.

Posterior theta activity reveals an early signal of self-face recognition

Self-related visual information, especially one's own face and name, are processed in a specific, prioritized way. However, the spatio-temporal brain dynamics of self-prioritization have remained elusive. Moreover, it has been unclear whether this prioritization is an effect of enhancement and amplification, or rather a facilitating automatization of processing self-referential information. In this EEG study, 25 married women (who changed their surnames after marriage, so that their past and present surnames could be used as stimuli) performed a detection task with faces and names from five categories: self, self from the past, friend, famous, and unknown person. The aim was to determine the temporal and spatial characteristics of early electrophysiological markers of self-referential processing. We report results of event-related component (ERP) and time-frequency analyses. In the ERPs, the earliest self-relevance effect was displayed only 300 ms after stimulus onset in the midfrontal N2, and later in the parietal P3b, independently of the stimulus type. No self-relevance effect was found on the N170 component. However, local theta power at the occipito-temporal (visual) areas and inter-regional theta phase coherence between the visual and midfrontal areas showed that self-relevance differentiation of faces began already about 100-300 ms after stimulus onset. No such early effects were found for names. The results are discussed in terms of the time-course, functional localization, stimulus-specificity, and automatization of self-prioritization.

The effect of sad mood on early sensory event-related potentials to task-irrelevant faces

It has been shown that the perceiver's mood affects the perception of emotional faces, but it is not known how mood affects preattentive brain responses to emotional facial expressions. To examine the question, we experimentally induced sad and neutral mood in healthy adults before presenting them with task-irrelevant pictures of faces while an electroencephalography was recorded. Sad, happy, and neutral faces were presented to the participants in an ignore oddball condition. Differential responses (emotional - neutral) for the P1, N170, and P2 amplitudes were extracted and compared between neutral and sad mood conditions. Emotional facial expressions modulated all the components, and an interaction effect of expression by mood was found for P1: an emotional modulation to happy faces, which was found in neutral mood condition, disappeared in sad mood condition. For N170 and P2, we found larger response amplitudes for both emotional faces, regardless of the mood. The results add to the previous behavioral findings showing that mood already affects low-level cortical feature encoding of task-irrelevant faces.

The time course of categorical perception of facial expressions

Decoding emotions on others' faces is one of the most important functions of the human brain, which has been widely studied in cognitive neuroscience. However, the precise time course of facial expression categorization in the human brain is still a matter of debate. Here we used an original paradigm to measure categorical perception of facial expression changes during event-related potentials (ERPs) recording, in which a face stimulus dynamically switched either to a different expression (between-category condition) or to the same expression (within-category condition), the physical distance between the two successive faces being equal across conditions. The switch between faces generated a negative differential potential peaking at around 160 ms over occipito-temporal regions, similar in term of latency and topography to the well-known face-selective N170 component. This response was larger in the condition where the switch occurred between faces that were perceived as having different facial expressions compared to the same expression. In addition, happy expressions were categorized around 20 ms faster than fearful expressions (respectively, 135 and 156 ms). These findings provide evidence that changes of facial expressions are categorically perceived as early as 160 ms following stimulus onset over the occipito-temporal cortex.

Can faces affect object-based attention? Evidence from online experiments

This study tested how human faces affect object-based attention (OBA) through two online experiments in a modified double-rectangle paradigm. The results of Experiment 1 revealed that faces did not elicit the OBA effect as non-face objects, which was caused by a longer response time (RT) when attention is focused on faces relative to non-face objects. In addition, by observing faster RTs when attention was engaged horizontally rather than vertically, we found a significant horizontal attention bias, which might override the OBA effect if vertical rectangles were the only items presented; these results were replicated in Experiment 2 (using only vertical rectangles) after directly measuring horizontal bias and excluding its influence on the OBA effect. This study suggested that faces cannot elicit the same-object advantage in the double-rectangle paradigm and provided a method to measure the OBA effect free from horizontal bias.

The Study of Face Processing in Social Anxiety Disorder Based on Face-Specific N170 Component

Although previous studies showed that social anxiety disorder (SAD) exhibits the attentional bias for angry faces, few studies investigated effective face recognition combined with event-related potential (ERP) technique in SAD patients, especially the treatment effect. This study examines the differences in face processing in SAD patients before and after treatment and healthy control people (H-group). High-density EEG scans were registered in response to emotional schematic faces, particularly interested in the face processing N170 component. Analysis of N170 amplitude revealed a larger N170 for P-group-pre in response to inverted and upright stimuli than H-group in the right hemisphere. The result of the intragroup t-test showed that N170 was delayed for inverted relative to upright faces only in P-group-post and H-group but not in P-group-pre. Remarkably, the results of ANOVAs manifested that emotional expression cannot modulate N170 for SAD patients. Besides, the N170-based asymmetry index (AI) was introduced to analyze the left- and right-hemisphere dominance of N170 for three groups. It was found that, with the improvement of patients' treatment, the value of AI_{N170−base  d} presented a decreasing trend. These results together suggested that there was no inversion effect observed for patients with SAD. The change in the value of AI_{N170−base  d} can be used as potential electrophysiological markers for the diagnosis and treatment effects on patients with SAD.

Turning to the negative: attention allocation to emotional faces in adolescents with dysregulation profile-an event-related potential study

Patients with irritability, temper outbursts, hyperactivity and mood swings often meet the dysregulation profile (DP) of the Child Behavior Checklist (CBCL) or the Strengths and Difficulties Questionnaire (SDQ), which have been investigated over the past few decades. While the DP has emerged as a transdiagnostic marker with a negative impact on therapeutic outcome and psychosocial functioning, little is known about its underlying mechanisms such as attention and emotion regulation processes. In this study, we tested whether adolescent psychiatric patients (n = 27) with the SDQ-DP show impaired emotional face processing for task-irrelevant stimuli compared to psychiatric patients without the SDQ-DP (n = 30) and non-clinical adolescents (n = 21). Facial processing was tested with event-related potential (ERP) measures known to be modulated by attention (i.e., P1, N1, N170, P2, and Nc) during a modified Attention Network Task, to which task-irrelevant emotional stimuli (sad, fearful, and neutral faces) were added prior to the actual trial. The results reveal group differences in the orienting and in the conflicting network. Patients with DP showed a less efficient orienting network and the clinical control group showed a less efficient conflicting network. Moreover, patients with the dysregulation profile had a shorter N1/N170 latency than did the two control groups, suggesting that dysregulation in adolescents is associated with a faster but less arousing encoding of (task-irrelevant) emotional information and less top-down control.

Time-resolved classification of dog brain signals reveals early processing of faces, species and emotion

Dogs process faces and emotional expressions much like humans, but the time windows important for face processing in dogs are largely unknown. By combining our non-invasive electroencephalography (EEG) protocol on dogs with machine-learning algorithms, we show category-specific dog brain responses to pictures of human and dog facial expressions, objects, and phase-scrambled faces. We trained a support vector machine classifier with spatiotemporal EEG data to discriminate between responses to pairs of images. The classification accuracy was highest for humans or dogs vs. scrambled images, with most informative time intervals of 100–140 ms and 240–280 ms. We also detected a response sensitive to threatening dog faces at 30–40 ms; generally, responses differentiating emotional expressions were found at 130–170 ms, and differentiation of faces from objects occurred at 120–130 ms. The cortical sources underlying the highest-amplitude EEG signals were localized to the dog visual cortex.

Emotional reactivity and regulation in 5- to 8-year-old children: An ERP study of own-age face processing

The school-age years is a period of increasing social interaction with peers and development of emotion regulation in facilitating that interaction. This study was an investigation of the neural correlates of emotional reactivity and reappraisal in typically developing school-age children elicited by threatening facial expressions of same-aged peers. This experimental paradigm is novel in eliciting event-related brain potentials (ERPs) to social stimuli that are ecologically valid to the everyday life of children. ERPs of 5- to 8-year-old children (N = 41, 18 females) were elicited by threatening (i.e., angry and fearful) and neutral child facial expressions, which were preceded by audio contextual cues. Three conditions differed in audio-image pairing: neutral context-neutral expression (neutral condition), negative context-threatening expression (threat condition), and reappraisal context-threatening expression (reappraisal condition). In addition, parental reporting of childhood temperament was collected to determine if elicited ERP morphologies were associated with temperamental dimensions of negative affect, extraversion, and effortful control. Elicitation of the P100 and N170 did not largely differ between conditions; however, amplitude of the late positive potential (LPP), a marker of heightened emotional reactivity and attention, was greater for threatening faces relative to neutral faces. During the reappraisal condition, no differences in ERP activity was observed compared to the threat condition. Neural substrates of emotional reactivity to social threat from peers were evident; however, the lack of ERP modulation facilitating reappraisal and the lack of strong associations between ERP morphology and temperamental dimensions is indicative of heterogeneity in LPP elicitation underlying emotion regulation in children.

Spatial attention affects the early processing of neutral versus fearful faces when they are task-irrelevant: a classifier study of the EEG C1 component

EEG studies suggest that the emotional content of visual stimuli is processed rapidly. In particular, the C1 component, which occurs up to 100 ms after stimulus onset and likely reflects activity in primary visual cortex V1, has been reported to be sensitive to emotional faces. However, difficulties replicating these results have been reported. We hypothesized that the nature of the task and attentional condition are key to reconcile the conflicting findings. We report three experiments of EEG activity during the C1 time range elicited by peripherally presented neutral and fearful faces under various attentional conditions: the faces were spatially attended or unattended and were either task-relevant or not. Using traditional event-related potential analysis, we found that the early activity changed depending on facial expression, attentional condition, and task. In addition, we trained classifiers to discriminate the different conditions from the EEG signals. Although the classifiers were not able to discriminate between facial expressions in any condition, they uncovered differences between spatially attended and unattended faces but solely when these were task-irrelevant. In addition, this effect was only present for neutral faces. Our study provides further indication that attention and task are key parameters when measuring early differences between emotional and neutral visual stimuli.

Distinct brain representations of processed and unprocessed foods

Among all of the stimuli surrounding us, food is arguably the most rewarding for the essential role it plays in our survival. In previous visual recognition research, it has already been demonstrated that the brain not only differentiates edible stimuli from non-edible stimuli but also is endowed with the ability to detect foods' idiosyncratic properties such as energy content. Given the contribution of the cooked diet to human evolution, in the present study we investigated whether the brain is sensitive to the level of processing food underwent, based solely on its visual appearance. We thus recorded visual evoked potentials (VEPs) from normal-weight healthy volunteers who viewed color images of unprocessed and processed foods equated in caloric content. Results showed that VEPs and underlying neural sources differed as early as 130 ms post-image onset when participants viewed unprocessed versus processed foods, suggesting a within-category early discrimination of food stimuli. Responses to unprocessed foods engaged the inferior frontal and temporal regions and the premotor cortices. In contrast, viewing processed foods led to the recruitment of occipito-temporal cortices bilaterally, consistently with other motivationally relevant stimuli. This is the first evidence of diverging brain responses to food as a function of the transformation undergone during its preparation that provides insights on the spatiotemporal dynamics of food recognition.

Oxytocin Enhances the Neural Efficiency of Social Perception

Face perception is a highly conserved process that directs our attention from infancy and is supported by specialized neural circuitry. Oxytocin (OT) can increase accuracy and detection of emotional faces, but these effects are mediated by valence, individual differences, and context. We investigated the temporal dynamics of OT’s influence on the neural substrates of face perception using event related potentials (ERPs). In a double blind, placebo controlled within-subject design, 21 healthy male adults inhaled OT or placebo and underwent ERP imaging during two face processing tasks. Experiment 1 investigated effects of OT on neural correlates of fearful vs. neutral facial expressions, and Experiment 2 manipulated point-of-gaze to neutral faces. In Experiment 1, we found that OT reduced N170 latency to fearful faces. In Experiment 2, N170 latency was decreased when participant gaze was directed to the eyes of neutral faces; however, there were no OT-associated effects in response to different facial features. Findings suggest OT modulates early stages of social perception for socially complex information such as emotional faces relative to neutral. These results are consistent with models suggesting OT impacts the salience of socially informative cues during processing, which leads to downstream effects in behavior. Future work should examine how OT affects neural processes underlying basic components of social behavior (such as, face perception) while varying emotional expression of stimuli or comparing different characteristics of participants (e.g., gender, personality traits).

Length of Hair Affects P1 and N170 Latencies for Perception of Women's Faces

This study investigated the relationship between length of hair in facial stimuli and latency and amplitude of the P1 and N170 components of event-related potentials during facial perception. Electroencephalography was recorded from 21 Japanese participants (four men, 17 women) who were shown pictures of faces with one of three lengths of hair: long, medium length, or short. In addition, we used both fixed-size and variable-size blocks. In fixed-size blocks, the three types of stimuli were matched to have the same overall size; in variable-size blocks, long hair stimuli were the biggest, medium length hair stimuli were medium sized, and short hair stimuli were the smallest. We analyzed P1 latency and amplitude using two-way (6 × 2) repeated-measures analysis of variance over length of hair and electrode; N170 latency and amplitude were analyzed using three-way (6 × 2 × 2) repeated-measures analysis of variance over length of hair, hemisphere, and electrode. The latency of P1 to faces with short hair in variable-size blocks was significantly longer than that to the other five stimulus types ( p < .01 for four of the other types; p = .083 for medium length hair in variable-size blocks). The latency of N170 to faces with long hair in variable-size blocks was significantly shorter than that to faces with medium length hair and short hair in variable-size blocks ( p = .026 and p = .086, respectively). These results indicate that length of hair influenced P1 and N170 latency, supporting the notion that length of hair is a significant external facial feature. Because long hair attracted participants' attention, there was early perceptual processing of this feature. In contrast, because short hair did not attract attention, perceptual processing of this feature was late.

Spatiotemporal dynamics in human visual cortex rapidly encode the emotional content of faces

Recognizing emotion in faces is important in human interaction and survival, yet existing studies do not paint a consistent picture of the neural representation supporting this task. To address this, we collected magnetoencephalography (MEG) data while participants passively viewed happy, angry and neutral faces. Using time-resolved decoding of sensor-level data, we show that responses to angry faces can be discriminated from happy and neutral faces as early as 90 ms after stimulus onset and only 10 ms later than faces can be discriminated from scrambled stimuli, even in the absence of differences in evoked responses. Time-resolved relevance patterns in source space track expression-related information from the visual cortex (100 ms) to higher-level temporal and frontal areas (200-500 ms). Together, our results point to a system optimised for rapid processing of emotional faces and preferentially tuned to threat, consistent with the important evolutionary role that such a system must have played in the development of human social interactions.

Differential Effects of Intranasal Vasopressin on the Processing of Adult and Infant Cues: An ERP Study

Arginine vasopressin (AVP) is a powerful regulator of various social behaviors across many species. However, seemingly contradictory effects of AVP have been found in both animal and human studies, e.g., promoting aggression on one hand and facilitating social bonding on the other hand. Therefore, we hypothesize that the role of AVP in social behaviors is context-dependent. To this end, we examined the modulatory effect of AVP on male’s behavioral and neural responses to infant and adult cues. After intranasal and double-blind treatment of AVP or placebo, male participants were asked to rate their subjective approaching willingness to infant and adult faces in specific contexts informed by cue words while EEG recording. Our results showed that AVP treatment increased approaching ratings to neutral and positive other-gender adult faces compared to emotional matched same-gender adult faces, and to negative girl faces compared to negative boy faces. Furthermore, compared to placebo treatment, AVP treatment induced larger N1 amplitudes to neutral cues associated with both adults and infants, whereas AVP treatment only sustained pronounced late positive potential amplitudes to neutral cues of infants but not adults. Those findings implicate differential roles of AVP in the processing of adult- and infant-related cues and thus lend support to the context-dependent account.

Automatic Processing of Changes in Facial Emotions in Dysphoria: A Magnetoencephalography Study

It is not known to what extent the automatic encoding and change detection of peripherally presented facial emotion is altered in dysphoria. The negative bias in automatic face processing in particular has rarely been studied. We used magnetoencephalography (MEG) to record automatic brain responses to happy and sad faces in dysphoric (Beck's Depression Inventory ≥ 13) and control participants. Stimuli were presented in a passive oddball condition, which allowed potential negative bias in dysphoria at different stages of face processing (M100, M170, and M300) and alterations of change detection (visual mismatch negativity, vMMN) to be investigated. The magnetic counterpart of the vMMN was elicited at all stages of face processing, indexing automatic deviance detection in facial emotions. The M170 amplitude was modulated by emotion, response amplitudes being larger for sad faces than happy faces. Group differences were found for the M300, and they were indexed by two different interaction effects. At the left occipital region of interest, the dysphoric group had larger amplitudes for sad than happy deviant faces, reflecting negative bias in deviance detection, which was not found in the control group. On the other hand, the dysphoric group showed no vMMN to changes in facial emotions, while the vMMN was observed in the control group at the right occipital region of interest. Our results indicate that there is a negative bias in automatic visual deviance detection, but also a general change detection deficit in dysphoria.

The influence of spatial frequency content on facial expression processing: An ERP study using rapid serial visual presentation

Spatial frequency (SF) contents have been shown to play an important role in emotion perception. This study employed event-related potentials (ERPs) to explore the time course of neural dynamics involved in the processing of facial expression conveying specific SF information. Participants completed a dual-target rapid serial visual presentation (RSVP) task, in which SF-filtered happy, fearful, and neutral faces were presented. The face-sensitive N170 component distinguished emotional (happy and fearful) faces from neutral faces in a low spatial frequency (LSF) condition, while only happy faces were distinguished from neutral faces in a high spatial frequency (HSF) condition. The later P3 component differentiated between the three types of emotional faces in both LSF and HSF conditions. Furthermore, LSF information elicited larger P1 amplitudes than did HSF information, while HSF information elicited larger N170 and P3 amplitudes than did LSF information. Taken together, these results suggest that emotion perception is selectively tuned to distinctive SF contents at different temporal processing stages.

The special status of sad infant faces: age and valence differences in adults' cortical face processing

Understanding the relative and joint prioritization of age- and valence-related face characteristics in adults’ cortical face processing remains elusive because these two characteristics have not been manipulated in a single study of neural face processing. We used electroencephalography to investigate adults’ P1, N170, P2 and LPP responses to infant and adult faces with happy and sad facial expressions. Viewing infant vs adult faces was associated with significantly larger P1, N170, P2 and LPP responses, with hemisphere and/or participant gender moderating this effect in select cases. Sad faces were associated with significantly larger N170 responses than happy faces. Sad infant faces were associated with significantly larger N170 responses in the right hemisphere than all other combinations of face age and face valence characteristics. We discuss the relative and joint neural prioritization of infant face characteristics and negative facial affect, and their biological value as distinct caregiving and social cues.

Positive Classification Advantage: Tracing the Time Course Based on Brain Oscillation

The present study aimed to explore the modulation of frequency bands (alpha, beta, theta) underlying the positive facial expressions classification advantage within different post-stimulus time intervals (100–200 ms, 200–300 ms, 300–400 ms). For this purpose, we recorded electroencephalogram (EEG) activity during an emotion discrimination task for happy, sad and neutral faces. The correlation between the non-phase-locked power of frequency bands and reaction times (RTs) was assessed. The results revealed that beta played a major role in positive classification advantage (PCA) within the 100–200 and 300–400 ms intervals, whereas theta was important within the 200–300 ms interval. We propose that the beta band modulated the neutral and emotional face classification process, and that the theta band modulated for happy and sad face classification.

Neural signatures of dynamic emotion constructs in the human brain

How is emotion represented in the brain: is it categorical or along dimensions? In the present study, we applied multivariate pattern analysis (MVPA) to magnetoencephalography (MEG) to study the brain's temporally unfolding representations of different emotion constructs. First, participants rated 525 images on the dimensions of valence and arousal and by intensity of discrete emotion categories (happiness, sadness, fear, disgust, and sadness). Thirteen new participants then viewed subsets of these images within an MEG scanner. We used Representational Similarity Analysis (RSA) to compare behavioral ratings to the unfolding neural representation of the stimuli in the brain. Ratings of valence and arousal explained significant proportions of the MEG data, even after corrections for low-level image properties. Additionally, behavioral ratings of the discrete emotions fear, disgust, and happiness significantly predicted early neural representations, whereas rating models of anger and sadness did not. Different emotion constructs also showed unique temporal signatures. Fear and disgust - both highly arousing and negative - were rapidly discriminated by the brain, but disgust was represented for an extended period of time relative to fear. Overall, our findings suggest that 1) dimensions of valence and arousal are quickly represented by the brain, as are some discrete emotions, and 2) different emotion constructs exhibit unique temporal dynamics. We discuss implications of these findings for theoretical understanding of emotion and for the interplay of discrete and dimensional aspects of emotional experience.

Basic Emotions in Human Neuroscience: Neuroimaging and Beyond

The existence of so-called ‘basic emotions’ and their defining attributes represents a long lasting and yet unsettled issue in psychology. Recently, neuroimaging evidence, especially related to the advent of neuroimaging meta-analytic methods, has revitalized this debate in the endeavor of systems and human neuroscience. The core theme focuses on the existence of unique neural bases that are specific and characteristic for each instance of basic emotion. Here we review this evidence, outlining contradictory findings, strengths and limits of different approaches. Constructionism dismisses the existence of dedicated neural structures for basic emotions, considering that the assumption of a one-to-one relationship between neural structures and their functions is central to basic emotion theories. While these critiques are useful to pinpoint current limitations of basic emotions theories, we argue that they do not always appear equally generative in fostering new testable accounts on how the brain relates to affective functions. We then consider evidence beyond PET and fMRI, including results concerning the relation between basic emotions and awareness and data from neuropsychology on patients with focal brain damage. Evidence from lesion studies are indeed particularly informative, as they are able to bring correlational evidence typical of neuroimaging studies to causation, thereby characterizing which brain structures are necessary for, rather than simply related to, basic emotion processing. These other studies shed light on attributes often ascribed to basic emotions, such as automaticity of perception, quick onset, and brief duration. Overall, we consider that evidence in favor of the neurobiological underpinnings of basic emotions outweighs dismissive approaches. In fact, the concept of basic emotions can still be fruitful, if updated to current neurobiological knowledge that overcomes traditional one-to-one localization of functions in the brain. In particular, we propose that the structure-function relationship between brain and emotions is better described in terms of pluripotentiality, which refers to the fact that one neural structure can fulfill multiple functions, depending on the functional network and pattern of co-activations displayed at any given moment.

Emotional cue validity effects: The role of neurocognitive responses to emotion

The beneficial effect of valid compared to invalid cues on attention performance is a basic attentional mechanism, but the impact of emotional content on cue validity is poorly understood. We tested whether the effect of cue validity on attention performance differed when cues were angry, happy, or neutral faces. Moreover, we used scalp-recorded event-related potentials (ERPs) reflecting the capture of early visual attention (P1, N170) to test whether effects were strengthened when neurocognitive responses to angry or happy cues were enhanced (larger P1 and N170 amplitudes). Twenty-five participants completed a modified flanker task using emotional face cues to measure the effects of emotion on conflict interference. Attention performance was enhanced following valid versus invalid cues, but effects did not differ by emotion cue type. However, for participants showing relatively larger N170 amplitudes to angry face cues, attention performance was specifically disrupted on those trials. Conversely, participants with relatively larger N170 amplitudes to happy face cues showed facilitated performance across all valid trials. These findings suggest that individual neurocognitive sensitivities to emotion predict the impact of emotional content on the basic attentional phenomenon of cue validity.

The effect of constraining eye-contact during dynamic emotional face perception-an fMRI study

Eye-contact modifies how we perceive emotions and modulates activity in the social brain network. Here, using fMRI, we demonstrate that adding a fixation cross in the eye region of dynamic facial emotional stimuli significantly increases activation in the social brain of healthy, neurotypical participants when compared with activation for the exact same stimuli observed in a free-viewing mode. In addition, using PPI analysis, we show that the degree of amygdala connectivity with the rest of the brain is enhanced for the constrained view for all emotions tested except for fear, and that anxiety and alexithymia modulate the strength of amygdala connectivity for each emotion differently. Finally, we show that autistic traits have opposite effects on amygdala connectivity for fearful and angry emotional expressions, suggesting that these emotions should be treated separately in studies investigating facial emotion processing.

Neural correlates of facial expression processing during a detection task: An ERP study

Given finite attentional resources, how emotional aspects of stimuli are processed automatically is controversial. Present study examined the time-course for automatic processing of facial expression by assessing N170, and late positive potentials (LPPs) of event-related potentials (ERPs) using a modified rapid serial visual presentation (RSVP) paradigm. Observers were required to confirm a certain house image and to detect whether a face image was presented at the end of a series of pictures. There were no significant main effects on emotional type for P1 amplitudes, whereas happy and fearful expressions elicited larger N170 amplitudes than neutral expressions. Significantly different LPP amplitudes were elicited depending on the type of emotional facial expressions (fear > happy > neutral). These results indicated that threatening priority was absent but discrimination of expressive vs. neutral faces occurred in implicit emotional tasks, at approximately 250 ms post-stimulus. Moreover, the three types of expressions were discriminated during the later stages of processing. Encoding emotional information of faces can be automated to a relatively higher degree, when attentional resources are mostly allocated to superficial analyzing.

Classification of Emotional Expressions Is Affected by Inversion: Behavioral and Electrophysiological Evidence

It has been shown that emotionally positive facial expressions are recognized substantially faster than emotionally negative facial expressions, the positive classification advantage (PCA). In this experiment we explored the involvement of configural computations while processing positive and negative faces in an expression categorization task using artificial faces. Analyzing the reaction times (RTs), we found that happy faces were categorized more quickly than sad faces (PCA) and this effect disappeared for inverted faces. Event-related potentials (ERPs) data showed that the face-sensitive N170 component was larger for sad than for happy faces only at upright condition and that face inversion significantly enhanced N170 amplitudes only for happy faces. Moreover, the happy faces elicited shorter N170 latency than did the sad faces, whereas for inverted condition the N170 latency did not differ between happy and sad faces. Finally, the significant positive correlation between the RTs and the latency of the N170 was not found for N170 amplitudes. Because the configural computation was task-irrelevant in the present study, these behavioral and ERP data indicated that one of the sources of PCA is the configural analysis applied by default while categorizing facial emotions.

Event-Related Potentials of Bottom-Up and Top-Down Processing of Emotional Faces

INTRODUCTION

Emotional stimulus is processed automatically in a bottom-up way or can be processed voluntarily in a top-down way. Imaging studies have indicated that bottom-up and top-down processing are mediated through different neural systems. However, temporal differentiation of top-down versus bottom-up processing of facial emotional expressions has remained to be clarified. The present study aimed to explore the time course of these processes as indexed by the emotion-specific P100 and late positive potential (LPP) event-related potential (ERP) components in a group of healthy women.

METHODS

Fourteen female students of Alzahra University, Tehran, Iran aged 18-30 years, voluntarily participated in the study. The subjects completed 2 overt and covert emotional tasks during ERP acquisition.

RESULTS

The results indicated that fearful expressions significantly produced greater P100 amplitude compared to other expressions. Moreover, the P100 findings showed an interaction between emotion and processing conditions. Further analysis indicated that within the overt condition, fearful expressions elicited more P100 amplitude compared to other emotional expressions. Also, overt conditions created significantly more LPP latencies and amplitudes compared to covert conditions.

CONCLUSION

Based on the results, early perceptual processing of fearful face expressions is enhanced in top-down way compared to bottom-up way. It also suggests that P100 may reflect an attentional bias toward fearful emotions. However, no such differentiation was observed within later processing stages of face expressions, as indexed by the ERP LPP component, in a top-down versus bottom-up way. Overall, this study provides a basis for further exploring of bottom-up and top-down processes underlying emotion and may be typically helpful for investigating the temporal characteristics associated with impaired emotional processing in psychiatric disorders.

The Language, Tone and Prosody of Emotions: Neural Substrates and Dynamics of Spoken-Word Emotion Perception

Rapid assessment of emotions is important for detecting and prioritizing salient input. Emotions are conveyed in spoken words via verbal and non-verbal channels that are mutually informative and unveil in parallel over time, but the neural dynamics and interactions of these processes are not well understood. In this paper, we review the literature on emotion perception in faces, written words, and voices, as a basis for understanding the functional organization of emotion perception in spoken words. The characteristics of visual and auditory routes to the amygdala—a subcortical center for emotion perception—are compared across these stimulus classes in terms of neural dynamics, hemispheric lateralization, and functionality. Converging results from neuroimaging, electrophysiological, and lesion studies suggest the existence of an afferent route to the amygdala and primary visual cortex for fast and subliminal processing of coarse emotional face cues. We suggest that a fast route to the amygdala may also function for brief non-verbal vocalizations (e.g., laugh, cry), in which emotional category is conveyed effectively by voice tone and intensity. However, emotional prosody which evolves on longer time scales and is conveyed by fine-grained spectral cues appears to be processed via a slower, indirect cortical route. For verbal emotional content, the bulk of current evidence, indicating predominant left lateralization of the amygdala response and timing of emotional effects attributable to speeded lexical access, is more consistent with an indirect cortical route to the amygdala. Top-down linguistic modulation may play an important role for prioritized perception of emotions in words. Understanding the neural dynamics and interactions of emotion and language perception is important for selecting potent stimuli and devising effective training and/or treatment approaches for the alleviation of emotional dysfunction across a range of neuropsychiatric states.

Facial Cosmetics Exert a Greater Influence on Processing of the Mouth Relative to the Eyes: Evidence from the N170 Event-Related Potential Component

Cosmetic makeup significantly influences facial perception. Because faces consist of similar physical structures, cosmetic makeup is typically used to highlight individual features, particularly those of the eyes (i.e., eye shadow) and mouth (i.e., lipstick). Though event-related potentials have been utilized to study various aspects of facial processing, the influence of cosmetics on specific ERP components remains unclear. The present study aimed to investigate the relationship between the application of cosmetic makeup and the amplitudes of the P1 and N170 event-related potential components during facial perception tasks. Moreover, the influence of visual perception on N170 amplitude, was evaluated under three makeup conditions: Eye Shadow, Lipstick, and No Makeup. Electroencephalography was used to monitor 17 participants who were exposed to visual stimuli under each these three makeup conditions. The results of the present study subsequently demonstrated that the Lipstick condition elicited a significantly greater N170 amplitude than the No Makeup condition, while P1 amplitude was unaffected by any of the conditions. Such findings indicate that the application of cosmetic makeup alters general facial perception but exerts no influence on the perception of low-level visual features. Collectively, these results support the notion that the application of makeup induces subtle alterations in the processing of facial stimuli, with a particular effect on the processing of specific facial components (i.e., the mouth), as reflected by changes in N170 amplitude.

Biological Computation Indexes of Brain Oscillations in Unattended Facial Expression Processing Based on Event-Related Synchronization/Desynchronization

Estimation of human emotions from Electroencephalogram (EEG) signals plays a vital role in affective Brain Computer Interface (BCI). The present study investigated the different event-related synchronization (ERS) and event-related desynchronization (ERD) of typical brain oscillations in processing Facial Expressions under nonattentional condition. The results show that the lower-frequency bands are mainly used to update Facial Expressions and distinguish the deviant stimuli from the standard ones, whereas the higher-frequency bands are relevant to automatically processing different Facial Expressions. Accordingly, we set up the relations between each brain oscillation and processing unattended Facial Expressions by the measures of ERD and ERS. This research first reveals the contributions of each frequency band for comprehension of Facial Expressions in preattentive stage. It also evidences that participants have emotional experience under nonattentional condition. Therefore, the user's emotional state under nonattentional condition can be recognized in real time by the ERD/ERS computation indexes of different frequency bands of brain oscillations, which can be used in affective BCI to provide the user with more natural and friendly ways.

Affective Priming by Simple Geometric Shapes: Evidence from Event-related Brain Potentials

Previous work has demonstrated that simple geometric shapes may convey emotional meaning using various experimental paradigms. However, whether affective meaning of simple geometric shapes can be automatically activated and influence the evaluations of subsequent stimulus is still unclear. Thus the present study employed an affective priming paradigm to investigate whether and how two geometric shapes (circle vs. downward triangle) impact on the affective processing of subsequently presented faces (Experiment 1) and words (Experiment 2). At behavioral level, no significant effect of affective congruency was found. However, ERP results in Experiment 1 and 2 showed a typical effect of affective congruency. The LPP elicited by affectively incongruent trials was larger compared to congruent trials. Our results provide support for the notion that downward triangle is perceived as negative and circle as positive and their emotional meaning can be activated automatically and then exert an influence on the electrophysiological processing of subsequent stimuli. The lack of significant congruent effect in behavioral measures and the inversed N400 congruent effect might reveal that the affective meaning of geometric shapes is weak because they are just abstract threatening cues rather than real threat. In addition, because no male participants are included in the present study, our findings are limited to females.

Effect of Affective Personality Information on Face Processing: Evidence from ERPs

This study explored the extent to which there are the neural correlates of the affective personality influence on face processing using event-related potentials (ERPs). In the learning phase, participants viewed a target individual’s face (expression neutral or faint smile) paired with either negative, neutral or positive sentences describing previous typical behavior of the target. In the following EEG testing phase, participants completed gender judgments of the learned faces. Statistical analyses were conducted on measures of neural activity during the gender judgment task. Repeated measures ANOVA of ERP data showed that faces described as having a negative personality elicited larger N170 than did those with a neutral or positive description. The early posterior negativity (EPN) showed the same result pattern, with larger amplitudes for faces paired with negative personality than for others. The size of the late positive potential was larger for faces paired with positive personality than for those with neutral and negative personality. The current study indicates that affective personality information is associated with an automatic, top–down modulation on face processing.

The Interface Between Emotion and Attention: A Review of Evidence from Psychology and Neuroscience

This review addresses the interconnections between emotional and attentional processing, with an emphasis on both behavioral and neuroscientific findings. Are emotional stimuli encoded automatically, and what does that mean? How are emotional stimuli selected for enhanced processing within a limited capacity system? Evidence suggests a two-stage process: First, emotional significance is evaluated preattentively by a subcortical circuit involving the amygdala; and second, stimuli deemed emotionally significant are given priority in the competition for access to selective attention. This process involves bottom-up inputs from the amygdala as well as top-down influences from frontal lobe regions involved in goal setting and maintaining representations in working memory. The review highlights limitations in the current literature, directions for fruitful future research, and the need to move beyond simple dichotomies such as “cognition” versus “emotion.”

The Idea Is Good, but…: Failure to Replicate Associations of Oxytocinergic Polymorphisms with Face-Inversion in the N170

BACKGROUND

In event-related potentials, the N170 manifests itself especially in reaction to faces. In the healthy population, face-inversion leads to stronger negative amplitudes and prolonged latencies of the N170, effects not being present in patients with autism-spectrum-disorder (ASD). ASD has frequently been associated with differences in oxytocinergic neurotransmission. This ERP-study aimed to investigate the face-inversion effect in association with oxytocinergic candidate genes. It was expected that risk-allele-carriers of the oxytocin-receptor-gene-polymorphism (rs53576) and of CD38 (rs379863) responded similar to upright and inverted faces as persons with ASD. Additionally, reactions to different facial emotional expressions were studied. As there have been difficulties with replications of those molecular genetic association studies, we aimed to replicate our findings in a second study.

METHOD

Seventy-two male subjects in the first-, and seventy-eight young male subjects in the replication-study conducted a face-inversion-paradigm, while recording EEG. DNA was extracted from buccal cells.

RESULTS

Results revealed stronger N170-amplitudes and longer latencies in reaction to inverted faces in comparison to upright ones. Furthermore, effects of emotion on N170 were evident. Those effects were present in the first and in the second study. Whereas we found molecular-genetic associations of oxytocinergic polymorphisms with the N170 in the first study, we failed to do so in the replication sample.

CONCLUSION

Results indicate that a deeper theoretical understanding of this research-field is needed, in order to generate possible explanations for these findings. Results, furthermore, support the hypotheses that success of reproducibility is correlated with strength of lower original p-values and larger effect sizes in the original study.

Deficits in Early Stages of Face Processing in Schizophrenia: A Systematic Review of the P100 Component

INTRODUCTION

Schizophrenia is associated with deficits in face and affect recognition, which contribute to broader social functioning deficits. The present aim was to conduct a meta-analysis of early face processing in schizophrenia, as indexed by the P100 event-related potential component.

METHODS

Twelve studies (n = 328 patients with schizophrenia, n = 330 healthy controls) of the P100 component during face processing were evaluated by calculating Cohen's d for each study and overall weighted mean effect size (ES). In additional exploratory analyses, moderating influences of method and design were investigated, and the P100 component during face processing was evaluated based on valence: 5 studies (n = 225 patients, n = 225 controls) included neutral stimuli, 5 studies (n = 225 patients, n = 225 controls) included happy stimuli, and 4 studies (n = 209 patients, n = 209 controls) included fearful stimuli.

RESULTS

The amplitude of the P100 to face stimuli was smaller in patients relative to controls (ES = .41, P < .01). Methodological or design differences did not account for heterogeneity in ES. When split by valence, results indicate smaller P100 in patients relative to control subjects in response to neutral (ES = .32, P < .001) and happy (ES = .21, P < .05) stimuli, whereas there was no difference in response to fearful faces (ES = .09, P > .05).

DISCUSSION

The results indicate that P100 amplitude in response to faces is smaller in patients with schizophrenia, showing that socially relevant visual processing deficits begin earlier in processing than previously suggested. Additionally, the exploratory analyses suggest emotional specificity in these deficits. Ramifications for our understanding of face processing deficits and treatment development are discussed.

Role of the first and second person perspective for control of behaviour: Understanding other people's facial expressions

Humans typically make probabilistic inferences about another person's affective state based on her/his bodily movements such as emotional facial expressions, emblematic gestures and whole body movements. Furthermore, humans deduce tentative predictions about the other person's intentions. Thus, the first person perspective of a subject is supplemented by the second person perspective involving theory of mind and empathy. Neuroimaging investigations have shown that the medial and lateral frontal cortex are critical nodes in the circuits underlying theory of mind, empathy, as well as intention of action. It is suggested that personal perspective taking in social interactions is paradigmatic for the capability of humans to generate probabilistic accounts of the outside world that underlie a person's control of behaviour.

Emotional facial expressions evoke faster orienting responses, but weaker emotional responses at neural and behavioural levels compared to scenes: A simultaneous EEG and facial EMG study

In the current study, electroencephalography (EEG) was recorded simultaneously with facial electromyography (fEMG) to determine whether emotional faces and emotional scenes are processed differently at the neural level. In addition, it was investigated whether these differences can be observed at the behavioural level via spontaneous facial muscle activity. Emotional content of the stimuli did not affect early P1 activity. Emotional faces elicited enhanced amplitudes of the face-sensitive N170 component, while its counterpart, the scene-related N100, was not sensitive to emotional content of scenes. At 220-280ms, the early posterior negativity (EPN) was enhanced only slightly for fearful as compared to neutral or happy faces. However, its amplitudes were significantly enhanced during processing of scenes with positive content, particularly over the right hemisphere. Scenes of positive content also elicited enhanced spontaneous zygomatic activity from 500-750ms onwards, while happy faces elicited no such changes. Contrastingly, both fearful faces and negative scenes elicited enhanced spontaneous corrugator activity at 500-750ms after stimulus onset. However, relative to baseline EMG changes occurred earlier for faces (250ms) than for scenes (500ms) whereas for scenes activity changes were more pronounced over the whole viewing period. Taking into account all effects, the data suggests that emotional facial expressions evoke faster attentional orienting, but weaker affective neural activity and emotional behavioural responses compared to emotional scenes.

Effects of an Attachment-Based Intervention on Child Protective Services--Referred Mothers' Event-Related Potentials to Children's Emotions

This study examined the neurobiology of maternal sensitivity to children's emotions among mothers involved with Child Protective Services (CPS) and low-risk comparison mothers (M(age) = 31.6 years). CPS-referred mothers participated in the Attachment and Biobehavioral Catch-up (ABC) intervention or a control intervention. Mothers' event-related potentials (ERPs) were measured while they categorized images of children with crying, laughing, and neutral expressions. CPS-referred ABC mothers (n = 19) and low-risk comparison mothers (n = 30) showed a larger enhancement of ERP responses for emotional faces relative to neutral faces than CPS-referred control mothers (n = 21). Additionally, the magnitude of ERP responses to emotional faces was associated with observed maternal sensitivity. Findings add to the understanding of the neurobiology of deficits in parenting and suggest that these deficits are changeable through a parenting intervention.

Time pressure inhibits dynamic advantage in the classification of facial expressions of emotion

Recent studies suggest an advantage in the recognition of dynamic over static facial expressions of emotion. Here, we explored the differences in the processing of static and dynamic faces under condition of time pressure. A group of 18 participants classified static and dynamic facial expressions (angry, happy, and neutral). In order to increase the goal-directed attention, instructions emphasized speed and announced time pressure in the interval for the response (maximal 600 ms). Participants responded faster and more accurately in the static than in the dynamic condition. Event-related potentials (ERPs) showed larger amplitude of the P1 (90–130 ms) and LPC (300–600 ms) components for dynamic relative to static stimuli, indicating enhanced early visual processing and emotional attention. On the other hand, the N170 was more negative in static relative to dynamic faces, suggesting better structural encoding for static faces under time pressure. The present study shows some advantages in the processing of static over dynamic facial expressions of emotion when the top-down (goal-driven) attention is strengthened.

An Intracranial EEG Study of the Neural Dynamics of Musical Valence Processing

The processing of valence is known to recruit the amygdala, orbitofrontal cortex, and relevant sensory areas. However, how these regions interact remains unclear. We recorded cortical electrical activity from 7 epileptic patients implanted with depth electrodes for presurgical evaluation while they listened to positively and negatively valenced musical chords. Time-frequency analysis suggested a specific role of the orbitofrontal cortex in the processing of positively valenced stimuli while, most importantly, Granger causality analysis revealed that the amygdala tends to drive both the orbitofrontal cortex and the auditory cortex in theta and alpha frequency bands, during the processing of valenced stimuli. Results from the current study show the amygdala to be a critical hub in the emotion processing network: specifically one that influences not only the higher order areas involved in the evaluation of a stimulus's emotional value but also the sensory cortical areas involved in the processing of its low-level acoustic features.

Three stages of emotional word processing: an ERP study with rapid serial visual presentation

Rapid responses to emotional words play a crucial role in social communication. This study employed event-related potentials to examine the time course of neural dynamics involved in emotional word processing. Participants performed a dual-target task in which positive, negative and neutral adjectives were rapidly presented. The early occipital P1 was found larger when elicited by negative words, indicating that the first stage of emotional word processing mainly differentiates between non-threatening and potentially threatening information. The N170 and the early posterior negativity were larger for positive and negative words, reflecting the emotional/non-emotional discrimination stage of word processing. The late positive component not only distinguished emotional words from neutral words, but also differentiated between positive and negative words. This represents the third stage of emotional word processing, the emotion separation. Present results indicated that, similar with the three-stage model of facial expression processing; the neural processing of emotional words can also be divided into three stages. These findings prompt us to believe that the nature of emotion can be analyzed by the brain independent of stimulus type, and that the three-stage scheme may be a common model for emotional information processing in the context of limited attentional resources.

Temporal and spatial neural dynamics in the perception of basic emotions from complex scenes

The different temporal dynamics of emotions are critical to understand their evolutionary role in the regulation of interactions with the surrounding environment. Here, we investigated the temporal dynamics underlying the perception of four basic emotions from complex scenes varying in valence and arousal (fear, disgust, happiness and sadness) with the millisecond time resolution of Electroencephalography (EEG). Event-related potentials were computed and each emotion showed a specific temporal profile, as revealed by distinct time segments of significant differences from the neutral scenes. Fear perception elicited significant activity at the earliest time segments, followed by disgust, happiness and sadness. Moreover, fear, disgust and happiness were characterized by two time segments of significant activity, whereas sadness showed only one long-latency time segment of activity. Multidimensional scaling was used to assess the correspondence between neural temporal dynamics and the subjective experience elicited by the four emotions in a subsequent behavioral task. We found a high coherence between these two classes of data, indicating that psychological categories defining emotions have a close correspondence at the brain level in terms of neural temporal dynamics. Finally, we localized the brain regions of time-dependent activity for each emotion and time segment with the low-resolution brain electromagnetic tomography. Fear and disgust showed widely distributed activations, predominantly in the right hemisphere. Happiness activated a number of areas mostly in the left hemisphere, whereas sadness showed a limited number of active areas at late latency. The present findings indicate that the neural signature of basic emotions can emerge as the byproduct of dynamic spatiotemporal brain networks as investigated with millisecond-range resolution, rather than in time-independent areas involved uniquely in the processing one specific emotion.

Phasic boosting of auditory perception by visual emotion

Emotionally negative stimuli boost perceptual processes. There is little known, however, about the timing of this modulation. The present study aims at elucidating the phasic effects of, emotional processing on auditory processing within subsequent time-windows of visual emotional, processing in humans. We recorded the electroencephalogram (EEG) while participants responded to a, discrimination task of faces with neutral or fearful expressions. A brief complex tone, which subjects, were instructed to ignore, was displayed concomitantly, but with different asynchronies respective to, the image onset. Analyses of the N1 auditory event-related potential (ERP) revealed enhanced brain, responses in presence of fearful faces. Importantly, this effect occurred at picture-tone asynchronies of, 100 and 150ms, but not when these were displayed simultaneously, or at 50ms or 200ms asynchrony. These results confirm the existence of a fast-operating crossmodal effect of visual emotion on auditory, processing, suggesting a phasic variation according to the time-course of emotional processing.