Neural correlates of facial expression processing during a detection task: An ERP study

Influence of stimulus manipulation on conscious awareness of emotional facial expressions in the match-to-sample paradigm

The conscious perception of emotional facial expressions plays an indispensable role in social interaction. However, previous psychological studies have reported inconsistent findings regarding whether conscious awareness is greater for emotional expressions than for neutral expressions. Furthermore, whether this phenomenon is attributable to emotional or visual factors remains unknown. To investigate these issues, we conducted five psychological experiments to test the conscious perception of emotional and neutral facial expressions using the match-to-sample paradigm. Facial stimuli were momentarily presented in the peripheral visual fields while participants read simultaneously presented letters in the central visual fields. The participants selected a perceived face from nine samples. The results of all experiments demonstrated that emotional expressions were more accurately identified than neutral expressions. Furthermore, Experiment 4 showed that angry expressions were identified more accurately than anti-angry expressions, which expressed neutral emotions with comparable physical changes to angry expressions. Experiment 5, testing the interaction between emotional expression and face direction, showed that angry expressions looking toward participants were more accurately identified than those looking away from participants, even though they were physically identical. These results suggest that the conscious awareness of emotional facial expressions is enhanced by their emotional significance.

Neural correlates of conscious processing of emotional faces: Evidence from event-related potentials

There is a theoretical debate between the early and late neural correlates of consciousness (NCCs). Previous studies using neutral face stimuli supported an early NCC and suggested that visual awareness negativity (VAN) is associated with consciousness, while late positivity (LP) reflects post-perceptual activity. However, emotional faces may help to examine the relationship between LP and consciousness due to the differences in late processing between emotional and neutral faces. To explore the effects of facial emotional information on NCCs, the present study manipulated consciousness with the inattentional blindness paradigm and used happy, fearful, and neutral faces as visual stimuli. The results showed that the conscious processing of emotional faces was correlated with VAN and LP, while the conscious processing of neutral faces was associated with VAN. First, the results suggest that VAN is an NCC, and the relationship between LP and consciousness is affected by facial emotional information. Second, VAN reflects the early perceptual experience of emotional faces, whereas LP may reflect the late conscious processing of emotional faces. Furthermore, source localization analysis showed that the LPs of emotional faces were mainly located in the frontal and parietal lobes, whereas those of neutral faces showed no significant activation. This suggests that facial emotional information may affect the brain regions associated with conscious processing. Time-frequency analysis showed that conscious processing is related to the enhancement of alpha and theta oscillation, indicating that conscious processing may be associated with the suppression of irrelevant stimuli. Overall, the present study suggests that the integration of the theories that support early and late NCCs helps explain the conscious processing of emotional faces.

Emotional Valence in the Eye Region Modulates the Attentional Blink in a Task-Dependent Manner: Evidence from Event-Related Potentials

Although emotional expressions conveyed by the eye regions are processed efficiently, little is known regarding the relationship between emotional processing of isolated eye regions and temporal attention. In this study, we conducted three rapid serial visual presentation (RSVP) experiments with varying task demands (emotion discrimination, eye detection, eyes ignored) related to the first target (T1) to investigate how the perception of emotional valence in the eye region (T1: happy, neutral, fearful) impacts the identification of a second target (T2: neutral houses). Event-related potential (ERP) findings indicated that fearful stimuli reliably increased N170 amplitude regardless of the emotional relevance of task demands. The P3 component exhibited enhanced responses to happy and fearful stimuli in the emotion discrimination task and to happy eye regions in the eye detection task. Analysis of T2-related ERPs within the attentional blink period revealed that T2 houses preceded by fearful and happy stimuli elicited larger N2 and P3 amplitudes than those preceded by neutral stimuli only in the emotion discrimination task. Together, these findings indicate that attention to affective content conveyed by the eyes can not only amplify the perceptual analysis of emotional eye regions but also facilitate the processing of a subsequent target.

Event‐related potential response to drivers' facial expressions in an online car‐hailing scene

Recognizing facial expressions is crucial for adaptive social interaction. Prior empirical research on facial expression processing has primarily focused on isolated faces; however, facial expressions appear embedded in surrounding scenes in everyday life. In this study, we attempted to demonstrate how the online car-hailing scene affects the processing of facial expression. This study examined the processing of drivers' facial expressions in scenes by recording event-related potentials, in which neutral or happy faces embedded in online car-hailing orders were constructed (with type of vehicle, driver rating, driver surname, and level of reputation controlled). A total of 35 female volunteers participated in this experiment and were asked to judge which facial expressions that emerged in scenes of online car-hailing were more trustworthy. The results revealed an interaction between facial expression scenes, brain areas, and electrode sites in the late positive potential, which indicated that happy faces elicited larger amplitudes than did neutral ones in the parietal areas and that scenes with happy facial expressions had shorter latencies than did those with neutral ones. As expected, the late positive potential evoked by happy facial expressions in a scene was larger than that evoked by neutral ones, which reflected motivated attention and motivational response processes. This study highlights the importance of scenes as context in the study of facial expression processing.

Behavioral and neurophysiological correlates of emotional face processing in borderline personality disorder: are there differences between men and women?

Emotional dysregulation is a core feature of borderline personality disorder (BPD); it is, for example, known to influence one's ability to read other people's facial expressions. We investigated behavioral and neurophysiological foundations of emotional face processing in individuals with BPD and in healthy controls, taking participants' sex into account. 62 individuals with BPD (25 men, 37 women) and 49 healthy controls (20 men, 29 women) completed an emotion classification task with faces depicting blends of angry and happy expressions while the electroencephalogram was recorded. The cortical activity (late positive potential, P3/LPP) was evaluated using source modeling. Compared to healthy controls, individuals with BPD responded slower to happy but not to angry faces; further, they showed more anger ratings in happy but not in angry faces, especially in those with high ambiguity. Men had lower anger ratings than women and responded slower to angry but not happy faces. The P3/LPP was larger in healthy controls than in individuals with BPD, and larger in women than in men; moreover, women but not men produced enlarged P3/LPP responses to angry vs. happy faces. Sex did not interact with behavioral or P3/LPP-related differences between healthy controls and individuals with BPD. Together, BPD-related alterations in behavioral and P3/LPP correlates of emotional face processing exist in both men and women, supposedly without sex-related interactions. Results point to a general 'negativity bias' in women. Source modeling is well suited to investigate effects of participant and stimulus characteristics on the P3/LPP generators.

The effect of fearful faces on the attentional blink is modulated by emotional task relevance: An event-related potential study

A fearful face as second visual target (T2) was detected better than a neutral T2 in a Rapid Serial Visual Presentation (RSVP) task. The advantage of fear over neutral emotion was originally attributed to a limited-capacity mechanism, in which fearful stimuli are prioritized for attention over neutral stimuli. However, more recent studies have shown that the prioritization of the processing of fear is strongly dependent on the emotional task relevance. Combining the RSVP task and Garner's paradigm, by varying the expression (fearful and neutral faces) and the emotional task relevance of the T2 (relevance: emotion classification task; irrelevance: gender classification task), this study aims to investigate the role of emotional task relevance on the advantage of fear during an RSVP task in which participants have to identify two visual targets in a stream of distractors. The behavioral results revealed that there was no significant effect of the expression on the task performance in the gender classification task. Fearful faces were easier to detect than neutral faces, but the T2 accuracy of fearful faces was lower than that of neutral faces in the emotion classification task. Furthermore, we found that the vertex positive potential and P100 components were enhanced for fearful faces compared to neutral faces independent of the emotional task relevance. For the P300 component, there was no significant difference in the gender classification task, but fearful faces elicited enhanced P300 amplitudes compared to neutral faces in the emotion classification task. These results indicated that the early processing of fear is automatic, while the late processing of fear is dependent on the emotional task relevance under limited attentional resources.

An ERP study on facial emotion processing in young people with subjective memory complaints

Subjective memory complaints (SMCs) are commonly related to aging, but they are also presented by young adults. Their neurophysiological mechanisms are not thoroughly understood, although some aspects related to affective state have been mentioned. Here, we investigated whether facial emotion processing is different in young people with (n = 41) and without (n = 39) SMCs who were exposed to positive, negative, and neutral faces, by recording the event-related potential (ERP) activity. From the ERP activity, the N170 (an index of face processing) and the LPP (an index of motivated attention) components were extracted. Regarding the N170, results showed less amplitude for positive and neutral faces in the participants with SMCs than in those without SMCs. Moreover, women with SMCs displayed longer latencies for neutral faces than women without SMCs. No significant differences were found between the groups in the LPP component. Together, our findings suggest deficits in an early stage of facial emotion processing in young people with SMCs, and they emphasize the importance of further examining affective dimensions.

The early processing of fearful and happy facial expressions is independent of task demands - Support from mass univariate analyses

Most ERP studies on facial expressions of emotion have yielded inconsistent results regarding the time course of emotion effects and their possible modulation by task demands. Most studies have used classical statistical methods with a high likelihood of type I and type II errors, which can be limited with Mass Univariate statistics. FMUT and LIMO are currently the only two available toolboxes for Mass Univariate analysis of ERP data and use different fundamental statistics. Yet, no direct comparison of their output has been performed on the same dataset. Given the current push to transition to robust statistics to increase results replicability, here we compared the output of these toolboxes on data previously analyzed using classic approaches (Itier & Neath-Tavares, 2017). The early (0-352 ms) processing of fearful, happy, and neutral faces was investigated under three tasks in a within-subject design that also controlled gaze fixation location. Both toolboxes revealed main effects of emotion and task but neither yielded an interaction between the two, confirming the early processing of fear and happy expressions is largely independent of task demands. Both toolboxes found virtually no difference between neutral and happy expressions, while fearful (compared to neutral and happy) expressions modulated the N170 and EPN but elicited maximum effects after the N170 peak, around 190 ms. Similarities and differences in the spatial and temporal extent of these effects are discussed in comparison to the published classical analysis and the rest of the ERP literature.

Emotion processing in children who do and do not stutter: An ERP study of electrocortical reactivity and regulation to peer facial expressions

PURPOSE

Event-related brain potentials (ERPs) were used to investigate the neural correlates of emotion processing in 5- to 8-year-old children who do and do not stutter.

METHODS

Participants were presented with an audio contextual cue followed by images of threatening (angry/fearful) and neutral facial expressions from similarly aged peers. Three conditions differed in audio-image pairing: neutral context-neutral expression (neutral condition), negative context-threatening expression (threat condition), and reappraisal context-threatening expression (reappraisal condition). These conditions reflected social stimuli that are ecologically valid to the everyday life of children.

RESULTS

P100, N170, and late positive potential (LPP) ERP components were elicited over parietal and occipital electrodes. The threat condition elicited an increased LPP mean amplitude compared to the neutral condition across our participants, suggesting increased emotional reactivity to threatening facial expressions. In addition, LPP amplitude decreased during the reappraisal condition- evidence of emotion regulation. No group differences were observed in the mean amplitude of ERP components between children who do and do not stutter. Furthermore, dimensions of childhood temperament and stuttering severity were not strongly correlated with LPP elicitation.

CONCLUSION

These findings are suggestive that, at this young age, children who stutter exhibit typical brain activation underlying emotional reactivity and regulation to social threat from peer facial expressions.

Multimodal Recognition of Emotions in Music and Facial Expressions

The aim of the study was to investigate the neural processing of congruent vs. incongruent affective audiovisual information (facial expressions and music) by means of ERPs (Event Related Potentials) recordings. Stimuli were 200 infant faces displaying Happiness, Relaxation, Sadness, Distress and 32 piano musical pieces conveying the same emotional states (as specifically assessed). Music and faces were presented simultaneously, and paired so that in half cases they were emotionally congruent or incongruent. Twenty subjects were told to pay attention and respond to infrequent targets (adult neutral faces) while their EEG was recorded from 128 channels. The face-related N170 (160-180 ms) component was the earliest response affected by the emotional content of faces (particularly by distress), while visual P300 (250-450 ms) and auditory N400 (350-550 ms) responses were specifically modulated by the emotional content of both facial expressions and musical pieces. Face/music emotional incongruence elicited a wide N400 negativity indicating the detection of a mismatch in the expressed emotion. A swLORETA inverse solution applied to N400 (difference wave Incong. - Cong.), showed the crucial role of Inferior and Superior Temporal Gyri in the multimodal representation of emotional information extracted from faces and music. Furthermore, the prefrontal cortex (superior and medial, BA 10) was also strongly active, possibly supporting working memory. The data hints at a common system for representing emotional information derived by social cognition and music processing, including uncus and cuneus.

Emotional interference during conflict resolution depends on task context

Evidence is currently mixed regarding the way in which cognitive conflict modulates the effect of emotion on task performance. The present study aimed to address methodological differences across previous studies and investigate the conditions under which interference from emotional stimuli can either be elicited or eliminated under high cognitive conflict. Four behavioural experiments were conducted with a university sample using a gender-discrimination stimulus-response compatibility task. In line with our previous findings, Experiment 1 found that when emotion and cognitive conflict conditions were blocked, emotional faces increased reaction time interference during response compatible trials (low conflict) but not response incompatible trials (high conflict). However, when conflict and emotion conditions were randomised in different configurations across Experiments 2 (all trials randomised), 3 (emotion blocked, compatibility randomised) and 4 (compatibility blocked, emotion randomised), emotion interfered with task performance across both high and low conflict trials. These results suggest that predictability of both compatibility and emotion is required in order to obtain reduced emotional interference under high cognitive conflict. Consistent with prior reports, a top-down anticipatory control mechanism seems to be engaged in the presence of negative emotion when there are incompatible stimulus-response mappings.

Gender-specific effects of vasopressin on human social communication: An ERP study

The quick and efficient perception of facial expressions represents a special and fundamental capacity of humans to engage in social communication. Here, we examined the effects of vasopressin (AVP, a neuropeptide) on the processing of same- and other-gender facial expressions among males and females. After receiving either AVP or placebo (PBO) intranasally in a randomized and double-blind manner, participants were asked to rate their approachability to facial expressions while event-related potentials (ERPs) were recorded. Males rated lower approachability scores to neutral and positive male faces relative to the scores to emotion-matched female faces after AVP but not following PBO administration. These behavioral effects were correlated with the AVP-induced increased P1 and decreased N170 responses to male faces among male participants. Females rated higher approachability scores to negative female faces than the scores to negative male faces after AVP but not following PBO treatment. These results suggest that AVP decreases friendly responses to neutral/positive male faces in males and increases friendly responses to negative female faces in females. Overall, these results demonstrate the gender-specific effects of AVP in response to same- and other-gender facial expressions, indicating there are sex- and context-dependent effects of AVP on socioemotional processes.

Differential Effects of Intranasal Vasopressin on the Processing of Adult and Infant Cues: An ERP Study

Arginine vasopressin (AVP) is a powerful regulator of various social behaviors across many species. However, seemingly contradictory effects of AVP have been found in both animal and human studies, e.g., promoting aggression on one hand and facilitating social bonding on the other hand. Therefore, we hypothesize that the role of AVP in social behaviors is context-dependent. To this end, we examined the modulatory effect of AVP on male’s behavioral and neural responses to infant and adult cues. After intranasal and double-blind treatment of AVP or placebo, male participants were asked to rate their subjective approaching willingness to infant and adult faces in specific contexts informed by cue words while EEG recording. Our results showed that AVP treatment increased approaching ratings to neutral and positive other-gender adult faces compared to emotional matched same-gender adult faces, and to negative girl faces compared to negative boy faces. Furthermore, compared to placebo treatment, AVP treatment induced larger N1 amplitudes to neutral cues associated with both adults and infants, whereas AVP treatment only sustained pronounced late positive potential amplitudes to neutral cues of infants but not adults. Those findings implicate differential roles of AVP in the processing of adult- and infant-related cues and thus lend support to the context-dependent account.