Face and emotional expression processing under continuous perceptual load tasks: An ERP study

Is processing superiority a universal trait for all threats? Divergent impacts of fearful, angry, and disgusted faces on attentional capture

Fearful, angry, and disgusted facial expressions are evolutionarily salient and convey different types of threat signals. However, it remains unclear whether these three expressions impact sensory perception and attention in the same way. The present ERP study investigated the temporal dynamics underlying the processing of different types of threatening faces and the impact of attentional resources employed during a perceptual load task. Participants were asked to judge the length of bars superimposed over faces presented in the center of the screen. A mass univariate statistical approach was used to analyze the EEG data. Behaviorally, task accuracy was significantly reduced following exposure to fearful faces relative to neutral distractors, independent of perceptual load. The ERP results revealed that the P1 amplitude over the right hemisphere was found to be enhanced for fearful relative to disgusted faces, reflecting the rapid and coarse detection of fearful cues. The N170 responses elicited by fearful, angry, and disgusted faces were larger than those elicited by neutral faces, suggesting the largely automatic and preferential processing of threats. Furthermore, the early posterior negativity (EPN) component yielded increased responses to fearful and angry faces, indicating prioritized attention to stimuli representing acute threats. Additionally, perceptual load exerted a pronounced influence on the EPN and late positive potential (LPP), with larger responses observed in the low perceptual load condition, indicating goal-directed cognitive processing. Overall, the early sensory processing of fearful, angry, and disgusted faces is characterized by differential sensitivity in capturing attention automatically, despite the importance of these facial signals for survival. Fearful faces produce a strong interference effect and are processed with higher priority than angry and disgusted ones.

Implicit weight bias: shared neural substrates for overweight and angry facial expressions revealed by cross-adaptation

The perception of facial expression plays a crucial role in social communication, and it is known to be influenced by various facial cues. Previous studies have reported both positive and negative biases toward overweight individuals. It is unclear whether facial cues, such as facial weight, bias facial expression perception. Combining psychophysics and event-related potential technology, the current study adopted a cross-adaptation paradigm to examine this issue. The psychophysical results of Experiments 1A and 1B revealed a bidirectional cross-adaptation effect between overweight and angry faces. Adapting to overweight faces decreased the likelihood of perceiving ambiguous emotional expressions as angry compared to adapting to normal-weight faces. Likewise, exposure to angry faces subsequently caused normal-weight faces to appear thinner. These findings were corroborated by bidirectional event-related potential results, showing that adaptation to overweight faces relative to normal-weight faces modulated the event-related potential responses of emotionally ambiguous facial expression (Experiment 2A); vice versa, adaptation to angry faces relative to neutral faces modulated the event-related potential responses of ambiguous faces in facial weight (Experiment 2B). Our study provides direct evidence associating overweight faces with facial expression, suggesting at least partly common neural substrates for the perception of overweight and angry faces.

Recognition and evaluation of mental workload in different stages of perceptual and cognitive information processing using a multimodal approach

This study explores the effects of different perceptual and cognitive information processing stages on mental workload by assessing multimodal indicators of mental workload such as the NASA-TLX, task performance, ERPs and eye movements. Repeated measures ANOVA of the data showed that among ERP indicators, P1, N1 and N2 amplitudes were sensitive to perceptual load (P-load), P3 amplitude was sensitive to P-load only in the prefrontal region during high cognitive load (C-load) states, and P3 amplitude in the occipital and parietal regions was sensitive to C-load. Among the eye movement indicators, blink frequency was sensitive to P-load in all C-load states, but to C-load in only low P-load states; pupil diameter and blink duration were sensitive to both P-load and C-load. Based on the above indicators, the k-nearest neighbours (KNN) algorithm was used to propose a classification method for the four different mental workload states with an accuracy of 97.89%.Practitioner summary: Based on the results of this study, it is possible to implement the monitoring of mental workload states and optimise brain task allocation in operations involving high mental workload, such as human-computer interaction.

Beyond facial expressions: A systematic review on effects of emotional relevance of faces on the N170

The N170 is the most prominent electrophysiological signature of face processing. While facial expressions reliably modulate the N170, there is considerable variance in N170 modulations by other sources of emotional relevance. Therefore, we systematically review and discuss this research area using different methods to manipulate the emotional relevance of inherently neutral faces. These methods were categorized into (1) existing pre-experimental affective person knowledge (e.g., negative attitudes towards outgroup faces), (2) experimentally instructed affective person knowledge (e.g., negative person information), (3) contingency-based affective learning (e.g., fear-conditioning), or (4) the immediate affective context (e.g., emotional information directly preceding the face presentation). For all categories except the immediate affective context category, the majority of studies reported significantly increased N170 amplitudes depending on the emotional relevance of faces. Furthermore, the potentiated N170 was observed across different attention conditions, supporting the role of the emotional relevance of faces on the early prioritized processing of configural facial information, regardless of low-level differences. However, we identified several open research questions and suggest venues for further research.

Visual perceptual load and processing of somatosensory stimuli in primary and secondary somatosensory cortices

Load theory assumes that neural activation to distractors in early sensory cortices is modulated by the perceptual load of a main task, regardless of whether task and distractor share the same sensory modality or not. While several studies have investigated the question of load effects on distractor processing in early sensory areas, there is no functional magnetic resonance imaging (fMRI) study regarding load effects on somatosensory stimuli. Here, we used fMRI to investigate effects of visual perceptual load on neural responses to somatosensory stimuli applied to the wrist in a study with 44 participants. Perceptual load was manipulated by an established sustained visual detection task, which avoided simultaneous target and distractor presentations. Load was operationalized by detection difficulty of subtle or clear color changes of one of 12 rotating dots. While all somatosensory stimuli led to activation in somatosensory areas SI and SII, we found no statistically significant difference in brain activation to these stimuli under high compared to low sustained visual load. Moreover, exploratory Bayesian analyses supported the absence of differences. Thus, our findings suggest a resistance of somatosensory processing to at least some forms of visual perceptual load, possibly due to behavioural relevance of discrete somatosensory stimuli and separable attentional resources for the somatosensory and visual modality.

The effects of visual perceptual load on detection performance and event-related potentials to auditory stimuli

Load Theory states that perceptual load prevents, or at least reduces, the processing of task-unrelated stimuli. This study systematically examined the detection and neural processing of auditory stimuli unrelated to a visual foreground task. The visual task was designed to create continuous perceptual load, alternated between low and high load, and contained performance feedback to motivate participants to focus on the visual task instead of the auditory stimuli presented in the background. The auditory stimuli varied in intensity, and participants signaled their subjective perception of these stimuli without receiving feedback. Depending on stimulus intensity, we observed load effects on detection performance and P3 amplitudes of the event-related potential (ERP). N1 amplitudes were unaffected by perceptual load, as tested by Bayesian statistics. Findings suggest that visual perceptual load affects the processing of auditory stimuli in a late time window, which is associated with a lower probability of reported awareness of these stimuli.

Potentiated early neural responses to fearful faces are not driven by specific face parts

Prioritized processing of fearful compared to neutral faces is reflected in increased amplitudes of components of the event-related potential (ERP). It is unknown whether specific face parts drive these modulations. Here, we investigated the contributions of face parts on ERPs to task-irrelevant fearful and neutral faces using an ERP-dependent facial decoding technique and a large sample of participants (N = 83). Classical ERP analyses showed typical and robust increases of N170 and EPN amplitudes by fearful relative to neutral faces. Facial decoding further showed that the absolute amplitude of these components, as well as the P1, was driven by the low-frequency contrast of specific face parts. However, the difference between fearful and neutral faces was not driven by any specific face part, as supported by Bayesian statistics. Furthermore, there were no correlations between trait anxiety and main effects or interactions. These results suggest that increased N170 and EPN amplitudes to task-irrelevant fearful compared to neutral faces are not driven by specific facial regions but represent a holistic face processing effect.

The effects of visual working memory load on detection and neural processing of task-unrelated auditory stimuli

While perceptual load has been proposed to reduce the processing of task-unrelated stimuli, theoretical arguments and empirical findings for other forms of task load are inconclusive. Here, we systematically investigated the detection and neural processing of auditory stimuli varying in stimulus intensity during a stimuli-unrelated visual working memory task alternating between low and high load. We found, depending on stimulus strength, decreased stimulus detection and reduced P3, but unaffected N1 amplitudes of the event-related potential to auditory stimuli under high as compared to low load. In contrast, load independent awareness effects were observed during both early (N1) and late (P3) time windows. Findings suggest a late neural effect of visual working memory load on auditory stimuli leading to lower probability of reported awareness of these stimuli.

Testing stimulus exposure time as the critical factor of increased EPN and LPP amplitudes for fearful faces during perceptual distraction tasks

Fearful facial expressions are prioritized across different information processing stages, as evident in early, intermediate, and late components of event-related brain potentials (ERPs). Recent studies showed that, in contrast to early N170 modulations, mid-latency (Early Posterior Negativity, EPN) and late (Late Positive Potential, LPP) emotional modulations depend on the attended perceptual feature. Nevertheless, several studies reported significant differences between emotional and neutral faces for the EPN or LPP components during distraction tasks. One cause for these conflicting findings might be that when faces are presented sufficiently long, participants attend to task-irrelevant features of the faces. In this registered report, we tested whether the presentation duration of faces is the critical factor for differences between reported emotional modulations during perceptual distraction tasks. To this end, 48 participants were required to discriminate the orientation of lines overlaid onto fearful or neutral faces, while face presentation varied (100 msec, 300 msec, 1,000 msec, 2,000 msec). While participants did not need to pay attention to the faces, we observed main effects of emotion for the EPN and LPP, but no interaction between emotion and presentation duration. Of note, unregistered exploratory tests per presentation duration showed no significant EPN and LPP emotion differences during short durations (100 and 300 msec) but significant differences with longer durations. While the presentation duration seems not to be a critical factor for EPN and LPP emotion effects, future studies are needed to investigate the role of threshold effects and the applied analytic designs to explain conflicting findings in the literature.

Estimation Strategy Utilization Is Modulated by Implicit Emotion Regulation: Evidence from Behavioral and Event-Related Potentials Studies

A large number of studies have studied the influence of emotional experience on an individual's estimation performance, but the influence of implicit emotion regulation is still unknown. Participants were asked to complete the following tasks in order: idiom matching task, multiplication computational estimation task (MCE task), gender judgment task (GJ task), and emotional experience intensity assessment task. The words matching task was adopted to achieve the purpose of implicit emotion regulation (implicit reappraisal and implicit suppression). Behavioral results showed that implicit reappraisal and implicit suppression equally contributed to improving an individual's estimation speed (but not ACC (accuracy)). The MCE task related ERP (event-related potential) results showed that the influence of implicit emotion regulation on estimation consisted of two phases. In the first phase (encoding phase), implicit reappraisal both enhanced (larger P1 amplitudes) and weakened (smaller N170 amplitudes) an individual's encoding sensitivity, while implicit suppression enhanced an individual's encoding sensitivity (larger P1 amplitudes). In the second phase (estimation strategies retrieval phase), implicit reappraisal (but not implicit suppression) cost more attention resources (larger LPC2 and LPC3 amplitudes). The present study suggested that both implicit reappraisal and implicit suppression contributed to improving an individual's estimation performance, and the regulation effect of implicit suppression (vs. implicit reappraisal) was better.

Emotional Valence in the Eye Region Modulates the Attentional Blink in a Task-Dependent Manner: Evidence from Event-Related Potentials

Although emotional expressions conveyed by the eye regions are processed efficiently, little is known regarding the relationship between emotional processing of isolated eye regions and temporal attention. In this study, we conducted three rapid serial visual presentation (RSVP) experiments with varying task demands (emotion discrimination, eye detection, eyes ignored) related to the first target (T1) to investigate how the perception of emotional valence in the eye region (T1: happy, neutral, fearful) impacts the identification of a second target (T2: neutral houses). Event-related potential (ERP) findings indicated that fearful stimuli reliably increased N170 amplitude regardless of the emotional relevance of task demands. The P3 component exhibited enhanced responses to happy and fearful stimuli in the emotion discrimination task and to happy eye regions in the eye detection task. Analysis of T2-related ERPs within the attentional blink period revealed that T2 houses preceded by fearful and happy stimuli elicited larger N2 and P3 amplitudes than those preceded by neutral stimuli only in the emotion discrimination task. Together, these findings indicate that attention to affective content conveyed by the eyes can not only amplify the perceptual analysis of emotional eye regions but also facilitate the processing of a subsequent target.

Om chanting modulates the processing of negative stimuli: Behavioral and electrophysiological evidence

Previous studies have suggested that Om chanting, a type of meditation, can relieve individuals' negative emotions. However, the dynamic aspects of neural processes in the processing of the negative stimuli while Om chanting are still unclear. In this study, we recruited 33 healthy undergraduate students without meditation experience and recorded event-related potentials (ERPs) to unpleasant and neutral images when they performed Om chanting and viewing task. The behavioral results showed that the unpleasant images were rated as less unpleasant and arousing in the condition of Om chanting than while passive viewing, and the rates were not different between the two conditions for the neutral images. Analyses of the ERP responses to the emotional stimuli revealed that Om chanting decreased P1 and late positive potential (LPP) amplitudes for the neutral images but not for the unpleasant images. We speculated that Om chanting might reduce vigilance to the neutral stimuli, whereas for the negative stimuli, they automatically captured all available attentional resources and led to a failure in observing the regulating effect of Om chanting. These observations suggest that Om chanting modulates individuals' affective evaluations to the negative stimuli and alters early visual and late neural processing of the stimuli.

Effects of task load, spatial attention, and trait anxiety on neuronal responses to fearful and neutral faces

There is an ongoing debate on how different components of the event-related potential (ERP) to threat-related facial expressions are modulated by attentional conditions and interindividual differences in trait anxiety. In the current study (N = 80), we examined ERPs to centrally presented, task-irrelevant fearful and neutral faces, while participants had to solve a face-unrelated visual task, which differed in difficulty and spatial position. Critically, we used a fixation-controlled experimental design and ensured the spatial attention manipulation by spectral analysis of the EEG. Besides the factors emotion, spatial attention, and perceptual load, we also investigated correlations between trait anxiety and ERPs. While P1 emotion effects were insignificant, the N170 was increased to fearful faces regardless of load and spatial attention conditions. During the EPN time window, a significantly increased negativity for fearful faces was observed only during low load and spatial attention to the face. We found no significant relationship between ERPs and trait anxiety, questioning the hypothesis of a general hypersensitivity toward fearful expressions in anxious individuals. These results show a high resistance of the N170 amplitude increase for fearful faces to spatial attention and task load manipulations. By contrast, the EPN modulation by fearful faces index a resource-dependent stage of the ERP, requiring both spatial attention at the location of faces and low load of the face-irrelevant task.

Do Subliminal Fearful Facial Expressions Capture Attention?

In two experiments, we tested whether fearful facial expressions capture attention in an awareness-independent fashion. In Experiment 1, participants searched for a visible neutral face presented at one of two positions. Prior to the target, a backward-masked and, thus, invisible emotional (fearful/disgusted) or neutral face was presented as a cue, either at target position or away from the target position. If negative emotional faces capture attention in a stimulus-driven way, we would have expected a cueing effect: better performance where fearful or disgusted facial cues were presented at target position than away from the target. However, no evidence of capture of attention was found, neither in behavior (response times or error rates), nor in event-related lateralizations (N2pc). In Experiment 2, we went one step further and used fearful faces as visible targets, too. Thereby, we sought to boost awareness-independent capture of attention by fearful faces. However, still, we found no significant attention-capture effect. Our results show that fearful facial expressions do not capture attention in an awareness-independent way. Results are discussed in light of existing theories.

The role of phase and orientation for ERP modulations of spectrum-manipulated fearful and neutral faces

Prioritized processing of fearful compared to neutral faces has been proposed to result from evolutionary adaptation of the contrast sensitivity function (CSF) to the features of emotionally relevant faces and/or vice versa. However, it is unknown whether a stimulus merely has to feature the amplitude spectrum of a fearful face to be prioritized or whether the relevant spatial frequencies have to occur with specific phases and orientations. Prioritized processing is indexed by specific increases of Event-Related Potentials (ERPs) of the EEG and occurs throughout different early processing stages, indexed by emotion-related modulations of the P1, N170, and EPN. In this pre-registered study, we manipulated phase and amplitude properties of the Fourier spectra of neutral and fearful faces to test the effect of phase coherence (PC, face vs. scramble) and orientation coherence (OC, original vs. rotational average) and their interactions with differential emotion processing. We found that differential emotion processing was not present at the level of P1 but strongly affected N170 and EPN. In both cases, intact phase coherence was required for enhanced processing of fearful faces. OC did not interact with emotion. While faces produced the typical N170 effect, we observed a reversed effect for scrambles. Additional exploratory independent component analysis (ICA) suggests that this reversal could signal a mismatch between an early "perceptual hypothesis" and feedback of configural information. In line with our expectations, fearful-neutral differences for the N170 and EPN depend on configural information, i.e., recognizable faces.

Arithmetic performance is modulated by cognitive reappraisal and expression suppression: Evidence from behavioral and ERP findings

Previous studies have shown that negative emotional experiences are detrimental to individual's arithmetic performance, and no study has investigated whether such negative influence can be improved through emotional regulation. This study aimed to explore the relationship between emotion regulation and individual's performance in completing arithmetic problem verification task. Participants were asked to complete arithmetic problem verification task using the down-up strategy (e.g., doing 30 × 70 = 2100 for 34 × 67), under neutral, happy, and fear priming conditions, during which they were asked to regulate their emotional experience by using cognitive reappraisal or expressive suppression. Behavioral results showed that, under happy priming condition, cognitive reappraisal (but not expression suppression) contributed to improving individual's estimation speed. Additionally, under happy and fear priming conditions, cognitive reappraisal (vs. expression suppression) reduced individuals' emotional experience intensity. The arithmetic problem verification task-related ERP results showed that the P1 amplitudes elicited by using cognitive reappraisal (vs. freely view) were larger under happy and fear priming conditions, but the P1 amplitudes elicited by using expression suppression were larger only under happy condition. Meanwhile, the corresponding N170 amplitudes were smaller when using cognitive reappraisal and expression suppression (vs. freely view). Additionally, using cognitive reappraisal (vs. expression suppression) cost less cognitive resource (smaller LPC amplitudes). The present study indicated that both cognitive reappraisal and expression suppression contributed to improving individual's arithmetic performance, and the regulation effect of cognitive reappraisal was better than that of expression suppression.

Spatiotemporal Dynamics of Covert Versus Overt Processing of Happy, Fearful and Sad Facial Expressions

Behavioral and electrophysiological correlates of the influence of task demands on the processing of happy, sad, and fearful expressions were investigated in a within-subjects study that compared a perceptual distraction condition with task-irrelevant faces (e.g., covert emotion task) to an emotion task-relevant categorization condition (e.g., overt emotion task). A state-of-the-art non-parametric mass univariate analysis method was used to address the limitations of previous studies. Behaviorally, participants responded faster to overtly categorized happy faces and were slower and less accurate to categorize sad and fearful faces; there were no behavioral differences in the covert task. Event-related potential (ERP) responses to the emotional expressions included the N170 (140-180 ms), which was enhanced by emotion irrespective of task, with happy and sad expressions eliciting greater amplitudes than neutral expressions. EPN (200-400 ms) amplitude was modulated by task, with greater voltages in the overt condition, and by emotion, however, there was no interaction of emotion and task. ERP activity was modulated by emotion as a function of task only at a late processing stage, which included the LPP (500-800 ms), with fearful and sad faces showing greater amplitude enhancements than happy faces. This study reveals that affective content does not necessarily require attention in the early stages of face processing, supporting recent evidence that the core and extended parts of the face processing system act in parallel, rather than serially. The role of voluntary attention starts at an intermediate stage, and fully modulates the response to emotional content in the final stage of processing.

Increased early and late neuronal responses to aversively conditioned faces across different attentional conditions

Faces with emotional information-by virtue of their expression or their history of affective learning-are prioritized during neuronal processing as compared to neutral faces. Classical conditioning studies have shown that aversively conditioned (CS+) faces potentiate different face processing stages as evidenced by increased early and late event-related potential (ERPs) components. However, it is unknown whether and how ERP modulations depend on certain attentional conditions. To examine this question, this preregistered study investigated ERPs to faces paired with aversive screams or neutral sounds under three tasks with increasing attention to CS + relevant features of the face: Participants (N = 40) had to discriminate either the orientation of superimposed lines, perceived gender, or the CS association. We found potentiation of the N170, the Early Posterior Negativity (EPN), and, most remarkably, the Late Positive Potential (LPP) to CS + faces regardless of task condition. This finding suggests that, in contrast to other types of emotional information and learning, classical conditioning boosts early and late processing stages, even if no explicit attention to the face information or the CS association is required.

Early reduction of sensory processing within the visual cortex when switching from internal to external attention

The neurocognitive process underlying attention switches between external (perception-based) and internal (memory-based) attention is poorly characterized. Previous research has found that when participants switch attention either between two perception-based tasks (within-domain switches) or between a memory- and a perception-based task (between-domain switches), a substantial and similar processing cost was observed compared to the repetition of the same task (Verschooren, Schindler, De Raedt, & Pourtois, 2019). Here, we recorded 64-channel EEG while participants carried out within- versus between-domain switches of attention. ERP results showed that during early sensory processing, a marked P1 attenuation was associated with both switch types, suggesting that switching was associated with an early bottleneck during information processing. This early gating effect was stronger when switching from an internal to an external task, compared to switching between external tasks, suggesting different top-down requirements for them. These findings are in line with earlier proposals in the literature.