Time-dependent effects of perceptual load on processing fearful and neutral faces

Is processing superiority a universal trait for all threats? Divergent impacts of fearful, angry, and disgusted faces on attentional capture

Fearful, angry, and disgusted facial expressions are evolutionarily salient and convey different types of threat signals. However, it remains unclear whether these three expressions impact sensory perception and attention in the same way. The present ERP study investigated the temporal dynamics underlying the processing of different types of threatening faces and the impact of attentional resources employed during a perceptual load task. Participants were asked to judge the length of bars superimposed over faces presented in the center of the screen. A mass univariate statistical approach was used to analyze the EEG data. Behaviorally, task accuracy was significantly reduced following exposure to fearful faces relative to neutral distractors, independent of perceptual load. The ERP results revealed that the P1 amplitude over the right hemisphere was found to be enhanced for fearful relative to disgusted faces, reflecting the rapid and coarse detection of fearful cues. The N170 responses elicited by fearful, angry, and disgusted faces were larger than those elicited by neutral faces, suggesting the largely automatic and preferential processing of threats. Furthermore, the early posterior negativity (EPN) component yielded increased responses to fearful and angry faces, indicating prioritized attention to stimuli representing acute threats. Additionally, perceptual load exerted a pronounced influence on the EPN and late positive potential (LPP), with larger responses observed in the low perceptual load condition, indicating goal-directed cognitive processing. Overall, the early sensory processing of fearful, angry, and disgusted faces is characterized by differential sensitivity in capturing attention automatically, despite the importance of these facial signals for survival. Fearful faces produce a strong interference effect and are processed with higher priority than angry and disgusted ones.

How and when social evaluative feedback is processed in the brain: A systematic review on ERP studies

Social evaluative feedback informs the receiver of the other's views, which may contain judgments of personality-related traits and/or the level of likability. Such kinds of social evaluative feedback are of particular importance to humans. Event-related potentials (ERPs) can directly measure where in the processing stream feedback valence, expectancy, or contextual relevance modulate information processing. This review provides an overview and systematization of studies and early, mid-latency, and late ERP effects. Early effects were inconsistently reported for all factors. Feedback valence effects are more consistently reported for specific mid-latency ERPs (Reward Positivity, RewP, and Early Posterior Negativity, EPN) and late positivities (P3 and Late Positive Potential, LPP). Unexpected feedback consistently increased the Feedback Related Negativity (FRN) and, less consistently, decreased P3 amplitudes. Contextual relevance of the sender (e.g., human vs computer sender) or self-relatedness increased mid-latency to late ERPs. Interactions between valence and other factors were less often found, arising during mid-latency stages, where most consistent interactions showed larger EPN and P3 amplitude differences for valent feedback in a more relevant context. The ERP findings highlight that social evaluative feedback is consistently differentiated during mid-latency processing stages. The review discusses the relevance of findings, possible shortcomings of different experimental designs, and open questions. Furthermore, we suggest concrete venues for future research.

Recognition and evaluation of mental workload in different stages of perceptual and cognitive information processing using a multimodal approach

This study explores the effects of different perceptual and cognitive information processing stages on mental workload by assessing multimodal indicators of mental workload such as the NASA-TLX, task performance, ERPs and eye movements. Repeated measures ANOVA of the data showed that among ERP indicators, P1, N1 and N2 amplitudes were sensitive to perceptual load (P-load), P3 amplitude was sensitive to P-load only in the prefrontal region during high cognitive load (C-load) states, and P3 amplitude in the occipital and parietal regions was sensitive to C-load. Among the eye movement indicators, blink frequency was sensitive to P-load in all C-load states, but to C-load in only low P-load states; pupil diameter and blink duration were sensitive to both P-load and C-load. Based on the above indicators, the k-nearest neighbours (KNN) algorithm was used to propose a classification method for the four different mental workload states with an accuracy of 97.89%.Practitioner summary: Based on the results of this study, it is possible to implement the monitoring of mental workload states and optimise brain task allocation in operations involving high mental workload, such as human-computer interaction.

Fixation-related electrical potentials during a free visual search task reveal the timing of visual awareness

It has been repeatedly claimed that emotional faces readily capture attention, and that they may be processed without awareness. Yet some observations cast doubt on these assertions. Part of the problem may lie in the experimental paradigms employed. Here, we used a free viewing visual search task during electroencephalographic recordings, where participants searched for either fearful or neutral facial expressions among distractor expressions. Fixation-related potentials were computed for fearful and neutral targets and the response compared for stimuli consciously reported or not. We showed that awareness was associated with an electrophysiological negativity starting at around 110 ms, while emotional expressions were distinguished on the N170 and early posterior negativity only when stimuli were consciously reported. These results suggest that during unconstrained visual search, the earliest electrical correlate of awareness may emerge as early as 110 ms, and fixating at an emotional face without reporting it may not produce any unconscious processing.

The effects of visual perceptual load on detection performance and event-related potentials to auditory stimuli

Load Theory states that perceptual load prevents, or at least reduces, the processing of task-unrelated stimuli. This study systematically examined the detection and neural processing of auditory stimuli unrelated to a visual foreground task. The visual task was designed to create continuous perceptual load, alternated between low and high load, and contained performance feedback to motivate participants to focus on the visual task instead of the auditory stimuli presented in the background. The auditory stimuli varied in intensity, and participants signaled their subjective perception of these stimuli without receiving feedback. Depending on stimulus intensity, we observed load effects on detection performance and P3 amplitudes of the event-related potential (ERP). N1 amplitudes were unaffected by perceptual load, as tested by Bayesian statistics. Findings suggest that visual perceptual load affects the processing of auditory stimuli in a late time window, which is associated with a lower probability of reported awareness of these stimuli.

Potentiated early neural responses to fearful faces are not driven by specific face parts

Prioritized processing of fearful compared to neutral faces is reflected in increased amplitudes of components of the event-related potential (ERP). It is unknown whether specific face parts drive these modulations. Here, we investigated the contributions of face parts on ERPs to task-irrelevant fearful and neutral faces using an ERP-dependent facial decoding technique and a large sample of participants (N = 83). Classical ERP analyses showed typical and robust increases of N170 and EPN amplitudes by fearful relative to neutral faces. Facial decoding further showed that the absolute amplitude of these components, as well as the P1, was driven by the low-frequency contrast of specific face parts. However, the difference between fearful and neutral faces was not driven by any specific face part, as supported by Bayesian statistics. Furthermore, there were no correlations between trait anxiety and main effects or interactions. These results suggest that increased N170 and EPN amplitudes to task-irrelevant fearful compared to neutral faces are not driven by specific facial regions but represent a holistic face processing effect.

Testing stimulus exposure time as the critical factor of increased EPN and LPP amplitudes for fearful faces during perceptual distraction tasks

Fearful facial expressions are prioritized across different information processing stages, as evident in early, intermediate, and late components of event-related brain potentials (ERPs). Recent studies showed that, in contrast to early N170 modulations, mid-latency (Early Posterior Negativity, EPN) and late (Late Positive Potential, LPP) emotional modulations depend on the attended perceptual feature. Nevertheless, several studies reported significant differences between emotional and neutral faces for the EPN or LPP components during distraction tasks. One cause for these conflicting findings might be that when faces are presented sufficiently long, participants attend to task-irrelevant features of the faces. In this registered report, we tested whether the presentation duration of faces is the critical factor for differences between reported emotional modulations during perceptual distraction tasks. To this end, 48 participants were required to discriminate the orientation of lines overlaid onto fearful or neutral faces, while face presentation varied (100 msec, 300 msec, 1,000 msec, 2,000 msec). While participants did not need to pay attention to the faces, we observed main effects of emotion for the EPN and LPP, but no interaction between emotion and presentation duration. Of note, unregistered exploratory tests per presentation duration showed no significant EPN and LPP emotion differences during short durations (100 and 300 msec) but significant differences with longer durations. While the presentation duration seems not to be a critical factor for EPN and LPP emotion effects, future studies are needed to investigate the role of threshold effects and the applied analytic designs to explain conflicting findings in the literature.

Emotional Valence in the Eye Region Modulates the Attentional Blink in a Task-Dependent Manner: Evidence from Event-Related Potentials

Although emotional expressions conveyed by the eye regions are processed efficiently, little is known regarding the relationship between emotional processing of isolated eye regions and temporal attention. In this study, we conducted three rapid serial visual presentation (RSVP) experiments with varying task demands (emotion discrimination, eye detection, eyes ignored) related to the first target (T1) to investigate how the perception of emotional valence in the eye region (T1: happy, neutral, fearful) impacts the identification of a second target (T2: neutral houses). Event-related potential (ERP) findings indicated that fearful stimuli reliably increased N170 amplitude regardless of the emotional relevance of task demands. The P3 component exhibited enhanced responses to happy and fearful stimuli in the emotion discrimination task and to happy eye regions in the eye detection task. Analysis of T2-related ERPs within the attentional blink period revealed that T2 houses preceded by fearful and happy stimuli elicited larger N2 and P3 amplitudes than those preceded by neutral stimuli only in the emotion discrimination task. Together, these findings indicate that attention to affective content conveyed by the eyes can not only amplify the perceptual analysis of emotional eye regions but also facilitate the processing of a subsequent target.

No trait anxiety influences on early and late differential neuronal responses to aversively conditioned faces across three different tasks

The human brain's ability to quickly detect dangerous stimuli is crucial in selecting appropriate responses to possible threats. Trait anxiety has been suggested to moderate these processes on certain processing stages. To dissociate such different information-processing stages, research using classical conditioning has begun to examine event-related potentials (ERPs) in response to fear-conditioned (CS +) faces. However, the impact of trait anxiety on ERPs to fear-conditioned faces depending on specific task conditions is unknown. In this preregistered study, we measured ERPs to faces paired with aversive loud screams (CS +) or neutral sounds (CS −) in a large sample (N = 80) under three different task conditions. Participants had to discriminate face-irrelevant perceptual information, the gender of the faces, or the CS category. Results showed larger amplitudes in response to aversively conditioned faces for all examined ERPs, whereas interactions with the attended feature occurred for the P1 and the early posterior negativity (EPN). For the P1, larger CS + effects were observed during the perceptual distraction task, while the EPN was increased for CS + faces when deciding about the CS association. Remarkably, we found no significant correlations between ERPs and trait anxiety. Thus, fear-conditioning potentiates all ERP amplitudes, some processing stages being further modulated by the task. However, the finding that these ERP differences were not affected by individual differences in trait anxiety does not support theoretical accounts assuming increased threat processing or reduced threat discrimination depending on trait anxiety.

The ERP and psychophysical changes related to facial emotion perception by expertise in Japanese hospitality, "OMOTENASHI"

We investigated the emotion perception process based on hospitality expertise. Forty subjects were divided into the OMOTENASHI group working at inns considered to represent the spirit of hospitality, OMOTENASHI in Japan, and CONTROL group without experience in the hospitality industry. We presented neutral, happy, and angry faces to investigate P100 and N170 by these faces, and psychophysical changes by the favor rating test to evaluate emotional perception. In the favor rating test, the score was significantly smaller (less favorable) in OMOTENASHI than in CONTROL. Regarding event-related potential components, the maximum amplitude of P100 was significantly larger for a neutral face at the right occipital electrode in OMOTENASHI than in CONTROL, and it was significantly larger for an angry face at both occipital electrodes in OMOTENASHI than in CONTROL. However, the peak latency and maximum amplitude of N170 were not significantly different between OMOTENASHI and CONTROL at both temporal electrodes for each emotion condition. Differences on the favor rating test and P100 in OMOTENASHI suggested that workers at inns may more quickly notice and be more sensitive to the facial emotion of guests due to hospitality training, and/or that hospitality expertise may increase attention to emotion by top-down and/or bottom-up processing.

Effects of task load, spatial attention, and trait anxiety on neuronal responses to fearful and neutral faces

There is an ongoing debate on how different components of the event-related potential (ERP) to threat-related facial expressions are modulated by attentional conditions and interindividual differences in trait anxiety. In the current study (N = 80), we examined ERPs to centrally presented, task-irrelevant fearful and neutral faces, while participants had to solve a face-unrelated visual task, which differed in difficulty and spatial position. Critically, we used a fixation-controlled experimental design and ensured the spatial attention manipulation by spectral analysis of the EEG. Besides the factors emotion, spatial attention, and perceptual load, we also investigated correlations between trait anxiety and ERPs. While P1 emotion effects were insignificant, the N170 was increased to fearful faces regardless of load and spatial attention conditions. During the EPN time window, a significantly increased negativity for fearful faces was observed only during low load and spatial attention to the face. We found no significant relationship between ERPs and trait anxiety, questioning the hypothesis of a general hypersensitivity toward fearful expressions in anxious individuals. These results show a high resistance of the N170 amplitude increase for fearful faces to spatial attention and task load manipulations. By contrast, the EPN modulation by fearful faces index a resource-dependent stage of the ERP, requiring both spatial attention at the location of faces and low load of the face-irrelevant task.

Do Subliminal Fearful Facial Expressions Capture Attention?

In two experiments, we tested whether fearful facial expressions capture attention in an awareness-independent fashion. In Experiment 1, participants searched for a visible neutral face presented at one of two positions. Prior to the target, a backward-masked and, thus, invisible emotional (fearful/disgusted) or neutral face was presented as a cue, either at target position or away from the target position. If negative emotional faces capture attention in a stimulus-driven way, we would have expected a cueing effect: better performance where fearful or disgusted facial cues were presented at target position than away from the target. However, no evidence of capture of attention was found, neither in behavior (response times or error rates), nor in event-related lateralizations (N2pc). In Experiment 2, we went one step further and used fearful faces as visible targets, too. Thereby, we sought to boost awareness-independent capture of attention by fearful faces. However, still, we found no significant attention-capture effect. Our results show that fearful facial expressions do not capture attention in an awareness-independent way. Results are discussed in light of existing theories.

Top-Down Modulation of Early Visual Processing in V1: Dissociable Neurophysiological Effects of Spatial Attention, Attentional Load and Task-Relevance

Until today, there is an ongoing discussion if attention processes interact with the information processing stream already at the level of the C1, the earliest visual electrophysiological response of the cortex. We used two highly powered experiments (each N = 52) and examined the effects of task relevance, spatial attention, and attentional load on individual C1 amplitudes for the upper or lower visual hemifield. Bayesian models revealed evidence for the absence of load effects but substantial modulations by task-relevance and spatial attention. When the C1-eliciting stimulus was a task-irrelevant, interfering distracter, we observed increased C1 amplitudes for spatially unattended stimuli. For spatially attended stimuli, different effects of task-relevance for the two experiments were found. Follow-up exploratory single-trial analyses revealed that subtle but systematic deviations from the eye-gaze position at stimulus onset between conditions substantially influenced the effects of attention and task relevance on C1 amplitudes, especially for the upper visual field. For the subsequent P1 component, attentional modulations were clearly expressed and remained unaffected by these deviations. Collectively, these results suggest that spatial attention, unlike load or task relevance, can exert dissociable top-down modulatory effects at the C1 and P1 levels.

Nonlinear Effects of Linearly Increasing Perceptual Load on ERPs to Emotional Pictures

The prioritized processing of emotional as compared to neutral stimuli is reflected in enlarged event-related potentials (ERPs). However, perceptual load theory proposes that under conditions of high perceptual load, information processing is attenuated or abolished. The parametrical effects of load on ERPs to emotional pictures are unknown. To shed light on this question, the current preregistered ERP study (N = 30) systematically investigated the effects of load on ERPs to task-irrelevant negative, neutral, and positive pictures. Crucially, while perceptual input was held constant, perceptual load was systematically manipulated so that it increased linearly across 4 load levels, which was evident in behavioral data. In contrast, load effects on ERP differences between emotional and neutral stimuli did not follow a linear function. For the N1, early posterior negativity and late positive potential, a nonlinear function with reversed emotion effects at the third load level provided the best fit. These findings do not only show that perceptual load attenuates emotional picture processing but also suggest that active processes are initiated to reduce distraction by emotional information. Moreover, these effects of perceptual load on emotional ERP components appear to deviate from theoretically expected functions.

Charged With a Crime: The Neuronal Signature of Processing Negatively Evaluated Faces Under Different Attentional Conditions

Our brains rapidly respond to human faces and can differentiate between many identities, retrieving rich semantic emotional-knowledge information. Studies provide a mixed picture of how such information affects event-related potentials (ERPs). We systematically examined the effect of feature-based attention on ERP modulations to briefly presented faces of individuals associated with a crime. The tasks required participants ( N = 40 adults) to discriminate the orientation of lines overlaid onto the face, the age of the face, or emotional information associated with the face. Negative faces amplified the N170 ERP component during all tasks, whereas the early posterior negativity (EPN) and late positive potential (LPP) components were increased only when the emotional information was attended to. These findings suggest that during early configural analyses (N170), evaluative information potentiates face processing regardless of feature-based attention. During intermediate, only partially resource-dependent, processing stages (EPN) and late stages of elaborate stimulus processing (LPP), attention to the acquired emotional information is necessary for amplified processing of negatively evaluated faces.

Spatiotemporal Dynamics of Covert Versus Overt Processing of Happy, Fearful and Sad Facial Expressions

Behavioral and electrophysiological correlates of the influence of task demands on the processing of happy, sad, and fearful expressions were investigated in a within-subjects study that compared a perceptual distraction condition with task-irrelevant faces (e.g., covert emotion task) to an emotion task-relevant categorization condition (e.g., overt emotion task). A state-of-the-art non-parametric mass univariate analysis method was used to address the limitations of previous studies. Behaviorally, participants responded faster to overtly categorized happy faces and were slower and less accurate to categorize sad and fearful faces; there were no behavioral differences in the covert task. Event-related potential (ERP) responses to the emotional expressions included the N170 (140-180 ms), which was enhanced by emotion irrespective of task, with happy and sad expressions eliciting greater amplitudes than neutral expressions. EPN (200-400 ms) amplitude was modulated by task, with greater voltages in the overt condition, and by emotion, however, there was no interaction of emotion and task. ERP activity was modulated by emotion as a function of task only at a late processing stage, which included the LPP (500-800 ms), with fearful and sad faces showing greater amplitude enhancements than happy faces. This study reveals that affective content does not necessarily require attention in the early stages of face processing, supporting recent evidence that the core and extended parts of the face processing system act in parallel, rather than serially. The role of voluntary attention starts at an intermediate stage, and fully modulates the response to emotional content in the final stage of processing.

Increased early and late neuronal responses to aversively conditioned faces across different attentional conditions

Faces with emotional information-by virtue of their expression or their history of affective learning-are prioritized during neuronal processing as compared to neutral faces. Classical conditioning studies have shown that aversively conditioned (CS+) faces potentiate different face processing stages as evidenced by increased early and late event-related potential (ERPs) components. However, it is unknown whether and how ERP modulations depend on certain attentional conditions. To examine this question, this preregistered study investigated ERPs to faces paired with aversive screams or neutral sounds under three tasks with increasing attention to CS + relevant features of the face: Participants (N = 40) had to discriminate either the orientation of superimposed lines, perceived gender, or the CS association. We found potentiation of the N170, the Early Posterior Negativity (EPN), and, most remarkably, the Late Positive Potential (LPP) to CS + faces regardless of task condition. This finding suggests that, in contrast to other types of emotional information and learning, classical conditioning boosts early and late processing stages, even if no explicit attention to the face information or the CS association is required.

Reduced early fearful face processing during perceptual distraction in high trait anxious participants

Fearful facial expressions are prioritized across different stages of information processing as reflected by early, mid-latency, and late components of event-related brain potentials (ERP). Trait anxiety has been proposed to modulate these responses, but it is yet unclear how such modulations depend on feature-based attention. In this preregistered study (N = 80), we investigated the effects of trait anxiety on ERP differences between fearful and neutral faces across three different tasks. Participants had to discriminate either the orientation of lines overlaid onto the faces, the gender of the face, or the emotional expression, thus increasing attention to emotionally relevant facial features across the tasks. Fearful versus neutral faces elicited increased P1 and N170 amplitudes across tasks and potentiated amplitudes when attention was directed to faces (early posterior negativity [EPN]) or the expression (EPN and late positive potential). Higher trait anxiety was related to smaller EPN differences between fearful and neutral faces during the perceptual discrimination task. This early relationship suggests reduced instead of amplified processing of fearful faces for high trait anxious participants under perceptual distraction.

Face and emotional expression processing under continuous perceptual load tasks: An ERP study

High perceptual load is thought to impair already the early stages of visual processing of task-irrelevant visual stimuli. However, recent studies showed no effects of perceptual load on early ERPs in response to task-irrelevant emotional faces. In this preregistered EEG study (N = 40), we investigated the effects of continuous perceptual load on ERPs to fearful and neutral task-irrelevant faces and their phase-scrambled versions. Perceptual load did not modulate face or emotion effects for the P1 or N170. In contrast, larger face-scramble and fearful-neutral differentiation were found during low as compared to high load for the Early Posterior Negativity (EPN). Further, face-independent P1, but face-dependent N170 emotional modulations were observed. Taken together, our findings show that P1 and N170 face and emotional modulations are highly resistant to load manipulations, indicating a high degree of automaticity during this processing stage, whereas the EPN might represent a bottleneck in visual information processing.

Effects of low-level visual information and perceptual load on P1 and N170 responses to emotional expressions

Emotional facial expressions lead to modulations of early event-related potentials (ERPs). However, it has so far remained unclear how far these modulations represent face-specific effects rather than differences in low-level visual features, and to which extent they depend on available processing resources. To examine these questions, we conducted two preregistered independent experiments (N = 40 in each experiment) using different variants of a novel task that manipulates peripheral perceptual load across levels but keeps overall visual stimulation constant. At the display center, we presented task-irrelevant angry, neutral, and happy faces and their Fourier phase-scrambled versions, which preserved low-level visual features. The results of both studies showed load-independent P1 and N170 emotional expression effects. Importantly, by using Bayesian analyses we could confirm that these facial expression effects were face-independent for the P1 but not for the N170 component. We conclude that firstly, ERP modulations during the P1 interval strongly depend on low-level visual information, while the N170 modulation requires the processing of figural facial expression features. Secondly, both P1 and N170 modulations appear to be immune to a large range of variations in perceptual load.

Attentional conditions differentially affect early, intermediate and late neural responses to fearful and neutral faces

The processing of fearful facial expressions is prioritized by the human brain. This priority is maintained across various information processing stages as evident in early, intermediate and late components of event-related potentials (ERPs). However, emotional modulations are inconsistently reported for these different processing stages. In this pre-registered study, we investigated how feature-based attention differentially affects ERPs to fearful and neutral faces in 40 participants. The tasks required the participants to discriminate either the orientation of lines overlaid onto the face, the sex of the face or the face's emotional expression, increasing attention to emotion-related features. We found main effects of emotion for the N170, early posterior negativity (EPN) and late positive potential (LPP). While N170 emotional modulations were task-independent, interactions of emotion and task were observed for the EPN and LPP. While EPN emotion effects were found in the sex and emotion tasks, the LPP emotion effect was mainly driven by the emotion task. This study shows that early responses to fearful faces are task-independent (N170) and likely based on low-level and configural information while during later processing stages, attention to the face (EPN) or-more specifically-to the face's emotional expression (LPP) is crucial for reliable amplified processing of emotional faces.