Spatial Attention Shifting to Emotional Faces is Contingent on Awareness and Task Relevancy

Can templates-for-rejection suppress real-world affective objects in visual search?

Previous evidence has suggested that feature-based templates-for-rejection can be maintained in working memory to suppress matching features in the environment. Currently, this effect has only been demonstrated using abstract neutral shapes, meaning that it is unclear whether this generalizes to real-world images, including aversive stimuli. In the current investigation, participants searched amongst an array of real-world objects for a target, after being precued with either a distractor template, target template, or a no template baseline. In Experiment 1, where both distractor and target template cues were presented randomly on a trial-by-trial basis, there was moderate evidence of increased capture by aversive distractors after the distractor template cue. In Experiment 2a, however, when distractor templates were the only available cue and more time was given to encode the cue features, there was moderate evidence of effective distractor inhibition for real-world aversive and neutral stimuli. In Experiment 2b, when the task required a slower more effortful comparison of target features to stereotypical object representations, there was weaker evidence of inhibition, though there was still modest evidence suggesting effective inhibition of aversive distractors. A Bayesian meta-analysis revealed that across Experiment 2, aversive distractors showed strong cumulative evidence of effective inhibition, but inconsistent inhibition for neutral distractors. The results are interpreted from a rational search behaviour framework, which suggests that individuals utilize informative cues when they enable the most beneficial strategy and are accessible, and apply these to distractors when they cause sufficient disruption, either to search speed or emotional state.

Neurophysiological Pathways of Unconscious Emotion Processing in Depression: Insights from a simultaneous EEG-fMRI Measurement

BACKGROUND

Major depressive disorder (MDD) is characterized by strong emotional dysregulation. Mechanisms driving the negative affect in depression may be fast processes existing on an unconscious level.

METHODS

A priming task was conducted using simultaneous EEG-fMRI measurement involving presentation of facial expressions (happy, sad, neutral) to examine the neurophysiological pathway of biased unconscious emotion processing in MDD. Priming prior to a target emotion created unconscious (16.7 ms primer) and conscious (150 ms primer) trials. A large sample of N = 126 was recruited, containing healthy controls (HC; n = 66; 37 women) and MDD (n = 60; 31 women).

RESULTS

HC showed a shorter reaction time in happy, but not in sad or neutral trials compared to MDD. N170 amplitudes were lower in trials with unconscious compared to conscious primer presentation. N170 amplitudes correlated with cortical (right fusiform gyrus (FFG), right middle temporal gyrus, right inferior temporal gyrus, left supplementary motor area, right middle frontal gyrus) and subcortical brain regions (right amygdala). The strength of N170 and brain activity correlation increased when the stimulus was consciously presented. Presented emotions did not affect the correlation of N170 values and brain activity.

CONCLUSIONS

Our findings show that MDD may exhibit biased emotion regulation abilities at a behavioral and neurophysiological level. Face-sensitive event-related potentials demonstrate a correlation with heightened brain activity in regions associated with both face recognition (FFG) and emotion processing (amygdala). These findings are evident in both MDD and HC, with lower effect sizes in MDD indicating reduced emotion recognition and processing abilities.

Early Electrophysiological Correlates of Perceptual Consciousness Are Affected by Both Exogenous and Endogenous Attention

It has been proposed that visual awareness negativity (VAN), which is an early ERP component, constitutes a neural correlate of visual consciousness that is independent of perceptual and cognitive mechanisms. In the present study, we investigated whether VAN is indeed a specific marker of phenomenal awareness or rather reflects the involvement of attention. To this end, we reanalyzed data collected in a previously published EEG experiment in which awareness of visual stimuli and two aspects that define attentional involvement, namely, the inherent saliency and task relevance of a stimulus, were manipulated orthogonally. During the experimental procedure, participants (n = 41) were presented with images of faces that were backward-masked or unmasked, fearful or neutral, and defined as task-relevant targets or task-irrelevant distractors. Single-trial ERP analysis revealed that VAN was highly dependent on attentional manipulations in the early time window (140-200 msec), up to the point that the effect of awareness was not observed for attentionally irrelevant stimuli (i.e., neutral faces presented as distractors). In the late time window (200-350 msec), VAN was present in all attentional conditions, but its amplitude was significantly higher in response to fearful faces and task-relevant face images than in response to neutral ones and task-irrelevant ones, respectively. In conclusion, we demonstrate that the amplitude of VAN is highly dependent on both exogenous (stimulus saliency) and endogenous attention (task requirements). Our results challenge the view that VAN constitutes an attention-independent correlate of phenomenal awareness.

Decoding neural patterns for the processing of fearful faces under different visual awareness conditions: A multivariate pattern analysis

Previous studies have provided mixed findings regarding the nonconscious processing of fearful faces. Here, we used multivariate pattern analysis on electroencephalography data from three backward masking experiments to examine the processing of fearful faces under different visual awareness conditions. Three groups of participants were shown pairs of face images presented very briefly (for 16 ms) or for sufficiently long (for 266 ms), and completed tasks where the faces were either relevant to the experimental task (Experiment 1) or not (Experiments 2 and 3). Three main decoding analyses were performed. First, in the visual awareness decoding, the visibility of the faces, and hence participants' awareness of them, was maximally decodable in three time windows: 158-168 ms, 235-260 ms and 400-600 ms where the earlier neural patterns were generalized to the later stage activity. Second, we found that the spatial location of a fearful face in the face pairs was decodable, however only when the faces were consciously seen and task-relevant. Finally, we successfully decoded distinct neural patterns associated with the fearful-face-present conditions, compared to the fearful-face-absent conditions, and these patterns were decodable during both short and long presentations of the faces. Together, our results suggest that, while the processing of the spatial location of fearful faces requires awareness and task-relevancy, the mere presence of fearful faces can be processed even when visual awareness is highly restricted.

Differential modulation on neural activity related to flankers during face processing: A visual crowding study

In this visual crowding study, we manipulated the perceivability of a central crowded face (a fearful or a neutral face) by varying the similarity between the central face and the surrounding flanker stimuli. We presented participants with pairs of visual clutters and recorded their electroencephalography during an emotion judgement task. In an upright flanker condition where both the central target face and flanker faces were upright faces, participants were less likely to report seeing the target face, and their P300 was weakened, compared to a scrambled flanker condition where scrambled face images were used as flankers. Additionally, at ∼ 120 ms post-stimulus, a posterior negativity was found for the upright compared to scrambled flanker condition, however only for fearful face targets. We concluded that early neural responses seem to be affected by the perceptual characteristics of both target and flanker stimuli whereas later-stage neural activity is associated with post-perceptual evaluation of the stimuli in this visual crowding paradigm.

Beyond facial expressions: A systematic review on effects of emotional relevance of faces on the N170

The N170 is the most prominent electrophysiological signature of face processing. While facial expressions reliably modulate the N170, there is considerable variance in N170 modulations by other sources of emotional relevance. Therefore, we systematically review and discuss this research area using different methods to manipulate the emotional relevance of inherently neutral faces. These methods were categorized into (1) existing pre-experimental affective person knowledge (e.g., negative attitudes towards outgroup faces), (2) experimentally instructed affective person knowledge (e.g., negative person information), (3) contingency-based affective learning (e.g., fear-conditioning), or (4) the immediate affective context (e.g., emotional information directly preceding the face presentation). For all categories except the immediate affective context category, the majority of studies reported significantly increased N170 amplitudes depending on the emotional relevance of faces. Furthermore, the potentiated N170 was observed across different attention conditions, supporting the role of the emotional relevance of faces on the early prioritized processing of configural facial information, regardless of low-level differences. However, we identified several open research questions and suggest venues for further research.

Faces capture spatial attention only when we want them to: An inattentional blindness EEG study

Previous research on emotional face processing has shown that emotional faces such as fearful faces may be processed without visual awareness. However, evidence for nonconscious attention capture by fearful faces is limited. In fact, studies using sensory manipulation of awareness (e.g., backward masking paradigms) have shown that fearful faces do not attract attention during subliminal viewings nor when they were task-irrelevant. Here, we used a three-phase inattentional blindness paradigm and electroencephalography to examine whether faces (fearful and neutral) capture attention under different conditions of awareness and task-relevancy. We found that the electrophysiological marker for attention capture, the N2-posterior-contralateral (N2pc), was elicited by face stimuli only when participants were aware of the faces and when they were task-relevant (phase 3). When participants were unaware of the presence of faces (phase 1) or when the faces were irrelevant to the task (phase 2), no N2pc was observed. Together with our previous work, we concluded that fearful faces, or faces in general, do not attract attention unless we want them to.

Fixation-related electrical potentials during a free visual search task reveal the timing of visual awareness

It has been repeatedly claimed that emotional faces readily capture attention, and that they may be processed without awareness. Yet some observations cast doubt on these assertions. Part of the problem may lie in the experimental paradigms employed. Here, we used a free viewing visual search task during electroencephalographic recordings, where participants searched for either fearful or neutral facial expressions among distractor expressions. Fixation-related potentials were computed for fearful and neutral targets and the response compared for stimuli consciously reported or not. We showed that awareness was associated with an electrophysiological negativity starting at around 110 ms, while emotional expressions were distinguished on the N170 and early posterior negativity only when stimuli were consciously reported. These results suggest that during unconstrained visual search, the earliest electrical correlate of awareness may emerge as early as 110 ms, and fixating at an emotional face without reporting it may not produce any unconscious processing.

Reward History Modulates the Processing of Task-Irrelevant Emotional Faces in a Demanding Task

The aim of the current study was to examine how reward-associated emotional facial distractors could capture attentional resources in a demanding visual task using event-related potentials (ERPs). In the learning phase, a high- or low-reward probability was paired with angry, happy, or neutral faces. Then, in the test phase, participants performed a face-irrelevant task with no reward at stake, in which they needed to discriminate the length of two lines presented in the center of the screen while faces that were taken from the learning phase were used as distractors presented in the periphery. The behavioral results revealed no effect of distractor emotional valence since the emotional information was task-irrelevant. The ERP results in the test phase revealed a significant main effect of distractor emotional valence for the parieto-occipital P200 (170-230 ms); the mean amplitudes in both the angry- and happy-face conditions were more positive than the neutral-face condition. Moreover, we found that the high-reward association enhanced both the N170 (140-180 ms) and EPN (260-330 ms) relative to the low-reward association condition. Finally, the N2pc (270-320 ms) also exhibited enhanced neural activity in the high-reward condition compared to the low-reward condition. The absence of emotional effects indicated that task-irrelevant emotional facial stimuli did not impact behavioral or neural responses in this highly demanding task. However, reward-associated information was processed when attention was directed elsewhere, suggesting that the processing of reward-associated information worked more in an automatic way, irrespective of the top-down task demand.

Neural processing of lateralised task-irrelevant fearful faces under different awareness conditions

The neural fate of task-irrelevant emotional faces under different awareness conditions is poorly understood. Here, we examined the electrophysiological activity during an experiment where the location of target information (contrast-induced line) was manipulated orthogonally to the location of task-irrelevant fearful faces, under subliminal or supraliminal viewing conditions. We found that only target lines elicited an N2-posterior-contralateral (N2pc), indexing spatial attention shifting, in the supraliminal condition. No N2pc was found for the targets in the subliminal condition or for task-irrelevant fearful faces in either conditions. However, the mere presence of a fearful face enhanced early neural activity between 200 and 300 ms only in the subliminal condition. Additionally, the presence of a target line, but not a fearful face, enhanced the P3. Our results suggest that the N2pc is dependent on visual awareness and task-relevancy of the information and that laterally-presented task-irrelevant fearful expressions can be processed without awareness during early visual processing.

Attentional capture by fearful faces requires consciousness and is modulated by task-relevancy: A dot-probe EEG study

In the current EEG study, we used a dot-probe task in conjunction with backward masking to examine the neural activity underlying awareness and spatial processing of fearful faces and the neural processes for subsequent cued spatial targets. We presented face images under different viewing conditions (subliminal and supraliminal) and manipulated the relation between a fearful face in the pair and a subsequent target. Our mass univariate analysis showed that fearful faces elicit the N2-posterior-contralateral, indexing spatial attention capture, only when they are presented supraliminally. Consistent with this, the multivariate pattern analysis revealed a successful decoding of the location of the fearful face only in the supraliminal viewing condition. Additionally, the spatial attention capture by fearful faces modulated the processing of subsequent lateralised targets that were spatially congruent with the fearful face, in both al and electrophysiological data. There was no evidence for nonconscious processing of the fearful faces in the current paradigm. We conclude that spatial attentional capture by fearful faces requires visual awareness and it is modulated by top-down task demands.

The Effects of Spatial Attention Focus and Visual Awareness on the Processing of Fearful Faces: An ERP Study

Previous research on the relationship between attention and emotion processing have focused essentially on consciously-viewed, supraliminal stimuli, while the attention-emotion interplay remains unexplored in situations where visual awareness is restricted. Here, we presented participants with face pairs in a backward masking paradigm and examined the electrophysiological activity in response to fearful and neutral expressions under different conditions of attention (spatially attended vs. unattended) and stimulus visibility (subliminal vs. supraliminal). We found an enhanced N2 (visual awareness negativity -VAN-) and an enhanced P3 for supraliminal compared to subliminal faces. The VAN, indexing the early perceptual awareness, was enhanced when the faces were spatially attended compared to when they were unattended, showing that the VAN does not require spatial attention focus but can be enhanced by it. Fearful relative to neutral expressions enhanced the early neural activity (N2) regardless of spatial attention but only in the supraliminal viewing condition. However, fear-related enhancements on later neural activity (P3) were found when stimuli were both attended and presented supraliminally. These findings suggest that visual awareness is needed for emotion processing during both early and late stages. Spatial attention is required for emotion processing at the later stage but not at the early stage.

Spatial attention shifting to fearful faces depends on visual awareness in attentional blink: An ERP study

It remains unclear to date whether spatial attention towards emotional faces is contingent on, or independent of visual awareness. To investigate this question, a bilateral attentional blink paradigm was used in which lateralised fearful faces were presented at various levels of detectability. Twenty-six healthy participants were presented with two rapid serial streams of human faces, while they attempted to detect a pair of target faces (T2) displayed in close or distant succession of a first target pair (T1). Spatial attention shifting to the T2 fearful faces, indexed by the N2-posterior-contralateral component, was dependent on visual awareness and its magnitude covaried with the visual awareness negativity, a neural marker of awareness at the perceptual level. Additionally, information consolidation in working memory, indexed by the sustained posterior contralateral negativity, positively correlated with the level of visual awareness and spatial attention shifting. These findings demonstrate that spatial attention shifting to fearful faces depends on visual awareness, and these early processes are closely linked to information maintenance in working memory.