Face Processing is Gated by Visual Spatial Attention

Spatiotemporal Dynamics of Covert Versus Overt Processing of Happy, Fearful and Sad Facial Expressions

Behavioral and electrophysiological correlates of the influence of task demands on the processing of happy, sad, and fearful expressions were investigated in a within-subjects study that compared a perceptual distraction condition with task-irrelevant faces (e.g., covert emotion task) to an emotion task-relevant categorization condition (e.g., overt emotion task). A state-of-the-art non-parametric mass univariate analysis method was used to address the limitations of previous studies. Behaviorally, participants responded faster to overtly categorized happy faces and were slower and less accurate to categorize sad and fearful faces; there were no behavioral differences in the covert task. Event-related potential (ERP) responses to the emotional expressions included the N170 (140-180 ms), which was enhanced by emotion irrespective of task, with happy and sad expressions eliciting greater amplitudes than neutral expressions. EPN (200-400 ms) amplitude was modulated by task, with greater voltages in the overt condition, and by emotion, however, there was no interaction of emotion and task. ERP activity was modulated by emotion as a function of task only at a late processing stage, which included the LPP (500-800 ms), with fearful and sad faces showing greater amplitude enhancements than happy faces. This study reveals that affective content does not necessarily require attention in the early stages of face processing, supporting recent evidence that the core and extended parts of the face processing system act in parallel, rather than serially. The role of voluntary attention starts at an intermediate stage, and fully modulates the response to emotional content in the final stage of processing.

Irrelevant task suppresses the N170 of automatic attention allocation to fearful faces

Recent researches have provided evidence that stimulus-driven attentional bias for threats can be modulated by top-down goals. However, it is highlight essential to indicate whether and to what extent the top-down goals can affect the early stage of attention processing and its early neural mechanism. In this study, we collected electroencephalographic data from 28 healthy volunteers with a modified spatial cueing task. The results revealed that in the irrelevant task, there was no significant difference between the reaction time (RT) of the fearful and neutral faces. In the relevant task, we found that RT of fearful faces was faster than that of neutral faces in the valid cue condition, whereas the RT of fearful faces was slower than that of neutral faces in the invalid cue condition. The N170 component in our study showed a similar result compared with RT. Specifically, we noted that in the relevant task, fearful faces in the cue position of the target evoked a larger N170 amplitude than neutral faces, whereas this effect was suppressed in the irrelevant task. These results suggest that the irrelevant task may inhibit the early attention allocation to the fearful faces. Furthermore, the top-down goals can modulate the early attentional bias for threatening facial expressions.

Neurophysiological correlates of holistic face processing in adolescents with and without autism spectrum disorder

Background

Face processing has been found to be impaired in autism spectrum disorders (ASD). One hypothesis is that individuals with ASD engage in piecemeal compared to holistic face processing strategies. To investigate the role of possible impairments in holistic face processing in individuals with autism, the current study investigated behavioral and electroencephalography (EEG) correlates of face processing (P1/N170 and gamma-band activity) in adolescents with ASD and sex-, age-, and IQ-matched neurotypical controls.

Methods

Participants were presented with upright and inverted Mooney stimuli; black and white low information faces that are only perceived as faces when processed holistically. Participants indicated behaviorally the detection of a face. EEG was collected time-locked to the presentation of the stimuli.

Results

Adolescents with ASD perceived Mooney stimuli as faces suggesting ability to use holistic processing but displayed a lower face detection rate and slower response times. ERP components suggest slowed temporal processing of Mooney stimuli in the ASD compared to control group for P1 latency but no differences between groups for P1 amplitude and at the N170. Increases in gamma-band activity was similar during the perception of the Mooney images by group, but the ASD group showed prolonged temporal elevation in activity.

Conclusion

Overall, our results suggest that adolescents with ASD were able to utilize holistic processing to perceive a face within the Mooney stimuli. Delays in early processing, marked by the P1, and elongated elevation in gamma activity indicate that the neural systems supporting holistic processing are slightly altered suggesting a less automatic and less efficient facial processing system.

Trial registration

Non-applicable.

Electronic supplementary material

The online version of this article (10.1186/s11689-018-9244-y) contains supplementary material, which is available to authorized users.

The Neural Dynamics of Attentional Selection in Natural Scenes

The human visual system can only represent a small subset of the many objects present in cluttered scenes at any given time, such that objects compete for representation. Despite these processing limitations, the detection of object categories in cluttered natural scenes is remarkably rapid. How does the brain efficiently select goal-relevant objects from cluttered scenes? In the present study, we used multivariate decoding of magneto-encephalography (MEG) data to track the neural representation of within-scene objects as a function of top-down attentional set. Participants detected categorical targets (cars or people) in natural scenes. The presence of these categories within a scene was decoded from MEG sensor patterns by training linear classifiers on differentiating cars and people in isolation and testing these classifiers on scenes containing one of the two categories. The presence of a specific category in a scene could be reliably decoded from MEG response patterns as early as 160 ms, despite substantial scene clutter and variation in the visual appearance of each category. Strikingly, we find that these early categorical representations fully depend on the match between visual input and top-down attentional set: only objects that matched the current attentional set were processed to the category level within the first 200 ms after scene onset. A sensor-space searchlight analysis revealed that this early attention bias was localized to lateral occipitotemporal cortex, reflecting top-down modulation of visual processing. These results show that attention quickly resolves competition between objects in cluttered natural scenes, allowing for the rapid neural representation of goal-relevant objects.

SIGNIFICANCE STATEMENT

Efficient attentional selection is crucial in many everyday situations. For example, when driving a car, we need to quickly detect obstacles, such as pedestrians crossing the street, while ignoring irrelevant objects. How can humans efficiently perform such tasks, given the multitude of objects contained in real-world scenes? Here we used multivariate decoding of magnetoencephalogaphy data to characterize the neural underpinnings of attentional selection in natural scenes with high temporal precision. We show that brain activity quickly tracks the presence of objects in scenes, but crucially only for those objects that were immediately relevant for the participant. These results provide evidence for fast and efficient attentional selection that mediates the rapid detection of goal-relevant objects in real-world environments.

Relative expertise affects N170 during selective attention to superimposed face-character images

It remains unclear whether the N170 of ERPs reflects domain-specific or domain-general visual object processing. In this study, we used superimposed images of a face and a Chinese character such that participants' relative expertise for the two object types was either similar (Experiment 1 and 2) or different (Experiment 3). Experiment 1 showed that N170 amplitude was larger when participants attended to the character instead of the face of a face-character combination. This result was unchanged in Experiment 2, in which task difficulty was selectively increased for the face component of the combined stimuli. Experiment 3 showed that, although this N170 enhancement for attending to characters relative to faces persisted for false characters with recognizable parts, it disappeared for unrecognizable characters. Therefore, N170 amplitude was significantly greater for Chinese characters than for faces presented within a combined image, independent of the relative task difficulty. This result strongly calls N170 face selectivity into question, demonstrating that, contrary to the expectations established by a domain-specific account, N170 is modulated by expertise.

Dissociating Attention Effects from Categorical Perception with ERP Functional Microstates

When faces appear in our visual environment we naturally attend to them, possibly to the detriment of other visual information. Evidence from behavioural studies suggests that faces capture attention because they are more salient than other types of visual stimuli, reflecting a category-dependent modulation of attention. By contrast, neuroimaging data has led to a domain-specific account of face perception that rules out the direct contribution of attention, suggesting a dedicated neural network for face perception. Here we sought to dissociate effects of attention from categorical perception using Event Related Potentials. Participants viewed physically matched face and butterfly images, with each category acting as a target stimulus during different blocks in an oddball paradigm. Using a data-driven approach based on functional microstates, we show that the locus of endogenous attention effects with ERPs occurs in the N1 time range. Earlier categorical effects were also found around the level of the P1, reflecting either an exogenous increase in attention towards face stimuli, or a putative face-selective measure. Both category and attention effects were dissociable from one another hinting at the role that faces may play in early capturing of attention before top-down control of attention is observed. Our data support the conclusion that certain object categories, in this experiment, faces, may capture attention before top-down voluntary control of attention is initiated.

Transient Distraction and Attentional Control during a Sustained Selective Attention Task

Distracting stimuli in the environment can pull our attention away from our goal-directed tasks. fMRI studies have implicated regions in right frontal cortex as being particularly important for processing distractors [e.g., de Fockert, J. W., & Theeuwes, J. Role of frontal cortex in attentional capture by singleton distractors. Brain and Cognition, 80, 367-373, 2012; Demeter, E., Hernandez-Garcia, L., Sarter, M., & Lustig, C. Challenges to attention: A continuous arterial spin labeling (ASL) study of the effects of distraction on sustained attention. Neuroimage, 54, 1518-1529, 2011]. Less is known, however, about the timing and sequence of how right frontal or other brain regions respond selectively to distractors and how distractors impinge upon the cascade of processes related to detecting and processing behaviorally relevant target stimuli. Here we used EEG and ERPs to investigate the neural consequences of a perceptually salient but task-irrelevant distractor on the detection of rare target stimuli embedded in a rapid, serial visual presentation (RSVP) stream. We found that distractors that occur during the presentation of a target interfere behaviorally with detection of those targets, reflected by reduced detection rates, and that these missed targets show a reduced amplitude of the long-latency, detection-related P3 component. We also found that distractors elicited a right-lateralized frontal negativity beginning at 100 msec, whose amplitude negatively correlated across participants with their distraction-related behavioral impairment. Finally, we also quantified the instantaneous amplitude of the steady-state visual evoked potentials elicited by the RSVP stream and found that the occurrence of a distractor resulted in a transient amplitude decrement of the steady-state visual evoked potential, presumably reflecting the pull of attention away from the RSVP stream when distracting stimuli occur in the environment.

Double dissociation of configural and featural face processing on P1 and P2 components as a function of spatial attention

Face recognition relies on both configural and featural processing. Previous research has shown that P1 is sensitive to configural face processing, but it is unclear whether any component is sensitive to featural face processing; moreover, if there is such a component, its temporal sequence relative to P1 is unknown. Thus, to avoid confounding physical stimuli differences between configural and featural face processing on ERP components, a spatial attention paradigm was employed by instructing participants to attend an image stream (faces and houses) or an alphanumeric character stream. The interaction between attention and face processing type on P1 and P2 components indicates different mechanisms of configural and featural face processing as a function of spatial attention. The steady-state visual evoked potential (SSVEP) results clearly demonstrated that participants could selectively attend to different streams of information. Importantly, configural face processing elicited a larger posterior P1 (approximately 128 ms) than featural face processing, whereas P2 (approximately 248 ms) was greater for featural than for configural face processing under attended condition. The interaction between attention and face processing type (configural vs. featural) on P1 and P2 components indicates that there are different mechanisms of configural and featural face processing operating as functions of spatial attention. While the P1 result confirms previous findings separating configural and featural face processing, the newly observed P2 finding in the present study extends this separation to a double dissociation. Therefore, configural and featural face processing are modulated differently by spatial attention, and configural face processing precedes featural face processing.

Configural and featural face processing are differently modulated by attentional resources at early stages: an event-related potential study with rapid serial visual presentation

It is widely reported that face recognition relies on two dissociable mechanisms, the featural and the configural processing. However, it is unclear whether these two processing types involve different neural mechanisms and are differently modulated by attentional resources. Using the attentional blink (AB) paradigm, we aimed to investigate the effect of attentional resources on configural and featural face processing by recording event-related potentials (ERPs). The amount of attentional resources was manipulated as deficient or sufficient by presenting the second target (T2) in or out of the AB period, respectively. We found that in addition to a traditional P3 attention effect, the amplitude of N170/VPP to the T2 stimuli was also sensitive to attentional resources, suggesting that attention affects face processing at an earlier perceptual processing stage. More importantly, configural face processing elicited a larger posterior P1 compared to featural face processing, but only when the attentional resources were sufficient. In contrast, the anterior N1 was larger for configural relative to featural face processing only when the attentional resources were deficient. These results suggest that early stages of configural and featural face processing are differently modulated by attentional resources, possibly with different underlying mechanisms.

Disruption of visual awareness during the attentional blink is reflected by selective disruption of late-stage neural processing

Any information represented in the brain holds the potential to influence behavior. It is therefore of broad interest to determine the extent and quality of neural processing of stimulus input that occurs with and without awareness. The attentional blink is a useful tool for dissociating neural and behavioral measures of perceptual visual processing across conditions of awareness. The extent of higher-order visual information beyond basic sensory signaling that is processed during the attentional blink remains controversial. To determine what neural processing at the level of visual-object categorization occurs in the absence of awareness, electrophysiological responses to images of faces and houses were recorded both within and outside the attentional blink period during a rapid serial visual presentation stream. Electrophysiological results were sorted according to behavioral performance (correctly identified targets vs. missed targets) within these blink and nonblink periods. An early index of face-specific processing (the N170, 140- to 220-msec poststimulus) was observed regardless of whether the participant demonstrated awareness of the stimulus, whereas a later face-specific effect with the same topographic distribution (500- to 700-msec poststimulus) was only seen for accurate behavioral discrimination of the stimulus content. The present findings suggest a multistage process of object-category processing, with only the later phase being associated with explicit visual awareness.

Neural processing stages during object-substitution masking and their relationship to perceptual awareness

The extent of visual perceptual processing that occurs in the absence of awareness is as yet unclear. Here we examined event-related-potential (ERP) indices of visual and cognitive processes as awareness was manipulated through object-substitution masking (OSM), an awareness-disrupting effect that has been hypothesized to result from the disruption of reentrant signaling to low-level visual cortical areas. In OSM, a visual stimulus array is briefly presented that includes a parafoveal visual target denoted by a cue, typically consisting of several surrounding dots. When the offset of the target-surrounding cue dots is delayed relative to the rest of the array, a striking reduction in the perception of the target image surrounded by the dots is observed. Using faces and houses as the target stimuli, we found that successful OSM reduced or eliminated all the measured electrophysiological indices of visual processing stages after 130ms post-stimulus. More specifically, when targets were missed within the masked condition (i.e., on trials with effective OSM that disrupted awareness), we observed fully intact early feed-forward processing up through the visual extrastriate P1 ERP component peaking at 100ms, followed by reduced low-level activity over the occipital pole 130-170ms post-stimulus, reduced ERP indices of lateralized shifts of attention toward the parafoveal target, reduced object-generic visual processing, abolished object-category-specific (face-specific) processing, and reduced late visual short-term-memory processing activity. The results provide a comprehensive electrophysiological account of the neurocognitive underpinnings of effective OSM of visual-object images, including evidence for central roles of early reentrant signal disruption and insufficient visual attentional deployment.

The naked truth: the face and body sensitive N170 response is enhanced for nude bodies

Recent event-related potential studies have shown that the occipitotemporal N170 component - best known for its sensitivity to faces - is also sensitive to perception of human bodies. Considering that in the timescale of evolution clothing is a relatively new invention that hides the bodily features relevant for sexual selection and arousal, we investigated whether the early N170 brain response would be enhanced to nude over clothed bodies. In two experiments, we measured N170 responses to nude bodies, bodies wearing swimsuits, clothed bodies, faces, and control stimuli (cars). We found that the N170 amplitude was larger to opposite and same-sex nude vs. clothed bodies. Moreover, the N170 amplitude increased linearly as the amount of clothing decreased from full clothing via swimsuits to nude bodies. Strikingly, the N170 response to nude bodies was even greater than that to faces, and the N170 amplitude to bodies was independent of whether the face of the bodies was visible or not. All human stimuli evoked greater N170 responses than did the control stimulus. Autonomic measurements and self-evaluations showed that nude bodies were affectively more arousing compared to the other stimulus categories. We conclude that the early visual processing of human bodies is sensitive to the visibility of the sex-related features of human bodies and that the visual processing of other people's nude bodies is enhanced in the brain. This enhancement is likely to reflect affective arousal elicited by nude bodies. Such facilitated visual processing of other people's nude bodies is possibly beneficial in identifying potential mating partners and competitors, and for triggering sexual behavior.

Modeling Single-Trial ERP Reveals Modulation of Bottom-Up Face Visual Processing by Top-Down Task Constraints (in Some Subjects)

We studied how task constraints modulate the relationship between single-trial event-related potentials (ERPs) and image noise. Thirteen subjects performed two interleaved tasks: on different blocks, they saw the same stimuli, but they discriminated either between two faces or between two colors. Stimuli were two pictures of red or green faces that contained from 10 to 80% of phase noise, with 10% increments. Behavioral accuracy followed a noise dependent sigmoid in the identity task but was high and independent of noise level in the color task. EEG data recorded concurrently were analyzed using a single-trial ANCOVA: we assessed how changes in task constraints modulated ERP noise sensitivity while regressing out the main ERP differences due to identity, color, and task. Single-trial ERP sensitivity to image phase noise started at about 95-110 ms post-stimulus onset. Group analyses showed a significant reduction in noise sensitivity in the color task compared to the identity task from about 140 ms to 300 ms post-stimulus onset. However, statistical analyses in every subject revealed different results: significant task modulation occurred in 8/13 subjects, one showing an increase and seven showing a decrease in noise sensitivity in the color task. Onsets and durations of effects also differed between group and single-trial analyses: at any time point only a maximum of four subjects (31%) showed results consistent with group analyses. We provide detailed results for all 13 subjects, including a shift function analysis that revealed asymmetric task modulations of single-trial ERP distributions. We conclude that, during face processing, bottom-up sensitivity to phase noise can be modulated by top-down task constraints, in a broad window around the P2, at least in some subjects.

Sandwich masking eliminates both visual awareness of faces and face-specific brain activity through a feedforward mechanism

It is generally agreed that considerable amounts of low-level sensory processing of visual stimuli can occur without conscious awareness. On the other hand, the degree of higher level visual processing that occurs in the absence of awareness is as yet unclear. Here, event-related potential (ERP) measures of brain activity were recorded during a sandwich-masking paradigm, a commonly used approach for attenuating conscious awareness of visual stimulus content. In particular, the present study used a combination of ERP activation contrasts to track both early sensory-processing ERP components and face-specific N170 ERP activations, in trials with versus without awareness. The electrophysiological measures revealed that the sandwich masking abolished the early face-specific N170 neural response (peaking at ~170 ms post-stimulus), an effect that paralleled the abolition of awareness of face versus non-face image content. Furthermore, however, the masking appeared to render a strong attenuation of earlier feedforward visual sensory-processing signals. This early attenuation presumably resulted in insufficient information being fed into the higher level visual system pathways specific to object category processing, thus leading to unawareness of the visual object content. These results support a coupling of visual awareness and neural indices of face processing, while also demonstrating an early low-level mechanism of interference in sandwich masking.

The temporal dynamics of object processing in visual cortex during the transition from distributed to focused spatial attention

Several major cognitive neuroscience models have posited that focal spatial attention is required to integrate different features of an object to form a coherent perception of it within a complex visual scene. Although many behavioral studies have supported this view, some have suggested that complex perceptual discrimination can be performed even with substantially reduced focal spatial attention, calling into question the complexity of object representation that can be achieved without focused spatial attention. In the present study, we took a cognitive neuroscience approach to this problem by recording cognition-related brain activity both to help resolve the questions about the role of focal spatial attention in object categorization processes and to investigate the underlying neural mechanisms, focusing particularly on the temporal cascade of these attentional and perceptual processes in visual cortex. More specifically, we recorded electrical brain activity in humans engaged in a specially designed cued visual search paradigm to probe the object-related visual processing before and during the transition from distributed to focal spatial attention. The onset times of the color popout cueing information, indicating where within an object array the subject was to shift attention, was parametrically varied relative to the presentation of the array (i.e., either occurring simultaneously or being delayed by 50 or 100 msec). The electrophysiological results demonstrate that some levels of object-specific representation can be formed in parallel for multiple items across the visual field under spatially distributed attention, before focal spatial attention is allocated to any of them. The object discrimination process appears to be subsequently amplified as soon as focal spatial attention is directed to a specific location and object. This set of novel neurophysiological findings thus provides important new insights on fundamental issues that have been long-debated in cognitive neuroscience concerning both object-related processing and the role of attention.

Age-related changes in processing faces from detection to identification: ERP evidence

We examined the ability of people 70 to 90 years old to apply global, configural, and featural face-processing strategies. In addition we investigated age-related changes in the ability to categorize faces at basic, subordinate, and individual levels. Using the N170 potential as index of early face processing and the P300 component as index of categorical decision making and effort, we found significant age-related perceptual changes which slowed and somewhat impaired face processing. Specifically, older participants had problems integrating face features into global structures, demonstrating enhanced dependence on distal global information. They did not apply configural computations by default while processing faces which suggests that, unless identification is required, they process faces only at a basic level. These perceptual changes could be the cause for slower and less accurate subordinate categorization, particularly when it is based on details. At the neural levels face processing was not right-lateralized, reflecting excessive involvement of the left hemisphere in perception leading to a more general reduction of interhemispheric asymmetry. In addition we found excessive but nonselective activation of frontal regions adding support to the view that executive control and particularly inhibition of irrelevant input are reduced in the elderly.