Attention to faces modulates early face processing during low but not high face discriminability

Visual periodicity reveals distinct attentional signatures for face and non-face categories

Observers can selectively deploy attention to regions of space, moments in time, specific visual features, individual objects, and even specific high-level categories-for example, when keeping an eye out for dogs while jogging. Here, we exploited visual periodicity to examine how category-based attention differentially modulates selective neural processing of face and non-face categories. We combined electroencephalography with a novel frequency-tagging paradigm capable of capturing selective neural responses for multiple visual categories contained within the same rapid image stream (faces/birds in Exp 1; houses/birds in Exp 2). We found that the pattern of attentional enhancement and suppression for face-selective processing is unique compared to other object categories: Where attending to non-face objects strongly enhances their selective neural signals during a later stage of processing (300-500 ms), attentional enhancement of face-selective processing is both earlier and comparatively more modest. Moreover, only the selective neural response for faces appears to be actively suppressed by attending towards an alternate visual category. These results underscore the special status that faces hold within the human visual system, and highlight the utility of visual periodicity as a powerful tool for indexing selective neural processing of multiple visual categories contained within the same image sequence.

ObScene database: Semantic congruency norms for 898 pairs of object-scene pictures

Research on the interaction between object and scene processing has a long history in the fields of perception and visual memory. Most databases have established norms for pictures where the object is embedded in the scene. In this study, we provide a diverse and controlled stimulus set comprising real-world pictures of 375 objects (e.g., suitcase), 245 scenes (e.g., airport), and 898 object-scene pairs (e.g., suitcase-airport), with object and scene presented separately. Our goal was twofold. First, to create a database of object and scene pictures, normed for the same variables to have comparable measures for both types of pictures. Second, to acquire normative data for the semantic relationships between objects and scenes presented separately, which offers more flexibility in the use of the pictures and allows disentangling the processing of the object and its context (the scene). Along three experiments, participants evaluated each object or scene picture on name agreement, familiarity, and visual complexity, and rated object-scene pairs on semantic congruency. A total of 125 septuplets of one scene and six objects (three congruent, three incongruent), and 120 triplets of one object and two scenes (in congruent and incongruent pairings) were built. In future studies, these objects and scenes can be used separately or combined, while controlling for their key features. Additionally, as object-scene pairs received semantic congruency ratings along the entire scale, researchers may select among a wide range of congruency values. ObScene is a comprehensive and ecologically valid database, useful for psychology and neuroscience studies of visual object and scene processing.

The time course of processing handwritten words: An ERP investigation

Behavioral studies have shown that the legibility of handwritten script hinders visual word recognition. Furthermore, when compared with printed words, lexical effects (e.g., word-frequency effect) are magnified for less intelligible (difficult) handwriting (Barnhart and Goldinger, 2010; Perea et al., 2016). This boost has been interpreted in terms of greater influence of top-down mechanisms during visual word recognition. In the present experiment, we registered the participants' ERPs to uncover top-down processing effects on early perceptual encoding. Participants' behavioral and EEG responses were recorded to high- and low-frequency words that varied in script's legibility (printed, easy handwritten, difficult handwritten) in a lexical decision experiment. Behavioral results replicated previous findings: word-frequency effects were larger in difficult handwriting than in easy handwritten or printed conditions. Critically, the ERP data showed an early effect of word-frequency in the N170 that was restricted to the difficult-to-read handwritten condition. These results are interpreted in terms of increased attentional deployment when the bottom-up signal is weak (difficult handwritten stimuli). This attentional boost would enhance top-down effects (e.g., lexical effects) in the early stages of visual word processing.

Post-Saccadic Face Processing Is Modulated by Pre-Saccadic Preview: Evidence from Fixation-Related Potentials

Humans actively sample their environment with saccadic eye movements to bring relevant information into high-acuity foveal vision. Despite being lower in resolution, peripheral information is also available before each saccade. How the pre-saccadic extrafoveal preview of a visual object influences its post-saccadic processing is still an unanswered question. The current study investigated this question by simultaneously recording behavior and fixation-related brain potentials while human subjects made saccades to face stimuli. We manipulated the relationship between pre-saccadic "previews" and post-saccadic images to explicitly isolate the influences of the former. Subjects performed a gender discrimination task on a newly foveated face under three preview conditions: scrambled face, incongruent face (different identity from the foveated face), and congruent face (same identity). As expected, reaction times were faster after a congruent-face preview compared with a scrambled-face preview. Importantly, intact face previews (either incongruent or congruent) resulted in a massive reduction of post-saccadic neural responses. Specifically, we analyzed the classic face-selective N170 component at occipitotemporal electroencephalogram electrodes, which was still present in our experiments with active looking. However, the post-saccadic N170 was strongly attenuated following intact-face previews compared with the scrambled condition. This large and long-lasting decrease in evoked activity is consistent with a trans-saccadic mechanism of prediction that influences category-specific neural processing at the start of a new fixation. These findings constrain theories of visual stability and show that the extrafoveal preview methodology can be a useful tool to investigate its underlying mechanisms.SIGNIFICANCE STATEMENT Neural correlates of object recognition have traditionally been studied by flashing stimuli to the central visual field. This procedure differs in fundamental ways from natural vision, where viewers actively sample the environment with eye movements and also obtain a low-resolution preview of soon-to-be-fixated objects. Here we show that the N170, a classic electrophysiological marker of the structural encoding of faces, also occurs during a more natural viewing condition but is strongly reduced due to extrafoveal preprocessing (preview benefit). Our results therefore highlight the importance of peripheral vision during trans-saccadic processing in building a coherent and stable representation of the world around us.

Task Demands Modulate Effects of Threatening Faces on Early Perceptual Encoding

The threat capture hypothesis states that threatening stimuli are automatically processed with higher priority than non-threatening stimuli, irrespective of observer intentions or focus of attention. We evaluated the threat capture hypothesis with respect to the early perceptual stages of face processing. We focused on an electrophysiological marker of face processing (the lateralized N170) in response to neutral, happy, and angry facial expressions displayed in competition with a non-face stimulus (a house). We evaluated how effects of facial expression on the lateralized N170 were modulated by task demands. In the pixel task, participants were required to identify the gender of the face, which made the face task-relevant and entailed structural encoding of the face stimulus. In the pixel task, participants identified the location of a missing pixel in the fixation cross, which made the face task-irrelevant and placed it outside the focus of attention. When faces were relevant, the lateralized N170 to angry faces was enhanced compared to happy and neutral faces. When faces were irrelevant, facial expression had no effect. These results reveal the critical role of task demands on the preference for threatening faces, indicating that top-down, voluntary processing modulates the prioritization of threat.

Attenuated Face Processing during Mind Wandering

Mind wandering (MW) has been recently investigated in many studies. It has been suggested that, during MW, processing of perceptual stimuli is attenuated in favor of internal thoughts, a phenomenon referred to as perceptual decoupling. Perceptual decoupling has been investigated in ERP studies, which have used relatively simple perceptual stimuli, yet it remains unclear if MW can impact the perceptual processing of complex stimuli with real-world relevance. Here, we investigated the impact of MW on behavioral and neural responses to faces. Thirty-six participants completed a novel sustained attention to response task with faces. They were asked to respond to upright faces (nontargets) and withhold responses to inverted faces (targets) and to report intermittently if they were “On task” or “Off task.” Behavioral analyses revealed greater intraindividual coefficient of variation for nontarget faces preceding Off task versus On task. ERP analyses focused primarily on the N170 component associated with face processing but also included the P1 and P3 components. The results revealed attenuated amplitudes to nontarget faces preceding Off task versus On task for the N170, but not for the P3 or P1. These findings suggest decoupled visual processing of faces during MW, which has implications for social neuroscience research.

Selective Attention to Faces in a Rapid Visual Stream: Hemispheric Differences in Enhancement and Suppression of Category-selective Neural Activity

In daily life, efficient perceptual categorization of faces occurs in dynamic and highly complex visual environments. Yet the role of selective attention in guiding face categorization has predominantly been studied under sparse and static viewing conditions, with little focus on disentangling the impact of attentional enhancement and suppression. Here we show that attentional enhancement and suppression exert a differential impact on face categorization supported by the left and right hemispheres. We recorded 128-channel EEG while participants viewed a 6-Hz stream of object images (buildings, animals, objects, etc.) with a face image embedded as every fifth image (i.e., OOOOFOOOOFOOOOF…). We isolated face-selective activity by measuring the response at the face presentation frequency (i.e., 6 Hz/5 = 1.2 Hz) under three conditions: Attend Faces, in which participants monitored the sequence for instances of female faces; Attend Objects, in which they responded to instances of guitars; and Baseline, in which they performed an orthogonal task on the central fixation cross. During the orthogonal task, face-specific activity was predominantly centered over the right occipitotemporal region. Actively attending to faces enhanced face-selective activity much more evidently in the left hemisphere than in the right, whereas attending to objects suppressed the face-selective response in both hemispheres to a comparable extent. In addition, the time courses of attentional enhancement and suppression did not overlap. These results suggest the left and right hemispheres support face-selective processing in distinct ways—where the right hemisphere is mandatorily engaged by faces and the left hemisphere is more flexibly recruited to serve current tasks demands.

Interfacing With Faces: Perceptual Humanization and Dehumanization

This article links the visual perception of faces and social behavior. We argue that the ways in which people visually encode others’ faces—a rapid-fire perceptual categorization—can result in either humanizing or dehumanizing modes of perception. Our model suggests that these perceptual pathways channel subsequent social inferences and behavior. We focus on the construct of perceptual dehumanization, which involves a shift from configural to featural processing of human faces and, in turn, enables the infliction of harm, such as harsh punishments. We discuss visual attention as an antecedent of perceptual modes and consequent modes of social behavior and speculate about the functions of humanization and dehumanization in sustaining macro-level social structures.

Relative expertise affects N170 during selective attention to superimposed face-character images

It remains unclear whether the N170 of ERPs reflects domain-specific or domain-general visual object processing. In this study, we used superimposed images of a face and a Chinese character such that participants' relative expertise for the two object types was either similar (Experiment 1 and 2) or different (Experiment 3). Experiment 1 showed that N170 amplitude was larger when participants attended to the character instead of the face of a face-character combination. This result was unchanged in Experiment 2, in which task difficulty was selectively increased for the face component of the combined stimuli. Experiment 3 showed that, although this N170 enhancement for attending to characters relative to faces persisted for false characters with recognizable parts, it disappeared for unrecognizable characters. Therefore, N170 amplitude was significantly greater for Chinese characters than for faces presented within a combined image, independent of the relative task difficulty. This result strongly calls N170 face selectivity into question, demonstrating that, contrary to the expectations established by a domain-specific account, N170 is modulated by expertise.

Dissociating Attention Effects from Categorical Perception with ERP Functional Microstates

When faces appear in our visual environment we naturally attend to them, possibly to the detriment of other visual information. Evidence from behavioural studies suggests that faces capture attention because they are more salient than other types of visual stimuli, reflecting a category-dependent modulation of attention. By contrast, neuroimaging data has led to a domain-specific account of face perception that rules out the direct contribution of attention, suggesting a dedicated neural network for face perception. Here we sought to dissociate effects of attention from categorical perception using Event Related Potentials. Participants viewed physically matched face and butterfly images, with each category acting as a target stimulus during different blocks in an oddball paradigm. Using a data-driven approach based on functional microstates, we show that the locus of endogenous attention effects with ERPs occurs in the N1 time range. Earlier categorical effects were also found around the level of the P1, reflecting either an exogenous increase in attention towards face stimuli, or a putative face-selective measure. Both category and attention effects were dissociable from one another hinting at the role that faces may play in early capturing of attention before top-down control of attention is observed. Our data support the conclusion that certain object categories, in this experiment, faces, may capture attention before top-down voluntary control of attention is initiated.

Double dissociation of configural and featural face processing on P1 and P2 components as a function of spatial attention

Face recognition relies on both configural and featural processing. Previous research has shown that P1 is sensitive to configural face processing, but it is unclear whether any component is sensitive to featural face processing; moreover, if there is such a component, its temporal sequence relative to P1 is unknown. Thus, to avoid confounding physical stimuli differences between configural and featural face processing on ERP components, a spatial attention paradigm was employed by instructing participants to attend an image stream (faces and houses) or an alphanumeric character stream. The interaction between attention and face processing type on P1 and P2 components indicates different mechanisms of configural and featural face processing as a function of spatial attention. The steady-state visual evoked potential (SSVEP) results clearly demonstrated that participants could selectively attend to different streams of information. Importantly, configural face processing elicited a larger posterior P1 (approximately 128 ms) than featural face processing, whereas P2 (approximately 248 ms) was greater for featural than for configural face processing under attended condition. The interaction between attention and face processing type (configural vs. featural) on P1 and P2 components indicates that there are different mechanisms of configural and featural face processing operating as functions of spatial attention. While the P1 result confirms previous findings separating configural and featural face processing, the newly observed P2 finding in the present study extends this separation to a double dissociation. Therefore, configural and featural face processing are modulated differently by spatial attention, and configural face processing precedes featural face processing.

Neural Signatures of Conscious Face Perception in an Inattentional Blindness Paradigm

Previous studies suggest that early stages of face-specific processing are performed preattentively and unconsciously, whereas conscious perception emerges with late-stage (>300 ms) neuronal activity. A conflicting view, however, posits that attention is necessary for face-specific processing and that early-to-mid latency neural responses (∼ 100-300 ms) correspond more closely with perceptual awareness. The current study capitalized on a recently developed method for manipulating attention and conscious perception during EEG recording (modified inattentional blindness paradigm) and used face stimuli that elicit a well known marker of early face processing, the N170 event-related potential (ERP). In Phase 1 of the experiment, subjects performed a demanding distracter task while line drawings of faces and matched control stimuli were presented in the center of their view. When queried, half of the subjects reported no awareness of the faces and were deemed inattentionally blind. In Phase 2, subjects performed the same distracter task, but now consciously perceived the face stimuli due to the intervening questioning. In Phase 3, subjects performed a discrimination task on the faces. Two primary contrasts were made: aware versus unaware (equally task irrelevant) and task-relevant versus task-irrelevant (equally aware). The N170 and a subsequent ERP component, the visual awareness negativity (∼ 260-300 ms), were absent during inattentional blindness and present in the aware conditions. The P3b (> 300 ms) was absent for task-irrelevant faces, even when consciously perceived, and present only when the faces were task relevant. These results inform contemporary theories of conscious face perception in particular and visual attention and perceptual awareness in general.

Neural dynamics of prediction and surprise in infants

Prior expectations shape neural responses in sensory regions of the brain, consistent with a Bayesian predictive coding account of perception. Yet, it remains unclear whether such a mechanism is already functional during early stages of development. To address this issue, we study how the infant brain responds to prediction violations using a cross-modal cueing paradigm. We record electroencephalographic responses to expected and unexpected visual events preceded by auditory cues in 12-month-old infants. We find an increased response for unexpected events. However, this effect of prediction error is only observed during late processing stages associated with conscious access mechanisms. In contrast, early perceptual components reveal an amplification of neural responses for predicted relative to surprising events, suggesting that selective attention enhances perceptual processing for expected events. Taken together, these results demonstrate that cross-modal statistical regularities are used to generate predictions that differentially influence early and late neural responses in infants.

Prior expectations shape neural processing in the brain and violations of these expectations can have a profound influence on learning. Here the authors demonstrate that such predictive coding mechanisms are already functional in the brains of 12-month-old infants.

Buffering effect of money priming on negative emotions—An ERP study

Recent studies have accumulated evidences that merely reminding people of money could lead to behavioral changes including alleviating both physical pain and social distress. However, the underlying neural mechanism regarding such pain-buffering effect of money is not clear. In this paper, we applied event-related potentials (ERP) to investigate the neural effect of money reminders on induced negative emotions. Subjects were first primed of money images and subsequently viewing unpleasant pictures, while EEG was recorded. Behavioral results suggested a reduced sensitivity to unpleasant pictures after participants being reminded of money. ERP data showed that money priming, compared to neutral priming, generated a larger N2 in frontal and posterior areas, reflecting an endogenous mental conflict and the recruitment of attention resources, and a smaller late positive potential (LPP) in parietal and occipital regions, indicating a regulating process of negative emotions. Additionally, how brain responded to money and neutral stimuli were also examined, indexed by "N170-P2" complex. This study provided additional neurophysiological evidences to support previous behavioral researches on money priming and discussed the two separated neural dynamic stages involved in emotion regulation.

Brief exposure to aversive stimuli impairs visual selective attention

Although it is well established that stress can disrupt complex cognitive functions, relatively little is known about how it influences visual processing, especially in terms of visual selective attention. In the current study, we used highly aversive images, taken from the International Affective Picture System, to induce acute, low-intensity stress while participants performed a visual discrimination task. Consistent with prior research, we found that anticipation of aversive stimuli increased overall amplitude of the N170, suggesting an increase in early sensory gain. More importantly, we found that stress disrupted visual selective attention. While in no-stress blocks, the amplitude of the face-sensitive N170 was higher when participants attended to faces rather than scenes in face-scene overlay images; this effect was absent under stress. This was because of an increase in N170 amplitude in the scene-attend condition under stress. We interpret these findings as suggesting that even low-intensity acute stress can impair participants' ability to filter out task-irrelevant information. We discuss our findings in relation to how even brief exposure to low-intensity stress may adversely impact both healthy and clinical populations.

fMRI investigation of monocular pattern rivalry

In monocular pattern rivalry, a composite image is shown to both eyes. The patient experiences perceptual alternations in which the two stimulus components alternate in clarity or salience. We used fMRI at 3T to image brain activity while participants perceived monocular rivalry passively or indicated their percepts with a task. The stimulus patterns were left/right oblique gratings, face/house composites, or a nonrivalrous control stimulus that did not support the perception of transparency or image segmentation. All stimuli were matched for luminance, contrast, and color. Compared with the control stimulus, the cortical activation for passive viewing of grating rivalry included dorsal and ventral extrastriate cortex, superior and inferior parietal regions, and multiple sites in frontal cortex. When the BOLD signal for the object rivalry task was compared with the grating rivalry task, a similar whole-brain network was engaged, but with significantly greater activity in extrastriate regions, including V3, V3A, fusiform face area (FFA), and parahippocampal place area (PPA). In addition, for the object rivalry task, FFA activity was significantly greater during face-dominant periods whereas parahippocampal place area activity was greater during house-dominant periods. Our results demonstrate that slight stimulus changes that trigger monocular rivalry recruit a large whole-brain network, as previously identified for other forms of bistability. Moreover, the results indicate that rivalry for complex object stimuli preferentially engages extrastriate cortex. We also establish that even with natural viewing conditions, endogenous attentional fluctuations in monocular pattern rivalry will differentially drive object-category-specific cortex, similar to binocular rivalry, but without complete suppression of the nondominant image.

Spatial and temporal attention modulate the early stages of face processing: behavioural evidence from a reaching paradigm

A presently unresolved question within the face perception literature is whether attending to the location of a face modulates face processing (i.e. spatial attention). Opinions on this matter diverge along methodological lines – where neuroimaging studies have observed that the allocation of spatial attention serves to enhance the neural response to a face, findings from behavioural paradigms suggest face processing is carried out independently of spatial attention. In the present study, we reconcile this divide by using a continuous behavioural response measure that indexes face processing at a temporal resolution not available in discrete behavioural measures (e.g. button press). Using reaching trajectories as our response measure, we observed that although participants were able to process faces both when attended and unattended (as others have found), face processing was not impervious to attentional modulation. Attending to the face conferred clear benefits on sex-classification processes at less than 350ms of stimulus processing time. These findings constitute the first reliable demonstration of the modulatory effects of both spatial and temporal attention on face processing within a behavioural paradigm.

The naked truth: the face and body sensitive N170 response is enhanced for nude bodies

Recent event-related potential studies have shown that the occipitotemporal N170 component - best known for its sensitivity to faces - is also sensitive to perception of human bodies. Considering that in the timescale of evolution clothing is a relatively new invention that hides the bodily features relevant for sexual selection and arousal, we investigated whether the early N170 brain response would be enhanced to nude over clothed bodies. In two experiments, we measured N170 responses to nude bodies, bodies wearing swimsuits, clothed bodies, faces, and control stimuli (cars). We found that the N170 amplitude was larger to opposite and same-sex nude vs. clothed bodies. Moreover, the N170 amplitude increased linearly as the amount of clothing decreased from full clothing via swimsuits to nude bodies. Strikingly, the N170 response to nude bodies was even greater than that to faces, and the N170 amplitude to bodies was independent of whether the face of the bodies was visible or not. All human stimuli evoked greater N170 responses than did the control stimulus. Autonomic measurements and self-evaluations showed that nude bodies were affectively more arousing compared to the other stimulus categories. We conclude that the early visual processing of human bodies is sensitive to the visibility of the sex-related features of human bodies and that the visual processing of other people's nude bodies is enhanced in the brain. This enhancement is likely to reflect affective arousal elicited by nude bodies. Such facilitated visual processing of other people's nude bodies is possibly beneficial in identifying potential mating partners and competitors, and for triggering sexual behavior.

Working memory templates are maintained as feature-specific perceptual codes

Working memory (WM) representations serve as templates that guide behavior, but the neural basis of these templates remains elusive. We tested the hypothesis that WM templates are maintained by biasing activity in sensoriperceptual neurons that code for features of items being held in memory. Neural activity was recorded using event-related potentials (ERPs) as participants viewed a series of faces and responded when a face matched a target face held in WM. Our prediction was that if activity in neurons coding for the features of the target is preferentially weighted during maintenance of the target, then ERP activity evoked by a nontarget probe face should be commensurate with the visual similarity between target and probe. Visual similarity was operationalized as the degree of overlap in visual features between target and probe. A face-sensitive ERP response was modulated by target-probe similarity. Amplitude was largest for probes that were similar to the target, and decreased monotonically as a function of decreasing target-probe similarity. These results indicate that neural activity is weighted in favor of visual features that comprise an actively held memory representation. As such, our findings support the notion that WM templates rely on neural populations involved in forming percepts of memory items.

Distinct mechanisms for the impact of distraction and interruption on working memory in aging

Interference is known to negatively impact the ability to maintain information in working memory (WM), an effect that is exacerbated with aging. Here, we explore how distinct sources of interference, i.e., distraction (stimuli to-be-ignored) and interruption (stimuli requiring attention), differentially influence WM in younger and older adults. EEG was recorded while participants engaged in three versions of a delayed-recognition task: no interference, a distracting stimulus, and an interrupting stimulus presented during WM maintenance. Behaviorally, both types of interference negatively impacted WM accuracy in older adults significantly more than younger adults (with a larger deficit for interruptions). N170 latency measures revealed that the degree of processing both distractors and interruptors predicted WM accuracy in both populations. However, while WM impairments could be explained by excessive attention to distractors by older adults (a suppression deficit), impairment induced by interruption were not clearly mediated by age-related increases in attention to interruptors. These results suggest that distinct underlying mechanisms mediate the impact of different types of external interference on WM in normal aging.