Controlling for interstimulus perceptual variance abolishes N170 face selectivity

Electrophysiological correlates of face and object perception: A comparative analysis of 2D laboratory and virtual reality conditions

Human face perception is a specialized visual process with inherent social significance. The neural mechanisms reflecting this intricate cognitive process have evolved in spatially complex and emotionally rich environments. Previous research using VR to transfer an established face perception paradigm to realistic conditions has shown that the functional properties of face-sensitive neural correlates typically observed in the laboratory are attenuated outside the original modality. The present study builds on these results by comparing the perception of persons and objects under conventional laboratory (PC) and realistic conditions in VR. Adhering to established paradigms, the PC- and VR modalities both featured images of persons and cars alongside standard control images. To investigate the individual stages of realistic face processing, response times, the typical face-sensitive N170 component, and relevant subsequent components (L1, L2; pre-, post-response) were analyzed within and between modalities. The between-modality comparison of response times and component latencies revealed generally faster processing under realistic conditions. However, the obtained N170 latency and amplitude differences showed reduced discriminative capacity under realistic conditions during this early stage. These findings suggest that the effects commonly observed in the lab are specific to monitor-based presentations. Analyses of later and response-locked components showed specific neural mechanisms for identification and evaluation are employed when perceiving the stimuli under realistic conditions, reflected in discernible amplitude differences in response to faces and objects beyond the basic perceptual features. Conversely, the results do not provide evidence for comparable stimulus-specific perceptual processing pathways when viewing pictures of the stimuli under conventional laboratory conditions.

Putting people in context: ERP responses to bodies in natural scenes

The N190 is a body-sensitive ERP component that responds to images of human bodies in different poses. In natural settings, bodies vary in posture and appear within complex, cluttered environments, frequently with other people. In many studies, however, such variability is absent. How does the N190 response change when observers see images that incorporate these sources of variability? In two experiments (N = 16 each), we varied the natural appearance of upright and inverted bodies to examine how the N190 amplitude, latency, and the Body-Inversion Effect (BIE) were affected by natural variability. In Experiment 1, we varied the number of people present in upright and inverted naturalistic scenes such that only one body, a subitizable number of bodies, or a "crowd" was present. In Experiment 2, we varied the natural body appearance by presenting bodies either as silhouettes or with photographic detail. Further, we varied the natural background appearance by either removing it or presenting individual bodies within a rich environment. Using component-based analyses of the N190, we found that the number of bodies in a scene reduced the N190 amplitude, but didn't affect the BIE (Experiment 1). Naturalistic body and background appearance (Experiment 2) also affected the N190, such that component amplitude was dramatically reduced by naturalistic appearance. To complement this analysis, we examined the contribution of spatiotemporal features (i.e., electrode × time point amplitude) via SVM decoding. This technique allows us to examine which timepoints across the entire waveform contribute the most to successful decoding of body orientation in each condition. This analysis revealed that later timepoints (after 300ms) contribute most to successful orientation decoding. These results demonstrate that natural appearance variability affects body processing at the N190 and that later ERP components may make important contributions to body processing in natural scenes.

Beyond facial expressions: A systematic review on effects of emotional relevance of faces on the N170

The N170 is the most prominent electrophysiological signature of face processing. While facial expressions reliably modulate the N170, there is considerable variance in N170 modulations by other sources of emotional relevance. Therefore, we systematically review and discuss this research area using different methods to manipulate the emotional relevance of inherently neutral faces. These methods were categorized into (1) existing pre-experimental affective person knowledge (e.g., negative attitudes towards outgroup faces), (2) experimentally instructed affective person knowledge (e.g., negative person information), (3) contingency-based affective learning (e.g., fear-conditioning), or (4) the immediate affective context (e.g., emotional information directly preceding the face presentation). For all categories except the immediate affective context category, the majority of studies reported significantly increased N170 amplitudes depending on the emotional relevance of faces. Furthermore, the potentiated N170 was observed across different attention conditions, supporting the role of the emotional relevance of faces on the early prioritized processing of configural facial information, regardless of low-level differences. However, we identified several open research questions and suggest venues for further research.

Processing objects of perceptual expertise: Differential interhemispheric transmission efficiency but similar transmission direction advantages

Faces and Chinese characters are both objects of perceptual expertise. In this study, we investigated the characteristics of interhemispheric transmission times (IHTTs) in both transmission direction and transmission efficiency during the processing of objects of perceptual expertise. A total of 112 participants engaged in a divided visual field paradigm for faces, Chinese characters, and houses in both upright and inverted orientations. The N170 amplitudes elicited by the objects of perceptual expertise (faces and Chinese characters) involved in this study were larger than those elicited by the non-perceptual expertise objects (houses). We used the latencies of the N170 component of the event-related potential (ERP) recorded in the left and right hemispheres to calculate the IHTTs. For all objects, the N170-related IHTTs from the right to the left hemispheres were shorter than those in the opposite direction. Essentially, the N170-related IHTTs for faces were shorter, that is, more efficient than those for Chinese characters and houses. This result indicates that the IHTTs during perceptual expertise and non-perceptual expertise object processing share a common transmission direction advantage, but transmission efficiency is face-specific.

Instant Effects of Semantic Information on Visual Perception

Does our perception of an object change once we discover what function it serves? We showed human participants (n = 48, 31 females and 17 males) pictures of unfamiliar objects either together with keywords matching their function, leading to semantically informed perception, or together with nonmatching keywords, resulting in uninformed perception. We measured event-related potentials to investigate at which stages in the visual processing hierarchy these two types of object perception differed from one another. We found that semantically informed compared with uninformed perception was associated with larger amplitudes in the N170 component (150-200 ms), reduced amplitudes in the N400 component (400-700 ms), and a late decrease in alpha/beta band power. When the same objects were presented once more without any information, the N400 and event-related power effects persisted, and we also observed enlarged amplitudes in the P1 component (100-150 ms) in response to objects for which semantically informed perception had taken place. Consistent with previous work, this suggests that obtaining semantic information about previously unfamiliar objects alters aspects of their lower-level visual perception (P1 component), higher-level visual perception (N170 component), and semantic processing (N400 component, event-related power). Our study is the first to show that such effects occur instantly after semantic information has been provided for the first time, without requiring extensive learning.SIGNIFICANCE STATEMENT There has been a long-standing debate about whether or not higher-level cognitive capacities, such as semantic knowledge, can influence lower-level perceptual processing in a top-down fashion. Here we could show, for the first time, that information about the function of previously unfamiliar objects immediately influences cortical processing within less than 200 ms. Of note, this influence does not require training or experience with the objects and related semantic information. Therefore, our study is the first to show effects of cognition on perception while ruling out the possibility that prior knowledge merely acts by preactivating or altering stored visual representations. Instead, this knowledge seems to alter perception online, thus providing a compelling case against the impenetrability of perception by cognition.

Real-life relevant face perception is not captured by the N170 but reflected in later potentials: A comparison of 2D and virtual reality stimuli

The perception of faces is one of the most specialized visual processes in the human brain and has been investigated by means of the early event-related potential component N170. However, face perception has mostly been studied in the conventional laboratory, i.e., monitor setups, offering rather distal presentation of faces as planar 2D-images. Increasing spatial proximity through Virtual Reality (VR) allows to present 3D, real-life-sized persons at personal distance to participants, thus creating a feeling of social involvement and adding a self-relevant value to the presented faces. The present study compared the perception of persons under conventional laboratory conditions (PC) with realistic conditions in VR. Paralleling standard designs, pictures of unknown persons and standard control images were presented in a PC- and a VR-modality. To investigate how the mechanisms of face perception differ under realistic conditions from those under conventional laboratory conditions, the typical face-specific N170 and subsequent components were analyzed in both modalities. Consistent with previous laboratory research, the N170 lost discriminatory power when translated to realistic conditions, as it only discriminated faces and controls under laboratory conditions. Most interestingly, analysis of the later component [230–420 ms] revealed more differentiated face-specific processing in VR, as indicated by distinctive, stimulus-specific topographies. Complemented by source analysis, the results on later latencies show that face-specific neural mechanisms are applied only under realistic conditions (A video abstract is available in the Supplementary material and via YouTube: https://youtu.be/TF8wiPUrpSY).

A GAN model encoded by CapsEEGNet for visual EEG encoding and image reproduction

In last few decades, reading the human mind is an innovative topic in scientific research. Recent studies in neuroscience indicate that it is possible to decode the signals of the human brain based on the neuroimaging data. The work in this paper explores the possibility of building an end-to-end BCI system to learn and visualize the brain thoughts evoked by the stimulating images. To achieve this goal, it designs an experiment to collect the EEG signals evoked by randomly presented images. Based on these data, this work analyzes and compares the classification abilities by several improved methods, including the Transformer, CapsNet and the ensemble strategies. After obtaining the optimal method to be the encoder, this paper proposes a distribution-to-distribution mapping network to transform an encoded latent feature vector into a prior image feature vector. To visualize the brain thoughts, a pretrained IC-GAN model is used to receive these image feature vectors and generate images. Extensive experiments are carried out and the results show that the proposed method can effectively deal with the small sample data original from the less electrode channels. By examining the generated images coming from the EEG signals, it verifies that the proposed model is capable of reproducing the images seen by human eyes to some extent.

WildLab: A naturalistic free viewing experiment reveals previously unknown electroencephalography signatures of face processing

Neural mechanisms of face perception are predominantly studied in well-controlled experimental settings that involve random stimulus sequences and fixed eye positions. Although powerful, the employed paradigms are far from what constitutes natural vision. Here, we demonstrate the feasibility of ecologically more valid experimental paradigms using natural viewing behaviour, by combining a free viewing paradigm on natural scenes, free of photographer bias, with advanced data processing techniques that correct for overlap effects and co-varying non-linear dependencies of multiple eye movement parameters. We validate this approach by replicating classic N170 effects in neural responses, triggered by fixation onsets (fixation event-related potentials [fERPs]). Importantly, besides finding a strong correlation between both experiments, our more natural stimulus paradigm yielded smaller variability between subjects than the classic setup. Moving beyond classic temporal and spatial effect locations, our experiment furthermore revealed previously unknown signatures of face processing: This includes category-specific modulation of the event-related potential (ERP)'s amplitude even before fixation onset, as well as adaptation effects across subsequent fixations depending on their history.

The ERP and psychophysical changes related to facial emotion perception by expertise in Japanese hospitality, "OMOTENASHI"

We investigated the emotion perception process based on hospitality expertise. Forty subjects were divided into the OMOTENASHI group working at inns considered to represent the spirit of hospitality, OMOTENASHI in Japan, and CONTROL group without experience in the hospitality industry. We presented neutral, happy, and angry faces to investigate P100 and N170 by these faces, and psychophysical changes by the favor rating test to evaluate emotional perception. In the favor rating test, the score was significantly smaller (less favorable) in OMOTENASHI than in CONTROL. Regarding event-related potential components, the maximum amplitude of P100 was significantly larger for a neutral face at the right occipital electrode in OMOTENASHI than in CONTROL, and it was significantly larger for an angry face at both occipital electrodes in OMOTENASHI than in CONTROL. However, the peak latency and maximum amplitude of N170 were not significantly different between OMOTENASHI and CONTROL at both temporal electrodes for each emotion condition. Differences on the favor rating test and P100 in OMOTENASHI suggested that workers at inns may more quickly notice and be more sensitive to the facial emotion of guests due to hospitality training, and/or that hospitality expertise may increase attention to emotion by top-down and/or bottom-up processing.

The role of phase and orientation for ERP modulations of spectrum-manipulated fearful and neutral faces

Prioritized processing of fearful compared to neutral faces has been proposed to result from evolutionary adaptation of the contrast sensitivity function (CSF) to the features of emotionally relevant faces and/or vice versa. However, it is unknown whether a stimulus merely has to feature the amplitude spectrum of a fearful face to be prioritized or whether the relevant spatial frequencies have to occur with specific phases and orientations. Prioritized processing is indexed by specific increases of Event-Related Potentials (ERPs) of the EEG and occurs throughout different early processing stages, indexed by emotion-related modulations of the P1, N170, and EPN. In this pre-registered study, we manipulated phase and amplitude properties of the Fourier spectra of neutral and fearful faces to test the effect of phase coherence (PC, face vs. scramble) and orientation coherence (OC, original vs. rotational average) and their interactions with differential emotion processing. We found that differential emotion processing was not present at the level of P1 but strongly affected N170 and EPN. In both cases, intact phase coherence was required for enhanced processing of fearful faces. OC did not interact with emotion. While faces produced the typical N170 effect, we observed a reversed effect for scrambles. Additional exploratory independent component analysis (ICA) suggests that this reversal could signal a mismatch between an early "perceptual hypothesis" and feedback of configural information. In line with our expectations, fearful-neutral differences for the N170 and EPN depend on configural information, i.e., recognizable faces.

Nudging the N170 forward with prior stimulation-Bridging the gap between N170 and recognition potential

Evoked response potentials are often divided up into numerous components, each with their own body of literature. But is there less variety than we might suppose? In this study, we nudge one component into looking like another. Both the N170 and recognition potential (RP) are N1 components in response to familiar objects. However, the RP is often measured with a forward mask that ends at stimulus onset whereas the N170 is often measured with no masking at all. This study investigates how inter‐stimulus interval (ISI) may delay and distort the N170 into an RP by manipulating the temporal gap (ISI) between forward mask and target. The results revealed reverse relationships between the ISI on the one hand, and the N170 latency, single‐trial N1 jitter (an approximation of N1 width) and reaction time on the other hand. Importantly, we find that scalp topographies have a unique signature at the N1 peak across all conditions, from the longest gap (N170) to the shortest (RP). These findings prove that the mask‐delayed N1 is still the same N170, even under conditions that are normally associated with a different component like the RP. In general, our results suggest greater synthesis in the study of event related potential components.

Early ERP functions are indexed by lateralized effects to peripherally presented emotional faces and scrambles

A large body of research suggests that early event-related potentials (ERPs), such as the P1 and N1, are potentiated by attention and represent stimulus amplification. However, recent accounts suggest that the P1 is associated with inhibiting the irrelevant visual field evidenced by a pronounced ipsilateral P1 during sustained attention to peripherally presented stimuli. The current EEG study further investigated this issue to reveal how lateralized ERP findings are modulated by face and emotional information. Therefore, participants were asked to fixate the center of the screen and pay sustained attention either to the right or left visual field, where angry or neutral faces or their Fourier phase-scrambled versions were presented. We found a bilateral P1 to all stimuli with relatively increased, but delayed, ipsilateral P1 amplitudes to faces but not to scrambles. Explorative independent component analyses dissociated an earlier lateralized larger contralateral P1 from a later bilateral P1. By contrast, the N170 showed a contralateral enhancement to all stimuli, which was most pronounced for neutral faces attended in the left hemifield. Finally, increased contralateral alpha power was found for both attended hemifields but was not significantly related to poststimulus ERPs. These results provide evidence against a general inhibitory role of the P1 but suggest stimulus-specific relative enhancements of the ipsilateral P1 for the irrelevant visual hemifield. The lateralized N170, however, is associated with stimulus amplification as a function of facial features.

Attention and prediction modulations in expected and unexpected visuospatial trajectories

Humans are constantly exposed to a rich tapestry of visual information in a potentially changing environment. To cope with the computational burden this engenders, our perceptual system must use prior context to simultaneously prioritise stimuli of importance and suppress irrelevant surroundings. This study investigated the influence of prediction and attention in visual perception by investigating event-related potentials (ERPs) often associated with these processes, N170 and N2pc for prediction and attention, respectively. A contextual trajectory paradigm was used which violated visual predictions and neglected to predetermine areas of spatial interest, to account for the potentially unpredictable nature of a real-life visual scene. Participants (N = 36) viewed a visual display of cued and non-cued shapes rotating in a five-step predictable trajectory, with the fifth and final position of either the cued or non-cued shape occurring in a predictable or unpredictable spatial location. To investigate the predictive coding theory of attention we used factors of attention and prediction, whereby attention was manipulated as either cued or non-cued conditions, and prediction manipulated in either predictable or unpredictable conditions. Results showed both enhanced N170 and N2pc amplitudes to unpredictable compared to predictable stimuli. Stimulus cueing status also increased N170 amplitude, but this did not interact with stimulus predictability. The N2pc amplitude was not affected by stimulus cueing status. In accordance with previous research these results suggest the N170 is in part a visual prediction error response with respect to higher-level visual processes, and furthermore the N2pc may index attention reorientation. The results demonstrate prior context influences the sensitivity of the N170 and N2pc electrophysiological responses. These findings add further support to the role of N170 as a prediction error signal and suggest that the N2pc may reflect attentional reorientation in response to unpredicted stimulus locations.

Robot faces elicit responses intermediate to human faces and objects at face-sensitive ERP components

Face recognition is supported by selective neural mechanisms that are sensitive to various aspects of facial appearance. These include event-related potential (ERP) components like the P100 and the N170 which exhibit different patterns of selectivity for various aspects of facial appearance. Examining the boundary between faces and non-faces using these responses is one way to develop a more robust understanding of the representation of faces in extrastriate cortex and determine what critical properties an image must possess to be considered face-like. Robot faces are a particularly interesting stimulus class to examine because they can differ markedly from human faces in terms of shape, surface properties, and the configuration of facial features, but are also interpreted as social agents in a range of settings. In the current study, we thus chose to investigate how ERP responses to robot faces may differ from the response to human faces and non-face objects. In two experiments, we examined how the P100 and N170 responded to human faces, robot faces, and non-face objects (clocks). In Experiment 1, we found that robot faces elicit intermediate responses from face-sensitive components relative to non-face objects (clocks) and both real human faces and artificial human faces (computer-generated faces and dolls). These results suggest that while human-like inanimate faces (CG faces and dolls) are processed much like real faces, robot faces are dissimilar enough to human faces to be processed differently. In Experiment 2 we found that the face inversion effect was only partly evident in robot faces. We conclude that robot faces are an intermediate stimulus class that offers insight into the perceptual and cognitive factors that affect how social agents are identified and categorized.

Early alpha/beta oscillations reflect the formation of face-related expectations in the brain

Although statistical regularities in the environment often go explicitly unnoticed, traces of implicit learning are evident in our neural activity. Recent perspectives have offered evidence that both pre-stimulus oscillations and peri-stimulus event-related potentials are reliable biomarkers of implicit expectations arising from statistical learning. What remains ambiguous, however, is the origination and development of these implicit expectations. To address this lack of knowledge and determine the temporal constraints of expectation formation, pre-stimulus increases in alpha/beta power were investigated alongside a reduction in the N170 and a suppression in peri-/post-stimulus gamma power. Electroencephalography was acquired from naive participants who engaged in a gender classification task. Participants were uninformed, that eight face images were sorted into four reoccurring pairs which were pseudorandomly hidden amongst randomly occurring face images. We found a reduced N170 for statistically expected images at left parietal and temporo-parietal electrodes. Furthermore, enhanced gamma power following the presentation of random images emphasized the bottom-up processing of these arbitrary occurrences. In contrast, enhanced alpha/beta power was evident pre-stimulus for expected relative to random faces. A particularly interesting finding was the early onset of alpha/beta power enhancement which peaked immediately after the depiction of the predictive face. Hence, our findings propose an approximate timeframe throughout which consistent traces of enhanced alpha/beta power illustrate the early prioritisation of top-down processes to facilitate the development of implicitly cued face-related expectations.

The N300: An Index for Predictive Coding of Complex Visual Objects and Scenes

Predictive coding models can simulate known perceptual or neuronal phenomena, but there have been fewer attempts to identify a reliable neural signature of predictive coding for complex stimuli. In a pair of studies, we test whether the N300 component of the event-related potential, occurring 250–350-ms poststimulus-onset, has the response properties expected for such a signature of perceptual hypothesis testing at the level of whole objects and scenes. We show that N300 amplitudes are smaller to representative (“good exemplars”) compared with less representative (“bad exemplars”) items from natural scene categories. Integrating these results with patterns observed for objects, we establish that, across a variety of visual stimuli, the N300 is responsive to statistical regularity, or the degree to which the input is “expected” (either explicitly or implicitly) based on prior knowledge, with statistically regular images evoking a reduced response. Moreover, we show that the measure exhibits context-dependency; that is, we find the N300 sensitivity to category representativeness when stimuli are congruent with, but not when they are incongruent with, a category pre-cue. Thus, we argue that the N300 is the best candidate to date for an index of perceptual hypotheses testing for complex visual objects and scenes.

Face and emotional expression processing under continuous perceptual load tasks: An ERP study

High perceptual load is thought to impair already the early stages of visual processing of task-irrelevant visual stimuli. However, recent studies showed no effects of perceptual load on early ERPs in response to task-irrelevant emotional faces. In this preregistered EEG study (N = 40), we investigated the effects of continuous perceptual load on ERPs to fearful and neutral task-irrelevant faces and their phase-scrambled versions. Perceptual load did not modulate face or emotion effects for the P1 or N170. In contrast, larger face-scramble and fearful-neutral differentiation were found during low as compared to high load for the Early Posterior Negativity (EPN). Further, face-independent P1, but face-dependent N170 emotional modulations were observed. Taken together, our findings show that P1 and N170 face and emotional modulations are highly resistant to load manipulations, indicating a high degree of automaticity during this processing stage, whereas the EPN might represent a bottleneck in visual information processing.

Effects of low-level visual information and perceptual load on P1 and N170 responses to emotional expressions

Emotional facial expressions lead to modulations of early event-related potentials (ERPs). However, it has so far remained unclear how far these modulations represent face-specific effects rather than differences in low-level visual features, and to which extent they depend on available processing resources. To examine these questions, we conducted two preregistered independent experiments (N = 40 in each experiment) using different variants of a novel task that manipulates peripheral perceptual load across levels but keeps overall visual stimulation constant. At the display center, we presented task-irrelevant angry, neutral, and happy faces and their Fourier phase-scrambled versions, which preserved low-level visual features. The results of both studies showed load-independent P1 and N170 emotional expression effects. Importantly, by using Bayesian analyses we could confirm that these facial expression effects were face-independent for the P1 but not for the N170 component. We conclude that firstly, ERP modulations during the P1 interval strongly depend on low-level visual information, while the N170 modulation requires the processing of figural facial expression features. Secondly, both P1 and N170 modulations appear to be immune to a large range of variations in perceptual load.

Attentional and emotional brain response to message framing in context of green marketing

Background

Message framing plays an important role in advertising strategies and has been studied from various perspectives in different behavioral studies.

New method

This study employs the event-related potential technique to examine attentional and emotional brain processing as influenced by message framing in the context of green marketing.

Results

The behavioral results demonstrated that purchase preference was higher under positive framing compared to negative and neutral framing.

As per the event-related potential results, negative framing elicited a larger P1 component, which reveals that in the first stage of processing information, threatening information attracted more attention. In the second and third stage, N170 and P3, respectively, were higher for positive framing, demonstrating that there was more attention toward the processing of non-threatening emotional information.

Comparison with existing method: Message Framing has been previously examined with behavioral methods. We for the first time examined it with a neuroscientific method like Event Related Brain Potential technique in a green marketing context.

Conclusion

Our results compared to behavioral studies provide stronger evidence from underlying neural perspective for how message framing can be affected by attentional and emotional brain responses in the context of green marketing.

Attentional conditions differentially affect early, intermediate and late neural responses to fearful and neutral faces

The processing of fearful facial expressions is prioritized by the human brain. This priority is maintained across various information processing stages as evident in early, intermediate and late components of event-related potentials (ERPs). However, emotional modulations are inconsistently reported for these different processing stages. In this pre-registered study, we investigated how feature-based attention differentially affects ERPs to fearful and neutral faces in 40 participants. The tasks required the participants to discriminate either the orientation of lines overlaid onto the face, the sex of the face or the face's emotional expression, increasing attention to emotion-related features. We found main effects of emotion for the N170, early posterior negativity (EPN) and late positive potential (LPP). While N170 emotional modulations were task-independent, interactions of emotion and task were observed for the EPN and LPP. While EPN emotion effects were found in the sex and emotion tasks, the LPP emotion effect was mainly driven by the emotion task. This study shows that early responses to fearful faces are task-independent (N170) and likely based on low-level and configural information while during later processing stages, attention to the face (EPN) or-more specifically-to the face's emotional expression (LPP) is crucial for reliable amplified processing of emotional faces.

Prior knowledge promotes hippocampal separation but cortical assimilation in the left inferior frontal gyrus

An adaptive memory system rarely learns information tabula rasa, but rather builds on prior knowledge to facilitate learning. How prior knowledge influences the neural representation of novel associations remains unknown. Here, participants associated pairs of faces in two conditions: a famous, highly familiar face with a novel face or two novel faces while undergoing fMRI. We examine multivoxel activity patterns corresponding to individual faces before and after learning. The activity patterns representing members of famous-novel pairs becomes separated in the hippocampus, that is, more distinct from one another through learning, in striking contrast to paired novel faces that become similar. In the left inferior frontal gyrus, however, prior knowledge leads to integration, and in a specific direction: the representation of the novel face becomes similar to that of the famous face after learning, suggesting assimilation of new into old memories. We propose that hippocampal separation might resolve interference between existing and newly learned information, allowing cortical assimilation. Thus, associative learning with versus without prior knowledge relies on radically different computations.

Prior knowledge strongly impacts new learning, but its influence on the neural representation of novel information is unknown. Here, the authors show multiple neural codes for learning: prior knowledge leads to integrated cortical representations, while promoting hippocampal separation.

Neurophysiological substrates of configural face perception in schizotypy

Face perception is a highly developed function of the human visual system. Previous studies of event-related potentials (ERPs) have identified a face-selective ERP component (negative peak at about 170 ms after stimulus onset, N170) in healthy participants. In contrast, patients with schizophrenia exhibit reduced amplitude of the N170, which may represent a pathological deficit in the neurophysiology of face perception. Interestingly, healthy humans with schizophrenia-like experiences (schizotypy) also exhibit abnormal processing of face perception. Yet, it has remained unknown how schizotypy in healthy humans is associated with the neurophysiological substrates of face perception. Here, we recruited 35 healthy participants and assessed their schizotypy by the magical ideation rating scale. We used high-density electroencephalography to obtain ERPs elicited by a set of Mooney faces (face and non-face visual stimuli). We investigated median and mean reaction times and visual ERP components in response to the stimuli. We observed a significant difference in N170 amplitude between the two face-stimulus conditions and found that the measured schizotypy scores were significantly correlated with both reaction times and N170 amplitude in response to the face stimuli across all participants. Our results thus support the model of schizotypy as a manifestation of a continuum between healthy individuals and patients with schizophrenia, where the N170 impairment serves as a biomarker for the degree of pathology along this continuum.

Modulation of face- and emotion-selective ERPs by the three most common types of face image manipulations

In neuroscientific studies, the naturalness of face presentation differs; a third of published studies makes use of close-up full coloured faces, a third uses close-up grey-scaled faces and another third employs cutout grey-scaled faces. Whether and how these methodological choices affect emotion-sensitive components of the event-related brain potentials (ERPs) is yet unclear. Therefore, this pre-registered study examined ERP modulations to close-up full-coloured and grey-scaled faces as well as cutout fearful and neutral facial expressions, while attention was directed to no-face oddballs. Results revealed no interaction of face naturalness and emotion for any ERP component, but showed, however, large main effects for both factors. Specifically, fearful faces and decreasing face naturalness elicited substantially enlarged N170 and early posterior negativity amplitudes and lower face naturalness also resulted in a larger P1.This pattern reversed for the LPP, showing linear increases in LPP amplitudes with increasing naturalness. We observed no interaction of emotion with face naturalness, which suggests that face naturalness and emotion are decoded in parallel at these early stages. Researchers interested in strong modulations of early components should make use of cutout grey-scaled faces, while those interested in a pronounced late positivity should use close-up coloured faces.

Electro-cortical correlates of multisensory integration using ecologically valid emotional stimuli: Differential effects for fear and disgust

Multisensory integration (MSI) is crucial for human communication and social interaction and has been investigated in healthy populations and neurodevelopmental disorders. However, the use of stimuli with high ecological validity is sparse, especially in event-related potential (ERP) studies. The present study examined the ERP correlates of MSI in healthy adults using short (500 ms) ecologically valid professional actor-produced emotions of fear or disgust as vocal exclamation or facial expression (unimodal conditions) or both (bimodal condition). Behaviourally, our results show a general visual dominance effect (similarly fast responses following bimodal and visual stimuli) and an MSI-related speedup of responses only for fear. Electrophysiologically, both P100 and N170 showed MSI-related amplitude increases only following fear, but not disgust stimuli. Our results show for the first time that the known differential neural processing of fear and disgust also holds for the integration of dynamic auditory and visual information.

Length of Hair Affects P1 and N170 Latencies for Perception of Women's Faces

This study investigated the relationship between length of hair in facial stimuli and latency and amplitude of the P1 and N170 components of event-related potentials during facial perception. Electroencephalography was recorded from 21 Japanese participants (four men, 17 women) who were shown pictures of faces with one of three lengths of hair: long, medium length, or short. In addition, we used both fixed-size and variable-size blocks. In fixed-size blocks, the three types of stimuli were matched to have the same overall size; in variable-size blocks, long hair stimuli were the biggest, medium length hair stimuli were medium sized, and short hair stimuli were the smallest. We analyzed P1 latency and amplitude using two-way (6 × 2) repeated-measures analysis of variance over length of hair and electrode; N170 latency and amplitude were analyzed using three-way (6 × 2 × 2) repeated-measures analysis of variance over length of hair, hemisphere, and electrode. The latency of P1 to faces with short hair in variable-size blocks was significantly longer than that to the other five stimulus types ( p < .01 for four of the other types; p = .083 for medium length hair in variable-size blocks). The latency of N170 to faces with long hair in variable-size blocks was significantly shorter than that to faces with medium length hair and short hair in variable-size blocks ( p = .026 and p = .086, respectively). These results indicate that length of hair influenced P1 and N170 latency, supporting the notion that length of hair is a significant external facial feature. Because long hair attracted participants' attention, there was early perceptual processing of this feature. In contrast, because short hair did not attract attention, perceptual processing of this feature was late.

Impairments of emotional face processing in schizophrenia patients: Evidence from P100, N170 and P300 ERP components in a sample of auditory hallucinators

Patients with schizophrenia show impaired face and emotional expression processing that may be due to early perceptual deficits or late impairments in higher-order emotional facial recognition. This study examined event-related potentials (ERPs) in 23 patients with schizophrenia who experience auditory hallucinations and 19 healthy controls. EEG activity was recorded from 32 scalp sites positioned according to the 10-10 placement system. Linked left and right electrodes at the mastoids served as the reference. The P100, N170 and P300 were measured during an emotional facial identification task, which included neutral, joyful, sad, angry and fearful facial expressions and non-face stimuli (chairs). P100 was measured at O_1/2 and P_7/8. N170 was measured at P_7/8. P300 was measured at P_z. Patients with schizophrenia were slower at identifying all facial expressions, including neutral ones. They also showed less positive P100 amplitude to sad, angry and fearful facial expressions. N170 amplitudes were smaller in patients in response to neutral, joyful, sad, angry, and fearful facial expression. Patients showed less positive P300 mean amplitudes to all facial expressions, including neutral ones. Within-group comparisons showed that patients exhibited a different pattern of ERP modulation across facial expressions than controls for P100 and N170, but not for P300. Our findings are compatible with the idea that behavioural and electrophysiological face-processing deficits in schizophrenia arise from early-stage deficits in visual processing.

Early and late cortical responses to directly gazing faces are task dependent

Gender categorisation of human faces is facilitated when gaze is directed toward the observer (i.e., a direct gaze), compared with situations where gaze is averted or the eyes are closed (Macrae, Hood, Milne, Rowe, & Mason, Psychological Science, 13(5), 460-464, 2002). However, the temporal dynamics underlying this phenomenon remain to some extent unknown. Here, we used electroencephalography (EEG) to assess the neural correlates of this effect, focusing on the event-related potential (ERP) components known to be sensitive to gaze perception (i.e., P1, N170, and P3b). We first replicated the seminal findings of Macrae et al. (2002, Experiment 1) regarding facilitated gender discrimination, and subsequently measured the underlying neural responses. Our data revealed an early preferential processing of direct gaze as compared with averted gaze and closed eyes at the P1, which reverberated at the P3b (Experiment 2). Critically, using the same material, we failed to reproduce these effects when gender categorisation was not required (Experiment 3). Taken together, our data confirm that direct gaze enhances the early P1, as well as later cortical responses to face processing, although the effect appears to be task dependent.

Face-to-face: Perceived personal relevance amplifies face processing

The human face conveys emotional and social information, but it is not well understood how these two aspects influence face perception. In order to model a group situation, two faces displaying happy, neutral or angry expressions were presented. Importantly, faces were either facing the observer, or they were presented in profile view directed towards, or looking away from each other. In Experiment 1 (n = 64), face pairs were rated regarding perceived relevance, wish-to-interact, and displayed interactivity, as well as valence and arousal. All variables revealed main effects of facial expression (emotional > neutral), face orientation (facing observer > towards > away) and interactions showed that evaluation of emotional faces strongly varies with their orientation. Experiment 2 (n = 33) examined the temporal dynamics of perceptual-attentional processing of these face constellations with event-related potentials. Processing of emotional and neutral faces differed significantly in N170 amplitudes, early posterior negativity (EPN), and sustained positive potentials. Importantly, selective emotional face processing varied as a function of face orientation, indicating early emotion-specific (N170, EPN) and late threat-specific effects (LPP, sustained positivity). Taken together, perceived personal relevance to the observer-conveyed by facial expression and face direction-amplifies emotional face processing within triadic group situations.

Lateralized occipito-temporal N1 responses to images of salient distorted finger postures

For humans as social beings, other people’s hands are highly visually conspicuous. Exceptionally striking are hands in other than natural configuration which have been found to elicit distinct brain activation. Here we studied response strength and lateralization of this activation using event-related potentials (ERPs), in particular, occipito-temporal N1 responses as correlates of activation in extrastriate body area. Participants viewed computer-generated images of hands, half of them showing distorted fingers, the other half showing natural fingers. As control stimuli of similar geometric complexity, images of chairs were shown, half of them with distorted legs, half with standard legs. The contrast of interest was between distorted and natural/standard stimuli. For hands, stronger N1 responses were observed for distorted (vs natural) stimuli from 170 ms post stimulus. Such stronger N1 responses were found for distorted hands and absent for distorted chairs, therefore likely unrelated to visuospatial processing of the unusual distorted shapes. Rather, N1 modulation over both hemispheres – but robustly right-lateralized – could reflect distorted hands as emotionally laden stimuli. The results are in line with privileged visual processing of hands as highly salient body parts, with distortions engaging neural resources that are especially activated for biological stimuli in social perception.

The cortical dynamics of speaking: Lexical and phonological knowledge simultaneously recruit the frontal and temporal cortex within 200 ms

Language production models typically assume that retrieving a word for articulation is a sequential process with substantial functional delays between conceptual, lexical, phonological and motor processing, respectively. Nevertheless, explicit evidence contrasting the spatiotemporal dynamics between different word production components is scarce. Here, using anatomically constrained magnetoencephalography during overt meaningful speech production, we explore the speed with which lexico-semantic versus acoustic-articulatory information of a to-be-uttered word become first neurophysiologically manifest in the cerebral cortex. We demonstrate early modulations of brain activity by the lexical frequency of a word in the temporal cortex and the left inferior frontal gyrus, simultaneously with activity in the motor and the posterior superior temporal cortex reflecting articulatory-acoustic phonological features (+LABIAL vs. +CORONAL) of the word-initial speech sounds (e.g., Monkey vs. Donkey). The specific nature of the spatiotemporal pattern correlating with a word's frequency and initial phoneme demonstrates that, in the course of speech planning, lexico-semantic and phonological-articulatory processes emerge together rapidly, drawing in parallel on temporal and frontal cortex. This novel finding calls for revisions of current brain language theories of word production.

Individual Differences in the Speed of Facial Emotion Recognition Show Little Specificity but Are Strongly Related with General Mental Speed: Psychometric, Neural and Genetic Evidence

Facial identity and facial expression processing are crucial socio-emotional abilities but seem to show only limited psychometric uniqueness when the processing speed is considered in easy tasks. We applied a comprehensive measurement of processing speed and contrasted performance specificity in socio-emotional, social and non-social stimuli from an individual differences perspective. Performance in a multivariate task battery could be best modeled by a general speed factor and a first-order factor capturing some specific variance due to processing emotional facial expressions. We further tested equivalence of the relationships between speed factors and polymorphisms of dopamine and serotonin transporter genes. Results show that the speed factors are not only psychometrically equivalent but invariant in their relation with the Catechol-O-Methyl-Transferase (COMT) Val158Met polymorphism. However, the 5-HTTLPR/rs25531 serotonin polymorphism was related with the first-order factor of emotion perception speed, suggesting a specific genetic correlate of processing emotions. We further investigated the relationship between several components of event-related brain potentials with psychometric abilities, and tested emotion specific individual differences at the neurophysiological level. Results revealed swifter emotion perception abilities to go along with larger amplitudes of the P100 and the Early Posterior Negativity (EPN), when emotion processing was modeled on its own. However, after partialling out the shared variance of emotion perception speed with general processing speed-related abilities, brain-behavior relationships did not remain specific for emotion. Together, the present results suggest that speed abilities are strongly interrelated but show some specificity for emotion processing speed at the psychometric level. At both genetic and neurophysiological levels, emotion specificity depended on whether general cognition is taken into account or not. These findings keenly suggest that general speed abilities should be taken into account when the study of emotion recognition abilities is targeted in its specificity.

Brain potentials predict language selection before speech onset in bilinguals

Studies of language production in bilinguals have seldom considered the fact that language selection likely involves proactive control. Here, we show that Chinese-English bilinguals actively inhibit the language not-to-be used before the onset of a picture to be named. Depending on the nature of a directive cue, participants named a subsequent picture in their native language, in their second language, or remained silent. The cue elicited a contingent negative variation of event-related brain potentials, greater in amplitude when the cue announced a naming trial as compared to when it announced a silent trial. In addition, the negativity was greater in amplitude when the picture was to be named in English than in Chinese, suggesting that preparation for speech in the second language requires more inhibition than preparation for speech in the native language. This result is the first direct neurophysiological evidence consistent with proactive inhibitory control in bilingual production.

Visual integration of fear and anger emotional cues by children on the autism spectrum and neurotypical peers: An EEG study

Communication deficits in children with autism spectrum disorders (ASD) are often related to inefficient interpretation of emotional cues, which are conveyed visually through both facial expressions and body language. The present study examined ASD behavioral and ERP responses to emotional expressions of anger and fear, as conveyed by the face and body. Behavioral results showed significantly faster response times for the ASD than for the typically developing (TD) group when processing fear, but not anger, in isolated face expressions, isolated body expressions, and in the integration of the two. In addition, EEG data for the N170 and P1 indicated processing differences between fear and anger stimuli only in TD group, suggesting that individuals with ASD may not be distinguishing between emotional expressions. These results suggest that ASD children may employ a different neural mechanism for visual emotion recognition than their TD peers, possibly relying on inferential processing.

Relative expertise affects N170 during selective attention to superimposed face-character images

It remains unclear whether the N170 of ERPs reflects domain-specific or domain-general visual object processing. In this study, we used superimposed images of a face and a Chinese character such that participants' relative expertise for the two object types was either similar (Experiment 1 and 2) or different (Experiment 3). Experiment 1 showed that N170 amplitude was larger when participants attended to the character instead of the face of a face-character combination. This result was unchanged in Experiment 2, in which task difficulty was selectively increased for the face component of the combined stimuli. Experiment 3 showed that, although this N170 enhancement for attending to characters relative to faces persisted for false characters with recognizable parts, it disappeared for unrecognizable characters. Therefore, N170 amplitude was significantly greater for Chinese characters than for faces presented within a combined image, independent of the relative task difficulty. This result strongly calls N170 face selectivity into question, demonstrating that, contrary to the expectations established by a domain-specific account, N170 is modulated by expertise.

Dynamic Encoding of Acoustic Features in Neural Responses to Continuous Speech

Humans are unique in their ability to communicate using spoken language. However, it remains unclear how the speech signal is transformed and represented in the brain at different stages of the auditory pathway. In this study, we characterized electroencephalography responses to continuous speech by obtaining the time-locked responses to phoneme instances (phoneme-related potential). We showed that responses to different phoneme categories are organized by phonetic features. We found that each instance of a phoneme in continuous speech produces multiple distinguishable neural responses occurring as early as 50 ms and as late as 400 ms after the phoneme onset. Comparing the patterns of phoneme similarity in the neural responses and the acoustic signals confirms a repetitive appearance of acoustic distinctions of phonemes in the neural data. Analysis of the phonetic and speaker information in neural activations revealed that different time intervals jointly encode the acoustic similarity of both phonetic and speaker categories. These findings provide evidence for a dynamic neural transformation of low-level speech features as they propagate along the auditory pathway, and form an empirical framework to study the representational changes in learning, attention, and speech disorders.SIGNIFICANCE STATEMENT We characterized the properties of evoked neural responses to phoneme instances in continuous speech. We show that each instance of a phoneme in continuous speech produces several observable neural responses at different times occurring as early as 50 ms and as late as 400 ms after the phoneme onset. Each temporal event explicitly encodes the acoustic similarity of phonemes, and linguistic and nonlinguistic information are best represented at different time intervals. Finally, we show a joint encoding of phonetic and speaker information, where the neural representation of speakers is dependent on phoneme category. These findings provide compelling new evidence for dynamic processing of speech sounds in the auditory pathway.

Early Left Parietal Activity Elicited by Direct Gaze: A High-Density EEG Study

Gaze is one of the most important cues for human communication and social interaction. In particular, gaze contact is the most primary form of social contact and it is thought to capture attention. A very early-differentiated brain response to direct versus averted gaze has been hypothesized. Here, we used high-density electroencephalography to test this hypothesis. Topographical analysis allowed us to uncover a very early topographic modulation (40-80 ms) of event-related responses to faces with direct as compared to averted gaze. This modulation was obtained only in the condition where intact broadband faces-as opposed to high-pass or low-pas filtered faces-were presented. Source estimation indicated that this early modulation involved the posterior parietal region, encompassing the left precuneus and inferior parietal lobule. This supports the idea that it reflected an early orienting response to direct versus averted gaze. Accordingly, in a follow-up behavioural experiment, we found faster response times to the direct gaze than to the averted gaze broadband faces. In addition, classical evoked potential analysis showed that the N170 peak amplitude was larger for averted gaze than for direct gaze. Taken together, these results suggest that direct gaze may be detected at a very early processing stage, involving a parallel route to the ventral occipito-temporal route of face perceptual analysis.

Dissociating Attention Effects from Categorical Perception with ERP Functional Microstates

When faces appear in our visual environment we naturally attend to them, possibly to the detriment of other visual information. Evidence from behavioural studies suggests that faces capture attention because they are more salient than other types of visual stimuli, reflecting a category-dependent modulation of attention. By contrast, neuroimaging data has led to a domain-specific account of face perception that rules out the direct contribution of attention, suggesting a dedicated neural network for face perception. Here we sought to dissociate effects of attention from categorical perception using Event Related Potentials. Participants viewed physically matched face and butterfly images, with each category acting as a target stimulus during different blocks in an oddball paradigm. Using a data-driven approach based on functional microstates, we show that the locus of endogenous attention effects with ERPs occurs in the N1 time range. Earlier categorical effects were also found around the level of the P1, reflecting either an exogenous increase in attention towards face stimuli, or a putative face-selective measure. Both category and attention effects were dissociable from one another hinting at the role that faces may play in early capturing of attention before top-down control of attention is observed. Our data support the conclusion that certain object categories, in this experiment, faces, may capture attention before top-down voluntary control of attention is initiated.

Facial Cosmetics Exert a Greater Influence on Processing of the Mouth Relative to the Eyes: Evidence from the N170 Event-Related Potential Component

Cosmetic makeup significantly influences facial perception. Because faces consist of similar physical structures, cosmetic makeup is typically used to highlight individual features, particularly those of the eyes (i.e., eye shadow) and mouth (i.e., lipstick). Though event-related potentials have been utilized to study various aspects of facial processing, the influence of cosmetics on specific ERP components remains unclear. The present study aimed to investigate the relationship between the application of cosmetic makeup and the amplitudes of the P1 and N170 event-related potential components during facial perception tasks. Moreover, the influence of visual perception on N170 amplitude, was evaluated under three makeup conditions: Eye Shadow, Lipstick, and No Makeup. Electroencephalography was used to monitor 17 participants who were exposed to visual stimuli under each these three makeup conditions. The results of the present study subsequently demonstrated that the Lipstick condition elicited a significantly greater N170 amplitude than the No Makeup condition, while P1 amplitude was unaffected by any of the conditions. Such findings indicate that the application of cosmetic makeup alters general facial perception but exerts no influence on the perception of low-level visual features. Collectively, these results support the notion that the application of makeup induces subtle alterations in the processing of facial stimuli, with a particular effect on the processing of specific facial components (i.e., the mouth), as reflected by changes in N170 amplitude.

The Neurological Traces of Look-Alike Avatars

We designed an observational study where participants (n = 17) were exposed to pictures and look-alike avatars pictures of themselves, a familiar friend or an unfamiliar person. By measuring participants’ brain activity with electroencephalography (EEG), we found face-recognition event related potentials (ERPs) in the visual cortex, around 200–250 ms, to be prominent for the different familiarity levels. A less positive component was found for self-recognized pictures (P200) than pictures of others, showing similar effects in both real faces and look-alike avatars. A rapid adaptation in the same component was found when comparing the neural processing of avatar faces vs. real faces, as if avatars in general were assimilated as real face representations over time. ERP results also showed that in the case of the self-avatar, the P200 component correlated with more complex conscious encodings of self-representation, i.e., the difference in voltage in the P200 between the self-avatar and the self-picture was reduced in participants that felt the avatar looked like them. This study is put into context within the literature of self-recognition and face recognition in the visual cortex. Additionally, the implications of these results on look-alike avatars are discussed both for future virtual reality (VR) and neuroscience studies.

Putting Up a Big Front: Car Design and Size Affect Road-Crossing Behaviour

Previous research suggests that people tend to see faces in car fronts and that they attribute personality characteristics to car faces. In the present study we investigated whether car design influences pedestrian road-crossing behaviour. An immersive virtual reality environment with a zebra crossing scenario was used to determine a) whether the minimum accepted distance for crossing the street is larger for cars with a dominant appearance than for cars with a friendly appearance and b) whether the speed of dominant-looking cars is overestimated as compared to friendly-looking cars. Participants completed both tasks while either standing on the pavement or on the centre island. We found that people started to cross the road later in front of friendly-looking low-power cars compared to dominant-looking high-power cars, but only if the cars were relatively large in size. For small cars we found no effect of power. The speed of smaller cars was estimated to be higher compared to large cars (size-speed bias). Furthermore, there was an effect of starting position: From the centre island, participants entered the road significantly later (i. e. closer to the approaching car) and left the road later than when starting from the pavement. Similarly, the speed of the cars was estimated significantly lower when standing on the centre island compared to the pavement. To our knowledge, this is the first study to show that car fronts elicit responses on a behavioural level.