Face-sensitive processes one hundred milliseconds after picture onset

Electrophysiological correlates of face and object perception: A comparative analysis of 2D laboratory and virtual reality conditions

Human face perception is a specialized visual process with inherent social significance. The neural mechanisms reflecting this intricate cognitive process have evolved in spatially complex and emotionally rich environments. Previous research using VR to transfer an established face perception paradigm to realistic conditions has shown that the functional properties of face-sensitive neural correlates typically observed in the laboratory are attenuated outside the original modality. The present study builds on these results by comparing the perception of persons and objects under conventional laboratory (PC) and realistic conditions in VR. Adhering to established paradigms, the PC- and VR modalities both featured images of persons and cars alongside standard control images. To investigate the individual stages of realistic face processing, response times, the typical face-sensitive N170 component, and relevant subsequent components (L1, L2; pre-, post-response) were analyzed within and between modalities. The between-modality comparison of response times and component latencies revealed generally faster processing under realistic conditions. However, the obtained N170 latency and amplitude differences showed reduced discriminative capacity under realistic conditions during this early stage. These findings suggest that the effects commonly observed in the lab are specific to monitor-based presentations. Analyses of later and response-locked components showed specific neural mechanisms for identification and evaluation are employed when perceiving the stimuli under realistic conditions, reflected in discernible amplitude differences in response to faces and objects beyond the basic perceptual features. Conversely, the results do not provide evidence for comparable stimulus-specific perceptual processing pathways when viewing pictures of the stimuli under conventional laboratory conditions.

Spatial representation of multidimensional information in emotional faces revealed by fMRI

Face perception is a complex process that involves highly specialized procedures and mechanisms. Investigating into face perception can help us better understand how the brain processes fine-grained, multidimensional information. This research aimed to delve deeply into how different dimensions of facial information are represented in specific brain regions or through inter-regional connections via an implicit face recognition task. To capture the representation of various facial information in the brain, we employed support vector machine decoding, functional connectivity, and model-based representational similarity analysis on fMRI data, resulting in the identification of three crucial findings. Firstly, despite the implicit nature of the task, emotions were still represented in the brain, contrasting with all other facial information. Secondly, the connection between the medial amygdala and the parahippocampal gyrus was found to be essential for the representation of facial emotion in implicit tasks. Thirdly, in implicit tasks, arousal representation occurred in the parahippocampal gyrus, while valence depended on the connection between the primary visual cortex and the parahippocampal gyrus. In conclusion, these findings dissociate the neural mechanisms of emotional valence and arousal, revealing the precise spatial patterns of multidimensional information processing in faces.

SHAP value-based ERP analysis (SHERPA): Increasing the sensitivity of EEG signals with explainable AI methods

Conventionally, event-related potential (ERP) analysis relies on the researcher to identify the sensors and time points where an effect is expected. However, this approach is prone to bias and may limit the ability to detect unexpected effects or to investigate the full range of the electroencephalography (EEG) signal. Data-driven approaches circumvent this limitation, however, the multiple comparison problem and the statistical correction thereof affect both the sensitivity and specificity of the analysis. In this study, we present SHERPA - a novel approach based on explainable artificial intelligence (XAI) designed to provide the researcher with a straightforward and objective method to find relevant latency ranges and electrodes. SHERPA is comprised of a convolutional neural network (CNN) for classifying the conditions of the experiment and SHapley Additive exPlanations (SHAP) as a post hoc explainer to identify the important temporal and spatial features. A classical EEG face perception experiment is employed to validate the approach by comparing it to the established researcher- and data-driven approaches. Likewise, SHERPA identified an occipital cluster close to the temporal coordinates for the N170 effect expected. Most importantly, SHERPA allows quantifying the relevance of an ERP for a psychological mechanism by calculating an "importance score". Hence, SHERPA suggests the presence of a negative selection process at the early and later stages of processing. In conclusion, our new method not only offers an analysis approach suitable in situations with limited prior knowledge of the effect in question but also an increased sensitivity capable of distinguishing neural processes with high precision.

Nonlocal contrast calculated by the second order visual mechanisms and its significance in identifying facial emotions

Background: Previously obtained results indicate that faces are / preattentively/ detected in the visual scene very fast, and information on facial expression is rapidly extracted at the lower levels of the visual system. At the same time different facial attributes make different contributions in facial expression recognition. However, it is known, among the preattentive mechanisms there are none that would be selective for certain facial features, such as eyes or mouth. The aim of our study was to identify a candidate for the role of such a mechanism. Our assumption was that the most informative areas of the image are those characterized by spatial heterogeneity, particularly with nonlocal contrast changes. These areas may be identified / in the human visual system/ by the second-order visual / mechanisms/ filters selective to contrast modulations of brightness gradients. Methods: We developed a software program imitating the operation of these / mechanisms/ filters and finding areas of contrast heterogeneity in the image. Using this program, we extracted areas with maximum, minimum and medium contrast modulation amplitudes from the initial face images, then we used these to make three variants of one and the same face. The faces were demonstrated to the observers along with other objects synthesized the same way. The participants had to identify faces and define facial emotional expressions. Results: It was found that the greater is the contrast modulation amplitude of the areas shaping the face, the more precisely the emotion is identified. Conclusions: The results suggest that areas with a greater increase in nonlocal contrast are more informative in facial images, and the second-order visual / mechanisms/ filters can claim the role of /filters/ elements that detect areas of interest, attract visual attention and are windows through which subsequent levels of visual processing receive valuable information.

Instant Effects of Semantic Information on Visual Perception

Does our perception of an object change once we discover what function it serves? We showed human participants (n = 48, 31 females and 17 males) pictures of unfamiliar objects either together with keywords matching their function, leading to semantically informed perception, or together with nonmatching keywords, resulting in uninformed perception. We measured event-related potentials to investigate at which stages in the visual processing hierarchy these two types of object perception differed from one another. We found that semantically informed compared with uninformed perception was associated with larger amplitudes in the N170 component (150-200 ms), reduced amplitudes in the N400 component (400-700 ms), and a late decrease in alpha/beta band power. When the same objects were presented once more without any information, the N400 and event-related power effects persisted, and we also observed enlarged amplitudes in the P1 component (100-150 ms) in response to objects for which semantically informed perception had taken place. Consistent with previous work, this suggests that obtaining semantic information about previously unfamiliar objects alters aspects of their lower-level visual perception (P1 component), higher-level visual perception (N170 component), and semantic processing (N400 component, event-related power). Our study is the first to show that such effects occur instantly after semantic information has been provided for the first time, without requiring extensive learning.SIGNIFICANCE STATEMENT There has been a long-standing debate about whether or not higher-level cognitive capacities, such as semantic knowledge, can influence lower-level perceptual processing in a top-down fashion. Here we could show, for the first time, that information about the function of previously unfamiliar objects immediately influences cortical processing within less than 200 ms. Of note, this influence does not require training or experience with the objects and related semantic information. Therefore, our study is the first to show effects of cognition on perception while ruling out the possibility that prior knowledge merely acts by preactivating or altering stored visual representations. Instead, this knowledge seems to alter perception online, thus providing a compelling case against the impenetrability of perception by cognition.

Real-life relevant face perception is not captured by the N170 but reflected in later potentials: A comparison of 2D and virtual reality stimuli

The perception of faces is one of the most specialized visual processes in the human brain and has been investigated by means of the early event-related potential component N170. However, face perception has mostly been studied in the conventional laboratory, i.e., monitor setups, offering rather distal presentation of faces as planar 2D-images. Increasing spatial proximity through Virtual Reality (VR) allows to present 3D, real-life-sized persons at personal distance to participants, thus creating a feeling of social involvement and adding a self-relevant value to the presented faces. The present study compared the perception of persons under conventional laboratory conditions (PC) with realistic conditions in VR. Paralleling standard designs, pictures of unknown persons and standard control images were presented in a PC- and a VR-modality. To investigate how the mechanisms of face perception differ under realistic conditions from those under conventional laboratory conditions, the typical face-specific N170 and subsequent components were analyzed in both modalities. Consistent with previous laboratory research, the N170 lost discriminatory power when translated to realistic conditions, as it only discriminated faces and controls under laboratory conditions. Most interestingly, analysis of the later component [230–420 ms] revealed more differentiated face-specific processing in VR, as indicated by distinctive, stimulus-specific topographies. Complemented by source analysis, the results on later latencies show that face-specific neural mechanisms are applied only under realistic conditions (A video abstract is available in the Supplementary material and via YouTube: https://youtu.be/TF8wiPUrpSY).

WildLab: A naturalistic free viewing experiment reveals previously unknown electroencephalography signatures of face processing

Neural mechanisms of face perception are predominantly studied in well-controlled experimental settings that involve random stimulus sequences and fixed eye positions. Although powerful, the employed paradigms are far from what constitutes natural vision. Here, we demonstrate the feasibility of ecologically more valid experimental paradigms using natural viewing behaviour, by combining a free viewing paradigm on natural scenes, free of photographer bias, with advanced data processing techniques that correct for overlap effects and co-varying non-linear dependencies of multiple eye movement parameters. We validate this approach by replicating classic N170 effects in neural responses, triggered by fixation onsets (fixation event-related potentials [fERPs]). Importantly, besides finding a strong correlation between both experiments, our more natural stimulus paradigm yielded smaller variability between subjects than the classic setup. Moving beyond classic temporal and spatial effect locations, our experiment furthermore revealed previously unknown signatures of face processing: This includes category-specific modulation of the event-related potential (ERP)'s amplitude even before fixation onset, as well as adaptation effects across subsequent fixations depending on their history.

Preschoolers’ Sensitivity to Negative and Positive Emotional Facial Expressions: An ERP Study

The study examined processing differences for facial expressions (happy, angry, or neutral) and their repetition with early (P1, N170) and late (P3) event-related potentials (ERPs) in young children (N = 33). EEG was recorded while children observed sequentially presented pairs of facial expressions, which were either the same (repeated trials) or differed in their emotion (novel trials). We also correlated ERP amplitude differences with parental and child measures of socio-emotional competence (emotion recognition, empathy). P1 amplitudes were increased for angry and happy as compared to neutral expressions. We also detected larger P3 amplitudes for angry expressions as compared to happy or neutral expressions. Repetition effects were evident at early and late processing stages marked by reduced P1 amplitudes for repeated vs. novel happy expressions, but enhanced P3 amplitudes for repeated vs. novel facial expressions. N170 amplitudes were neither modulated by facial expressions nor their repetition. None of the repetition effects were associated with measures of socio-emotional competence. Taken together, negative facial expressions led to increased neural activations in early and later processing stages, indicative of enhanced saliency to potential threating stimuli in young children. Processing of repeated facial expression seem to be differential for early and late neural stages: Reduced activation was detected at early neural processing stages particularly for happy faces, indicative of effective processing for an emotion, which is most familiar within this age range. Contrary to our hypothesis, enhanced activity for repeated vs. novel expression independent of a particular emotion were detected at later processing stages, which may be linked to the creation of new memory traces. Early and late repetition effects are discussed in light of developmental and perceptual differences as well as task-specific load.

Are covered faces eye-catching for us? The impact of masks on attentional processing of self and other faces during the COVID-19 pandemic

During the COVID-19 pandemic, we have been confronted with faces covered by surgical-like masks. This raises a question about how our brains process this kind of visual information. Thus, the aims of the current study were twofold: (1) to investigate the role of attention in the processing of different types of faces with masks, and (2) to test whether such partial information about faces is treated similarly to fully visible faces. Participants were tasked with the simple detection of self-, close-other's, and unknown faces with and without a mask; this task relies on attentional processes. Event-related potential (ERP) findings revealed a similar impact of surgical-like masks for all faces: the amplitudes of early (P100) and late (P300, LPP) attention-related components were higher for faces with masks than for fully visible faces. Amplitudes of N170 were similar for covered and fully visible faces, and sources of brain activity were located in the fusiform gyri in both cases. Linear Discriminant Analysis (LDA) revealed that irrespective of whether the algorithm was trained to discriminate three types of faces either with or without masks, it was able to effectively discriminate faces that were not presented in the training phase.

Anterior cingulate cortex activity during attentional control corresponds with rumination in depression and social anxiety

Rumination and worry are transdiagnostic perseverative cognitions that have overlapping and distinct characteristics. While the mechanisms of perseverative cognitions remain incomplete, limited data indicate anterior cingulate cortex (ACC) as it relates to top-down functions contributes to perseverative cognitions in internalizing conditions. The current study extends this work in patients with major depression (n = 39) or social anxiety (n = 42). During fMRI, participants viewed images comprising letter strings superimposed on task-irrelevant threatening or neutral faces. To moderate task difficulty, there was a low perceptual load condition where the target letter was in a string of identical letters and high load condition with the target letter in a mixed letter string increasing task difficulty. Regions of interest (ROI) comprised dorsal ACC and rostral ACC. Bilateral amygdala was also examined. Results showed diagnostic groups had similar levels of rumination and worry. Exploratory hierarchical regression analysis comprising clinical measures, task performance, and the 4 ROIs revealed significantly less dorsal ACC engagement during low (vs. high) load to task-irrelevant faces corresponded with more rumination, but not worry, regardless of diagnostic status. However, the ACC finding did not survive Bonferroni correction. Preliminary results suggest dorsal ACC response during attentional control may serve as a transdiagnostic mechanism of rumination.

Nudging the N170 forward with prior stimulation-Bridging the gap between N170 and recognition potential

Evoked response potentials are often divided up into numerous components, each with their own body of literature. But is there less variety than we might suppose? In this study, we nudge one component into looking like another. Both the N170 and recognition potential (RP) are N1 components in response to familiar objects. However, the RP is often measured with a forward mask that ends at stimulus onset whereas the N170 is often measured with no masking at all. This study investigates how inter‐stimulus interval (ISI) may delay and distort the N170 into an RP by manipulating the temporal gap (ISI) between forward mask and target. The results revealed reverse relationships between the ISI on the one hand, and the N170 latency, single‐trial N1 jitter (an approximation of N1 width) and reaction time on the other hand. Importantly, we find that scalp topographies have a unique signature at the N1 peak across all conditions, from the longest gap (N170) to the shortest (RP). These findings prove that the mask‐delayed N1 is still the same N170, even under conditions that are normally associated with a different component like the RP. In general, our results suggest greater synthesis in the study of event related potential components.

Older Adults Automatically Detect Age of Older Adults' Photographs: A Visual Mismatch Negativity Study

The human face is one of the most frequently used stimuli in vMMN (visual mismatch negativity) research. Previous studies showed that vMMN is sensitive to facial emotions and gender, but investigations of age-related vMMN differences are relatively rare. The aim of this study was to investigate whether the models' age in photographs were automatically detected, even if the photographs were not parts of the ongoing task. Furthermore, we investigated age-related differences, and the possibility of different sensitivity to photographs of participants' own versus different ages. We recorded event-related potentials (ERPs) to faces of young and old models in younger (N = 20; 18-30 years) and older groups (N = 20; 60-75 years). The faces appeared around the location of the field of a tracking task. In sequences the young or the old faces were either frequent (standards) or infrequent (deviants). According to the results, a regular sequence of models' age is automatically registered, and faces violating the models' age elicited the vMMN component. However, in this study vMMN emerged only in the older group to same-age deviants. This finding is explained by the less effective inhibition of irrelevant stimuli in the elderly, and corresponds to own-age bias effect of recognition studies.

Gender differences in subliminal affective face priming: A high-density ERP study

INTRODUCTION

Subliminal affective priming effects (SAPEs) refer to the phenomenon by which the presentation of an affective prime stimulus influences the subsequent affective evaluation of a target stimulus. Previous studies have reported that unconsciously processed stimuli affect behavioral performance more than consciously processed stimuli. However, the impact of SAPEs on the face-specific N170 component is unclear. We studied how SAPEs for fearful faces affected the N170 for subsequent supraliminal target faces using event-related potentials (ERPs).

METHODS

Japanese adults (n = 44, 20 females) participated in this study. Subliminal prime faces (neutral or fearful) were presented for 17 ms, followed by a backward mask for 283 ms and 800 ms target faces (neutral, emotionally ambiguous, or fearful). 128-channel ERPs were recorded while participants judged the expression of target faces as neutral or fearful. Response rates and response times were also measured for assessing behavioral alterations.

RESULTS

Although the behavioral results revealed no evidence of SAPEs, we found gender-related SAPEs in right N170 amplitude. Specifically, female participants exhibited enhanced right N170 amplitude for emotionally neutral faces primed by fearful faces, while male participants exhibited decreased N170 amplitude in fearful prime trials with fearful target faces. Male participants exhibited significant correlations between N170 amplitude and behavioral response time in the fearful prime-neutral target condition.

CONCLUSIONS

Our ERP results suggest the existence of a gender difference in target-face processing preceded by subliminally presented face stimuli in the right occipito-temporal region.

The Neural Signature of Social Dominance Discrimination by Means of Fast Periodic Visual Stimulation

Hierarchy is a pervasive feature of social organization. The ability to rapidly discriminate hierarchical information is critical for social interaction. Here, we took advantage of a special technique in electroencephalography (EEG) known as fast periodic visual stimulation (FPVS). We used this technique, which captures the automatic perception of faces, to explore the neural signature of social dominance discrimination. A stream of computer-generated faces was presented at 6 Hz, i.e. six faces/second. In the experimental condition, faces alternated from high to low social dominance within a sequence, bringing about a frequency of interest of 3 Hz (6 Hz/2), i.e. three high/low dominance faces appeared in one second. In two control conditions, we presented faces which came exclusively from one of two hierarchical ranks (either lower or higher). Participants were asked to respond to information unrelated to this hierarchical information, namely pressing the spacebar when the fixation changes color. Results revealed a significant 3 Hz response for the experimental condition only. This response was located bilaterally in the occipito-temporal region, indicating discrimination of differences in social dominance. Through the use of FPVS, we provide electrophysiological evidence to show that social hierarchical information can be detected automatically.

Temporal Dynamics of the Neural Representation of Social Relationships

Humans can rapidly encode information from faces to support social judgments and facilitate interactions with others. We can also recall complex knowledge about those individuals, such as their social relationships with others, but the time course of this process has not been examined in detail. This study addressed the temporal dynamics of emerging visual and social relationship information using EEG and representational similarity analysis. Participants (female = 23, male = 10) became familiar with a 10-person social network, and were then shown faces of that network's members while EEG was recorded. To examine the temporal dynamics of the cognitive processes related to face perception, we compared the similarity structure of neural pattern responses to models of visual processing, face shape similarity, person identity, and social relationships. We found that all types of information are associated with neural patterns after a face is seen. Visual models became significant early after image onset, and identity across a change in facial expression was uniquely associated with neural patterns at several points throughout the time course. Additionally, a model reflecting perceived frequency of social interaction was present beginning at ∼110 ms, even in the absence of an explicit task to think about the relationships among the network members. This study highlights the speed and salience of social information relating to group dynamics that are present in the brain during person perception.SIGNIFICANCE STATEMENT We live our lives in social groups where complex relationships form among and around us. It is likely that some of the information about social relationships that we observe is integral during person perception, to better help us interact in differing situations with a variety of people. However, when exactly this information becomes relevant has been unclear. In this study, we present evidence that information reflecting observed relationships among a social network is spontaneously represented in whole-brain patterns shortly following presentation of a face. These results are consistent with neuroimaging studies showing spontaneous spatial representation of social network characteristics, and contribute novel insights into the timing of these neural processes.

Ambiguous at the second sight: Mixed facial expressions trigger late electrophysiological responses linked to lower social impressions

Social interactions require quick perception, interpretation, and categorization of faces, with facial features offering cues to emotions, intentions, and traits. Importantly, reactions to faces depend not only on their features but also on their processing fluency, with disfluent faces suffering social devaluation. The current research used electrophysiological (EEG) and behavioral measures to explore at what processing stage and under what conditions emotional ambiguity is detected in the brain and how it influences trustworthiness judgments. Participants viewed male and female faces ranging from pure anger, through mixed expressions, to pure happiness. They categorized each face along the experimental dimension (happy vs. angry) or a control dimension (gender). In the emotion-categorization condition, mixed (ambiguous) expressions were classified relatively slower, and their trustworthiness was rated relatively lower. EEG analyses revealed that early brain responses are independent of the categorization condition, with pure faces evoking larger P1/N1 responses than mixed expressions. Some late (728- 880 ms) brain responses from central-parietal sites also were independent of the categorization condition and presumably reflect familiarity of the emotion categories, with pure expressions evoking larger central-parietal LPP amplitude than mixed expressions. Interestingly, other late responses were sensitive to both expressive features and categorization task, with ambiguous faces evoking a larger LPP amplitude in frontal-medial sites around 560-660 ms but only in the emotion categorization task. Critically, these late responses from the frontal-medial cluster correlated with the reduction in trustworthiness judgments. Overall, the results suggest that ambiguity detection involves late, top-down processes and that it influences important social impressions.

Unique N170 signatures to words and faces in deaf ASL signers reflect experience-specific adaptations during early visual processing

Previous studies with deaf adults reported reduced N170 waveform asymmetry to visual words, a finding attributed to reduced phonological mapping in left-hemisphere temporal regions compared to hearing adults. An open question remains whether this pattern indeed results from reduced phonological processing or from general neurobiological adaptations in visual processing of deaf individuals. Deaf ASL signers and hearing nonsigners performed a same-different discrimination task with visually presented words, faces, or cars, while scalp EEG time-locked to the onset of the first item in each pair was recorded. For word recognition, the typical left-lateralized N170 in hearing participants and reduced left-sided asymmetry in deaf participants were replicated. The groups did not differ on word discrimination but better orthographic skill was associated with larger N170 in the right hemisphere only for deaf participants. Face recognition was characterized by unique N170 signatures for both groups, and deaf individuals exhibited superior face discrimination performance. Laterality or discrimination performance effects did not generalize to the N170 responses to cars, confirming that deaf signers are not inherently less lateralized in their electrophysiological responses to words and critically, giving support to the phonological mapping hypothesis. P1 was attenuated for deaf participants compared to the hearing, but in both groups, P1 selectively discriminated between highly learned familiar objects - words and faces versus less familiar objects - cars. The distinct electrophysiological signatures to words and faces reflected experience-driven adaptations to words and faces that do not generalize to object recognition.

Neural Sensitivity to Mutual Information in Intermediate-Complexity Face Features Changes during Childhood

One way in which face recognition develops during infancy and childhood is with regard to the visual information that contributes most to recognition judgments. Adult face recognition depends on critical features spanning a hierarchy of complexity, including low-level, intermediate, and high-level visual information. To date, the development of adult-like information biases for face recognition has focused on low-level features, which are computationally well-defined but low in complexity, and high-level features, which are high in complexity, but not defined precisely. To complement this existing literature, we examined the development of children’s neural responses to intermediate-level face features characterized using mutual information. Specifically, we examined children’s and adults’ sensitivity to varying levels of category diagnosticity at the P100 and N170 components. We found that during middle childhood, sensitivity to mutual information shifts from early components to later ones, which may indicate a critical restructuring of face recognition mechanisms that takes place over several years. This approach provides a useful bridge between the study of low- and high-level visual features for face recognition and suggests many intriguing questions for further investigation.

Length of Hair Affects P1 and N170 Latencies for Perception of Women's Faces

This study investigated the relationship between length of hair in facial stimuli and latency and amplitude of the P1 and N170 components of event-related potentials during facial perception. Electroencephalography was recorded from 21 Japanese participants (four men, 17 women) who were shown pictures of faces with one of three lengths of hair: long, medium length, or short. In addition, we used both fixed-size and variable-size blocks. In fixed-size blocks, the three types of stimuli were matched to have the same overall size; in variable-size blocks, long hair stimuli were the biggest, medium length hair stimuli were medium sized, and short hair stimuli were the smallest. We analyzed P1 latency and amplitude using two-way (6 × 2) repeated-measures analysis of variance over length of hair and electrode; N170 latency and amplitude were analyzed using three-way (6 × 2 × 2) repeated-measures analysis of variance over length of hair, hemisphere, and electrode. The latency of P1 to faces with short hair in variable-size blocks was significantly longer than that to the other five stimulus types ( p < .01 for four of the other types; p = .083 for medium length hair in variable-size blocks). The latency of N170 to faces with long hair in variable-size blocks was significantly shorter than that to faces with medium length hair and short hair in variable-size blocks ( p = .026 and p = .086, respectively). These results indicate that length of hair influenced P1 and N170 latency, supporting the notion that length of hair is a significant external facial feature. Because long hair attracted participants' attention, there was early perceptual processing of this feature. In contrast, because short hair did not attract attention, perceptual processing of this feature was late.

Neural sensitivity to natural texture statistics in infancy

During infancy, vision becomes tuned to environmental statistics. For example, infant face recognition "narrows" in response to the frequency of face categories in the visual world, inducing out-group effects that disadvantage other-race, other-species, and other-age face recognition. There are many other low-level statistical regularities in visual experience that infants may also become tuned to during this period. In particular, natural scenes have lawful properties that adults and children are sensitive to. To what extent do infants become tuned to these regularities during the first year of life? In particular, do infants exhibit evidence of perceptual narrowing that excludes atypical images from fluent processing? We examined this question by measuring 6- and 9-month-old infants' event-related potentials (ERPs) to natural and artificial textures created by: (a) Disrupting local statistics via contrast negation, (b) Disrupting global statistics via parametric texture synthesis, or (c) both of these. We predicted that younger infants' would be sensitive to both manipulations of natural appearance, but that older infants might not distinguish between different kinds of atypical images. Instead, we found that sensitivity to synthetic appearance is only evident late in infancy. We discuss what these results imply for our understanding of visual statistical learning in infancy.

The sequence of cortical activity inferred by response latency variability in the human ventral pathway of face processing

Variability in neuronal response latency has been typically considered caused by random noise. Previous studies of single cells and large neuronal populations have shown that the temporal variability tends to increase along the visual pathway. Inspired by these previous studies, we hypothesized that functional areas at later stages in the visual pathway of face processing would have larger variability in the response latency. To test this hypothesis, we used magnetoencephalographic data collected when subjects were presented with images of human faces. Faces are known to elicit a sequence of activity from the primary visual cortex to the fusiform gyrus. Our results revealed that the fusiform gyrus showed larger variability in the response latency compared to the calcarine fissure. Dynamic and spectral analyses of the latency variability indicated that the response latency in the fusiform gyrus was more variable than in the calcarine fissure between 70 ms and 200 ms after the stimulus onset and between 4 Hz and 40 Hz, respectively. The sequential processing of face information from the calcarine sulcus to the fusiform sulcus was more reliably detected based on sizes of the response variability than instants of the maximal response peaks. With two areas in the ventral visual pathway, we show that the variability in response latency across brain areas can be used to infer the sequence of cortical activity.

Brain Activity Related to the Judgment of Face-Likeness: Correlation between EEG and Face-Like Evaluation

Faces represent important information for social communication, because social information, such as face-color, expression, and gender, is obtained from faces. Therefore, individuals' tend to find faces unconsciously, even in objects. Why is face-likeness perceived in non-face objects? Previous event-related potential (ERP) studies showed that the P1 component (early visual processing), the N170 component (face detection), and the N250 component (personal detection) reflect the neural processing of faces. Inverted faces were reported to enhance the amplitude and delay the latency of P1 and N170. To investigate face-likeness processing in the brain, we explored the face-related components of the ERP through a face-like evaluation task using natural faces, cars, insects, and Arcimboldo paintings presented upright or inverted. We found a significant correlation between the inversion effect index and face-like scores in P1 in both hemispheres and in N170 in the right hemisphere. These results suggest that judgment of face-likeness occurs in a relatively early stage of face processing.

Using Event-Related Potentials and Startle to Evaluate Time Course in Anxiety and Depression

The National Institute of Mental Health's Research Domain Criteria initiative is a research framework designed toward understanding psychopathology as abnormalities of dimensional neurobehavioral constructs rather than in terms of DSM-defined categories. Research Domain Criteria constructs within the negative valence domain are particularly relevant for understanding anxiety and depressive disorders, which are pervasive, debilitating, and characterized by negative processing bias. One important direction for Research Domain Criteria research is investigating processes and parameters related to the time course (or chronometry) of negative valenced constructs. Two reliable methods for assessing chronometry are event-related potentials (ERPs) and startle blink. In this qualitative review, we examine ERP and startle studies of individuals with anxiety or depression or individuals vulnerable to affective disorders. The aim of the review is to highlight how these methods can inform the role of chronometry in the spectrum of anxiety and depression. ERP studies examining different chronometry facets of negative valenced responses have shown that transdiagnostic groups of individuals with internalizing psychopathologies exhibit abnormalities at early stages of processing. Startle reactivity studies have robustly differentiated fear-based disorders (e.g., panic disorder, social phobia) from other anxiety disorders (e.g., generalized anxiety disorder) and have also shown that different internalizing phenotypes exhibit different patterns of habituation. Findings lend support to the value of ERP and startle measures in identifying groups that cut across conventional classification systems. We also highlight methodological issues that can aid in the validity and reproducibility of ERP and startle findings and, ultimately, in the goal of developing more precise models of anxiety and depression.

Lateralized occipito-temporal N1 responses to images of salient distorted finger postures

For humans as social beings, other people’s hands are highly visually conspicuous. Exceptionally striking are hands in other than natural configuration which have been found to elicit distinct brain activation. Here we studied response strength and lateralization of this activation using event-related potentials (ERPs), in particular, occipito-temporal N1 responses as correlates of activation in extrastriate body area. Participants viewed computer-generated images of hands, half of them showing distorted fingers, the other half showing natural fingers. As control stimuli of similar geometric complexity, images of chairs were shown, half of them with distorted legs, half with standard legs. The contrast of interest was between distorted and natural/standard stimuli. For hands, stronger N1 responses were observed for distorted (vs natural) stimuli from 170 ms post stimulus. Such stronger N1 responses were found for distorted hands and absent for distorted chairs, therefore likely unrelated to visuospatial processing of the unusual distorted shapes. Rather, N1 modulation over both hemispheres – but robustly right-lateralized – could reflect distorted hands as emotionally laden stimuli. The results are in line with privileged visual processing of hands as highly salient body parts, with distortions engaging neural resources that are especially activated for biological stimuli in social perception.

The Impact of Face Inversion on Animacy Categorization

Face animacy perception is categorical: Gradual changes in the real/artificial appearance of a face lead to nonlinear behavioral responses. Neural markers of face processing are also sensitive to face animacy, further suggesting that these are meaningful perceptual categories. Artificial faces also appear to be an “out-group” relative to real faces such that behavioral markers of expert-level processing are less evident with artificial faces than real ones. In the current study, we examined how categorical processing of real versus doll faces was impacted by the face inversion effect, which is one of the most robust markers of expert face processing. We examined how explicit categorization of faces drawn from a real/doll morph continuum was affected by face inversion (Experiment 1) and also how the response properties of the N170 were impacted by face animacy and inversion. We found that inversion does not change the position or steepness of the category boundary measured behaviorally. Further, neural markers of face processing are equally impacted by inversion regardless of whether they are elicited by real faces or doll faces. On balance, our results indicate that inversion has a limited impact on the categorical perception of face animacy.

Relative expertise affects N170 during selective attention to superimposed face-character images

It remains unclear whether the N170 of ERPs reflects domain-specific or domain-general visual object processing. In this study, we used superimposed images of a face and a Chinese character such that participants' relative expertise for the two object types was either similar (Experiment 1 and 2) or different (Experiment 3). Experiment 1 showed that N170 amplitude was larger when participants attended to the character instead of the face of a face-character combination. This result was unchanged in Experiment 2, in which task difficulty was selectively increased for the face component of the combined stimuli. Experiment 3 showed that, although this N170 enhancement for attending to characters relative to faces persisted for false characters with recognizable parts, it disappeared for unrecognizable characters. Therefore, N170 amplitude was significantly greater for Chinese characters than for faces presented within a combined image, independent of the relative task difficulty. This result strongly calls N170 face selectivity into question, demonstrating that, contrary to the expectations established by a domain-specific account, N170 is modulated by expertise.

Dissociating Attention Effects from Categorical Perception with ERP Functional Microstates

When faces appear in our visual environment we naturally attend to them, possibly to the detriment of other visual information. Evidence from behavioural studies suggests that faces capture attention because they are more salient than other types of visual stimuli, reflecting a category-dependent modulation of attention. By contrast, neuroimaging data has led to a domain-specific account of face perception that rules out the direct contribution of attention, suggesting a dedicated neural network for face perception. Here we sought to dissociate effects of attention from categorical perception using Event Related Potentials. Participants viewed physically matched face and butterfly images, with each category acting as a target stimulus during different blocks in an oddball paradigm. Using a data-driven approach based on functional microstates, we show that the locus of endogenous attention effects with ERPs occurs in the N1 time range. Earlier categorical effects were also found around the level of the P1, reflecting either an exogenous increase in attention towards face stimuli, or a putative face-selective measure. Both category and attention effects were dissociable from one another hinting at the role that faces may play in early capturing of attention before top-down control of attention is observed. Our data support the conclusion that certain object categories, in this experiment, faces, may capture attention before top-down voluntary control of attention is initiated.

Facial Cosmetics Exert a Greater Influence on Processing of the Mouth Relative to the Eyes: Evidence from the N170 Event-Related Potential Component

Cosmetic makeup significantly influences facial perception. Because faces consist of similar physical structures, cosmetic makeup is typically used to highlight individual features, particularly those of the eyes (i.e., eye shadow) and mouth (i.e., lipstick). Though event-related potentials have been utilized to study various aspects of facial processing, the influence of cosmetics on specific ERP components remains unclear. The present study aimed to investigate the relationship between the application of cosmetic makeup and the amplitudes of the P1 and N170 event-related potential components during facial perception tasks. Moreover, the influence of visual perception on N170 amplitude, was evaluated under three makeup conditions: Eye Shadow, Lipstick, and No Makeup. Electroencephalography was used to monitor 17 participants who were exposed to visual stimuli under each these three makeup conditions. The results of the present study subsequently demonstrated that the Lipstick condition elicited a significantly greater N170 amplitude than the No Makeup condition, while P1 amplitude was unaffected by any of the conditions. Such findings indicate that the application of cosmetic makeup alters general facial perception but exerts no influence on the perception of low-level visual features. Collectively, these results support the notion that the application of makeup induces subtle alterations in the processing of facial stimuli, with a particular effect on the processing of specific facial components (i.e., the mouth), as reflected by changes in N170 amplitude.

Putting Up a Big Front: Car Design and Size Affect Road-Crossing Behaviour

Previous research suggests that people tend to see faces in car fronts and that they attribute personality characteristics to car faces. In the present study we investigated whether car design influences pedestrian road-crossing behaviour. An immersive virtual reality environment with a zebra crossing scenario was used to determine a) whether the minimum accepted distance for crossing the street is larger for cars with a dominant appearance than for cars with a friendly appearance and b) whether the speed of dominant-looking cars is overestimated as compared to friendly-looking cars. Participants completed both tasks while either standing on the pavement or on the centre island. We found that people started to cross the road later in front of friendly-looking low-power cars compared to dominant-looking high-power cars, but only if the cars were relatively large in size. For small cars we found no effect of power. The speed of smaller cars was estimated to be higher compared to large cars (size-speed bias). Furthermore, there was an effect of starting position: From the centre island, participants entered the road significantly later (i. e. closer to the approaching car) and left the road later than when starting from the pavement. Similarly, the speed of the cars was estimated significantly lower when standing on the centre island compared to the pavement. To our knowledge, this is the first study to show that car fronts elicit responses on a behavioural level.

Social and nonsocial affective processing in schizophrenia - An ERP study

BACKGROUND

Despite social cognitive dysfunction that may be observed in patients with schizophrenia, the knowledge about social and nonsocial affective processing in schizophrenia is scant. The aim of this study was to examine neurophysiological and behavioural responses to neutral and negative stimuli with (faces, people) and without (animals, objects) social content in schizophrenia.

METHODS

Twenty-six patients with schizophrenia (SCZ) and 21 healthy controls (HC) completed a visual oddball paradigm with either negative or neutral pictures from the Nencki Affective Picture System (NAPS) as targets while EEG was recorded. Half of the stimuli within each category presented social content (faces, people).

RESULTS

Negative stimuli with social content produced lower N2 amplitude and higher mean LPP than any other type of stimuli in both groups. Despite differences in behavioural ratings and alterations in ERP processing of affective stimuli (lack of EPN differentiation, decreased P3 to neutral stimuli) SCZ were still able to respond to specific categories of stimuli similarly to HC.

CONCLUSIONS

The pattern of results suggests that with no additional emotion-related task demands patients with schizophrenia may present similar attentional engagement with negative social stimuli as healthy controls.

The Uncanny Valley: Existence and Explanations

More than 40 years ago, Japanese roboticist Masahiro Mori (1970/2005) proposed the “uncanny valley” hypothesis, which predicted a nonlinear relation between robots’ perceived human likeness and their likability. Although some studies have corroborated this hypothesis and proposed explanations for its existence, the evidence on both fronts has been mixed and open to debate. We first review the literature to ascertain whether the uncanny valley exists. We then try to explain the uncanny phenomenon by reviewing hypotheses derived from diverse theoretical and methodological perspectives within psychology and allied fields, including evolutionary, social, cognitive, and psychodynamic approaches. Next, we provide an evaluation and critique of these studies by focusing on their methodological limitations, leading us to question the accepted definition of the uncanny valley. We examine the definitions of human likeness and likability, and propose a statistical test to preliminarily quantify their nonlinear relation. We argue that the uncanny valley hypothesis is ultimately an engineering problem that bears on the possibility of building androids that may some day become indistinguishable from humans. In closing, we propose a dehumanization hypothesis to explain the uncanny phenomenon.

A robust and representative lower bound on object processing speed in humans

How early does the brain decode object categories? Addressing this question is critical to constrain the type of neuronal architecture supporting object categorization. In this context, much effort has been devoted to estimating face processing speed. With onsets estimated from 50 to 150 ms, the timing of the first face-sensitive responses in humans remains controversial. This controversy is due partially to the susceptibility of dynamic brain measurements to filtering distortions and analysis issues. Here, using distributions of single-trial event-related potentials (ERPs), causal filtering, statistical analyses at all electrodes and time points, and effective correction for multiple comparisons, we present evidence that the earliest categorical differences start around 90 ms following stimulus presentation. These results were obtained from a representative group of 120 participants, aged 18-81, who categorized images of faces and noise textures. The results were reliable across testing days, as determined by test-retest assessment in 74 of the participants. Furthermore, a control experiment showed similar ERP onsets for contrasts involving images of houses or white noise. Face onsets did not change with age, suggesting that face sensitivity occurs within 100 ms across the adult lifespan. Finally, the simplicity of the face-texture contrast, and the dominant midline distribution of the effects, suggest the face responses were evoked by relatively simple image properties and are not face specific. Our results provide a new lower benchmark for the earliest neuronal responses to complex objects in the human visual system.

Modulation of Alpha Oscillations in the Human EEG with Facial Preference

Facial preference that results from the processing of facial information plays an important role in social interactions as well as the selection of a mate, friend, candidate, or favorite actor. However, it still remains elusive which brain regions are implicated in the neural mechanisms underlying facial preference, and how neural activities in these regions are modulated during the formation of facial preference. In the present study, we investigated the modulation of electroencephalography (EEG) oscillatory power with facial preference. For the reliable assessments of facial preference, we designed a series of passive viewing and active choice tasks. In the former task, twenty-four face stimuli were passively viewed by participants for multiple times in random order. In the latter task, the same stimuli were then evaluated by participants for their facial preference judgments. In both tasks, significant differences between the preferred and non-preferred faces groups were found in alpha band power (8-13 Hz) but not in other frequency bands. The preferred faces generated more decreases in alpha power. During the passive viewing task, significant differences in alpha power between the preferred and non-preferred face groups were observed at the left frontal regions in the early (0.15-0.4 s) period during the 1-s presentation. By contrast, during the active choice task when participants consecutively watched the first and second face for 1 s and then selected the preferred one, an alpha power difference was found for the late (0.65-0.8 s) period over the whole brain during the first face presentation and over the posterior regions during the second face presentation. These results demonstrate that the modulation of alpha activity by facial preference is a top-down process, which requires additional cognitive resources to facilitate information processing of the preferred faces that capture more visual attention than the non-preferred faces.

N170 changes reflect competition between faces and identifiable characters during early visual processing

According to the neuronal recycling hypothesis, brain circuits can gain new functions through cultural learning, which are distinct from their evolutionarily established functions, creating competition between processes such as facial and identifiable character processing. In the present study, event-related potential (ERP) recording was used to examine electrophysiological correlates of identification levels of Chinese characters as well as the competition between facial and Chinese character processing after the characters were learnt. Twenty volunteers performed a lateralized face detection task, and N170 responses were recorded when the participants viewed only Chinese characters (identifiable or unidentifiable in Xiaozhuan font), or Chinese characters and faces concurrently. Viewing identifiable Chinese characters bilaterally elicited larger N170 amplitudes than viewing unidentifiable ones. N170 amplitudes in response to faces bilaterally declined when identifiable Chinese characters and faces were viewed concurrently as compared to viewing unidentifiable Chinese characters and faces concurrently. These results indicate that the N170 component is modulated by the observer's identification level of Chinese characters, and that identifiable Chinese characters compete with faces during early visual processing.

Phase noise reveals early category-specific modulation of the event-related potentials

Previous studies have found that the amplitude of the early event-related potential (ERP) components evoked by faces, such as N170 and P2, changes systematically as a function of noise added to the stimuli. This change has been linked to an increased perceptual processing demand and to enhanced difficulty in perceptual decision making about faces. However, to date it has not yet been tested whether noise manipulation affects the neural correlates of decisions about face and non-face stimuli similarly. To this end, we measured the ERPs for faces and cars at three different phase noise levels. Subjects performed the same two-alternative age-discrimination task on stimuli chosen from young–old morphing continua that were created from faces as well as cars and were calibrated to lead to similar performances at each noise-level. Adding phase noise to the stimuli reduced performance and enhanced response latency for the two categories to the same extent. Parallel to that, phase noise reduced the amplitude and prolonged the latency of the face-specific N170 component. The amplitude of the P1 showed category-specific noise dependence: it was enhanced over the right hemisphere for cars and over the left hemisphere for faces as a result of adding phase noise to the stimuli, but remained stable across noise levels for cars over the left and for faces over the right hemisphere. Moreover, noise modulation altered the category-selectivity of the N170, while the P2 ERP component, typically associated with task decision difficulty, was larger for the more noisy stimuli regardless of stimulus category. Our results suggest that the category-specificity of noise-induced modulations of ERP responses starts at around 100 ms post-stimulus.

Compound words prompt arbitrary semantic associations in conceptual memory

Linguistic relativity theory has received empirical support in domains such as color perception and object categorization. It is unknown, however, whether relations between words idiosyncratic to language impact non-verbal representations and conceptualizations. For instance, would one consider the concepts of horse and sea as related were it not for the existence of the compound seahorse? Here, we investigated such arbitrary conceptual relationships using a non-linguistic picture relatedness task in participants undergoing event-related brain potential recordings. Picture pairs arbitrarily related because of a compound and presented in the compound order elicited N400 amplitudes similar to unrelated pairs. Surprisingly, however, pictures presented in the reverse order (as in the sequence horse–sea) reduced N400 amplitudes significantly, demonstrating the existence of a link in memory between these two concepts otherwise unrelated. These results break new ground in the domain of linguistic relativity by revealing predicted semantic associations driven by lexical relations intrinsic to language.

The neural dynamics of face detection in the wild revealed by MVPA

Previous magnetoencephalography/electroencephalography (M/EEG) studies have suggested that face processing is extremely rapid, indeed faster than any other object category. Most studies, however, have been performed using centered, cropped stimuli presented on a blank background resulting in artificially low interstimulus variability. In contrast, the aim of the present study was to assess the underlying temporal dynamics of face detection presented in complex natural scenes. We recorded EEG activity while participants performed a rapid go/no-go categorization task in which they had to detect the presence of a human face. Subjects performed at ceiling (94.8% accuracy), and traditional event-related potential analyses revealed only modest modulations of the two main components classically associated with face processing (P100 and N170). A multivariate pattern analysis conducted across all EEG channels revealed that face category could, however, be readout very early, under 100 ms poststimulus onset. Decoding was linked to reaction time as early as 125 ms. Decoding accuracy did not increase monotonically; we report an increase during an initial 95-140 ms period followed by a plateau ∼140-185 ms-perhaps reflecting a transitory stabilization of the face information available-and a strong increase afterward. Further analyses conducted on individual images confirmed these phases, further suggesting that decoding accuracy may be initially driven by low-level stimulus properties. Such latencies appear to be surprisingly short given the complexity of the natural scenes and the large intraclass variability of the face stimuli used, suggesting that the visual system is highly optimized for the processing of natural scenes.

Neural correlates of own name and own face detection in autism spectrum disorder

Autism spectrum disorder (ASD) is a heterogeneous neurodevelopmental condition clinically characterized by social interaction and communication difficulties. To date, the majority of research efforts have focused on brain mechanisms underlying the deficits in interpersonal social cognition associated with ASD. Recent empirical and theoretical work has begun to reveal evidence for a reduced or even absent self-preference effect in patients with ASD. One may hypothesize that this is related to the impaired attentional processing of self-referential stimuli. The aim of our study was to test this hypothesis. We investigated the neural correlates of face and name detection in ASD. Four categories of face/name stimuli were used: own, close-other, famous, and unknown. Event-related potentials were recorded from 62 electrodes in 23 subjects with ASD and 23 matched control subjects. P100, N170, and P300 components were analyzed. The control group clearly showed a significant self-preference effect: higher P300 amplitude to the presentation of own face and own name than to the close-other, famous, and unknown categories, indicating preferential attentional engagement in processing of self-related information. In contrast, detection of both own and close-other's face and name in the ASD group was associated with enhanced P300, suggesting similar attention allocation for self and close-other related information. These findings suggest that attention allocation in the ASD group is modulated by the personal significance factor, and that the self-preference effect is absent if self is compared to close-other. These effects are similar for physical and non-physical aspects of the autistic self. In addition, lateralization of face and name processing is attenuated in ASD, suggesting atypical brain organization.

P1 and N170 components distinguish human-like and animal-like makeup stimuli

This study used event-related potentials to investigate the sensitivity of P1 and N170 components to human-like and animal-like makeup stimuli, which were derived from pictures of Peking opera characters. As predicted, human-like makeup stimuli elicited larger P1 and N170 amplitudes than did animal-like makeup stimuli. Interestingly, a right hemisphere advantage was observed for human-like but not for animal-like makeup stimuli. Dipole source analyses of 130-200-ms window showed that the bilateral fusiform face area may contribute to the differential sensitivity of the N170 component in response to human-like and animal-like makeup stimuli. The present study suggests that the amplitudes of both the P1 and the N170 are sensitive for the mouth component of face-like stimuli.

Transcranial direct current stimulation of the dorsolateral prefrontal cortex modulates repetition suppression to unfamiliar faces: an ERP study

Repeated visual processing of an unfamiliar face suppresses neural activity in face-specific areas of the occipito-temporal cortex. This "repetition suppression" (RS) is a primitive mechanism involved in learning of unfamiliar faces, which can be detected through amplitude reduction of the N170 event-related potential (ERP). The dorsolateral prefrontal cortex (DLPFC) exerts top-down influence on early visual processing. However, its contribution to N170 RS and learning of unfamiliar faces remains unclear. Transcranial direct current stimulation (tDCS) transiently increases or decreases cortical excitability, as a function of polarity. We hypothesized that DLPFC excitability modulation by tDCS would cause polarity-dependent modulations of N170 RS during encoding of unfamiliar faces. tDCS-induced N170 RS enhancement would improve long-term recognition reaction time (RT) and/or accuracy rates, whereas N170 RS impairment would compromise recognition ability. Participants underwent three tDCS conditions in random order at ∼72 hour intervals: right anodal/left cathodal, right cathodal/left anodal and sham. Immediately following tDCS conditions, an EEG was recorded during encoding of unfamiliar faces for assessment of P100 and N170 visual ERPs. The P3a component was analyzed to detect prefrontal function modulation. Recognition tasks were administered ∼72 hours following encoding. Results indicate the right anodal/left cathodal condition facilitated N170 RS and induced larger P3a amplitudes, leading to faster recognition RT. Conversely, the right cathodal/left anodal condition caused N170 amplitude and RTs to increase, and a delay in P3a latency. These data demonstrate that DLPFC excitability modulation can influence early visual encoding of unfamiliar faces, highlighting the importance of DLPFC in basic learning mechanisms.

Children's neural response to contrast-negated faces is species specific

Face recognition abilities develop dramatically during the first year of life, but comparatively little is known about the nature of face-specific perceptual development during early childhood. Face-specific effects of image appearance on recognition, including face inversion and contrast negation, are a useful means of understanding the functional properties of face perception developmentally. Here, we examined the generality of the impact of contrast negation on face perception during early childhood using event-related potentials (ERPs). Specifically, we recorded continuous electroencephalography (EEG) while adult participants and children between 4 and 6 years of age viewed human and non-human primate faces presented in either positive or negative contrast. We examined both the P100 and N170 components to determine whether or not sensitivity to contrast polarity was evident in face-sensitive components during early childhood and also whether or not that sensitivity was specific to species category. We found evidence of a species-specific effect of contrast negation at the N170, suggesting that by early childhood some aspects of face-specific processing have been restricted to a relatively narrow class of face stimuli. However, this effect is of the opposite sign relative to adults, suggesting that there is continued maturation of face-specific processing during childhood.