The faces you remember: Caricaturing shape facilitates brain processes reflecting the acquisition of new face representations

Effects of Inversion and Fixation Location on the Processing of Face and House Stimuli - A Mass Univariate Analysis

Most Event Related Potential studies investigating the time course of visual processing have focused mainly on the N170 component. Stimulus orientation affects the N170 amplitude for faces but not for objects, a finding interpreted as reflecting holistic/configural processing for faces and featural processing for objects. Furthermore, while recent studies suggest where on the face people fixate impacts the N170, fixation location effects have not been investigated in objects. A data-driven mass univariate analysis (all time points and electrodes) was used to investigate the time course of inversion and fixation location effects on the neural processing of faces and houses. Strong and widespread orientation effects were found for both faces and houses, from 100-350ms post-stimulus onset, including P1 and N170 components, and later, a finding arguing against a lack of holistic processing for houses. While no clear fixation effect was found for houses, fixation location strongly impacted face processing early, reflecting retinotopic mapping around the C2 and P1 components, and during the N170-P2 interval. Face inversion effects were also largest for nasion fixation around 120ms. The results support the view that facial feature integration (1) depends on which feature is being fixated and where the other features are situated in the visual field, (2) occurs maximally during the P1-N170 interval when fixation is on the nasion and (3) continues past 200ms, suggesting the N170 peak, where weak effects were found, might be an inflexion point between processes rather than the end of a feature integration into a whole process.

Microstate Analyses to Study face Processing in Healthy Individuals and Psychiatric Disorders: A Review of ERP Findings

Microstates represent brief periods of quasi-stable electroencephalography (EEG) scalp topography, offering insights into dynamic fluctuations in event-related potential (ERP) topographies. Despite this, there is a lack of a comprehensive systematic overview of microstate findings concerning cognitive face processing. This review aims to summarize ERP findings on face processing using microstate analyses and assess their effectiveness in characterizing face-related neural representations. A literature search was conducted for microstate ERP studies involving healthy individuals and psychiatric populations, utilizing PubMed, Google Scholar, Web of Science, PsychInfo, and Scopus databases. Twenty-two studies were identified, primarily focusing on healthy individuals (n = 16), with a smaller subset examining psychiatric populations (n = 6). The evidence reviewed in this study suggests that various microstates are consistently associated with distinct ERP stages involved in face processing, encompassing the processing of basic visual facial features to more complex functions such as analytical processing, facial recognition, and semantic representations. Furthermore, these studies shed light on atypical attentional neural mechanisms in Autism Spectrum Disorder (ASD), facial recognition deficits among emotional dysregulation disorders, and encoding and semantic dysfunctions in Post-Traumatic Stress Disorder (PTSD). In conclusion, this review underscores the practical utility of ERP microstate analyses in investigating face processing. Methodologies have evolved towards greater automation and data-driven approaches over time. Future research should aim to forecast clinical outcomes and conduct validation studies to directly demonstrate the efficacy of such analyses in inverse space.

Asymmetries in event-related potentials part 1: A systematic review of face processing studies

The human brain shows distinct lateralized activation patterns for a range of cognitive processes. One such function, which is thought to be lateralized to the right hemisphere (RH), is human face processing. Its importance for social communication and interaction has led to a plethora of studies investigating face processing in health and disease. Temporally highly resolved methods, like event-related potentials (ERPs), allow for a detailed characterization of different processing stages and their specific lateralization patterns. This systematic review aimed at disentangling some of the contradictory findings regarding the RH specialization in face processing focusing on ERP research in healthy participants. Two databases were searched for studies that investigated left and right electrodes while participants viewed (mostly neutral) facial stimuli. The included studies used a variety of different tasks, which ranged from passive viewing to memorizing faces. The final data selection highlights, that strongest lateralization to the RH was found for the N170, especially for right-handed young male participants. Left-handed, female, and older participants showed less consistent lateralization patterns. Other ERP components like the P1, P2, N2, P3, and the N400 were overall less clearly lateralized. The current review highlights that many of the assumed lateralization patterns are less clear than previously thought and that the variety of stimuli, tasks, and EEG setups used, might contribute to the ambiguous findings.

Investigating the neural effects of typicality and predictability for face and object stimuli

The brain calibrates itself based on the past stimulus diet, which makes frequently observed stimuli appear as typical (as opposed to uncommon stimuli, which appear as distinctive). Based on predictive processing theory, the brain should be more "prepared" for typical exemplars, because these contain information that has been encountered frequently, allowing it to economically represent items of that category. Thus, one could ask whether predictability and typicality of visual stimuli interact, or rather act in an additive manner. We adapted the design by Egner and colleagues (2010), who used cues to induce expectations about stimulus category (face vs. chair) occurrence during an orthogonal inversion detection task. We measured BOLD responses with fMRI in 35 participants. First, distinctive stimuli always elicited stronger responses than typical ones in all ROIs, and our whole-brain directional contrasts for the effects of typicality and distinctiveness converge with previous findings. Second and importantly, we could not replicate the interaction between category and predictability reported by Egner et al. (2010), which casts doubt on whether cueing designs are ideal to elicit reliable predictability effects. Third, likely as a consequence of the lack of predictability effects, we found no interaction between predictability and typicality in any of the four tested regions (bilateral fusiform face areas, lateral occipital complexes) when considering both categories, nor in the whole brain. We discuss the issue of replicability in neuroscience and sketch an agenda for how future studies might address the same question.

Bridging the gap between EEG and DCNNs reveals a fatigue mechanism of facial repetition suppression

Facial repetition suppression, a well-studied phenomenon characterized by decreased neural responses to repeated faces in visual cortices, remains a subject of ongoing debate regarding its underlying neural mechanisms. Our research harnesses advanced multivariate analysis techniques and the prowess of deep convolutional neural networks (DCNNs) in face recognition to bridge the gap between human electroencephalogram (EEG) data and DCNNs, especially in the context of facial repetition suppression. Our innovative reverse engineering approach, manipulating the neuronal activity in DCNNs and conducted representational comparisons between brain activations derived from human EEG and manipulated DCNN activations, provided insights into the underlying facial repetition suppression. Significantly, our findings advocate the fatigue mechanism as the dominant force behind the facial repetition suppression effect. Broadly, this integrative framework, bridging the human brain and DCNNs, offers a promising tool for simulating brain activity and making inferences regarding the neural mechanisms underpinning complex human behaviors.

Varying sex and identity of faces affects face categorization differently in humans and computational models

Our faces display socially important sex and identity information. How perceptually independent are these facial characteristics? Here, we used a sex categorization task to investigate how changing faces in terms of either their sex or identity affects sex categorization of those faces, whether these manipulations affect sex categorization similarly when the original faces were personally familiar or unknown, and, whether computational models trained for sex classification respond similarly to human observers. Our results show that varying faces along either sex or identity dimension affects their sex categorization. When the sex was swapped (e.g., female faces became male looking, Experiment 1), sex categorization performance was different from that with the original unchanged faces, and significantly more so for people who were familiar with the original faces than those who were not. When the identity of the faces was manipulated by caricaturing or anti-caricaturing them (these manipulations either augment or diminish idiosyncratic facial information, Experiment 2), sex categorization performance to caricatured, original, and anti-caricatured faces increased in that order, independently of face familiarity. Moreover, our face manipulations showed different effects upon computational models trained for sex classification and elicited different patterns of responses in humans and computational models. These results not only support the notion that the sex and identity of faces are processed integratively by human observers but also demonstrate that computational models of face categorization may not capture key characteristics of human face categorization.

Differences between high and low performers in face recognition in electrophysiological correlates of face familiarity and distance-to-norm

Valentine's influential norm-based multidimensional face-space model (nMDFS) predicts that perceived distinctiveness of a face increases with its distance to the norm. Occipito-temporal event-related potentials (ERPs) have been recently shown to respond selectively to variations in distance-to-norm (P200) or familiarity (N250, late negativity), respectively (Wuttke & Schweinberger, 2019). Despite growing evidence on interindividual differences in face perception skills at the behavioral level, little research has focused on their electrophysiological correlates. To reveal potential interindividual differences in face spaces, we contrasted high and low performers in face recognition in regards to distance-to-norm (P200) and familiarity (N250). We replicated both the P200 distance-to-norm and the N250 familiarity effect. Importantly, we observed: i) reduced responses in low compared to high performers of face recognition, especially in terms of smaller distance-to-norm effects in the P200, possibly indicating less 'expanded' face spaces in low compared to high performers; ii) increased N250 responses to familiar original faces in high performers, suggesting more robust face identity representations. In summary, these findings suggest the contribution of both early norm-based face coding and robust face representations to individual face recognition skills, and indicate that ERPs can offer a promising route to understand individual differences in face perception and their neurocognitive correlates.

Neural Correlates of Voice Learning with Distinctive and Non-Distinctive Faces

Recognizing people from their voices may be facilitated by a voice's distinctiveness, in a manner similar to that which has been reported for faces. However, little is known about the neural time-course of voice learning and the role of facial information in voice learning. Based on evidence for audiovisual integration in the recognition of familiar people, we studied the behavioral and electrophysiological correlates of voice learning associated with distinctive or non-distinctive faces. We repeated twelve unfamiliar voices uttering short sentences, together with either distinctive or non-distinctive faces (depicted before and during voice presentation) in six learning-test cycles. During learning, distinctive faces increased early visually-evoked (N170, P200, N250) potentials relative to non-distinctive faces, and face distinctiveness modulated voice-elicited slow EEG activity at the occipito-temporal and fronto-central electrodes. At the test, unimodally-presented voices previously learned with distinctive faces were classified more quickly than were voices learned with non-distinctive faces, and also more quickly than novel voices. Moreover, voices previously learned with faces elicited an N250-like component that was similar in topography to that typically observed for facial stimuli. The preliminary source localization of this voice-induced N250 was compatible with a source in the fusiform gyrus. Taken together, our findings provide support for a theory of early interaction between voice and face processing areas during both learning and voice recognition.

Neural timing of the other-race effect across the lifespan: A review

Face race influences the way we process faces, so that faces of a different ethnic group are processed for identity less efficiently than faces of one's ethnic group - a phenomenon known as the Other-Race Effect (ORE). Although widely replicated, the ORE is still poorly characterized in terms of its development and the underlying mechanisms. In the last two decades, the Event-Related Potential (ERP) technique has brought insight into the mechanisms underlying the ORE and has demonstrated potential to clarify its development. Here, we review the ERP evidence for a differential neural processing of own-race and other-race faces throughout the lifespan. In infants, race-related processing differences emerged at the N290 and P400 (structural encoding) stages. In children, race affected the P100 (early processing, attention) perceptual stage and was implicitly encoded at the N400 (semantic processing) stage. In adults, processing difficulties for other-race faces emerged at the N170 (structural encoding), P200 (configuration processing) and N250 (accessing individual representations) perceptual stages. Early in processing, race was implicitly encoded from other-race faces (N100, P200 attentional biases) and in-depth processing preferentially applied to own-race faces (N200 attentional bias). Encoding appeared less efficient (Dm effects) and retrieval less recollection-based (old/new effects) for other-race faces. Evidence admits the contribution of perceptual, attentional, and motivational processes to the development and functioning of the ORE, offering no conclusive support for perceptual or socio-cognitive accounts. Cross-racial and non-cross-racial studies provided convergent evidence. Future research would need to include less represented ethnic populations and the developmental population.

The role of the eye region for familiar face recognition: Evidence from spatial low-pass filtering and contrast negation

What information is used for familiar face recognition? While previous research suggests a particular importance of the eye region, information from the rest of the face also needs to be integrated. What type of information is used in conjunction with the eyes is largely unclear. In three experiments, participants were asked to recognise so-called face chimeras, in which the eye region was not manipulated, while the rest of the face was either presented in negative contrast (contrast chimeras) or low-pass filtered (blur chimeras). We show (1) that both chimeras are recognised substantially better than fully blurred faces, (2) that the recognition advantage for blur chimeras is specific to the eye region but cannot be explained by cues available in this part of the face alone, and (3) that a combination of negative contrast and blurring outside of the eye region eliminates the chimera advantage. We conclude that full-frequency but distorted surface reflectance cues (in contrast chimeras) or coarse shape information (in blur chimeras) can be used in combination with the eye region for effective face recognition. Our findings further suggest that the face recognition system can flexibly use both types of information, depending on availability.

fMRI evidence that hyper-caricatured faces activate object-selective cortex

Many brain imaging studies have looked at the cortical responses to object categories and faces. A popular way to manipulate face stimuli is by using a "face space," a high dimensional representation of individual face images, with the average face located at the origin. However, how the brain responds to faces that deviate substantially from average has not been much explored. Increasing the distance from the average (leading to increased caricaturing) could increase neural responses in face-selective regions, an idea supported by results from non-human primates. Here, we used a face space based on principal component analysis (PCA) to generate faces ranging from average to heavily caricatured. Using functional magnetic resonance imaging (fMRI), we first independently defined face-, object- and scene-selective areas with a localiser scan and then measured responses to parametrically caricatured faces. We also included conditions in which the images of faces were inverted. Interestingly in the right fusiform face area (FFA), we found that the patterns of fMRI response were more consistent as caricaturing increased. However, we found no consistent effect of either caricature level or facial inversion on the average fMRI response in the FFA or face-selective regions more broadly. In contrast, object-selective regions showed an increase in both the consistency of response pattern and the average fMRI response with increasing caricature level. This shows that caricatured faces recruit processing from regions typically defined as object-selective, possibly through enhancing low-level properties that are characteristic of objects.

Reading the mind in cartoon eyes: Comparing human versus cartoon emotion recognition in those with high and low levels of autistic traits

People who have a high degree of autistic traits often underperform on theory of mind tasks such as perspective-taking or facial emotion recognition compared to those with lower levels of autistic traits. However, some research suggests that this may not be the case if the agent they are evaluating is anthropomorphic (i.e. animal or cartoon) rather than typically human. The present studies examined the relation between facial emotion recognition and autistic trait profiles in over 750 adults using either a standard or cartoon version of the Reading the Mind in the Eyes (RME) test. Results showed that those scoring above the clinical cut off for autistic traits on the Autism Quotient performed significantly worse than those with the lowest levels of autistic traits on the standard RME, while scores across these groups did not differ substantially on the cartoon version of the task. These findings add further evidence that theory of mind ability such as facial emotion recognition is not at a global deficit in those with a high degree of autistic traits. Instead, differences in this ability may be specific to evaluating human agents.

Caricatured facial movements enhance perception of emotional facial expressions

Although faces “in the wild” constantly undergo complicated movements, humans adeptly perceive facial identity and expression. Previous studies, focusing mainly on identity, used photographic caricature to show that distinctive form increases perceived dissimilarity. We tested whether distinctive facial movements showed similar effects, and we focussed on both perception of expression and identity. We caricatured the movements of an animated computer head, using physical motion metrics extracted from videos. We verified that these “ground truth” metrics showed the expected effects: Caricature increased physical dissimilarity between faces differing in expression and those differing in identity. Like the ground truth dissimilarity, participants’ dissimilarity perception was increased by caricature when faces differed in expression. We found these perceived dissimilarities to reflect the “representational geometry” of the ground truth. However, neither of these findings held for faces differing in identity. These findings replicated across two paradigms: pairwise ratings and multiarrangement. In a final study, motion caricature did not improve recognition memory for identity, whether manipulated at study or test. We report several forms of converging evidence for spatiotemporal caricature effects on dissimilarity perception of different expressions. However, more work needs to be done to discover what identity-specific movements can enhance face identification.

Neurocognitive effects of a training program for poor face recognizers using shape and texture caricatures: A pilot investigation

Recent research suggested disproportional usage of shape information by people with poor face recognition, although texture information appears to be more important for familiar face recognition. Here, we tested a training program with faces that were selectively caricatured in either shape or texture parameters. Forty-eight young adults with poor face recognition skills (1 SD below the mean in at least 2/3 face processing tests: CFMT, GFMT, BFFT) were pseudo-randomly assigned to either one of two training groups or a control group (n = 16 each). Training comprised six sessions over three weeks. Per session, participants studied ten unfamiliar facial identities whose shape or texture characteristics were caricatured. Before and after training (or waiting in the control group), all participants completed EEG experiments on face learning and famous face recognition, and behavioral face processing tests. Results showed small but specific training-induced improvements: Whereas shape training improved face matching (training tasks, and to some extent GFMT), texture training elicited marked improvements in face learning (CFMT). Moreover, for the texture training group the N170 ERP was enhanced for novel faces post-training, suggesting training-induced changes in early markers of face processing. Although further research is necessary, this suggests that parameter-specific caricature training is a promising way to improve performance in people with poor face recognition skills.

Similar use of shape and texture cues for own- and other-race faces during face learning and recognition

Although the other-race effect (ORE; superior recognition of own- relative to other-race faces) is well established, the mechanisms underlying it are not well understood. We examined whether the ORE is attributable to differential use of shape and texture cues for own- vs. other-race faces. Shape cues are particularly important for detecting that an own-race face is unfamiliar, whereas texture cues are more important for recognizing familiar and newly learned own-race faces. We compared the influence of shape and texture cues on Caucasian participants' recognition of Caucasian and East Asian faces using two complementary approaches. In Experiment 1, participants studied veridical, shape-caricatured, or texture-caricatured faces and then were asked to recognize them in an old/new recognition task. In Experiment 2, all study faces were veridical and we independently removed the diagnosticity of shape (or texture) cues in the test phase by replacing original shape (or texture) with average shape (or texture). Despite an overall own-race advantage, participants' use of shape and texture cues was comparable for own- and other-race faces. These results suggest that the other-race effect is not attributable to qualitative differences in the use of shape and texture cues.

The perception of caricatured emotion in voice

Affective vocalisations such as screams and laughs can convey strong emotional content without verbal information. Previous research using morphed vocalisations (e.g. 25% fear/75% anger) has revealed categorical perception of emotion in voices, showing sudden shifts at emotion category boundaries. However, it is currently unknown how further modulation of vocalisations beyond the veridical emotion (e.g. 125% fear) affects perception. Caricatured facial expressions produce emotions that are perceived as more intense and distinctive, with faster recognition relative to the original and anti-caricatured (e.g. 75% fear) emotions, but a similar effect using vocal caricatures has not been previously examined. Furthermore, caricatures can play a key role in assessing how distinctiveness is identified, in particular by evaluating accounts of emotion perception with reference to prototypes (distance from the central stimulus) and exemplars (density of the stimulus space). Stimuli consisted of four emotions (anger, disgust, fear, and pleasure) morphed at 25% intervals between a neutral expression and each emotion from 25% to 125%, and between each pair of emotions. Emotion perception was assessed using emotion intensity ratings, valence and arousal ratings, speeded categorisation and paired similarity ratings. We report two key findings: 1) across tasks, there was a strongly linear effect of caricaturing, with caricatured emotions (125%) perceived as higher in emotion intensity and arousal, and recognised faster compared to the original emotion (100%) and anti-caricatures (25%-75%); 2) our results reveal evidence for a unique contribution of a prototype-based account in emotion recognition. We show for the first time that vocal caricature effects are comparable to those found previously with facial caricatures. The set of caricatured vocalisations provided open a promising line of research for investigating vocal affect perception and emotion processing deficits in clinical populations.

Neural Correlates of Own- and Other-Face Perception in Body Dysmorphic Disorder

BACKGROUND

Body dysmorphic disorder (BDD) is characterized by an excessive preoccupation with one or more perceived flaws in one's own appearance. Previous studies provided evidence for deficits in configural and holistic processing in BDD. Preliminary evidence suggests abnormalities at an early stage of visual processing. The present study is the first examining early neurocognitive perception of the own face in BDD by using electroencephalography (EEG). We investigated the face inversion effect, in which inverted (upside-down) faces are disproportionately poorly processed compared to upright faces. This effect reflects a disruption of configural and holistic processing, and in consequence a preponderance of featural face processing.

METHODS

We recorded face-sensitive event-related potentials (ERPs) in 16 BDD patients and 16 healthy controls, all unmedicated. Participants viewed upright and inverted (upside-down) images of their own face and an unfamiliar other face, each in two facial emotional expressions (neutral vs. smiling). We calculated the early ERP components P100, N170, P200, N250, and the late positive component (LPC), and compared amplitudes among both groups.

RESULTS

In the early P100, no face inversion effects were found in both groups. In the N170, both groups exhibited the common face inversion effects, with significantly larger N170 amplitudes for inverted than upright faces. In the P200, both groups exhibited larger inversion effects to other (relative to own) faces, with larger P200 amplitudes for other upright than inverted faces. In the N250, no significant group differences were found in face processing. In the LPC, both groups exhibited larger inversion effects to other (relative to own) faces, with larger LPC amplitudes for other inverted than upright faces. These overall patterns appeared to be comparable for both groups. Smaller inversion effects to own (relative to other) faces were observed in none of these components in BDD, relative to controls.

CONCLUSIONS

The findings suggest no evidence for abnormalities at all levels of early face processing in our observed sample of BDD patients. Further research should investigate the neural substrates underlying BDD symptomatology.

Caricaturing faces to improve identity recognition in low vision simulations: How effective is current-generation automatic assignment of landmark points?

PURPOSE

Previous behavioural studies demonstrate that face caricaturing can provide an effective image enhancement method for improving poor face identity perception in low vision simulations (e.g., age-related macular degeneration, bionic eye). To translate caricaturing usefully to patients, assignment of the multiple face landmark points needed to produce the caricatures needs to be fully automatised. Recent development in computer science allows automatic face landmark detection of 68 points in real time and in multiple viewpoints. However, previous demonstrations of the behavioural effectiveness of caricaturing have used higher-precision caricatures with 147 landmark points per face, assigned by hand. Here, we test the effectiveness of the auto-assigned 68-point caricatures. We also compare this to the hand-assigned 147-point caricatures.

METHOD

We assessed human perception of how different in identity pairs of faces appear, when veridical (uncaricatured), caricatured with 68-points, and caricatured with 147-points. Across two experiments, we tested two types of low-vision images: a simulation of blur, as experienced in macular degeneration (testing two blur levels); and a simulation of the phosphenised images seen in prosthetic vision (at three resolutions).

RESULTS

The 68-point caricatures produced significant improvements in identity discrimination relative to veridical. They were approximately 50% as effective as the 147-point caricatures.

CONCLUSION

Realistic translation to patients (e.g., via real time caricaturing with the enhanced signal sent to smart glasses or visual prosthetic) is approaching feasibility. For maximum effectiveness software needs to be able to assign landmark points tracing out all details of feature and face shape, to produce high-precision caricatures.

Asymmetric neural responses for facial expressions and anti-expressions

Face recognition requires identifying both the invariant characteristics that distinguish one individual from another and the variations within the individual that correspond to emotional expressions. Both have been postulated to be represented via a norm-based code, in which identity or expression are represented as deviations from an average or neutral prototype. We used Fast Periodic Visual Stimulation (FPVS) with electroencephalography (EEG) to compare neural responses for neutral faces, expressions and anti-expressions. Anti-expressions are created by projecting an expression (e.g. a happy face) through the neutral face to form the opposite facial shape (anti-happy). Thus expressions and anti-expressions differ from the norm by the same "configural" amount and thus have equivalent but opposite status with regard to their shape, but differ in their ecological validity. We examined whether neural responses to these complementary stimulus pairs were equivalent or asymmetric, and also tested for norm-based coding by comparing whether stronger responses are elicited by expressions and anti-expressions than neutral faces. Observers viewed 20 s sequences of 6 Hz alternations of neutral faces and expressions, neutral faces and anti-expressions, and expressions and anti-expressions. Responses were analyzed in the frequency domain. Significant responses at half the frequency of the presentation rate (3 Hz), indicating asymmetries in responses, were observed for all conditions. Inversion of the images reduced the size of this signal, indicating these asymmetries are not solely due to differences in the low-level properties of the images. While our results do not preclude a norm-based code for expressions, similar to identity, this representation (as measured by the FPVS EEG responses) may also include components sensitive to which configural distortions form meaningful expressions.

Enhancement of face-sensitive ERPs in older adults induced by face recognition training

A common cognitive problem reported by older people is compromised face recognition, which is often paralleled by age-related changes in face-sensitive and memory-related components in event-related brain potentials (ERPs). We developed a new training using photorealistic caricatures based on evidence that caricatures are beneficial for people with compromised face processing. Twenty-four older participants (62-75 yrs, 13 female) completed 12 training sessions (3 per week, 60 min each) and 24 older participants (61-76 yrs, 12 female) acted as controls. Before and after training (or waiting), participants took part in a diagnostic test battery for face processing abilities, and in ERP experiments on face learning and recognition. Although performance improvements during the training provided little evidence for generalization to other face processing tasks, ERPs showed substantial training-related enhancements of face-sensitive ERPs. Specifically, we observed marked increases of the N170, P200 and N250 components, which may indicate training-induced enhancement of face detection and activation of identity-specific representations. Thus, neuronal correlates of face processing are plastic in older age, and can be modulated by caricature training.

Distinct neural processes for the perception of familiar versus unfamiliar faces along the visual hierarchy revealed by EEG

Humans recognize faces with ease, despite the complexity of the task and of the visual system which underlies it. Different spatial regions, including both the core and extended face processing networks, and distinct temporal stages of processing have been implicated in face recognition, but there is ongoing controversy regarding the extent to which the mechanisms for recognizing a familiar face differ from those for an unfamiliar face. Here, we used electroencephalogram (EEG) and flicker SSVEP, a high signal-to-noise approach, and searchlight decoding methods to elucidate the mechanisms mediating the processing of familiar and unfamiliar faces in the time domain. Familiar and unfamiliar faces were presented periodically at 15 Hz, 6 Hz and 3.75 Hz either upright or inverted in separate blocks, with the rationale that faster frequencies require shorter processing times per image and tap into fundamentally different levels of visual processing. The 15 Hz trials, likely to reflect early visual processing, exhibited enhanced neural responses for familiar over unfamiliar face trials, but only when the faces were upright. In contrast, decoding methods revealed similar classification accuracies for upright and inverted faces for both familiar and unfamiliar faces. For the 6 Hz frequency, familiar faces had lower amplitude responses than unfamiliar faces, and decoding familiarity was more accurate for upright compared with inverted faces. Finally, the 3.75 Hz frequency revealed no main effects of familiarity, but decoding showed significant correlations with behavioral ratings of face familiarity, suggesting that activity evoked by this slow presentation frequency reflected higher-level, cognitive aspects of familiarity processing. This three-way dissociation between frequencies reveals that fundamentally different stages of the visual hierarchy are modulated by face familiarity. The combination of experimental and analytical approaches used here represent a novel method for elucidating spatio-temporal characteristics within the visual system.

Source Reconstruction of Brain Potentials Using Bayesian Model Averaging to Analyze Face Intra-Domain vs. Face-Occupation Cross-Domain Processing

We investigated the neural correlates of the access to and retrieval of face structure information in contrast to those concerning the access to and retrieval of person-related verbal information, triggered by faces. We experimentally induced stimulus familiarity via a systematic learning procedure including faces with and without associated verbal information. Then, we recorded event-related potentials (ERPs) in both intra-domain (face-feature) and cross-domain (face-occupation) matching tasks while N400-like responses were elicited by incorrect eyes-eyebrows completions and occupations, respectively. A novel Bayesian source reconstruction approach plus conjunction analysis of group effects revealed that in both cases the generated N170s were of similar amplitude but had different neural origin. Thus, whereas the N170 of faces was associated predominantly to right fusiform and occipital regions (the so-called “Fusiform Face Area”, “FFA” and “Occipital Face Area”, “OFA”, respectively), the N170 of occupations was associated to a bilateral very posterior activity, suggestive of basic perceptual processes. Importantly, the right-sided perceptual P200 and the face-related N250 were evoked exclusively in the intra-domain task, with sources in OFA and extensively in the fusiform region, respectively. Regarding later latencies, the intra-domain N400 seemed to be generated in right posterior brain regions encompassing mainly OFA and, to some extent, the FFA, likely reflecting neural operations triggered by structural incongruities. In turn, the cross-domain N400 was related to more anterior left-sided fusiform and temporal inferior sources, paralleling those described previously for the classic verbal N400. These results support the existence of differentiated neural streams for face structure and person-related verbal processing triggered by faces, which can be activated differentially according to specific task demands.

Differential effects of face-realism and emotion on event-related brain potentials and their implications for the uncanny valley theory

Cartoon characters are omnipresent in popular media. While few studies have scientifically investigated their processing, in computer graphics, efforts are made to increase realism. Yet, close approximations of reality have been suggested to evoke sometimes a feeling of eeriness, the “uncanny valley” effect. Here, we used high-density electroencephalography to investigate brain responses to professionally stylized happy, angry, and neutral character faces. We employed six face-stylization levels varying from abstract to realistic and investigated the N170, early posterior negativity (EPN), and late positive potential (LPP) event-related components. The face-specific N170 showed a u-shaped modulation, with stronger reactions towards both most abstract and most realistic compared to medium-stylized faces. For abstract faces, N170 was generated more occipitally than for real faces, implying stronger reliance on structural processing. Although emotional faces elicited highest amplitudes on both N170 and EPN, on the N170 realism and expression interacted. Finally, LPP increased linearly with face realism, reflecting activity increase in visual and parietal cortex for more realistic faces. Results reveal differential effects of face stylization on distinct face processing stages and suggest a perceptual basis to the uncanny valley hypothesis. They are discussed in relation to face perception, media design, and computer graphics.

The electrophysiological correlates of integrated face and body-part perception

Previous studies have suggested that the human visual system processes faces and bodies holistically-that is, the different body parts are integrated into a unified representation. However, the time course of this integrative process is less known. In the present study, we investigated this issue by recording event-related potentials evoked by a face and two hands presented simultaneously and in different configurations. When the hands were rotated to obtain a biologically implausible configuration, a reduction of the P2 amplitude was observed relative to the condition in which the face and hands were retained in their veridical configuration and were supplemented with visual cues to highlight further the overall body posture. Our results show that the P2 component is sensitive to manipulations affecting the configuration of face and hand stimuli and suggest that the P2 reflects the operation of perceptual mechanisms responsible for the integrated processing of visually presented body parts.

Electrophysiological correlates of face-evoked person knowledge

Face recognition includes identifying a face as perceptually familiar and recollecting biographical information, or person-knowledge, associated with the face. The majority of studies examining the neural basis of face recognition have confounded these stages by comparing brain responses evoked by novel and perceptually familiar famous faces. Here, we recorded EEG in two tasks in which subjects viewed two sets of faces that were equally perceptually familiar, but which had differing levels of associated person-knowledge. Our results dissociated the effects of person-knowledge from perceptual familiarity. Faces with associated biographical information elicited a larger ∼600ms centroparietal positivity in both a passive viewing task in which subjects viewed faces without explicitly responding, and an active question-answering task in which subjects indicated whether or not they knew particular facts about the faces. In the question task only, person-knowledge was associated with a negative ERP difference over right posterior scalp over the 170-450ms interval which appeared again at long latency (>900ms).

Effects of Caricaturing in Shape or Color on Familiarity Decisions for Familiar and Unfamiliar Faces

Recent evidence suggests that while reflectance information (including color) may be more diagnostic for familiar face recognition, shape may be more diagnostic for unfamiliar face identity processing. Moreover, event-related potential (ERP) findings suggest an earlier onset for neural processing of facial shape compared to reflectance. In the current study, we aimed to explore specifically the roles of facial shape and color in a familiarity decision task using pre-experimentally familiar (famous) and unfamiliar faces that were caricatured either in shape-only, color-only, or both (full; shape + color) by 15%, 30%, or 45%. We recorded accuracies, mean reaction times, and face-sensitive ERPs. Performance data revealed that shape caricaturing facilitated identity processing for unfamiliar faces only. In the ERP data, such effects of shape caricaturing emerged earlier than those of color caricaturing. Unsurprisingly, ERP effects were accentuated for larger levels of caricaturing. Overall, our findings corroborate the importance of shape for identity processing of unfamiliar faces and demonstrate an earlier onset of neural processing for facial shape compared to color.

Effects of attractiveness on face memory separated from distinctiveness: evidence from event-related brain potentials

The present study examined effects of attractiveness on behavioral and event-related potential (ERP) correlates of face memory. Extending previous reports, we controlled for potential moderating effects of distinctiveness, a variable known to affect memory. Attractive and unattractive faces were selected on the basis of a rating study, and were matched for distinctiveness. In a subsequent recognition memory experiment, we found more accurate memory for unattractive relative to attractive faces. Additionally, an attractiveness effect in the early posterior negativity (EPN) during learning, with larger amplitudes for attractive than unattractive faces, correlated significantly with the magnitude of the memory advantage for unattractive faces at test. These findings establish a contribution of attractiveness to face memory over and above the well-known effect of distinctiveness. Additionally, as the EPN is typically enhanced for affective stimuli, our ERP results imply that the processing of emotionally relevant attractive faces during learning may hamper their encoding into memory.

Adaptor identity modulates adaptation effects in familiar face identification and their neural correlates

Adaptation-related aftereffects (AEs) show how face perception can be altered by recent perceptual experiences. Along with contrastive behavioural biases, modulations of the early event-related potentials (ERPs) were typically reported on categorical levels. Nevertheless, the role of the adaptor stimulus per se for face identity-specific AEs is not completely understood and was therefore investigated in the present study. Participants were adapted to faces (S1s) varying systematically on a morphing continuum between pairs of famous identities (identities A and B), or to Fourier phase-randomized faces, and had to match the subsequently presented ambiguous faces (S2s; 50/50% identity A/B) to one of the respective original faces. We found that S1s identical with or near to the original identities led to strong contrastive biases with more identity B responses following A adaptation and vice versa. In addition, the closer S1s were to the 50/50% S2 on the morphing continuum, the smaller the magnitude of the AE was. The relation between S1s and AE was, however, not linear. Additionally, stronger AEs were accompanied by faster reaction times. Analyses of the simultaneously recorded ERPs revealed categorical adaptation effects starting at 100 ms post-stimulus onset, that were most pronounced at around 125-240 ms for occipito-temporal sites over both hemispheres. S1-specific amplitude modulations were found at around 300-400 ms. Response-specific analyses of ERPs showed reduced voltages starting at around 125 ms when the S1 biased perception in a contrastive way as compared to when it did not. Our results suggest that face identity AEs do not only depend on physical differences between S1 and S2, but also on perceptual factors, such as the ambiguity of S1. Furthermore, short-term plasticity of face identity processing might work in parallel to object-category processing, and is reflected in the first 400 ms of the ERP.

How experience shapes memory for faces: an event-related potential study on the own-age bias

Young adults more accurately remember own-age than older faces. We tested whether this own-age bias (OAB) is reduced by increased experience. Young experts (geriatric nurses) and controls performed a recognition experiment with young and old faces. Critically, while control participants demonstrated better memory for young faces, no OAB was observed in the experts. Event-related potentials revealed larger N170 and P2 amplitudes for young than old faces in both groups, suggesting no group differences during early perceptual processing. At test, N250 repetition effects were more anteriorily distributed for own- than other-age faces in control participants, whereas experts showed no corresponding effects. A larger late positive component (LPC) for old than young faces was observed in controls, but not in experts. Larger LPCs may reflect prolonged stimulus processing compromising memory retrieval. In sum, experience with other-age faces does not affect early perceptual processing, but modulates later stages related to memory retrieval.

What drives social in-group biases in face recognition memory? ERP evidence from the own-gender bias

It is well established that memory is more accurate for own-relative to other-race faces (own-race bias), which has been suggested to result from larger perceptual expertise for own-race faces. Previous studies also demonstrated better memory for own-relative to other-gender faces, which is less likely to result from differences in perceptual expertise, and rather may be related to social in-group vs out-group categorization. We examined neural correlates of the own-gender bias using event-related potentials (ERP). In a recognition memory experiment, both female and male participants remembered faces of their respective own gender more accurately compared with other-gender faces. ERPs during learning yielded significant differences between the subsequent memory effects (subsequently remembered - subsequently forgotten) for own-gender compared with other-gender faces in the occipito-temporal P2 and the central N200, whereas neither later subsequent memory effects nor ERP old/new effects at test reflected a neural correlate of the own-gender bias. We conclude that the own-gender bias is mainly related to study phase processes, which is in line with sociocognitive accounts.

The early development of face processing--what makes faces special?

In the present article we review behavioral and neurophysiological studies on face processing in adults and in early development. From the existing empirical and theoretical literature we derive three aspects that distinguish face processing from the processing of other visual object categories. Each of these aspects is discussed from a developmental perspective. First, faces are recognized and represented at the individual level rather than at the basic level. Second, humans typically acquire extensive expertise in individuating faces from early on in development. And third, more than other objects, faces are processed holistically. There is a quantitative difference in the amount of visual experience for faces and other object categories in that the amount of expertise typically acquired for faces is greater than that for other object categories. In addition, we discuss possible qualitative differences in experience for faces and objects. For instance, there is evidence for a sensitive period in infancy for building up a holistic face representation and for perceptual narrowing for faces of one's own species and race. We conclude our literature review with questions for future research, for instance, regarding the exact relationship between behavioral and neuronal markers of face processing across development.