Event-related potentials reveal the development of stable face representations from natural variability

Familiarity enhances functional connectivity between visual and nonvisual regions of the brain during natural viewing

We explored the neural correlates of familiarity with people and places using a naturalistic viewing paradigm. Neural responses were measured using functional magnetic resonance imaging, while participants viewed a movie taken from Game of Thrones. We compared inter-subject correlations and functional connectivity in participants who were either familiar or unfamiliar with the TV series. Higher inter-subject correlations were found between familiar participants in regions, beyond the visual brain, that are typically associated with the processing of semantic, episodic, and affective information. However, familiarity also increased functional connectivity between face and scene regions in the visual brain and the nonvisual regions of the familiarity network. To determine whether these regions play an important role in face recognition, we measured responses in participants with developmental prosopagnosia (DP). Consistent with a deficit in face recognition, the effect of familiarity was significantly attenuated across the familiarity network in DP. The effect of familiarity on functional connectivity between face regions and the familiarity network was also attenuated in DP. These results show that the neural response to familiarity involves an extended network of brain regions and that functional connectivity between visual and nonvisual regions of the brain plays an important role in the recognition of people and places during natural viewing.

How neural representations of newly learnt faces change over time: Event-related brain potential evidence for overnight consolidation

Previous experiments have shown that a brief encounter with a previously unfamiliar person leads to the establishment of new facial representations, which can be activated by completely novel pictures of the newly learnt face. The present study examined how stable such novel neural representations are over time, and, specifically, how they become consolidated within the first 24 h after learning. Using event-related brain potentials (ERPs) in a between-participants design, we demonstrate that clear face familiarity effects in the occipito-temporal N250 are evident immediately after learning. These effects then undergo change, with a nearly complete absence of familiarity-related ERP differences 4 h after the initial encounter. Critically, 24 h after learning, the original familiarity effect re-emerges. These findings suggest that the neural correlates of novel face representations are not stable over time but change during the first day after learning. The resulting pattern of change is consistent with a process of consolidation.

Your place or mine? The neural dynamics of personally familiar scene recognition suggests category independent familiarity encoding

Recognizing a stimulus as familiar is an important capacity in our everyday life. Recent investigation of visual processes has led to important insights into the nature of the neural representations of familiarity for human faces. Still, little is known about how familiarity affects the neural dynamics of non-face stimulus processing. Here we report the results of an EEG study, examining the representational dynamics of personally familiar scenes. Participants viewed highly variable images of their own apartments and unfamiliar ones, as well as personally familiar and unfamiliar faces. Multivariate pattern analyses were used to examine the time course of differential processing of familiar and unfamiliar stimuli. Time-resolved classification revealed that familiarity is decodable from the EEG data similarly for scenes and faces. The temporal dynamics showed delayed onsets and peaks for scenes as compared to faces. Familiarity information, starting at 200 ms, generalized across stimulus categories and led to a robust familiarity effect. In addition, familiarity enhanced category representations in early (250-300 ms) and later (>400 ms) processing stages. Our results extend previous face familiarity results to another stimulus category and suggest that familiarity as a construct can be understood as a general, stimulus-independent processing step during recognition.

Personal familiarity of faces, animals, objects, and scenes: Distinct perceptual and overlapping conceptual representations

While face, object, and scene recognition are often studied at a basic categorization level (e.g. "a face", "a car", "a kitchen"), we frequently recognise individual items of these categories as unique entities (e.g. "my mother", "my car", "my kitchen"). This recognition of individual identity is essential to appropriate behaviour in our world. However, relatively little is known about how we recognise individually familiar visual stimuli. Using event-related brain potentials, the present study examined whether and to what extent the underlying neural representations of personally familiar items are similar or different across different categories. In three experiments, we examined the recognition of personally highly familiar faces, animals, indoor scenes, and objects. We observed relatively distinct familiarity effects in an early time window (200-400 ms), with a clearly right-lateralized occipito-temporal scalp distribution for human faces and more bilateral and posterior distributions for other stimulus categories, presumably reflecting access to at least partly discrete visual long-term representations. In contrast, we found clearly overlapping familiarity effects in a later time window (starting 400 to 500 ms after stimulus onset), again with a mainly right occipito-temporal scalp distribution, for all stimulus categories. These later effects appear to reflect the sustained activation of conceptual properties relevant to any potential interaction. We conclude that familiarity for items from the various visual stimulus categories tested here is represented differently at the perceptual level, while relatively overlapping conceptual mechanisms allow for the preparation of impending potential interaction with the environment.

Explicit face memory abilities are positively related to the non-intentional encoding of faces: Behavioral and ERP evidence

Individual differences in face memory abilities have been shown to be related to individual differences in brain activity. The present study investigated brain-behavior relationships for the N250 component in event-related brain potentials, which is taken as a neural sign of face familiarity. We used a task in which a designated, typical target face and several (high- and low-distinctive) nontarget faces had to be distinguished during multiple presentations across a session. Separately, face memory/recognition abilities were measured with easy versus difficult tasks. We replicated an increase of the N250 amplitude to the target face across the session and observed a similar increase for the non-target faces, indicating the build-up of memory representations also for these faces. On the interindividual level, larger across-session N250 amplitude increases to low-distinctive non-target faces were related to faster face recognition as measured in an easy task. These findings indicate that non-intentional encoding of non-target faces into memory is associated with the swift recognition of explicitly learned faces; that is, there is shared variance of incidental and intentional face memory.

A neural marker of the human face identity familiarity effect

Human adults associate different views of an identity much better for familiar than for unfamiliar faces. However, a robust and consistent neural index of this behavioral face identity familiarity effect (FIFE)-not found in non-human primate species-is lacking. Here we provide such a neural FIFE index, measured implicitly and with one fixation per face. Fourteen participants viewed 70 s stimulation sequences of a large set (n = 40) of widely variable natural images of a face identity at a rate of 6 images/second (6 Hz). Different face identities appeared every 5th image (1.2 Hz). In a sequence, face images were either familiar (i.e., famous) or unfamiliar, participants performing a non-periodic task unrelated to face recognition. The face identity recognition response identified at 1.2 Hz over occipital-temporal regions in the frequency-domain electroencephalogram was 3.4 times larger for familiar than unfamiliar faces. The neural response to familiar faces-which emerged at about 180 ms following face onset-was significant in each individual but a case of prosopdysgnosia. Besides potential clinical and forensic applications to implicitly measure one's knowledge of a face identity, these findings open new perspectives to clarify the neurofunctional source of the FIFE and understand the nature of human face identity recognition.

Increased N250 elicited by facial familiarity: An ERP study including the face inversion effect and facial emotion processing

The present study aims to explore how familiarity modulates the neural processing of faces under different conditions: upright or inverted, neutral or emotional. To this purpose, 32 participants (25 female; age: M = 27.7 years, SD = 9.3) performed two face/emotion identification tasks during EEG recording. In the first task, to study facial processing, three different categories of facial stimuli were presented during a target detection task: famous familiar faces, faces of loved ones, and unfamiliar faces. To explore the face inversion effect according to each level of familiarity, these facial stimuli were also presented upside down. In the second task, to study emotional face processing, an emotional identification task on personally familiar and unfamiliar faces was conducted. The behavioural results showed an improved performance in the identification of facial expressions of emotion with the increase of facial familiarity, consistent with the previous literature. Regarding electrophysiological results, we found increased amplitudes of the P100, N170, and N250 for inverted compared to upright faces, independently of their degree of familiarity. Moreover, we did not find familiarity effects at the P100 and N170 time-windows, but we found that N250 amplitude was larger for personally familiar compared to unfamiliar faces. This result supports the reasoning that the facial familiarity increases the neural activity during the N250 time-window, which may be explained by the processing of additional information prompted by the viewing of our loved ones faces, in contrast to what happens with unfamiliar individuals.

Differences between high and low performers in face recognition in electrophysiological correlates of face familiarity and distance-to-norm

Valentine's influential norm-based multidimensional face-space model (nMDFS) predicts that perceived distinctiveness of a face increases with its distance to the norm. Occipito-temporal event-related potentials (ERPs) have been recently shown to respond selectively to variations in distance-to-norm (P200) or familiarity (N250, late negativity), respectively (Wuttke & Schweinberger, 2019). Despite growing evidence on interindividual differences in face perception skills at the behavioral level, little research has focused on their electrophysiological correlates. To reveal potential interindividual differences in face spaces, we contrasted high and low performers in face recognition in regards to distance-to-norm (P200) and familiarity (N250). We replicated both the P200 distance-to-norm and the N250 familiarity effect. Importantly, we observed: i) reduced responses in low compared to high performers of face recognition, especially in terms of smaller distance-to-norm effects in the P200, possibly indicating less 'expanded' face spaces in low compared to high performers; ii) increased N250 responses to familiar original faces in high performers, suggesting more robust face identity representations. In summary, these findings suggest the contribution of both early norm-based face coding and robust face representations to individual face recognition skills, and indicate that ERPs can offer a promising route to understand individual differences in face perception and their neurocognitive correlates.

Developing familiarity during the first eight months of knowing a person: A longitudinal EEG study on face and identity learning

It is well-established that familiar and unfamiliar faces are processed differently, but surprisingly little is known about how familiarity builds up over time and how novel faces gradually become represented in the brain. Here, we used event-related brain potentials (ERPs) in a pre-registered, longitudinal study to examine the neural processes accompanying face and identity learning during the first eight months of knowing a person. Specifically, we examined how increasing real-life familiarity affects visual recognition (N250 Familiarity Effect) and the integration of person-related knowledge (Sustained Familiarity Effect, SFE). Sixteen first-year undergraduates were tested in three sessions, approximately one, five, and eight months after the start of the academic year, with highly variable "ambient" images of a new friend they had met at university and of an unfamiliar person. We observed clear ERP familiarity effects for the new friend after one month of familiarity. While there was an increase in the N250 effect over the course of the study, no change in the SFE was observed. These results suggest that visual face representations develop faster relative to the integration of identity-specific knowledge.

Neural timing of the other-race effect across the lifespan: A review

Face race influences the way we process faces, so that faces of a different ethnic group are processed for identity less efficiently than faces of one's ethnic group - a phenomenon known as the Other-Race Effect (ORE). Although widely replicated, the ORE is still poorly characterized in terms of its development and the underlying mechanisms. In the last two decades, the Event-Related Potential (ERP) technique has brought insight into the mechanisms underlying the ORE and has demonstrated potential to clarify its development. Here, we review the ERP evidence for a differential neural processing of own-race and other-race faces throughout the lifespan. In infants, race-related processing differences emerged at the N290 and P400 (structural encoding) stages. In children, race affected the P100 (early processing, attention) perceptual stage and was implicitly encoded at the N400 (semantic processing) stage. In adults, processing difficulties for other-race faces emerged at the N170 (structural encoding), P200 (configuration processing) and N250 (accessing individual representations) perceptual stages. Early in processing, race was implicitly encoded from other-race faces (N100, P200 attentional biases) and in-depth processing preferentially applied to own-race faces (N200 attentional bias). Encoding appeared less efficient (Dm effects) and retrieval less recollection-based (old/new effects) for other-race faces. Evidence admits the contribution of perceptual, attentional, and motivational processes to the development and functioning of the ORE, offering no conclusive support for perceptual or socio-cognitive accounts. Cross-racial and non-cross-racial studies provided convergent evidence. Future research would need to include less represented ethnic populations and the developmental population.

Unfamiliar face matching ability predicts the slope of face learning

We provide the first examination of individual differences in the efficiency of face learning. Investigating individual differences in face learning can illuminate potential mechanisms and provide greater understanding of why certain individuals might be more efficient face learners. Participants completed two unfamiliar face matching tasks and a learning task in which learning was assessed after viewing 1, 3, 6, and 9 images of to-be-learned identities. Individual differences in the slope of face learning (i.e., increases in sensitivity to identity) were predicted by the ability to discriminate between matched (same-identity) vs. mismatched (different-identity) pairs of wholly unfamiliar faces. A Dual Process Signal Detection model showed that three parameters increased with learning: Familiarity (an unconscious type of memory that varies in strength), recollection-old (conscious recognition of a learned identity), and recollection-new (conscious/confident rejection of novel identities). Good (vs. poor) matchers had higher Recollection-Old scores throughout learning and showed a steeper increase in Recollection-New. We conclude that good matchers are better able to capitalize on exposure to within-person variability in appearance, an effect that is attributable to their conscious memory for both learned and novel faces. These results have applied implications and will inform contemporary and traditional models of face identification.

Stable individual differences in unfamiliar face identification: Evidence from simultaneous and sequential matching tasks

Matching identity in images of unfamiliar faces is difficult: Images of the same person can look different and images of different people can look similar. Recent studies have capitalized on individual differences in the ability to distinguish match (same ID) vs. mismatch (different IDs) face pairs to inform models of face recognition. We addressed two significant gaps in the literature by examining the stability of individual differences in both sensitivity to identity and response bias. In Study 1, 210 participants completed a battery of four tasks in each of two sessions separated by one week. Tasks varied in protocol (same/different, lineup, sorting) and stimulus characteristics (low vs. high within-person variability in appearance). In Study 2, 148 participants completed a battery of three tasks in a single session. Stimuli were presented simultaneously on some trials and sequentially on others, introducing short-term memory demands. Principal components analysis revealed two components that were stable across time and tasks: sensitivity to identity and bias. Analyses of response times suggest that individual differences in bias reflect decision-making processes. We discuss the implications of our findings in applied settings and for models of face recognition.

Face recognition ability is manifest in early dynamic decoding of face-orientation selectivity-Evidence from multi-variate pattern analysis of the neural response

Although humans are considered to be face experts, there is a well-established reliable variation in the degree to which neurotypical individuals are able to learn and recognise faces. While many behavioural studies have characterised these differences, studies that seek to relate the neuronal response to standardised behavioural measures of ability remain relatively scarce, particularly so for the time-resolved approaches and the early response to face stimuli. In the present study we make use of a relatively recent methodological advance, multi-variate pattern analysis (MVPA), to decode the time course of the neural response to faces compared to other object categories (inverted faces, objects). Importantly, for the first time, we directly relate metrics of this decoding assessed at the individual level to gold-standard measures of behavioural face processing ability assessed in an independent task. Thirty-nine participants completed the behavioural Cambridge Face Memory Test (CFMT), then viewed images of faces and houses (presented upright and inverted) while their neural activity was measured via electroencephalography. Significant decoding of both face orientation and face category were observed in all individual participants. Decoding of face orientation, a marker of more advanced face processing, was earlier and stronger in participants with higher levels of face expertise, while decoding of face category information was earlier but not stronger for individuals with greater face expertise. Taken together these results provide a marker of significant differences in the early neuronal response to faces from around 100 ms post stimulus as a function of behavioural expertise with faces.

How quickly do we learn new faces in everyday life? Neurophysiological evidence for face identity learning after a brief real-life encounter

Faces learnt in a single experimental session elicit a familiarity effect in event-related brain potentials (ERPs), with more negative amplitudes for newly learnt relative to unfamiliar faces in the N250 component. However, no ERP study has examined face learning following a brief real-life encounter, and it is not clear how long it takes to learn new faces in such ecologically more valid conditions. To investigate these questions, the present study examined whether robust image-independent representations, as reflected in the N250 familiarity effect, could be established after a brief unconstrained social interaction by analysing the ERPs elicited by highly variable images of the newly learnt identity and an unfamiliar person. Significant N250 familiarity effects were observed after a 30-min (Experiment 1) and a 10-min (Experiment 2) encounter, and a trend was observed after 5 min of learning (Experiment 3), demonstrating that 5-10 min of exposure were sufficient for the initial establishment of image-independent representations. Additionally, the magnitude of the effects reported after 10 and 30 min was comparable suggesting that the first 10 min of a social encounter might be crucial, with extra 20 min from the same encounter not adding further benefit for the initial formation of robust face representations.

From Pictures to the People in Them: Averaging Within-Person Variability Leads to Face Familiarization

Familiar faces can be confidently recognized despite sometimes radical changes in their appearance. Exposure to within-person variability-differences in facial characteristics over successive encounters-contributes to face familiarization. Research also suggests that viewers create mental averages of the different views of faces they encounter while learning them. Averaging over within-person variability is thus a promising mechanism for face familiarization. In Experiment 1, 153 Canadian undergraduates (88 female; age: M = 21 years, SD = 5.24) learned six target identities from eight different photos of each target interspersed among 32 distractor identities. Face-matching accuracy improved similarly irrespective of awareness of the target's identity, confirming that target faces presented among distractors can be learned incidentally. In Experiment 2, 170 Canadian undergraduates (125 female; age: M = 22.6 years, SD = 6.02) were tested using a novel indirect measure of learning. The results show that viewers update a mental average of a person's face as it becomes learned. Our findings are the first to show how averaging within-person variability over time leads to face familiarization.

Atypical prosopagnosia following right hemispheric stroke: A 23-year follow-up study with M.T

Most findings on prosopagnosia to date suggest preserved voice recognition in prosopagnosia (except in cases with bilateral lesions). Here we report a follow-up examination on M.T., suffering from acquired prosopagnosia following a large unilateral right-hemispheric lesion in frontal, parietal, and anterior temporal areas excluding core ventral occipitotemporal face areas. Twenty-three years after initial testing we reassessed face and object recognition skills [Henke, K., Schweinberger, S. R., Grigo, A., Klos, T., & Sommer, W. (1998). Specificity of face recognition: Recognition of exemplars of non-face objects in prosopagnosia. Cortex, 34(2), 289-296]; [Schweinberger, S. R., Klos, T., & Sommer, W. (1995). Covert face recognition in prosopagnosia - A dissociable function? Cortex, 31(3), 517-529] and additionally studied voice recognition. Confirming the persistence of deficits, M.T. exhibited substantial impairments in famous face recognition and memory for learned faces, but preserved face matching and object recognition skills. Critically, he showed substantially impaired voice recognition skills. These findings are congruent with the ideas that (i) prosopagnosia after right anterior temporal lesions can persist over long periods > 20 years, and that (ii) such lesions can be associated with both facial and vocal deficits in person recognition.

Neurocognitive effects of a training program for poor face recognizers using shape and texture caricatures: A pilot investigation

Recent research suggested disproportional usage of shape information by people with poor face recognition, although texture information appears to be more important for familiar face recognition. Here, we tested a training program with faces that were selectively caricatured in either shape or texture parameters. Forty-eight young adults with poor face recognition skills (1 SD below the mean in at least 2/3 face processing tests: CFMT, GFMT, BFFT) were pseudo-randomly assigned to either one of two training groups or a control group (n = 16 each). Training comprised six sessions over three weeks. Per session, participants studied ten unfamiliar facial identities whose shape or texture characteristics were caricatured. Before and after training (or waiting in the control group), all participants completed EEG experiments on face learning and famous face recognition, and behavioral face processing tests. Results showed small but specific training-induced improvements: Whereas shape training improved face matching (training tasks, and to some extent GFMT), texture training elicited marked improvements in face learning (CFMT). Moreover, for the texture training group the N170 ERP was enhanced for novel faces post-training, suggesting training-induced changes in early markers of face processing. Although further research is necessary, this suggests that parameter-specific caricature training is a promising way to improve performance in people with poor face recognition skills.

Detecting a viewer's familiarity with a face: Evidence from event-related brain potentials and classifier analyses

Human observers recognize the faces of people they know efficiently and without apparent effort. Consequently, recognizing a familiar face is often assumed to be an automatic process beyond voluntary control. However, there are circumstances in which a person might seek to hide their recognition of a particular face. The present study therefore used event-related potentials (ERPs) and a classifier based on logistic regression to determine if it is possible to detect whether a viewer is familiar with a particular face, regardless of whether the participant is willing to acknowledge it or not. In three experiments, participants were presented with highly variable "ambient" images of personally familiar and unfamiliar faces, while performing an incidental butterfly detection task (Experiment 1), an explicit familiarity judgment task (Experiment 2), and a concealed familiarity task in which they were asked to deny familiarity with one truly known facial identity while acknowledging familiarity with a second known identity (Experiment 3). In all three experiments, we observed substantially more negative ERP amplitudes at occipito-temporal electrodes for familiar relative to unfamiliar faces starting approximately 200 ms after stimulus onset. Both the earlier N250 familiarity effect, reflecting visual recognition of a known face, and the later sustained familiarity effect, reflecting the integration of visual with additional identity-specific information, were similar across experiments and thus independent of task demands. These results were further supported by the classifier analysis. We conclude that ERP correlates of familiar face recognition are largely independent of voluntary control and discuss potential applications in forensic settings.

The N250 event-related potential as an index of face familiarity: a replication study

The neural correlates of face individuation-the acquisition of memory representations for novel faces-have been studied only in coarse detail and disregarding individual differences between learners. In their seminal study, Tanaka et al. (Tanaka et al. 2006 J. Cogn. Neurosci. 18, 1488-1497. (doi:10.1162/jocn.2006.18.9.1488)) required the identification of a particular novel face across 70 trials and found that the N250 component in the EEG event-related potentials became more negative from the first to the second half of the experiment, where it reached a similar amplitude as a well-known face. We were unable to directly replicate this finding in our study when we used the original split of trials. However, when we applied a different split of trials we observed very similar changes in N250 amplitude. We conclude that the N250 component is indeed sensitive to the build-up of a robust representation of a face in memory; the time course of this process appears to vary as a function of variables that may be determined in future research.

Mapping the Featural and Holistic Face Processing of Bad and Good Face Recognizers

Individual abilities in face recognition (good versus bad recognizers) were explored by means of event-related potentials (ERPs). The adaptation response profile of the N170 component to whole faces, eyes and mouths was used in order to highlight the crucial role of individual abilities in identity repetition processes for unfamiliar faces. The main point of this study is to underline the importance of characterizing the performance (bad or good) of the participants and to show that behaviorally selected groups might reveal neural differences. Good recognizers showed selective right hemisphere N170 repetition effects for whole faces and not for features. On the contrary, bad recognizers showed a general repetition effect not specifically related to faces and more pronounced processing for features. These findings suggest a different contribution of holistic and featural analysis in bad and good performers. In conclusion, we propose that the N170 might be used as a tool to tease apart face encoding processes as a function of individual differences.

Inhibition of the occipital face area modulates the electrophysiological signals of face familiarity: A combined cTBS-EEG study

The occipital face area (OFA) is hierarchically one of the first stages of the face processing network. It has originally been thought to be involved in early, structural processing steps, but currently more and more studies challenge this view and propose that it also takes part in higher level face processing, such as identification and recognition. Here we tested whether the OFA is involved in the initial steps of recognition memory and plays a causal role in the differential processing of familiar and unfamiliar faces. We used an offline, inhibitory continuous theta-burst stimulation (cTBS) protocol over the right OFA and the vertex as control site. Electroencephalographic (EEG) recording of event-related potentials (ERPs), elicited by visually presented familiar (famous) and unfamiliar faces was performed before and after stimulation. We observed a difference in ERPs for famous and unfamiliar faces in a time-window corresponding to the N250 component. Importantly, this difference was significantly increased by cTBS of the right OFA, suggesting its causal role in the differential processing of familiar and unfamiliar faces. The enhancement occurred focally, at electrodes close to the right hemispheric cTBS site, as well as over similar occipito-temporal sites of the contralateral hemisphere. To the best of our knowledge, this is the first study showing the causal role of the rOFA in the differential processing of familiar and unfamiliar faces, using combined cTBS and EEG recording methods. These results are discussed with respect to the nature of familiar face representations, supported by an extensive, bilateral network.

The N170 is Sensitive to Long-term (Personal) Familiarity of a Face Identity

The N170 is a large deflection of the human electroencephalogram (EEG), peaking at about 170 milliseconds over the occipito-temporal cortex after the sudden onset of a face stimulus. The N170 reflects perceptual awareness of a face and its onset corresponds to the emergence of reliable face-selectivity in the human brain. However, whether sensitivity to the long-term familiarity of a face identity emerges already at this early time-point remains debated. Here we provide a brief survey of the 45 published studies comparing the N170 response to unfamiliar and familiar (famous, experimentally familiarized, personally familiar and own) faces. Even though effects of familiarity on the N170 are relatively small and inconsistent across studies, this overview indicates that face familiarity significantly increases the N170 amplitude. This effect is especially present for personally familiar faces, learned in natural conditions. In the human brain, effects linked to familiarity with specific facial identities therefore appear to emerge between 150 and 200 ms in occipito-temporal brain regions, i.e., shortly after the onset of face-selectivity but at the same time as the earliest high-level effects of immediate unfamiliar face identity repetition. This observation challenges standard neurocognitive models with a clear-cut distinction between perceptual and memory stages in human face recognition.

Subliminal affective priming effect: Dissociated processes for intense versus normal facial expressions

Positive vs. negative intense-facial expressions are difficult to explicitly distinguish; yet, whether they dissociate when subliminally presented remains unclear. Through three experiments using affective priming paradigms, we assessed how intense facial expressions, when presented briefly (17 ms) and masked, influenced following neutral ambiguous words (Experiment 1) or visible facial expressions (Experiments 2&3). We also compared these results with those of using normal facial expressions as primes in each experiment. All experiments indicated masked affective priming effects (biasing valence judgement of neutral words or facilitating reaction time to faces with the same valence as the prime) in normal facial expression, but not those intense ones. Experiment 3 using event related potentials (ERPs) further revealed that two ERP components N250 and LPP were consistent with behavioral changes in the normal condition (larger when valences of primes and targets were different), but inconsistent in the intense condition. Taken together, our results provided behavioral and neural evidence for distinctive processing between normal and intense facial expressions under masked condition.

Understanding face identification through within-person variability in appearance: Introduction to a virtual special issue

In the effort to determine the cognitive processes underlying the identification of faces, the dissimilarities between images of different people have long been studied. In contrast, the inherent variability between different images of the same face has either been treated as a nuisance variable that should be eliminated from psychological experiments or it has not been considered at all. Over the past decade, research efforts have increased substantially to demonstrate that this within-person variation is meaningful and can give insight into various processes of face identification, such as identity matching, face learning, and familiar face recognition. In this virtual special issue of the Quarterly Journal of Experimental Psychology, we explain the importance of within-person variability for face identification and bring together recent relevant articles published in the journal.

Unitization of internal and external features contributes to associative recognition for faces: Evidence from modulations of the FN400

Associative recognition requires discriminating between old items and conjunction lures constructed by recombining elements from two different study items. This task can be solved not only by recollection but also by familiarity if the to-be-remembered stimuli are perceived as a unitized representation. In two event-related potential (ERP) studies, we provide evidence for the integration of internal and external facial features by showing that the early frontal old-new effect (considered a correlate of familiarity) is modulated by the specific combination of facial features. Participants studied faces consisting of internal features (eyes, eyebrows, nose, and mouth) paired with external features (hair, head shape, and ears). During the testing phase, intact, recombined, and new faces were presented. Recombined faces consisted of internal and external features taken from two different studied faces. The results showed that at the frontal sites, during the time window from 300 to 500 ms, ERPs to intact faces were more positive than those to new and recombined faces; the latter two did not differ from one another. The late parietal effect was observed only after a more extended study phase in Experiment 2. We take the modulation of the early frontal old-new effect as evidence for the contribution of familiarity to associative recognition for combinations of internal and external facial features.

Characterizing the neural signature of face processing in Williams syndrome via multivariate pattern analysis and event related potentials

Face recognition ability is often reported to be a relative strength in Williams syndrome (WS). Yet methodological issues associated with the supporting research, and evidence that atypical face processing mechanisms may drive outcomes 'in the typical range', challenge these simplistic characterisations of this important social ability. Detailed investigations of face processing abilities in WS both at a behavioural and neural level provide critical insights. Here, we behaviourally characterised face recognition ability in 18 individuals with WS comparatively to typically developing children and adult control groups. A subset of 11 participants with WS as well as chronologically age matched typical adults further took part in an EEG task where they were asked to attentively view a series of upright and inverted faces and houses. State-of-the-art multivariate pattern analysis (MVPA) was used alongside standard ERP analysis to obtain a detailed characterisation of the neural profile associated with 1) viewing faces as an overall category (by examining neural activity associated with upright faces and houses), and to 2) the canonical upright configuration of a face, critically associated with expertise in typical development and often linked with holistic processing (upright and inverted faces). Our results show that while face recognition ability is not on average at a chronological age-appropriate level in individuals with WS, it nonetheless appears to be a relative strength within their cognitive profile. Furthermore, all participants with WS revealed a differential pattern of neural activity to faces compared to objects, showing a distinct response to faces as a category, as well as a differential neural pattern for upright vs. inverted faces. Nonetheless, an atypical profile of face orientation classification was found in WS, suggesting that this group differs from typical individuals in their face processing mechanisms. Through this innovative application of MVPA, alongside the high temporal resolution of EEG, we provide important new insights into the neural processing of faces in WS.

Does culture shape face perception in autism? Cross-cultural evidence of the own-race advantage from the UK and Japan

Autism spectrum disorders (ASD) are associated with face perception atypicalities, and atypical experience with faces has been proposed as an underlying explanation. Studying the own-race advantage (ORA) for face recognition can reveal the effect of experience on face perception in ASD, although the small number of studies in the area present mixed findings. This study probed the ORA in ASD by comparing two cultural groups simultaneously for the first time. Children with ASD in the UK (N = 16) and Japan (N = 26) were compared with age- and ability-matched typically developing (TD) children in the UK (N = 16) and Japan (N = 26). Participants completed a two-alternative forced-choice task, whereby they had to recognize a just seen face from a foil which was manipulated in one of four ways (IC: identity change; EE: easy eyes; HE: hard eyes; HM: hard mouth). Face stimuli were Asian and Caucasian, and thus the same stimuli were own and other race depending on the cultural group. The ASD groups in the UK and Japan did not show impaired face recognition abilities, or impairments with recognizing faces depending on manipulations to the eye region, and importantly they showed an ORA. There was considerable heterogeneity in the presence of the ORA in ASD and TD and also across cultures. Children in Japan had higher accuracy than children in the UK, and TD children in Japan did not show an ORA. This cross-cultural study challenges the view that atypical experiences with faces lead to a reduced/absent ORA in ASD.

Learning own- and other-race facial identities: Testing implicit recognition with event-related brain potentials

Exposure to varying images of the same person can encourage the formation of a representation that is sufficiently robust to allow recognition of previously unseen images of this person. While behavioural work suggests that face identity learning is harder for other-race faces, the present experiment investigated the neural correlates underlying own- and other-race face learning. Participants sorted own- and other-race identities into separate identity clusters and were further familiarised with these identities in a matching task. Subsequently, we compared event-related brain potentials (ERPs) in an implicit recognition (butterfly detection) task for learnt and previously unseen identities. We observed better sorting and matching for own- than other-race identities, and behavioural learning effects were restricted to own-race identities. Similarly, the N170 ERP component showed clear learning effects for own-race faces only. The N250, a component more closely associated with face learning was more negative for learnt than novel identities. ERP findings thus suggests a processing advantage for own-race identities at an early perceptual level whereas later correlates of identity learning were unaffected by ethnicity. These results suggest learning advantages for own-race identities, which underscores the importance of perceptual expertise in the own-race bias.

Two Sides of Face Learning: Improving Between-Identity Discrimination While Tolerating More Within-Person Variability in Appearance

Two photos of an unfamiliar face are often perceived as belonging to different people—an error that disappears when a face is familiar. Face learning has been characterized as increased tolerance of within-person variability in appearance and is facilitated by exposure to such variability (e.g., differences in expression, lighting, and aesthetics). We hypothesized that increased tolerance of variability in appearance might lead to reduced discrimination and that misidentifications would be reduced if a face was learned in the context of a similar-looking identity. After validating our stimuli (Experiments 1a and 1b), we conducted three experiments investigating face learning. In two of these, participants learned three faces (Experiment 2: 15 images/identity and Experiment 3: 5 images/identity), two of which were similar. In a recognition task, misidentifications did not change as a function of similarity, although participants recognized more images of the target in Experiment 2 (i.e., after learning 15 images). In Experiment 4, participants learned one identity and the number of images studied varied across groups. Recognition of new images increased with the number of images studied, with no changes in false alarms; sensitivity (A′) marginally increased. The results suggest that recognition and discrimination reflect separable processes with minimal influence of between-person similarity on discrimination.

Learning own- and other-race facial identities from natural variability

Exposure to multiple varying face images of the same person encourages the formation of identity representations which are sufficiently robust to allow subsequent recognition from new, never-before seen images. While recent studies suggest that identity information is initially harder to perceive in images of other- relative to own-race identities, it remains unclear whether these difficulties propagate to face learning, that is, to the formation of robust face representations. We report two experiments in which Caucasian and East Asian participants sorted multiple images of own- and other-race persons according to identity in an implicit learning task and subsequently either matched novel images of learnt and previously unseen faces for identity (Experiment 1) or made old/new decisions for new images of learnt and unfamiliar identities (Experiment 2). Caucasian participants demonstrated own-race advantages during sorting, matching, and old/new recognition, while corresponding effects were absent in East Asian participants with substantial other-race expertise. Surprisingly, East Asian participants showed enhanced learning for other-race identities during matching in Experiment 1, which may reflect their increased motivation to individuate other-race faces. Thus, our results highlight the importance of perceptual expertise for own- and other-race processing, but may also lend support to recent suggestions on how expertise and socio-cognitive factors can interact.

An objective, sensitive and ecologically valid neural measure of rapid human individual face recognition

Humans may be the only species able to rapidly and automatically recognize a familiar face identity in a crowd of unfamiliar faces, an important social skill. Here, by combining electroencephalography (EEG) and fast periodic visual stimulation (FPVS), we introduce an ecologically valid, objective and sensitive neural measure of this human individual face recognition function. Natural images of various unfamiliar faces are presented at a fast rate of 6 Hz, allowing one fixation per face, with variable natural images of a highly familiar face identity, a celebrity, appearing every seven images (0.86 Hz). Following a few minutes of stimulation, a high signal-to-noise ratio neural response reflecting the generalized discrimination of the familiar face identity from unfamiliar faces is observed over the occipito-temporal cortex at 0.86 Hz and harmonics. When face images are presented upside-down, the individual familiar face recognition response is negligible, being reduced by a factor of 5 over occipito-temporal regions. Differences in the magnitude of the individual face recognition response across different familiar face identities suggest that factors such as exposure, within-person variability and distinctiveness mediate this response. Our findings of a biological marker for fast and automatic recognition of individual familiar faces with ecological stimuli open an avenue for understanding this function, its development and neural basis in neurotypical individual brains along with its pathology. This should also have implications for the use of facial recognition measures in forensic science.

Familiarity revealed by involuntary eye movements on the fringe of awareness

Involuntary eye movements during fixation of gaze are typically transiently inhibited following stimulus onset. This oculomotor inhibition (OMI), which includes microsaccades and spontaneous eye blinks, is modulated by stimulus saliency and anticipation, but it is currently unknown whether it is sensitive to familiarity. To investigate this, we measured the OMI while observers passively viewed a slideshow of one familiar and 7 unfamiliar facial images presented briefly at 1 Hz in random order. Since the initial experiments indicated that OMI was occasionally insensitive to familiarity when the facial images were highly visible, and to prevent top-down strategies and potential biases, we limited visibility by backward masking making the faces barely visible or at the fringe of awareness. Under these conditions, we found prolonged inhibition of both microsaccades and eye-blinks, as well as earlier onset of microsaccade inhibition with familiarity. These findings demonstrate, for the first time, the sensitivity of OMI to familiarity. Because this is based on involuntary eye movements and can be measured on the fringe of awareness and in passive viewing, our results provide direct evidence that OMI can be used as a novel physiological measure for studying hidden memories with potential implications for health, legal, and security purposes.

A Robust Neural Index of High Face Familiarity

Humans are remarkably accurate at recognizing familiar faces, whereas their ability to recognize, or even match, unfamiliar faces is much poorer. However, previous research has failed to identify neural correlates of this striking behavioral difference. Here, we found a clear difference in brain potentials elicited by highly familiar faces versus unfamiliar faces. This effect starts 200 ms after stimulus onset and reaches its maximum at 400 to 600 ms. This sustained-familiarity effect was substantially larger than previous candidates for a neural familiarity marker and was detected in almost all participants, representing a reliable index of high familiarity. Whereas its scalp distribution was consistent with a generator in the ventral visual pathway, its modulation by repetition and degree of familiarity suggests an integration of affective and visual information.

Same-gender distractors are not so easy to reject: ERP evidence of gender categorization

Social categorization appears to be an automatic process that occurs during person perception. Understanding social categorization better is important because mere categorization can lead to stereotype activation and, in turn, to discrimination. In the present study we used a novel approach to examine event-related potentials (ERPs) of gender categorization in the “Who said what?” memory paradigm, thus allowing for a more in-depth understanding of the specific mechanisms underlying identity versus categorization processing. After observing video clips showing a “discussion” among female and male targets, participants were shown individual statements, each accompanied by one of the discussants’ faces. While we measured ERPs, participants had to decide whether or not a given statement had previously been made by the person with the accompanying face. In same-person trials, statements were paired with the correct person, whereas in the distractor trials, either a same-gender or a different-gender distractor was shown. As expected, participants were able to reject different-gender distractors faster than same-gender distractors, and they were more likely to falsely choose yes for a same-gender than for a different-gender distractor. Both findings indicate gender-based categorization. ERPs, analyzed in a 300- to 400-ms time window at occipito-temporal channels, indicated more negative amplitudes for yes responses both for the same person and for same-gender distractors, relative to different-gender distractors. Overall, these results show gender-based categorization even when the task was to assess the identifying information in a gender-neutral context. These findings are interpreted as showing that gender categorization occurs automatically during person perception, but later than race- or age-based categorization.

Familiarity and Within-Person Facial Variability: The Importance of the Internal and External Features

As faces become familiar, we come to rely more on their internal features for recognition and matching tasks. Here, we assess whether this same pattern is also observed for a card sorting task. Participants sorted photos showing either the full face, only the internal features, or only the external features into multiple piles, one pile per identity. In Experiments 1 and 2, we showed the standard advantage for familiar faces-sorting was more accurate and showed very few errors in comparison with unfamiliar faces. However, for both familiar and unfamiliar faces, sorting was less accurate for external features and equivalent for internal and full faces. In Experiment 3, we asked whether external features can ever be used to make an accurate sort. Using familiar faces and instructions on the number of identities present, we nevertheless found worse performance for the external in comparison with the internal features, suggesting that less identity information was available in the former. Taken together, we show that full faces and internal features are similarly informative with regard to identity. In comparison, external features contain less identity information and produce worse card sorting performance. This research extends current thinking on the shift in focus, both in attention and importance, toward the internal features and away from the external features as familiarity with a face increases.