Towards an exemplar model of face processing: the effects of race and distinctiveness

Prosopagnosia and the role of face-sensitive areas in race perception

Race is rapidly and effortlessly extracted from faces. Previous fMRI studies have reported race-related modulations in the bilateral Fusiform Face Areas (FFAs) and Occipital Face Areas (OFAs) during the categorization of faces by race. However, our recent findings revealed a comparable Other-Race Categorization Advantage between a well-studied case of pure acquired prosopagnosia-patient PS-and healthy controls. Notably, PS demonstrated faster categorization by race of other-compared to same-race faces, similar to healthy participants, despite sustaining lesions in the right OFA (rOFA) and left FFA (lFFA). This observation suggests that race processing can occur effectively even with damage to core face-sensitive regions, challenging the functional significance of race-related activations in the rOFA and lFFA observed in healthy individuals with fMRI. To address this apparent contradiction, we tested PS and age-matched controls during the categorization by race of same- to other-race morphed faces. Our data showed that PS required more visual information to accurately categorize racially ambiguous faces, indicating that intact rOFA and/or lFFA are crucial for extracting fine-grained racial information. These results refine our understanding of the functional roles of these key cortical regions and offer novel insights into the neural mechanisms underlying the perception of face race and prosopagnosia.

The role of expansion and adaptability of face-space for individual differences in face identity processing

Individuals can strongly vary in their ability to process face identity. Understanding the mechanisms driving these differences is important for theoretical development, and in clinical and applied contexts. Here we investigate the role of face-space properties in relation to individual face identity processing skills. We consider two fundamental properties of face-space: expansion (how distant from each other similar faces are located in such space) and adaptability (the degree to which these distances change over time). Fifty-two participants performed a face detection task, with faces systematically varying in their location in face-space, and a comprehensive face identity processing test battery. We replicate previous results indicating a detection advantage for typical, as compared with distinctive faces. Critically, we find that neither our measure of face-space expansion nor that of face-space adaptability are related to individual face processing abilities. While future studies might benefit from the use of more sensitive measures of face-space properties, these results suggest that the two examined here do not contribute to individual differences in face processing abilities as previous studies suggest.

The neural dynamics of familiar face recognition

Humans are highly efficient at recognising familiar faces. However, previous EEG/ERP research has given a partial and fragmented account of the neural basis of this remarkable ability. We argue that this is related to insufficient consideration of fundamental characteristics of familiar face recognition. These include image-independence (recognition across different pictures), levels of familiarity (familiar faces vary hugely in duration and intensity of our exposure to them), automaticity (we cannot voluntarily withhold from recognising a familiar face), and domain-selectivity (the degree to which face familiarity effects are selective). We review recent EEG/ERP work, combining uni- and multivariate methods, that has systematically targeted these shortcomings. We present a theoretical account of familiar face recognition, dividing it into early visual, domain-sensitive and domain-general phases, and integrating image-independence and levels of familiarity. Our account incorporates classic and more recent concepts, such as multi-dimensional face representation and course-to-fine processing. While several questions remain to be addressed, this new account represents a major step forward in our understanding of the neurophysiological basis of familiar face recognition.

Semantic Content in Face Representation: Essential for Proficient Recognition of Unfamiliar Faces by Good Recognizers

Face recognition is adapted to achieve goals of social interactions, which rely on further processing of the semantic information of faces, beyond visual computations. Here, we explored the semantic content of face representation apart from visual component, and tested their relations to face recognition performance. Specifically, we propose that enhanced visual or semantic coding could underlie the advantage of familiar over unfamiliar faces recognition, as well as the superior recognition of skilled face recognizers. We asked participants to freely describe familiar/unfamiliar faces using words or phrases, and converted these descriptions into semantic vectors. Face semantics were transformed into quantifiable face vectors by aggregating these word/phrase vectors. We also extracted visual features from a deep convolutional neural network and obtained the visual representation of familiar/unfamiliar faces. Semantic and visual representations were used to predict perceptual representation generated from a behavior rating task separately in different groups (bad/good face recognizers in familiar-face/unfamiliar-face conditions). Comparisons revealed that although long-term memory facilitated visual feature extraction for familiar faces compared to unfamiliar faces, good recognizers compensated for this disparity by incorporating more semantic information for unfamiliar faces, a strategy not observed in bad recognizers. This study highlights the significance of semantics in recognizing unfamiliar faces.

EEG evidence for racial ingroup bias in collective empathy for pain

Previous research on racial ingroup bias in empathy for pain focused on neural responses to a single person's suffering. It is unclear whether empathy for simultaneously perceived multiple individuals' pain (denoted as collective empathy in this study) is also sensitive to perceived racial identities of empathy targets. We addressed this issue by recording electroencephalography from Chinese adults who responded to racial identities of 2 × 2 arrays of Asian or White faces in which 4 faces, 1 face, or no face showed painful expressions. Participants reported greater feelings of others' pain and their own unpleasantness when viewing 4 compared to 1 (or no) painful faces. Behavioral responses to racial identities of faces revealed decreased speeds of information acquisition when responding to the face arrays with 4 (vs. 1 or no) painful expressions of Asian (but not White) faces. Moreover, Asian compared to White face arrays with 4 (vs. 1 or no) painful expressions elicited a larger positive neural response at 160-190 ms (P2) at the frontal/central electrodes and enhanced alpha synchronizations at 288-1,000 ms at the central electrodes. Our findings provide evidence for racial ingroup biases in collective empathy for pain and unravel its relevant neural underpinnings.

The speed of race

When asked to categorize faces according to 'race', people typically categorize other-race faces faster than faces belonging to their own race. This 'Other Race Categorization Advantage' is thought to reflect enhanced sensitivity to early visual signals characteristic of other-race faces, and can manifest within 200 ms of face presentation. However, recent research has highlighted the importance of signal intensity in this effect, where visual-degradation of the face images significantly enhances the effect and exposes a behavioural threshold at very low levels of visual quality where other-race visual signals are able to be perceived while same-race signals are not. The current study investigated the effect of signal intensity in race categorization processes in the brain through electroencephalography and in accuracy/reaction times. While replicating the previously observed enhancement of the other-race categorization advantage, we also found enhanced sensitivity to other-race faces in early P1 peaks, as well as later N170 and N250 peaks. These effects, however, related to the varying levels of signal intensity in the face stimuli, suggesting that race categorization may involve different types of perceptual and neural processes rather than one discrete process. The speed at which race is perceived depends on the intensity of the face signal.

Explicit face memory abilities are positively related to the non-intentional encoding of faces: Behavioral and ERP evidence

Individual differences in face memory abilities have been shown to be related to individual differences in brain activity. The present study investigated brain-behavior relationships for the N250 component in event-related brain potentials, which is taken as a neural sign of face familiarity. We used a task in which a designated, typical target face and several (high- and low-distinctive) nontarget faces had to be distinguished during multiple presentations across a session. Separately, face memory/recognition abilities were measured with easy versus difficult tasks. We replicated an increase of the N250 amplitude to the target face across the session and observed a similar increase for the non-target faces, indicating the build-up of memory representations also for these faces. On the interindividual level, larger across-session N250 amplitude increases to low-distinctive non-target faces were related to faster face recognition as measured in an easy task. These findings indicate that non-intentional encoding of non-target faces into memory is associated with the swift recognition of explicitly learned faces; that is, there is shared variance of incidental and intentional face memory.

On reliability of annotations in contextual emotion imagery

We documented the relabeling process for a subset of a renowned database for emotion-in-context recognition, with the aim of promoting reliability in final labels. To this end, emotion categories were organized into eight groups, while a large number of participants was requested for tagging. A strict control strategy was performed along the experiments, whose duration was 13.45 minutes average per day. Annotators were free to participate in any of the daily experiments (the average number of participants was 28), and a Z-Score filtering technique was implemented to keep trustworthiness of annotations. As a result, the value of the agreement parameter Fleiss' Kapa increasingly varied from slight to almost perfect, revealing a coherent diversity of the experiments. Our results support the hypothesis that a small number of categories and a large number of voters benefit reliability of annotations in contextual emotion imagery.

Differences Between East Asians and Westerners in the Mental Representations and Visual Information Extraction Involved in the Decoding of Pain Facial Expression Intensity

Effectively communicating pain is crucial for human beings. Facial expressions are one of the most specific forms of behavior associated with pain, but the way culture shapes expectations about the intensity with which pain is typically facially conveyed, and the visual strategies deployed to decode pain intensity in facial expressions, is poorly understood. The present study used a data-driven approach to compare two cultures, namely East Asians and Westerners, with respect to their mental representations of pain facial expressions (experiment 1, N=60; experiment 2, N=74) and their visual information utilization during the discrimination of facial expressions of pain of different intensities (experiment 3; N=60). Results reveal that compared to Westerners, East Asians expect more intense pain expressions (experiments 1 and 2), need more signal, and do not rely as much as Westerners on core facial features of pain expressions to discriminate between pain intensities (experiment 3). Together, those findings suggest that cultural norms regarding socially accepted pain behaviors shape the expectations about pain facial expressions and decoding visual strategies. Furthermore, they highlight the complexity of emotional facial expressions and the importance of studying pain communication in multicultural settings.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-023-00186-1.

Prosopagnosia does not abolish other-race effects

Healthy observers recognize more accurately same-than other-race faces (i.e., the Same-Race Recognition Advantage - SRRA) but categorize them by race more slowly than other-race faces (i.e., the Other-Race Categorization Advantage - ORCA). Several fMRI studies reported discrepant bilateral activations in the Fusiform Face Area (FFA) and Occipital Face Area (OFA) correlating with both effects. However, due to the very nature and limits of fMRI results, whether these face-sensitive regions play an unequivocal causal role in those other-race effects remains to be clarified. To this aim, we tested PS, a well-studied pure case of acquired prosopagnosia with lesions encompassing the left FFA and the right OFA. PS, healthy age-matched and young adults performed two recognition and three categorization by race tasks, respectively using Western Caucasian and East Asian faces normalized for their low-level properties with and without-external features, as well as in naturalistic settings. As expected, PS was slower and less accurate than the controls. Crucially, however, the magnitudes of her SRRA and ORCA were comparable to the controls in all the tasks. Our data show that prosopagnosia does not abolish other-race effects, as an intact face system, the left FFA and/or right OFA are not critical for eliciting the SRRA and ORCA. Race is a strong visual and social signal that is encoded in a large neural face-sensitive network, robustly tuned for processing same-race faces.

Is the Other-Race Effect in Working Memory Due to Attentional Refreshing?

The other-race effect is the observation that faces from another ethnicity induce worst recall performance than faces from one's own ethnicity. This effect has been defined as a type of familiarity effect, with more familiar faces better recalled than less familiar faces. In this study, we tested the hypothesis that a working memory maintenance mechanism called attentional refreshing mediates the other-race effect and that faces from one's own ethnicity are refreshed more efficiently than faces from other ethnicities. In two experiments, face ethnicity was orthogonally manipulated with cognitive load of a concurrent processing task in a complex-span paradigm (Exp. 1) and with the memory load in a Brown-Peterson paradigm (Exp. 2). Both cognitive and memory load effects are indices of the functioning of attentional refreshing. Testing Caucasian young adults, Caucasian and East-Asian faces were contrasted. Results from both experiments were congruent and against our initial hypothesis. The other-race effect in working memory does not appear to be supported by attentional refreshing. Furthermore, the results are congruent with the idea that faces as a whole are not maintained in working memory via attentional refreshing.

The role of task demands in racial face encoding

People more accurately remember faces of their own racial group compared to faces of other racial groups; this phenomenon is called the other-race effect. To date, numerous researchers have devoted themselves to exploring the reasons for this other-race effect, and they have posited several theoretical explanations. One integrated explanation is the categorization-individuation model, which addresses two primary ways (categorization and individuation) of racial face processing and emphasizes the emergence of these two ways during the encoding stage. Learning-recognition and racial categorization tasks are two classical tasks used to explore racial face processing. Event-related potentials can facilitate investigation of the encoding differences of own- and other-race faces under these two typical task demands. Unfortunately, to date, results have been mixed. In the current study, we investigated whether categorization and individuation differ for own- and other-race faces during the encoding stage by using racial categorization and learning-recognition tasks. We found that task demands not only influence the encoding of racial faces, but also have a more profound effect in the encoding stage of recognition tasks for other-race faces. More specifically, own-race faces demonstrate deeper structural encoding than other-race faces, with less attentional involvement. Moreover, recognitions tasks might ask for more individual-level encoding, requiring more attentional resources in the early stage that may be maintained until relatively late stages. Our results provide some evidence concerning task selection for future racial face studies and establish a groundwork for a unified interpretation of racial face encoding.

Social contacts and loneliness affect the own age bias for emotional faces

Individuals are better at recognizing faces of their own age group (Own Age Bias) but it is unclear whether this bias occurs also for emotional faces and to what extent is affected by loneliness. Young individuals (N = 235) completed an age categorization task on faces of young and old individuals showing neutral, happy, and angry expressions. After a filler task, they categorized as seen or novel the original set of faces intermixed with a new set. Findings showed an Own Age Bias for novel young faces but no evidence that emotion eliminates it. Recognition accuracy was better for emotional faces, but the two factors did not interact. Importantly, low loneliness was linked to an Own Age Bias for novel happy faces. These findings are discussed in the context of current theoretical accounts of the Own Age Bias and of the effects of loneliness on attention and memory.

Enhanced source memory for cheaters with higher resemblance to own-culture typical faces

Recent evidence suggests that culture-specific face typicality has an impact on making trait judgments. Additionally, facial resemblance to one's culture-typical faces causes them to be perceived as reliable, less dangerous, and more accurately recognized. When judging persons from other cultural origins, one's own culture's face standards might shape inferences, behavior, and memory. In this study, the partners' facial resemblance to participants' culturally typical faces was manipulated using target faces, considered to be higher or lower, similar to people living in the participants' hometown. Participants were asked to invest in a company together with partners who have a higher and lower resemblance to their own-culture typical faces in a cooperation game. The results showed that facial resemblance to own-culture typical faces affected investment preferences. Partners with a higher resemblance to own-culture typical faces were more correctly distinguished in the old-new recognition memory task. The study found that partners with a higher resemblance to own-culture typical faces had a source memory advantage for cheating behaviors. These results confirmed that a higher resemblance to own-culture typical faces provide an advantage in cross-cultural interactions, allowing them to become better recognized. Additionally, enhanced source memory for cheaters with higher resemblance to own-culture typical faces may indicate a flexible cognitive system that is sensitive to information that violates social expectations.

Event-related brain potential correlates of the other-race effect: A review

People are better at remembering own-race relative to other-race faces. Here, we review event-related brain potential (ERP) correlates of this so-called other-'race' effect (ORE) by discussing three critical aspects that characterize the neural signature of this phenomenon. First, difficulties with other-race faces initially emerge during perceptual processing, which is indexed by an increased N170. Second, as evidenced by 'difference due to subsequent memory' effects, more effortful processing of other-race faces is needed for successful encoding into long-term memory. Third, ERP old/new effects reveal that a stronger engagement of processing resources is also required for successful retrieval of other-race faces from memory. The ERP evidence available to date thus suggests widespread ethnicity-related modulations during both perceptual and mnemonic processing stages. We further discuss how findings from the ORE compared with potentially related memory biases (e.g. other-gender or other-age effects) and how ERP findings inform the ongoing debate regarding the mechanisms underlying the ORE. Finally, we outline open questions and potential future directions with an emphasis on using multiple, ecologically more valid 'ambient' images for each face to assess the ORE in paradigms that capture identity rather than image recognition.

Asymmetrical responding to male versus female other-race categories in 9- to 12-month-old infants

Faces can be categorized along various dimensions including gender or race, an ability developing in infancy. Infant categorization studies have focused on facial attributes in isolation, but the interaction between these attributes remains poorly understood. Experiment 1 examined gender categorization of other-race faces in 9- and 12-month-old White infants. Nine- and 12-month-olds were familiarized with Asian male or female faces, and tested with a novel exemplar from the familiarized category paired with a novel exemplar from a novel category. Both age groups showed novel category preferences for novel Asian female faces after familiarization with Asian male faces, but showed no novel category preference for novel Asian male faces after familiarization with Asian female faces. This categorization asymmetry was not due to a spontaneous preference hindering novel category reaction (Experiment 2), and both age groups displayed difficulty discriminating among male, but not female, other-race faces (Experiment 3). These results indicate that category formation for male other-race faces is mediated by categorical perception. Overall, the findings suggest that even by 12 months of age, infants are not fully able to form gender category representations of other-race faces, responding categorically to male, but not female, other-race faces.

Race categorization in noise

People are typically faster to categorize the race of a face if it belongs to a race different from their own. This Other Race Categorization Advantage (ORCA) is thought to reflect an enhanced sensitivity to the visual race signals of other race faces, leading to faster response times. The current study investigated this sensitivity in a cross-cultural sample of Swiss and Japanese observers with a race categorization task using faces that had been parametrically degraded of visual structure, with normalized luminance and contrast. While Swiss observers exhibited an increasingly strong ORCA in both reaction time and accuracy as the face images were visually degraded up to 20% structural coherence, the Japanese observers manifested this pattern most distinctly when the faces were fully structurally-intact. Critically, for both observer groups, there was a clear accuracy effect at the 20% structural coherence level, indicating that the enhanced sensitivity to other race visual signals persists in significantly degraded stimuli. These results suggest that different cultural groups may rely on and extract distinct types of visual race signals during categorization, which may depend on the available visual information. Nevertheless, heavily degraded stimuli specifically favor the perception of other race faces, indicating that the visual system is tuned by experience and is sensitive to the detection of unfamiliar signals.

Gene-Environment Interactions in Face Categorization: Oxytocin Receptor Genotype x Childcare Experience Shortens Reaction Time

Human faces capture attention, provide information about group belonging, and elicit automatic prepared responses. Early experiences with other-race faces play a critical role in acquiring face expertise, but the exact mechanism through which early experience exerts its influence is still to be elucidated. Genetic factors and a multi-ethnic context are likely involved, but their specific influences have not been explored. This study investigated how oxytocin receptor gene (OXTR) genotypes and childcare experience interacted to regulate face categorization in adults. Information about single nucleotide polymorphisms of OXTR (rs53576) and experiences with own- and other-race child caregivers was collected from 89 Singaporean adults, who completed a visual categorization task with own- versus other-race faces. Participants were grouped into A/A homozygotes and G carriers and assigned a score to account for their type of child caregiver experience. A multivariate linear regression model was used to estimate the effect of genetic group, child caregiver experience, and their interaction on categorization reaction time. A significant interaction of genetic group and child caregiver experience (t = 2.48, p = 0.015), as well as main effects of both genetic group (t = −2.17, p = 0.033) and child caregiver experience (t = −4.29, p < 0.001) emerged. Post-hoc analysis revealed that the correlation between categorization reaction time and child caregiver experience was significantly different between the two genetic groups. A significant gene x environment interaction on face categorization appears to represent an indirect pathway through which genes and experiences interact to shape mature social sensitivity to faces in human adults.

Cross-cultural evidence of a space-ethnicity association in face categorisation

According to a space-valence association, individuals tend to relate negatively- and positively-connoted stimuli with the left and right side of space, respectively. So far, only a few studies have explored whether this phenomenon can also emerge for social dimensions associated with facial stimuli. Here, we adopted a cross-cultural approach and conducted two experiments with the main aim to test whether a left–right space-valence association can also emerge for other- vs. own-race faces. Asian Japanese (Experiment 1) and White Italian (Experiment 2) participants engaged in a speeded binary classification task in which a central placed face had to be classified as either Asian or White. Manual responses were provided through a left- vs. right-side button. In both experiments, other-race faces elicited faster responses than own-race faces, in line with the well-documented other-race categorisation advantage. Moreover, evidence of an association between space and ethnic membership also arose and, interestingly, was similar in both groups. Indeed, Asian faces were responded to faster with the right-side key than with the left-side key, whereas response side had no effect for White faces. These results are discussed with regard to possible cross-cultural differences in group perception.

The enfacement illusion in autism spectrum disorder: How interpersonal multisensory stimulation influences facial recognition of the self

At its most basic level, the sense of self is built upon awareness of one's body and the face holds special significance as the individual's most important and distinctive physical feature. Multimodal sensory integration is pivotal to experiencing one's own body as a coherent visual "self" representation is formed and maintained by matching felt and observed sensorimotor experiences in the mirror. While difficulties in individual facial identity recognition and in both self-referential cognition and empathy are frequently reported in individuals with autism spectrum disorder (ASD), studying the effect of multimodal sensory stimulation in this population is of relevant interest. The present study investigates for the first time the specific effect on Interpersonal Multisensory Stimulation (IMS) on face self-recognition in a sample of 30 adults with (n = 15) and without (n = 15) ASD, matched on age and sex. The results demonstrate atypical self-face recognition and absence of IMS effects (enfacement illusion) in adults with ASD compared to controls, indicating that multisensory integration failed in updating cognitive representations of one's own face among persons with this disorder. The results are discussed in the light of other findings indicating alterations in body enfacement illusion and automatic imitation in ASD as well as in the context of the theories of procedural perception and multisensory integration alterations.

The Episodic Prototypes Model (EPM): On the nature and genesis of facial representations

Faces undergo massive changes over time and life events. We need a mental representation which is flexible enough to cope with the existing visual varieties, but which is also stable enough to be the basis for valid recognition. Two main theoretical frameworks exist to describe facial representations: prototype models assuming one central item comprising all visual experiences of a face, and exemplar models assuming single representations of each visual experience of a face. We introduce a much more ecological valid model dealing with episodic prototypes (the Episodic Prototypes Model—EPM), where faces are represented by a low number of prototypes that refer to specific Episodes of Life (EoL, e.g., early adulthood, mature age) during which the facial appearance shows only moderate variation. Such an episodic view of mental representation allows for efficient storage, as the number of needed prototypes is relatively low, and it allows for the needed variation within a prototype that keeps the everyday and steadily ongoing changes across a certain period of time. Studies 1–3 provide evidence that facial representations are highly dependent on temporal aspects which is in accord with EoL, and that individual learning history generates the structure and content of respective prototypes. In Study 4, we used implicit measures (RT) in a face verification task to investigate the postulated power of the EPM. We could demonstrate that episodic prototypes clearly outperformed visual depictions of exhaustive prototypes, supporting the general idea of our approach.

Rapid saccadic categorization of other-race faces

The human visual system is very fast and efficient at extracting socially relevant information from faces. Visual studies employing foveated faces have consistently reported faster categorization by race response times for other-race compared with same-race faces. However, in everyday life we typically encounter faces outside the foveated visual field. In study 1, we explored whether and how race is categorized extrafoveally in same- and other-race faces normalized for low-level properties by tracking eye movements of Western Caucasian and East Asian observers in a saccadic response task. The results show that not only are people sensitive to race in faces presented outside of central vision, but the speed advantage in categorizing other-race faces occurs astonishingly quickly in as little as 200 ms. Critically, this visual categorization process was approximately 300 ms faster than the typical button press responses on centrally presented foveated faces. Study 2 investigated the genesis of the extrafoveal saccadic response speed advantage by comparing the influences of the response modality (button presses and saccadic responses), as well as the potential contribution of the impoverished low-spatial frequency spectrum characterizing extrafoveal visual information processing. Button press race categorization was not significantly faster with reconstructed retinal-filtered low spatial frequency faces, regardless of the visual field presentation. The speed of race categorization was significantly boosted only by extrafoveal saccades and not centrally foveated faces. Race is a potent, rapid, and effective visual signal transmitted by faces used for the categorization of ingroup/outgroup members. This fast universal visual categorization can occur outside central vision, igniting a cascade of social processes.

Is Face Age Mapped Asymmetrically onto Space? Insights from a SNARC-like Task

The magnitude associated with a stimulus can be spatially connoted, with relatively smaller and larger magnitudes that would be represented on the left and on the right side of space, respectively. According to recent evidence, this space–magnitude association could reflect specific brain asymmetries. In this study, we explored whether such an association can also emerge for face age, assuming that responders should represent relatively younger and older adult faces on the left and on the right, respectively. A sample of young adults performed a speeded binary classification task aimed at categorising the age of a centrally placed adult face stimulus as either younger or older than the age of a reference face. A left-side and a right-side response key were used to collect manual responses. Overall, older faces were categorised faster than younger faces, and response latencies decreased with the absolute difference between the age of the target stimulus and the age of the reference, in line with a distance effect. However, no evidence of a left-to-right spatial representation of face age emerged. Taken together, these results suggest that face age is mapped onto space differently from other magnitudes.

Can anxiety and race interact to influence face-recognition accuracy? A systematic literature review

Wrongful convictions continue to occur through eyewitness misidentification. Recognising what factors, or interaction between factors, affect face-recognition is therefore imperative. Extensive research indicates that face-recognition accuracy is impacted by anxiety and by race. Limited research, however, has examined how these factors interact to potentially exacerbate face-recognition deficits. Brigham (2008) suggests that anxiety exacerbates other-race face-recognition deficits. Conversely, Attentional Control Theory predicts that anxiety exacerbates deficits for all faces. This systematic review examined existing studies investigating the possible interaction between anxiety and face-race to compare these theories. Recent studies included in this review found that both anxiety and race influence face-recognition accuracy but found no interaction. Potential moderators existing in reviewed studies, however, might have influenced their results. Separately, in some studies reviewed, anxiety induced during retrieval impacted recognition, contrasting with the conclusions of previous reviews. Recommendations for future research are given to address moderators potentially impacting results observed previously.

Sensitivity to face animacy and inversion in childhood: Evidence from EEG data

Adults exhibit relative behavioral difficulties in processing inanimate, artificial faces compared to real human faces, with implications for using artificial faces in research and designing artificial social agents. However, the developmental trajectory of inanimate face perception is unknown. To address this gap, we used electroencephalography to investigate inanimate faces processing in cross-sectional groups of 5-10-year-old children and adults. A face inversion manipulation was used to test whether face animacy processing relies on expert face processing strategies. Groups of 5-7-year-olds (N = 18), 8-10-year-olds (N = 18), and adults (N = 16) watched pictures of real or doll faces presented in an upright or inverted orientation. Analyses of event-related potentials revealed larger N170 amplitudes in response to doll faces, irrespective of age group or face orientation. Thus, the N170 is sensitive to face animacy by 5-7 years of age, but such sensitivity may not reflect high-level, expert face processing. Multivariate pattern analyses of the EEG signal additionally assessed whether animacy information could be reliably extracted during face processing. Face orientation, but not face animacy, could be reliably decoded from occipitotemporal channels in children and adults. Face animacy could be decoded from whole scalp channels in adults, but not children. Together, these results suggest that 5-10-year-old children exhibit some sensitivity to face animacy over occipitotemporal regions that is comparable to adults.

Multidimensional Face Representation in a Deep Convolutional Neural Network Reveals the Mechanism Underlying AI Racism

The increasingly popular application of AI runs the risk of amplifying social bias, such as classifying non-white faces as animals. Recent research has largely attributed this bias to the training data implemented. However, the underlying mechanism is poorly understood; therefore, strategies to rectify the bias are unresolved. Here, we examined a typical deep convolutional neural network (DCNN), VGG-Face, which was trained with a face dataset consisting of more white faces than black and Asian faces. The transfer learning result showed significantly better performance in identifying white faces, similar to the well-known social bias in humans, the other-race effect (ORE). To test whether the effect resulted from the imbalance of face images, we retrained the VGG-Face with a dataset containing more Asian faces, and found a reverse ORE that the newly-trained VGG-Face preferred Asian faces over white faces in identification accuracy. Additionally, when the number of Asian faces and white faces were matched in the dataset, the DCNN did not show any bias. To further examine how imbalanced image input led to the ORE, we performed a representational similarity analysis on VGG-Face's activation. We found that when the dataset contained more white faces, the representation of white faces was more distinct, indexed by smaller in-group similarity and larger representational Euclidean distance. That is, white faces were scattered more sparsely in the representational face space of the VGG-Face than the other faces. Importantly, the distinctiveness of faces was positively correlated with identification accuracy, which explained the ORE observed in the VGG-Face. In summary, our study revealed the mechanism underlying the ORE in DCNNs, which provides a novel approach to studying AI ethics. In addition, the face multidimensional representation theory discovered in humans was also applicable to DCNNs, advocating for future studies to apply more cognitive theories to understand DCNNs' behavior.

The left-side bias is not unique to own-race face processing

Humans show a clear left-side bias in face processing. A chimeric face constructed with the left side (from the viewer's perspective) of a face and its mirror image is usually rated as more resemblant to the original face than a chimeric face constructed with the right side of the same face. Previous studies have characterized the left-side bias mainly with own-race faces, but it remains unclear whether this effect is race specific or if it reflects an universal visual expertise. One hundred and five Chinese students completed two versions of a chimeric face-identification task. The results revealed a significant left-side bias for both own-race (Chinese) and other-race (Caucasian) faces, suggesting that the left-side bias reflects an universal visual expertise in face processing.

Attractiveness and distinctiveness between speakers' voices in naturalistic speech and their faces are uncorrelated

Facial attractiveness has been linked to the averageness (or typicality) of a face and, more tentatively, to a speaker's vocal attractiveness, via the 'honest signal' hypothesis, holding that attractiveness signals good genes. In four experiments, we assessed ratings for attractiveness and two common measures of distinctiveness ('distinctiveness-in-the-crowd', DITC and 'deviation-based distinctiveness', DEV) for faces and voices (simple vowels, or more naturalistic sentences) from 64 young adult speakers (32 female). Consistent and substantial negative correlations between attractiveness and DEV generally supported the averageness account of attractiveness, for both voices and faces. By contrast, and indicating that both measures of distinctiveness reflect different constructs, correlations between attractiveness and DITC were numerically positive for faces (though small and non-significant), and significant for voices in sentence stimuli. Between faces and voices, distinctiveness ratings were uncorrelated. Remarkably, and at variance with the honest signal hypothesis, vocal and facial attractiveness were also uncorrelated in all analyses involving naturalistic, i.e. sentence-based, speech. This result pattern was confirmed using a new set of stimuli and raters (experiment 5). Overall, while our findings strongly support an averageness account of attractiveness for both domains, they provide no evidence for an honest signal account of facial and vocal attractiveness in complex naturalistic speech.

Recognising other-race faces is more effortful: The effect of individuation instructions on encoding-related ERP Dm effects

Humans are better at recognising faces from their own vs. another ethnic background. Socio-cognitive theories of this own-race bias (ORB) propose that reduced recognition of other-race faces results from less motivation to attend to individuating information during encoding. Accordingly, individuation instructions that explain the phenomenon and instruct participants to attend to other-race faces during learning attenuate or eliminate the ORB. However, it is still unclear how exactly such instructions affect other-race face processing. We addressed this question by investigating encoding-related event-related brain potentials, contrasting neural activity of subsequently remembered and forgotten items (Dm effects). In line with socio-cognitive accounts, individuation instructions reduced the ORB. Critically, instructions increased Dm effects for other-race faces, suggesting that more processing resources were allocated to these faces during encoding. Thus, compensating for reduced experience with other-race faces is possible to some extent, but additional resources are needed to decrease difficulties resulting from a lack of perceptual expertise.

Face-voice space: Integrating visual and auditory cues in judgments of person distinctiveness

Faces and voices each convey multiple cues enabling us to tell people apart. Research on face and voice distinctiveness commonly utilizes multidimensional space to represent these complex, perceptual abilities. We extend this framework to examine how a combined face-voice space would relate to its constituent face and voice spaces. Participants rated videos of speakers for their dissimilarity in face only, voice only, and face-voice together conditions. Multiple dimensional scaling (MDS) and regression analyses showed that whereas face-voice space more closely resembled face space, indicating visual dominance, face-voice distinctiveness was best characterized by a multiplicative integration of face-only and voice-only distinctiveness, indicating that auditory and visual cues are used interactively in person-distinctiveness judgments. Further, the multiplicative integration could not be explained by the small correlation found between face-only and voice-only distinctiveness. As an exploratory analysis, we next identified auditory and visual features that correlated with the dimensions in the MDS solutions. Features pertaining to facial width, lip movement, spectral centroid, fundamental frequency, and loudness variation were identified as important features in face-voice space. We discuss the implications of our findings in terms of person perception, recognition, and face-voice matching abilities.

A developmental investigation of the other-race categorization advantage in a multiracial population: Contrasting social categorization and perceptual expertise accounts

Most prior studies of the other-race categorization advantage have been conducted in predominantly monoracial societies. This limitation has left open the question of whether tendencies to more rapidly and accurately categorize other-race faces reflect social categorization (own-race vs. other-race) or perceptual expertise (frequent exposure vs. infrequent exposure). To address this question, we tested Malay and Malaysian Chinese children (9- and 10-year-olds) and adults on (a) own-race faces (i.e., Malay faces for Malay participants and Chinese faces for Malaysian Chinese participants), (b) high-frequency other-race faces (i.e., Chinese faces for Malay participants and Malay faces for Malaysian Chinese participants), and (c) low-frequency other-race faces (i.e., Caucasian faces). Whereas the other-race categorization advantage was in evidence in the accuracy data of Malay adults, other aspects of performance were supportive of either the social categorization or perceptual expertise accounts and were dependent on the race (Malay vs. Chinese) or age (child vs. adult) of the participants. Of particular significance is the finding that Malaysian Chinese children and adults categorized own-race Chinese faces more rapidly than high-frequency other-race Malay faces. Thus, in accord with a perceptual expertise account, the other-race categorization advantage seems to be more an advantage for racial categories of lesser experience regardless of whether these face categories are own-race or other-race.

Evidence of an Own-Age Bias in Facial Emotion Recognition for Adolescents With and Without Autism Spectrum Disorder

A common interpretation of the face-processing deficits associated with autism spectrum disorder (ASD) is that they arise from a failure to develop normative levels of perceptual expertise. One indicator of perceptual expertise for faces is the own-age bias, operationalized as a processing advantage for faces of one's own age, presumably due to more frequent contact and experience. This effect is especially evident in domains of face recognition memory but less commonly investigated in social-emotional expertise (e.g., facial emotion recognition; FER), where individuals with ASD have shown consistent deficits. In the present study, we investigated whether a FER task would elicit an own-age bias for individuals with and without ASD and explored how the magnitude of an own-age bias may differ as a function of ASD status and symptoms. Ninety-two adolescents (63 male) between the ages of 11 and 14 years completed the child- and adult-face subtests of a standardized FER task. Overall FER accuracy was found to differ by ASD severity, reflecting poorer performance for those with increased symptoms. Results also indicated that an own-age bias was evident, reflecting greater FER performance for child compared to adult faces, for all adolescents regardless of ASD status or symptoms. However, the strength of the observed own-age bias did not differ by ASD status or severity. Findings suggest that face processing abilities of adolescents with ASD may be influenced by experience with specific categories of stimuli, similar to their typically developing peers.

The Role of Extraversion, IQ and Contact in the Own-Ethnicity Face Recognition Bias

While IQ is weakly related to the overall face recognition (Shakeshaft & Plomin, 2015), it plays a larger role in the processing of misaligned faces in the composite face task (Zhu et al., 2010). This type of stimuli are relatively novel and may reflect the involvement of intelligence in the processing of infrequently encountered faces, such as those of other-ethnicities. Extraversion is associated with increased eye contact which signifies less viewing of diagnostic features for Black faces. Using an old/new recognition paradigm, we found that IQ negatively correlated with the magnitude of the own-ethnicity bias (OEB) and that this relationship was moderated by contact with people from another ethnicity. We interpret these results in terms of IQ enhancing the ability to process novel stimuli by utilising multiple forms of coding. Extraversion was positively correlated with the OEB in White participants and negatively correlated with the OEB in Black participants suggesting that extraverts have lower attention to diagnostic facial features of Black faces, leading to poorer recognition of Black faces in both White and Black participants, thereby contributing to the relative OEB in these participants. The OEB is dependent on participant variables such as intelligence and extraversion.

Learning own- and other-race facial identities: Testing implicit recognition with event-related brain potentials

Exposure to varying images of the same person can encourage the formation of a representation that is sufficiently robust to allow recognition of previously unseen images of this person. While behavioural work suggests that face identity learning is harder for other-race faces, the present experiment investigated the neural correlates underlying own- and other-race face learning. Participants sorted own- and other-race identities into separate identity clusters and were further familiarised with these identities in a matching task. Subsequently, we compared event-related brain potentials (ERPs) in an implicit recognition (butterfly detection) task for learnt and previously unseen identities. We observed better sorting and matching for own- than other-race identities, and behavioural learning effects were restricted to own-race identities. Similarly, the N170 ERP component showed clear learning effects for own-race faces only. The N250, a component more closely associated with face learning was more negative for learnt than novel identities. ERP findings thus suggests a processing advantage for own-race identities at an early perceptual level whereas later correlates of identity learning were unaffected by ethnicity. These results suggest learning advantages for own-race identities, which underscores the importance of perceptual expertise in the own-race bias.

Learning own- and other-race facial identities from natural variability

Exposure to multiple varying face images of the same person encourages the formation of identity representations which are sufficiently robust to allow subsequent recognition from new, never-before seen images. While recent studies suggest that identity information is initially harder to perceive in images of other- relative to own-race identities, it remains unclear whether these difficulties propagate to face learning, that is, to the formation of robust face representations. We report two experiments in which Caucasian and East Asian participants sorted multiple images of own- and other-race persons according to identity in an implicit learning task and subsequently either matched novel images of learnt and previously unseen faces for identity (Experiment 1) or made old/new decisions for new images of learnt and unfamiliar identities (Experiment 2). Caucasian participants demonstrated own-race advantages during sorting, matching, and old/new recognition, while corresponding effects were absent in East Asian participants with substantial other-race expertise. Surprisingly, East Asian participants showed enhanced learning for other-race identities during matching in Experiment 1, which may reflect their increased motivation to individuate other-race faces. Thus, our results highlight the importance of perceptual expertise for own- and other-race processing, but may also lend support to recent suggestions on how expertise and socio-cognitive factors can interact.

Intentionally remembering or forgetting own- and other-race faces: Evidence from directed forgetting

People are better at remembering faces of their own relative to another ethnic group. This so-called own-race bias (ORB) has been explained in terms of differential perceptual expertise for own- and other-race faces or, alternatively, as resulting from socio-cognitive factors. To test predictions derived from the latter account, we examined item-method directed forgetting (DF), a paradigm sensitive to an intentional modulation of memory, for faces belonging to different ethnic and social groups. In a series of five experiments, participants during learning received cues following each face to either remember or forget the item, but at test were required to recognize all items irrespective of instruction. In Experiments 1 and 5, Caucasian participants showed DF for own-race faces only while, in Experiment 2, East Asian participants with considerable expertise for Caucasian faces demonstrated DF for own- and other-race faces. Experiments 3 and 4 found clear DF for social in- and outgroup faces. These results suggest that a modulation of face memory by motivational processes is limited to faces with which we have acquired perceptual expertise. Thus, motivation alone is not sufficient to modulate memory for other-race faces and cannot fully explain the ORB.