Computational perspectives on the other-race effect

Why is the literature on first impressions so focused on White faces?

We spontaneously attribute to strangers a wide variety of character traits based on their facial appearance. While these first impressions have little or no basis in reality, they exert a strong influence over our behaviour. Cognitive scientists have revealed a great deal about first impressions from faces including their factor structure, the cues on which they are based, the neurocognitive mechanisms responsible, and their developmental trajectory. In this field, authors frequently strive to remove as much ethnic variability from stimulus sets as possible. Typically, this convention means that participants are asked to judge the likely traits of White faces only. In the present article, we consider four possible reasons for the lack of facial diversity in this literature and find that it is unjustified. Next, we illustrate how the focus on White faces has undermined scientific efforts to understand first impressions from faces and argue that it reinforces socially regressive ideas about 'race' and status. We go on to articulate our concern that opportunities may be lost to leverage the knowledge derived from the study of first impressions against the dire consequences of prejudice and discrimination. Finally, we highlight some promising developments in the field.

Can anxiety and race interact to influence face-recognition accuracy? A systematic literature review

Wrongful convictions continue to occur through eyewitness misidentification. Recognising what factors, or interaction between factors, affect face-recognition is therefore imperative. Extensive research indicates that face-recognition accuracy is impacted by anxiety and by race. Limited research, however, has examined how these factors interact to potentially exacerbate face-recognition deficits. Brigham (2008) suggests that anxiety exacerbates other-race face-recognition deficits. Conversely, Attentional Control Theory predicts that anxiety exacerbates deficits for all faces. This systematic review examined existing studies investigating the possible interaction between anxiety and face-race to compare these theories. Recent studies included in this review found that both anxiety and race influence face-recognition accuracy but found no interaction. Potential moderators existing in reviewed studies, however, might have influenced their results. Separately, in some studies reviewed, anxiety induced during retrieval impacted recognition, contrasting with the conclusions of previous reviews. Recommendations for future research are given to address moderators potentially impacting results observed previously.

Are you from North or South India? A hard face-classification task reveals systematic representational differences between humans and machines

We make a rich variety of judgments on faces, but the underlying features are poorly understood. Here we describe a challenging geographical-origin classification problem that elucidates feature representations in both humans and machine algorithms. In Experiment 1, we collected a diverse set of 1,647 faces from India labeled with their fine-grained geographical origin (North vs. South India), characterized the categorization performance of 129 human subjects on these faces, and compared this with the performance of machine vision algorithms. Our main finding is that while many machine algorithms achieved an overall performance comparable to that of humans (64%), their error patterns across faces were qualitatively different despite training. To elucidate the face parts used by humans for classification, we trained linear classifiers on overcomplete sets of features derived from each face part. This revealed mouth shape to be the most discriminative part compared to eyes, nose, or external contour. In Experiment 2, we confirmed that humans relied the most on mouth shape for classification using an additional experiment in which subjects classified faces with occluded parts. In Experiment 3, we compared human performance for briefly viewed faces and for inverted faces. Interestingly, human performance on inverted faces was predicted better by computational models compared to upright faces, suggesting that humans use relatively more generic features on inverted faces. Taken together, our results show that studying hard classification tasks can lead to useful insights into both machine and human vision.

Facial and self-report questionnaire measures capture different aspects of romantic partner preferences

Romantic relationship researchers often use self-report measures of partner preferences based on verbal questionnaires. These questionnaires show that partner preferences involve an evaluation in terms of underlying factors of vitality-attractiveness, status-resources, and warmth-trustworthiness. However, when people first encounter a potential partner, they can usually derive a wealth of impressions from their face, and little is known about the relationship between verbal self-reports and impressions derived from faces. We conducted four studies investigating potential parallels and differences between facial impressions and verbal self-reports. Study 1 showed that when evaluating highly variable everyday face images in a context that does not require considering them as potential partners, participants can reliably perceive the traits and factors that are related to partner preferences. However, despite being capable of these nuanced evaluations, Study 2 found that when asked to evaluate images of faces as potential romantic partners, participants' preferences become dominated by attractiveness-related concerns. Study 3 confirmed this dominance of facial attractiveness using morphed face-like images. Study 4 showed that attractiveness dominates partner preferences for faces even when task instructions imply that warmth-trustworthiness or status-resources should be of primary importance. In contrast to verbal questionnaire measures of partner preferences, evaluations of faces focus heavily on attractiveness, whereas questionnaire self-reports tend on average to prioritize warmth-trustworthiness over attractiveness. Evaluations of faces and verbal self-report measures therefore capture different aspects of partner preferences.

Crucial information for efficient face searching by humans and Japanese macaques

Humans can efficiently detect a face among non-face objects, but few studies of this ability have been conducted in animals. Here, in Japanese macaques and humans, we examined visual searching for a face and explored what factors contribute to efficient facial information processing. Subjects were asked to search for an odd target among the different numbers of distracters. Faces of the subjects' own species, the backs of the head of the subjects' own species, faces of the subjects' closely related species or race, and faces of species that are clearly different from the subjects' own species were used as the target. Both the macaques and humans detected a face of their own species more efficiently than a face from a clearly different species. Similar efficient detections were confirmed for the faces of the subjects' closely related species or race. These results suggest that conspecific faces and faces that share morphological similarity with conspecific faces can be detected efficiently among non-face objects by both humans and Japanese macaques. In another experiment, facial recognition efficiency was observed when the subjects searched for own-species faces that had lower-spatial-frequency components compared to faces with higher-spatial-frequency components. It seems reasonable that the ability to search efficiently for faces by using holistic face processing is derived from fundamental social cognition abilities that are broadly shared among species.

Facial First Impressions of Partner Preference Traits: Trustworthiness, Status, and Attractiveness

This research used the minimal exposure paradigm to examine facial first impressions of traits of trustworthiness, status, and attractiveness, considered important in verbal models of partner preferences. Heterosexual participants rated opposite-sex faces comprising either naturalistic images or youthful-looking averaged faces on trustworthiness, status, and attractiveness following 33, 100, and 500 ms masked presentation. The pattern masks were phase scrambled to provide the same overall color composition, brightness, and spatial frequency content as the presented faces. Trustworthiness, status, and attractiveness judgments were all reliable at above-chance levels even at 33 ms presentation, and extra time (100 or 500 ms) only led to modest improvement in the correspondence with an independent set of time-unconstrained judgments. The increasing prevalence of online images and internet-based relationships make these findings timely and important.

Learning Race from Face: A Survey

Faces convey a wealth of social signals, including race, expression, identity, age and gender, all of which have attracted increasing attention from multi-disciplinary research, such as psychology, neuroscience, computer science, to name a few. Gleaned from recent advances in computer vision, computer graphics, and machine learning, computational intelligence based racial face analysis has been particularly popular due to its significant potential and broader impacts in extensive real-world applications, such as security and defense, surveillance, human computer interface (HCI), biometric-based identification, among others. These studies raise an important question: How implicit, non-declarative racial category can be conceptually modeled and quantitatively inferred from the face? Nevertheless, race classification is challenging due to its ambiguity and complexity depending on context and criteria. To address this challenge, recently, significant efforts have been reported toward race detection and categorization in the community. This survey provides a comprehensive and critical review of the state-of-the-art advances in face-race perception, principles, algorithms, and applications. We first discuss race perception problem formulation and motivation, while highlighting the conceptual potentials of racial face processing. Next, taxonomy of feature representational models, algorithms, performance and racial databases are presented with systematic discussions within the unified learning scenario. Finally, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potentially important cross-cutting themes and research directions for the issue of learning race from face.

Own-race and own-species advantages in face perception: a computational view

The frequency to which an organism is exposed to a particular type of face influences recognition performance. For example, Asians are better in individuating Asian than Caucasian faces, known as the own-race advantage. Similarly, humans in general are better in individuating human than monkey faces, known as the own-species advantage. It is an open question whether the underlying mechanisms causing these effects are similar. We hypothesize that these processes are governed by neural plasticity of the face discrimination system to retain optimal discrimination performance in its environment. Using common face features derived from a set of images from various face classes, we show that maximizing the feature variance between different individuals while ensuring minimal variance within individuals achieved good discrimination performances on own-class faces when selecting a subset of feature dimensions. Further, the selected subset of features does not necessarily lead to an optimal performance on the other class of faces. Thus, the face discrimination system continuously re-optimizes its space constraint face representation to optimize recognition performance on the current distribution of faces in its environment. This model can account for both, the own-race and own-species advantages. We name this approach Space Constraint Optimized Representational Embedding (SCORE).