Intra- and extra-personal variability in person recognition

The importance of internal and external features in recognizing faces that vary in familiarity and race

Familiar and unfamiliar faces are recognized in fundamentally different ways. One way in which recognition differs is in terms of the features that facilitate recognition: previous studies have shown that familiar face recognition depends more on internal facial features (i.e., eyes, nose and mouth), whereas unfamiliar face recognition depends more on external facial features (i.e., hair, ears and contour). However, very few studies have examined the recognition of faces that vary in both familiarity and race, and the reliance on different facial features, whilst also using faces that incorporate natural within-person variability. In the current study, we used an online version of the card sorting task to assess adults’ (n = 258) recognition of faces that varied in familiarity and race when presented with either the whole face, internal features only, or external features only. Adults better recognized familiar faces than unfamiliar faces in both the whole face and the internal features only conditions, but not in the external features only condition. Reasons why adults did not show an own-race advantage in recognition are discussed.

Partitioning natural face image variability emphasises within-identity over between-identity representation for understanding accurate recognition

Accurately recognising faces enables social interactions. In recent years it has become clear that people's accuracy differs markedly depending on viewer's familiarity with a face and their individual skill, but the cognitive and neural bases of these accuracy differences are not understood. We examined cognitive representations underlying these accuracy differences by measuring similarity ratings to natural facial image variation. Natural variation was sampled from uncontrolled images on the internet to reflect the appearance of faces as they are encountered in daily life. Using image averaging, and inspired by the computation of Analysis of Variance, we partitioned this variation into differences between faces (between-identity variation) and differences between photos of the same face (within-identity variation). This allowed us to compare modulation of these two sources of variation attributable to: (i) a person's familiarity with a face and, (ii) their face recognition ability. Contrary to prevailing accounts of human face recognition and perceptual learning, we found that modulation of within-identity variation - rather than between-identity variation - was associated with high accuracy. First, familiarity modulated similarity ratings to within-identity variation more than to between-face variation. Second, viewers that are extremely accurate in face recognition - 'super-recognisers' - differed from typical perceivers mostly in their ratings of within-identity variation, compared to between-identity variation. In a final computational analysis, we found evidence that transformations of between- and within-identity variation make separable contributions to perceptual expertise in face recognition. We conclude that inter- and intra-individual accuracy differences primarily arise from differences in the representation of within-identity image variation.

Contrast Negation Impairs Sorting Unfamiliar Faces by Identity: A Comparison With Original (Contrast-Positive) and Stretched Images

Recognition of unfamiliar faces is difficult in part due to variations in expressions, angles, and image quality. Studies suggest shape and surface properties play varied roles in face learning, and identification of unfamiliar faces uses diagnostic pigmentation/surface reflectance relative to shape information. Here, participants sorted photo-cards of unfamiliar faces by identity, which were shown in their original, stretched, and contrast-negated forms, to examine the utility of diagnostic shape and surface properties in sorting unfamiliar faces by identity. In four experiments, we varied the presentation order of conditions (contrast-negated first or original first with stretched second across experiments) and whether the same or different photo-cards were seen across conditions. Stretching the images did not impair performance in any measures relative to other conditions. Contrast negation generally exacerbated poor sorting by identity compared with the other conditions. However, seeing the contrast-negated photo-cards last mitigated some of the effects of contrast negation. Together, results suggest an important role for surface properties such as pigmentation and reflectance for sorting by identity and add to literatures on informational content and appearance variability in discrimination of facial identity.

Body Language Influences on Facial Identification at Passport Control: An Exploration in Virtual Reality

Person identification at airports requires the matching of a passport photograph to its bearer. One aim of this process is to find identity impostors, who use valid identity documents of similar-looking people to avoid detection. In psychology, this process has been studied extensively with static pairs of face photographs that require identity match (same person shown) versus mismatch (two different people) decisions. However, this approach provides a limited proxy for studying how other factors, such as nonverbal behaviour, affect this task. The current study investigated the influence of body language on facial identity matching within a virtual reality airport environment, by manipulating activity levels of person avatars queueing at passport control. In a series of six experiments, detection of identity mismatches was unaffected when observers were not instructed to utilise body language. By contrast, under explicit instruction to look out for unusual body language, these cues enhanced detection of mismatches but also increased false classification of matches. This effect was driven by increased activity levels rather than body language that simply differed from the behaviour of the majority of passengers. The implications and limitations of these findings are discussed.

Discrimination and recognition of faces with changed configuration

Subtle metric differences in facial configuration, such as between-person variation in the distances between the eyes, have been used widely in psychology to explain face recognition. However, these studies of configuration have typically utilized unfamiliar faces rather than the familiar faces that the process of recognition ultimately seeks to explain. This study investigates whether face recognition relies on the metric information presumed in configural theory, by manipulating the interocular distance in both unfamiliar and familiar faces. In Experiment 1, observers were asked to detect which face in a pair was presented with its configuration intact. In Experiment 2, this discrimination task was repeated with faces presented individually, and observers were also asked to make familiarity categorizations to the same stimuli. In both experiments, familiarity determined detection of faces in their original configuration, and also enhanced identity categorization in Experiment 2. However, discrimination of configuration was generally low. In turn, recognition accuracy was generally high irrespective of configuration condition. Moreover, observers most sensitive to configuration during discrimination did not appear to rely on this information for recognition of familiar faces. These results demonstrate that configuration theory provides limited explanatory power for the recognition of familiar faces.

Agent identity drives adaptive encoding of biological motion into working memory

To engage in normal social interactions, we have to encode human biological motions (BMs, e.g., walking and jumping), which is one of the most salient and biologically significant types of kinetic information encountered in everyday life, into working memory (WM). Critically, each BM in real life is produced by a distinct person, carrying a dynamic motion signature (i.e., identity). Whether agent identity influences the WM processing of BMs remains unknown. Here, we addressed this question by examining whether memorizing BMs with different identities promoted the WM processing of task-irrelevant clothing colors. Two opposing hypotheses were tested: (a) WM only stores the target action (element-based hypothesis) and (b) WM stores both action and irrelevant clothing color (event-based hypothesis), interpreting each BM as an event. We required participants to memorize actions that either performed by one agent or distinct agents, while ignoring clothing colors. Then we examined whether the irrelevant color was also stored in WM by probing a distracting effect: If the color was extracted into WM, the change of irrelevant color in the probe would lead to a significant distracting effect on action performance. We found that WM encoding of BMs was adaptive: Once the memorized actions had different identities, WM adopted an event-based encoding mode regardless of memory load and probe identity (Experiment 1, different-identity group of Experiment 2, and Experiment 3). However, WM used an element-based encoding mode when memorized-actions shared the same identity (same-identity group of Experiment 2) or were inverted (Experiment 4). Overall, these findings imply that agent identity information has a significant effect on the WM processing of BMs.

Social Evaluation of Faces Across Gender and Familiarity

Models of social evaluation aim to capture the information people use to form first impressions of unfamiliar others. However, little is currently known about the relationship between perceived traits across gender. In Study 1, we asked viewers to provide ratings of key social dimensions (dominance, trustworthiness, etc.) for multiple images of 40 unfamiliar identities. We observed clear sex differences in the perception of dominance—with negative evaluations of high dominance in unfamiliar females but not males. In Study 2, we used the social evaluation context to investigate the key predictions about the importance of pictorial information in familiar and unfamiliar face processing. We compared the consistency of ratings attributed to different images of the same identities and demonstrated that ratings of images depicting the same familiar identity are more tightly clustered than those of unfamiliar identities. Such results imply a shift from image rating to person rating with increased familiarity, a finding which generalises results previously observed in studies of identification.

Face perception: A brief journey through recent discoveries and current directions

Faces are a rich source of information about the people around us. Identity, state of mind, emotions, intentions, age, gender, ethnic background, attractiveness and a host of other attributes about an individual can be gleaned from a face. When face perception fails, dramatic psycho-social consequences can follow at the individual level, as in the case of prosopagnosic parents who are unable to recognize their children at school pick-up. At the species level, social interaction patterns are shaped by human face perception abilities. The computational feat of recognizing faces and facial attributes, and the challenges overcome by the human brain to achieve this feat, have fascinated generations of vision researchers. In this paper, we present a brief overview of some of the milestones of discovery as well as outline a selected set of current directions and open questions on this topic.

The Effects of Blur and Inversion on the Recognition of Ambient Face Images

When viewing unfamiliar faces that vary in expressions, angles, and image quality, observers make many recognition errors. Specifically, in unconstrained identity-sorting tasks, observers struggle to cope with variation across different images of the same person while succeeding at telling different people apart. The use of ambient face images in this simple card-sorting task reveals the magnitude of these face recognition errors and suggests a useful platform to reexamine the nature of face processing using naturalistic stimuli. In the present study, we chose to investigate the impact of two basic stimulus manipulations (image blur and face inversion) on identity sorting with ambient images. Although these manipulations are both known to affect face processing when well-controlled, frontally viewed face images are used, examining how they affect performance for ambient images is an important step toward linking the large body of research using controlled face images to more ecologically valid viewing conditions. Briefly, we observed a high cost of image blur regardless of blur magnitude, and a strong inversion effect that affected observers’ sensitivity to extrapersonal variability but did not affect the number of unique identities they estimated were present in the set of images presented to them.

Familiarity and Within-Person Facial Variability: The Importance of the Internal and External Features

As faces become familiar, we come to rely more on their internal features for recognition and matching tasks. Here, we assess whether this same pattern is also observed for a card sorting task. Participants sorted photos showing either the full face, only the internal features, or only the external features into multiple piles, one pile per identity. In Experiments 1 and 2, we showed the standard advantage for familiar faces-sorting was more accurate and showed very few errors in comparison with unfamiliar faces. However, for both familiar and unfamiliar faces, sorting was less accurate for external features and equivalent for internal and full faces. In Experiment 3, we asked whether external features can ever be used to make an accurate sort. Using familiar faces and instructions on the number of identities present, we nevertheless found worse performance for the external in comparison with the internal features, suggesting that less identity information was available in the former. Taken together, we show that full faces and internal features are similarly informative with regard to identity. In comparison, external features contain less identity information and produce worse card sorting performance. This research extends current thinking on the shift in focus, both in attention and importance, toward the internal features and away from the external features as familiarity with a face increases.