Improving Identity Matching of Newly Encountered Faces: Effects of Multi-image Training

Multiple images captured from a single encounter do not promote face learning

Viewing multiple images of a newly encountered face improves recognition of that identity in new instances. Studies examining face learning have presented high-variability (HV) images that incorporate changes that occur from moment-to-moment (e.g., head orientation and expression) and over time (e.g., lighting, hairstyle, and health). We examined whether low-variability (LV) images (i.e., images that incorporate only moment-to-moment changes) also promote generalisation of learning such that novel instances are recognised. Participants viewed a single image, six LV images, or six HV images of a target identity before being asked to recognise novel images of that identity in a face matching task (training stimuli remained visible) or a memory task (training stimuli were removed). In Experiment 1 (n = 71), participants indicated which image(s) in 8-image arrays belonged to the target identity. In Experiment 2 (n = 73), participants indicated whether sequentially presented images belonged to the target identity. Relative to the single-image condition, sensitivity to identity improved and response biases were less conservative in the HV condition; we found no evidence of generalisation of learning in the LV condition regardless of testing protocol. Our findings suggest that day-to-day variability in appearance plays an essential role in acquiring expertise with a novel face.

Ensemble coding of facial identity is robust, but may not contribute to face learning

Ensemble coding - the rapid extraction of a perceptual average - has been proposed as a potential mechanism underlying face learning. We tested this proposal across five pre-registered experiments in which four ambient images of an identity were presented in the study phase. In Experiments 1 and 2a-c, participants were asked whether a test image was in the study array; these experiments examined the robustness of ensemble coding. Experiment 1 replicated ensemble coding in an online sample; participants recognize images from the study array and the average of those images. Experiments 2a-c provide evidence that ensemble coding meets several criteria of a possible learning mechanism: It is robust to changes in head orientation (± 60o), survives a short (30s) delay, and persists when images of two identities are interleaved during the study phase. Experiment 3 examined whether ensemble coding is sufficient for face learning (i.e., facilitates recognition of novel images of a target identity). Each study array comprised four ambient images (variability + average), a single image, or an average of four images (average only). Participants were asked whether a novel test image showed the identity from a study array. Performance was best in the four-image condition, with no difference between the single-image and average-only conditions. We conclude that ensemble coding of facial identity is robust but that the perceptual average per se is not sufficient for face learning.

Provoked overt recognition in acquired prosopagnosia using multiple different images of famous faces

Provoked overt recognition refers to the fact that patients with acquired prosopagnosia can sometimes recognize faces when presented in arrays of individuals from the same category (e.g., actors or politicians). We ask whether a prosopagnosic patient might experience recognition when presented with multiple different images of the same face simultaneously. Over two sessions, patient Herschel, a 66-year-old British man with acquired prosopagnosia, viewed face images individually or in arrays. On several occasions he failed to recognize single photos of an individual but successfully identified that person when the same photos were presented together. For example, Herschel failed to recognize any individual images of King Charles or Paul McCartney but recognised both in arrays of the same photos. Like reports based on category membership, overt recognition was transient and inconsistent. These findings are discussed in terms of models of covert recognition, alongside more recent research on within-person variability for face perception.

Perceptual similarity modulates effects of learning from variability on face recognition

Face recognition is a challenging classification task that humans perform effortlessly for familiar faces. Recent studies have emphasized the importance of exposure to high variability appearances of the same identity to perform this task. However, these studies did not explicitly measure the perceptual similarity between the learned images and the images presented at test, which may account for the advantage of learning from high variability. Particularly, randomly selected test images are more likely to be perceptually similar to learned high variability images, and dissimilar to learned low variability images. Here we dissociated effects of learning from variability and study-test perceptual similarity, by collecting human similarity ratings for the study and test images. Using these measures, we independently manipulated the variability between the learning images and their perceptual similarity to the test images. Different groups of participants learned face identities from a low or high variability set of images. The learning phase was followed by a face matching test (Experiment 1) or a face recognition task (Experiment 2) that presented novel images of the learned identities that were perceptually dissimilar or similar to the learned images. Results of both experiments show that perceptual similarity between study and test, rather than image variability at learning per se, predicts face recognition. We conclude that learning from high variability improves face recognition for perceptually similar but not for perceptually dissimilar images. These findings may not be specific to faces and should be similarly evaluated for other domains.

Children's ability to recognize their parent's face improves with age

Adults are experts at recognizing familiar faces across images that incorporate natural within-person variability in appearance (i.e., ambient images). Little is known about children's ability to do so. In the current study, we investigated whether 4- to 7-year-olds (n = 56) could recognize images of their own parent-a person with whom children have had abundant exposure in a variety of different contexts. Children were asked to identify images of their parent that were intermixed with images of other people. We included images of each parent taken both before and after their child was born to manipulate how close the images were to the child's own experience. When viewing before-birth images, 4- and 5-year-olds were less sensitive to identity than were older children; sensitivity did not differ when viewing images taken after the child was born. These findings suggest that with even the most familiar face, 4- and 5-year-olds have difficulty recognizing instances that go beyond their direct experience. We discuss two factors that may contribute to the prolonged development of familiar face recognition.

Face Recognition by Humans and Machines: Three Fundamental Advances from Deep Learning

Deep learning models currently achieve human levels of performance on real-world face recognition tasks. We review scientific progress in understanding human face processing using computational approaches based on deep learning. This review is organized around three fundamental advances. First, deep networks trained for face identification generate a representation that retains structured information about the face (e.g., identity, demographics, appearance, social traits, expression) and the input image (e.g., viewpoint, illumination). This forces us to rethink the universe of possible solutions to the problem of inverse optics in vision. Second, deep learning models indicate that high-level visual representations of faces cannot be understood in terms of interpretable features. This has implications for understanding neural tuning and population coding in the high-level visual cortex. Third, learning in deep networks is a multistep process that forces theoretical consideration of diverse categories of learning that can overlap, accumulate over time, and interact. Diverse learning types are needed to model the development of human face processing skills, cross-race effects, and familiarity with individual faces.

GFMT2: A psychometric measure of face matching ability

We present an expanded version of a widely used measure of unfamiliar face matching ability, the Glasgow Face Matching Test (GFMT). The GFMT2 is created using the same source database as the original test but makes five key improvements. First, the test items include variation in head angle, pose, expression and subject-to-camera distance, making the new test more difficult and more representative of challenges in everyday face identification tasks. Second, short and long versions of the test each contain two forms that are calibrated to be of equal difficulty, allowing repeat tests to be performed to examine effects of training interventions. Third, the short-form tests contain no repeating face identities, thereby removing any confounding effects of familiarity that may have been present in the original test. Fourth, separate short versions are created to target exceptionally high performing or exceptionally low performing individuals using established psychometric principles. Fifth, all tests are implemented in an executable program, allowing them to be administered automatically. All tests are available free for scientific use via www.gfmt2.org .

Why the other-race effect matters: Poor recognition of other-race faces impacts everyday social interactions

What happens to everyday social interactions when other-race recognition fails? Here, we provide the first formal investigation of this question. We gave East Asian international students (N = 89) a questionnaire concerning their experiences of the other-race effect (ORE) in Australia, and a laboratory test of their objective other-race face recognition deficit using the Cambridge Face Memory Test (CFMT). As a 'perpetrator' of the ORE, participants reported that their problems telling apart Caucasian people contributed significantly to difficulties socializing with them. Moreover, the severity of this problem correlated with their ORE on the CFMT. As a 'victim' of the ORE, participants reported that Caucasians' problems telling them apart also contributed to difficulties socializing. Further, 81% of participants had been confused with other Asians by a Caucasian authority figure (e.g., university tutor, workplace boss), resulting in varying levels of upset/difficulty. When compared to previously established contributors to international students' high rates of social isolation, ORE-related problems were perceived as equally important as the language barrier and only moderately less important than cultural differences. We conclude that the real-world impact of the ORE extends beyond previously identified specialized settings (eyewitness testimony, security), to common everyday situations experienced by all humans.

When experience does not promote expertise: security professionals fail to detect low prevalence fake IDs

Professional screeners frequently verify photograph IDs in such industries as professional security, bar tending, and sales of age-restricted materials. Moreover, security screening is a vital tool for law enforcement in the search for missing or wanted persons. Nevertheless, previous research demonstrates that novice participants fail to spot fake IDs when they are rare (i.e., the low prevalence effect; LPE). To address whether this phenomenon also occurs with professional screeners, we conducted three experiments. Experiment 1 compared security professional and non-professionals. Experiment 2 compared bar-security professionals, access-security professionals, and non-professionals. Finally, Experiment 3 added a newly created Professional Identity Training Questionnaire to determine whether and how aspects of professionals’ employment predict ID-matching accuracy. Across all three experiments, all participants were susceptible to the LPE regardless of professional status. Neither length/type of professional experience nor length/type of training experience affected ID verification performance. We discuss task performance and survey responses with aims to acknowledge and address this potential problem in real-world screening scenarios.

Facial identity across the lifespan

We can recognise people that we know across their lifespan. We see family members age, and we can recognise celebrities across long careers. How is this possible, despite the very large facial changes that occur as people get older? Here we analyse the statistical properties of faces as they age, sampling photos of the same people from their 20s to their 70s. Across a number of simulations, we observe that individuals' faces retain some idiosyncratic physical properties across the adult lifespan that can be used to support moderate levels of age-independent recognition. However, we found that models based exclusively on image-similarity only achieved limited success in recognising faces across age. In contrast, more robust recognition was achieved with the introduction of a minimal top-down familiarisation procedure. Such models can incorporate the within-person variability associated with a particular individual to show a surprisingly high level of generalisation, even across the lifespan. The analysis of this variability reveals a powerful statistical tool for understanding recognition, and demonstrates how visual representations may support operations typically thought to require conceptual properties.

Learning own- and other-race facial identities: Testing implicit recognition with event-related brain potentials

Exposure to varying images of the same person can encourage the formation of a representation that is sufficiently robust to allow recognition of previously unseen images of this person. While behavioural work suggests that face identity learning is harder for other-race faces, the present experiment investigated the neural correlates underlying own- and other-race face learning. Participants sorted own- and other-race identities into separate identity clusters and were further familiarised with these identities in a matching task. Subsequently, we compared event-related brain potentials (ERPs) in an implicit recognition (butterfly detection) task for learnt and previously unseen identities. We observed better sorting and matching for own- than other-race identities, and behavioural learning effects were restricted to own-race identities. Similarly, the N170 ERP component showed clear learning effects for own-race faces only. The N250, a component more closely associated with face learning was more negative for learnt than novel identities. ERP findings thus suggests a processing advantage for own-race identities at an early perceptual level whereas later correlates of identity learning were unaffected by ethnicity. These results suggest learning advantages for own-race identities, which underscores the importance of perceptual expertise in the own-race bias.

Two Sides of Face Learning: Improving Between-Identity Discrimination While Tolerating More Within-Person Variability in Appearance

Two photos of an unfamiliar face are often perceived as belonging to different people—an error that disappears when a face is familiar. Face learning has been characterized as increased tolerance of within-person variability in appearance and is facilitated by exposure to such variability (e.g., differences in expression, lighting, and aesthetics). We hypothesized that increased tolerance of variability in appearance might lead to reduced discrimination and that misidentifications would be reduced if a face was learned in the context of a similar-looking identity. After validating our stimuli (Experiments 1a and 1b), we conducted three experiments investigating face learning. In two of these, participants learned three faces (Experiment 2: 15 images/identity and Experiment 3: 5 images/identity), two of which were similar. In a recognition task, misidentifications did not change as a function of similarity, although participants recognized more images of the target in Experiment 2 (i.e., after learning 15 images). In Experiment 4, participants learned one identity and the number of images studied varied across groups. Recognition of new images increased with the number of images studied, with no changes in false alarms; sensitivity (A′) marginally increased. The results suggest that recognition and discrimination reflect separable processes with minimal influence of between-person similarity on discrimination.

Use-inspired basic research on individual differences in face identification: implications for criminal investigation and security

This journal is dedicated to "use-inspired basic research" where a problem in the world shapes the hypotheses for study in the laboratory. This review considers the role of individual variation in face identification and the challenges and opportunities this presents in security and criminal investigations. We show how theoretical work conducted on individual variation in face identification has, in part, been stimulated by situations presented in the real world. In turn, we review the contribution of theoretical work on individual variation in face processing and how this may help shape the practical identification of faces in applied situations. We consider two cases in detail. The first case is that of security officers; gatekeepers who use facial ID to grant entry or deny access. One applied example, where much research has been conducted, is passport control officers who are asked to match a person in front of them to a photograph shown on their ID. What happens if they are poor at making such face matching decisions and can they be trained to improve their performance? Second, we outline the case of "super-recognisers", people who are excellent at face recognition. Here it is interesting to consider whether these individuals can be strategically allocated to security and criminal roles, to maximise the identification of suspects. We conclude that individual differences are one of the largest documented sources of error in face matching and face recognition but more work is needed to account for these differences within theoretical models of face processing.

Becoming Familiar With a Newly Encountered Face: Evidence of an Own-Race Advantage

Adults' ability to match identity in images of unfamiliar faces is impaired for other- compared with own-race faces; their ability to match identity in images of familiar faces is independent of face race. Exposure to within-person variability in appearance plays a key role in face learning. Past research suggests that children need exposure to higher levels of variability than adults to learn a new face-a difference that has been attributed to experience. We predicted that adults' limited experience with other-race faces would result in their needing exposure to higher levels of variability when learning other- compared with own-race faces. We introduced adults to four new identities (two own-race; two other-race) in one of the three conditions: a single image, a low-variability video (filmed on 1 day), or a high-variability video (filmed across 3 days). Adults' ability to recognize new instances of learned identities improved in the low-variability condition for own-race faces but only in the high-variability condition for other-race faces. We discuss learning mechanisms that might drive this difference-a difference we attribute to experience.