Individual differences in the detection, matching and memory of faces

Are contemporary facial recognition algorithms making human facial comparison performance worse?

Facial recognition plays a vital role in several security and law enforcement workflows, such as passport control and criminal investigations. The identification process typically involves a facial recognition system comparing an image against a large database of faces to return a list of probable matches, called a candidate list, for review. A human then looks at the returned images to determine whether there is a match. Most evaluations of these systems tend to examine the performance of the algorithm or human in isolation, not accounting for the interaction that occurs in operational contexts. To ensure optimal whole system performance, it is important to understand how the output produced by an algorithm can impact human performance. Anecdotal claims have been made by users of facial recognition systems that the images being returned by new algorithms in these systems have become more similar in appearance compared to old algorithms, making their job of determining the presence of a match more difficult. This paper explores whether these claims are true and whether the latest facial recognition algorithms decrease human performance compared to an old algorithm from the same company. We examined the performance of 40 novice participants on 120 face matching trials. Each trial required the participant to compare a face image against a candidate list containing eight possible matches returned by either a new or old algorithm (60 trials of each). Overall, participants were more likely to make errors when presented with a candidate list from a new algorithm. Specifically, they were more likely to misidentify an incorrect identity as a match. Participants were more accurate, confident, and faster on candidate lists from the older algorithm. These findings suggest that new algorithms are generating more plausible matches, making the task of determining a match harder for humans. We propose strategies to potentially improve performance and recommendations for future research.

Face Feature Change Detection Ability in Developmental Prosopagnosia and Super-Recognisers

In non-clinical populations, facial features (eyes, nose, mouth) may vary in their contribution to face identity perception. Changes to whole faces are easier to detect than changes to individual features, and eye changes are typically easier to detect than mouth changes, which in turn are easier to detect than nose changes. However, how this differs for people with face recognition difficulties (developmental prosopagnosia; DP) and for individuals with superior face recognition abilities (super-recognisers; SR) is not clear; although findings from previous studies have suggested differences, the nature of this difference is not understood. The aim of this study was to examine whether differences in the ability to detect feature changes in DPs and SRs were (a) quantitative, meaning that the pattern across feature changes remained the same but there was an overall upwards or downwards shift in performance, or (b) qualitative, meaning that the pattern across feature changes was different. Using a change detection task in which individual face features (eyes, nose, mouth) changed between sequentially presented faces, we found that while prosopagnosics showed a quantitative difference in performance with a downwards shift across all conditions, super-recognisers only showed qualitative differences: they were better able to detect when the face was the same and were marginally (but not non-significantly) worse at detecting when the eyes changed. Further, the only condition which distinguished between the three groups was the ability to identify when the same face was presented, with SRs being better than controls, and controls being better than DPs. Our findings suggest that, in feature-matching tasks, differences for DPs are due to them being overall worse at the task, while SRs use a qualitatively different strategy.

The relation between holistic processing as measured by three composite tasks and face processing: A latent variable modeling approach

We investigated the relationship between holistic processing and face processing using a latent variables approach. Three versions of the composite paradigm were used to measure holistic processing: Vanderbilt Holistic Face Processing Test, a sequential composite matching task, and a simultaneous composite matching task. Three tasks were used to measure face perception and face memory abilities respectively. We had three pairs of tasks such that within each pair (of memory and perception task), the stimuli involved, the requirement for matching across viewpoints, etc., are the same, such that the only difference is whether perception or memory is taxed. There were no significant correlations between the different versions of the composite task. We discovered no evidence to support a distinction between face perception and face memory, suggesting the existence of a general face processing factor. Finally, there was no evidence that holistic processing (as captured by either of the three composite tasks) is predictive of better face processing per se, casting doubts on the role of holistic processing in differentiating different levels of efficiency in face processing.

The influence of familiarity on memory for faces and mask wearing

During the COVID-19 pandemic, the wearing of face masks became mandatory in public areas or at workplaces in many countries. While offering protection, the coverage of large parts of our face (nose, mouth and chin) may have consequences for face recognition. This seems especially important in the context of contact tracing which can require memory of familiar and unfamiliar contacts and whether they were wearing a mask. In this study, we tested how well participants perform at remembering faces and whether they wore a mask, and if this depends on familiarity. Our results show that: (a) Participants remembered familiar faces better than unfamiliar ones, regardless of mask wearing. (b) Participants remembered unmasked faces better than masked faces, regardless of familiarity. (c) Participants were significantly worse at remembering whether an unfamiliar face was wearing a mask or not-even if they indicated remembering the face. (d) Participants showed a bias to indicate no memory of unfamiliar faces. (e) Participants showed a bias to indicate that unfamiliar faces wore a mask, even if they did not. In sum, it was harder to remember both, the identity of unfamiliar faces and whether they wore a mask. These findings have practical relevance for contact tracing and epidemic control.

Face processing in police service: the relationship between laboratory-based assessment of face processing abilities and performance in a real-world identity matching task

In the present study, we investigated whether police officers’ performance in searching for unfamiliar faces in a video-based real-world task is predicted by laboratory-based face processing tests that are typically used to assess individual differences in face processing abilities. Specifically, perceptual performance in the field was operationalized via the identification of target individuals in self-made close-circuit television (CCTV) video tapes. Police officers’ abilities in the laboratory were measured by the Cambridge Face Memory Test long form (CFMT+). We hypothesized that the CFMT+ predicts individual differences in the CCTV task performance. A total of N = 186 police officers of the Rhineland-Palatinate State Police participated in the study (i.e., N = 139 novice and advanced cadets with either 3 months, 15 months or 24 months of pre-service experience; N = 47 experienced police officers with three years of pre-service experience and at least two years of full-service experience, who participated in the assessment center of the special police forces, specifically the surveillance and technical unit). Results revealed that the CFMT+ explained variance in the CCTV task. In sample 1, CFMT+ scores predicted hits, but not false alarms. In contrast, in sample 2, CFMT+ scores were correlated with both hits and false alarms. From a theoretical perspective, we discuss factors that might explain CCTV task performance. From a practical perspective, we recommend that personnel selection processes investigating individual differences of police officers’ face processing abilities should comprise of two steps. At first, laboratory-based tests of face processing abilities should be applied. Subsequently, to validate laboratory-based individual differences in face processing abilities, we recommend that work samples such as CCTV tasks from the field should be added.

Psychophysical profiles in super-recognizers

Facial identity matching ability varies widely, ranging from prosopagnosic individuals (who exhibit profound impairments in face cognition/processing) to so-called super-recognizers (SRs), possessing exceptional capacities. Yet, despite the often consequential nature of face matching decisions—such as identity verification in security critical settings—ability assessments tendentially rely on simple performance metrics on a handful of heterogeneously related subprocesses, or in some cases only a single measured subprocess. Unfortunately, methodologies of this ilk leave contributions of stimulus information to observed variations in ability largely un(der)specified. Moreover, they are inadequate for addressing the qualitative or quantitative nature of differences between SRs’ abilities and those of the general population. Here, therefore, we sought to investigate individual differences—among SRs identified using a novel conservative diagnostic framework, and neurotypical controls—by systematically varying retinal availability, bandwidth, and orientation of faces’ spatial frequency content in two face matching experiments. Psychophysical evaluations of these parameters’ contributions to ability reveal that SRs more consistently exploit the same spatial frequency information, rather than suggesting qualitatively different profiles between control observers and SRs. These findings stress the importance of optimizing procedures for SR identification, for example by including measures quantifying the consistency of individuals’ behavior.

A quantitative meta-analysis of face recognition deficits in autism: 40 years of research

The ability to recognize an individual face is essential to human social interaction. Even subtle errors in this process can have huge implications for the way we relate to social partners. Because autism spectrum disorder (ASD) is characterized by deficits in social interaction, researchers have theorized about the potential role of atypical face identity processing to the symptom profile of ASD for more than 40 years. We conducted an empirical meta-analysis of this large literature to determine whether and to what extent face identity processing is atypical in ASD compared to typically developing (TD) individuals. We also tested the hypotheses that the deficit is selective to face identity recognition, not perception, and that methodological variation across studies moderates the magnitude of the estimated deficit. We identified 112 studies (5,390 participants) that generated 172 effect sizes from both recognition (k = 119) and discrimination (k = 53) paradigms. We used state-of-the-art approaches for assessing the validity and robustness of the analyses. We found comparable and large deficits in ASD for both face identity recognition (Hedge's g = -0.86) and discrimination (Hedge's g = -0.82). This means that the score of an average ASD individual is nearly 1 SD below the average TD individual on tasks assessing both aspects of face identity processing. These deficits generalize across age groups, sex, IQ scores, and task paradigms. These findings suggest that deficits in face identity processing may represent a core deficit in ASD. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Behavioural and cognitive mechanisms of Developmental Topographical Disorientation

Individuals affected by Developmental Topographical Disorientation (DTD) get lost on a daily basis, even in the most familiar of surroundings such as their neighbourhood, the building where they have worked for many years, and, in extreme cases, even in their own homes. Individuals with DTD report a lifelong selective inability to orient despite otherwise well-preserved general cognitive functions, and the absence of any acquired brain injury or neurological condition, with general intelligence reported to be within the normal range. To date, the mechanisms underlying such a selective developmental condition remain unknown. Here, we report the findings of a 10-year-long study investigating the behavioural and cognitive mechanisms of DTD in a large sample of 1211 cases. We describe the demographics, heritability pattern, self-reported and objective spatial abilities, and some personality traits of individuals with DTD as compared to a sample of 1624 healthy controls; importantly, we test the specific hypothesis that the presence of DTD is significantly related to the inability of the individuals to form a mental representation of the spatial surroundings (i.e., a cognitive map). We found that individuals with DTD reported relatively greater levels of neuroticism and negative affect, and rated themselves more poorly on self-report measures of memory and imagery skills related to objects, faces, and places. While performing interactive tasks, as a group, the individuals with DTD performed slightly worse on a scene-based perspective-taking task, and, notably struggled to solve tasks that demand the generation and use of a cognitive map. These novel findings help define the phenotype of DTD, and lay the foundation for future studies of the neurological and genetic mechanisms of this lifelong condition.

Birthweight predicts individual differences in adult face recognition ability

It has long been known that premature birth and/or low birthweight can lead to general difficulties in cognitive and emotional functioning throughout childhood. However, the influence of these factors on more specific processes has seldom been addressed, despite their potential to account for wide individual differences in performance that often appear innate. Here, we examined the influence of gestation and birthweight on adults' face perception and face memory skills. Performance on both sub-processes was predicted by birthweight and birthweight-for-gestation, but not gestation alone. Evidence was also found for the domain-specificity of these effects: No perinatal measure correlated with performance on object perception or memory tasks, but they were related to the size of the face inversion effect on the perceptual test. This evidence indicates a novel, very early influence on individual differences in face recognition ability, which persists into adulthood, influences face-processing strategy itself, and may be domain-specific.

Body Language Influences on Facial Identification at Passport Control: An Exploration in Virtual Reality

Person identification at airports requires the matching of a passport photograph to its bearer. One aim of this process is to find identity impostors, who use valid identity documents of similar-looking people to avoid detection. In psychology, this process has been studied extensively with static pairs of face photographs that require identity match (same person shown) versus mismatch (two different people) decisions. However, this approach provides a limited proxy for studying how other factors, such as nonverbal behaviour, affect this task. The current study investigated the influence of body language on facial identity matching within a virtual reality airport environment, by manipulating activity levels of person avatars queueing at passport control. In a series of six experiments, detection of identity mismatches was unaffected when observers were not instructed to utilise body language. By contrast, under explicit instruction to look out for unusual body language, these cues enhanced detection of mismatches but also increased false classification of matches. This effect was driven by increased activity levels rather than body language that simply differed from the behaviour of the majority of passengers. The implications and limitations of these findings are discussed.

Differences between and within individuals, and subprocesses of face cognition: implications for theory, research and personnel selection

Recent investigations of individual differences have demonstrated striking variability in performance both within the same subprocess in face cognition (e.g. face perception), but also between two different subprocesses (i.e. face perception versus face recognition) that are assessed using different tasks (face matching versus face memory). Such differences between and within individuals between and within laboratory tests raise practical challenges. This applies in particular to the development of screening tests for the selection of personnel in real-world settings where faces are routinely processed, such as at passport control. The aim of this study, therefore, was to examine the performance profiles of individuals within and across two different subprocesses of face cognition: face perception and face recognition. To this end, 146 individuals completed four different tests of face matching-one novel tool for assessing proficiency in face perception, as well as three established measures-and two benchmark tests of face memory probing face recognition. In addition to correlational analyses, we further scrutinized individual performance profiles of the highest and lowest performing observers identified per test, as well as across all tests. Overall, a number of correlations emerged between tests. However, there was limited evidence at the individual level to suggest that high proficiency in one test generalized to other tests measuring the same subprocess, as well as those that measured a different subprocess. Beyond emphasizing the need to honour inter-individual differences through careful multivariate assessment in the laboratory, our findings have real-world implications: combinations of tests that most accurately map the task(s) and processes of interest are required for personnel selection.

Normative data for two challenging tests of face matching under ecological conditions

Background

Unfamiliar face processing is an ability that varies considerably between individuals. Numerous studies have aimed to identify its underlying determinants using controlled experimental procedures. While such tests can isolate variables that influence face processing, they usually involve somewhat unrealistic situations and optimized face images as stimulus material. As a consequence, the extent to which the performance observed under laboratory settings is informative for predicting real-life proficiency remains unclear.

Results

We present normative data for two ecologically valid but underused tests of face matching: the Yearbook Test (YBT) and the Facial Identity Card Sorting Test (FICST). The YBT (n = 252) measures identity matching across substantial age-related changes in facial appearance, while the FICST (n = 218) assesses the ability to process unfamiliar facial identity despite superficial image variations. To determine the predictive value of both tests, a subsample of our cohort (n = 181) also completed a commonly used test of face recognition and two tests of face perception (the long form of the Cambridge Face Memory Test (CFMT+), the Expertise in Facial Comparison Test (EFCT) and the Person Identification Challenge Test (PICT)).

Conclusions

Focusing on the top performers identified independently per test, we made two important observations: 1) YBT and FICST performance can predict CFMT+ scores and vice versa; and 2) EFCT and PICT scores neither reliably predict superior performance in ecologically meaningful and challenging tests of face matching, nor in the most commonly used test of face recognition. These findings emphasize the necessity for using challenging and ecologically relevant, and thus highly sensitive, tasks of unfamiliar face processing to identify high-performing individuals in the normal population.

Caucasian Infants' Attentional Orienting to Own- and Other-Race Faces

Infants show preferential attention toward faces and detect faces embedded within complex naturalistic scenes. Newborn infants are insensitive to race, but rapidly develop differential processing of own- and other-race faces. In the present study, we investigated the development of attentional orienting toward own- and other-race faces embedded within naturalistic scenes. Infants aged six-, nine- and twelve-months did not show differences in the speed of orienting to own- and other race faces, but other-race faces held infants' visual attention for longer. We also found a clear developmental progression in attentional capture and holding, with older infants orienting to faces faster and fixating them for longer. Results are interpreted within the context of the two-process model of face processing.

Super-recognizers: From the lab to the world and back again

The recent discovery of individuals with superior face processing ability has sparked considerable interest amongst cognitive scientists and practitioners alike. These ‘Super‐recognizers’ (SRs) offer clues to the underlying processes responsible for high levels of face processing ability. It has been claimed that they can help make societies safer and fairer by improving accuracy of facial identity processing in real‐world tasks, for example when identifying suspects from Closed Circuit Television or performing security‐critical identity verification tasks. Here, we argue that the current understanding of superior face processing does not justify widespread interest in SR deployment: There are relatively few studies of SRs and no evidence that high accuracy on laboratory‐based tests translates directly to operational deployment. Using simulated data, we show that modest accuracy benefits can be expected from deploying SRs on the basis of ideally calibrated laboratory tests. Attaining more substantial benefits will require greater levels of communication and collaboration between psychologists and practitioners. We propose that translational and reverse‐translational approaches to knowledge development are critical to advance current understanding and to enable optimal deployment of SRs in society. Finally, we outline knowledge gaps that this approach can help address.

Person Identification from Drones by Humans: Insights from Cognitive Psychology

The deployment of unmanned aerial vehicles (i.e., drones) in military and police operations implies that drones can provide footage that is of sufficient quality to enable the recognition of strategic targets, criminal suspects, and missing persons. On the contrary, evidence from Cognitive Psychology suggests that such identity judgements by humans are already difficult under ideal conditions, and are even more challenging with drone surveillance footage. In this review, we outline the psychological literature on person identification for readers who are interested in the real-world application of drones. We specifically focus on factors that are likely to affect identification performance from drone-recorded footage, such as image quality, and additional person-related information from the body and gait. Based on this work, we suggest that person identification from drones is likely to be very challenging indeed, and that performance in laboratory settings is still very likely to underestimate the difficulty of this task in real-world settings.