Face matching and self-insight A Registered Report investigating individual differences in metacognitive sensitivity, efficiency, and bias

Deciding whether two face photographs depict the same person or not can be a challenging task, and there are substantial individual differences in face matching ability. Far less is known about differences in metacognitive ability; that is, how well people can estimate the quality of their own face matching judgements. The purpose of this Registered Report was to determine the relationship between first-order performance in a face matching task, and three metacognitive measures: metacognitive sensitivity (the information exploited by metacognition), metacognitive efficiency (the quality of metacognitive processing itself), and metacognitive bias (the overall tendency towards high or low confidence). Participants completed a 200-trial unfamiliar face matching task, providing a second-order (metacognitive) confidence rating after each first-order (cognitive) response. Cognitive performance and metacognitive sensitivity showed a substantial positive association (rs = .60), suggesting that they were based on overlapping information, with poorer performers having lower quality information available. Once these differences in the information available to metacognition were accounted for, the quality of metacognitive processing itself (metacognitive efficiency) did not vary systematically with cognitive performance (rs = .00). Moreover, poor performers were not less or more confident overall than good performers, as cognitive performance and metacognitive bias did not correlate significantly (rs = .11). These findings allow us to move beyond consideration of group-level insight and inform regarding individual differences in cognitive and metacognitive abilities.

The Oxford Face Matching Test: Short-form alternative

A recently published test of face perception, the Oxford Face Matching Test, asks participants to make two judgements: whether two faces are of the same individual and how perceptually similar the two faces are. In this study, we sought to determine to what extent the test can be shortened by removing the perceptual similarity judgements and whether this affects test performance. In Experiment 1, participants completed two versions of the test, with and without similarity judgements, in separate sessions in counterbalanced order. The version without similarity judgements took approximately 40% less time to complete. Performance on the matching judgements did not differ across versions and the correlation in accuracy across the two versions was comparable with the originally reported test-retest reliability value. Experiment 2 validated the version without similarity judgements against other measures, demonstrating moderate relationships with other face matching, memory, and self-report face perception measures. These data indicate that a test version without the similarity judgements can substantially reduce administration time without affecting test performance.

Investigating people's metacognitive insight into their own face abilities

Within the domain of face processing, researchers have been interested in quantifying the relationship between objective (i.e., performance on laboratory tests of recognition and matching) and subjective measures of ability (typically, self-report questionnaires). Put simply, do people show high levels of metacognitive insight into their own abilities with faces? Although several studies have suggested that the association between these two types of measures may only be moderate, there remain several important issues that require consideration before this question can be sensibly investigated. First, specificity is needed regarding both objective and subjective measurements because both tend to span a wide range of potentially separable abilities. Second, experimental tasks appear to focus on different contexts to those tapped in self-report questionnaire items. Third, recent issues with statistical approaches and visualisation can result in numerical artefacts and misinterpretations. Finally, the sizes of population-level insights suggested by recent work provide only limited information regarding individuals within these populations, and so researchers aiming to identify people at the extremes of ability must be careful when drawing conclusions. Taken together, we argue that more attention to these issues is needed when attempting to investigate metacognitive insight within this domain.

Catching wanted people at the border: prospective person memory and face matching in border control decisions

Border control officers may be on the lookout for wanted people while they verify that travellers match their passport photos. We developed a novel experimental paradigm to investigate whether people are more likely to report that someone is wanted if they also believe that person is using a fraudulent passport. In two experiments, undergraduate students assumed the role of a border control officer and completed multiple "shifts" of a face matching task designed to simulate a passport verification check. Before each shift participants viewed posters of wanted people and were instructed to report any sightings if a wanted person appeared in any of the images during the passport check. Participants were more likely to say an individual was wanted if they also believed the person did not match their passport image. In addition, the accuracy of wanted person sightings was reduced for trials with nonmatching passports compared to trials with matching passports. This suggests wanted people with matching passports were easier to spot because participants had an additional image to compare with their memory of the person in the wanted poster.

Image variability and face matching

This study investigates whether variability in perceived trait judgements disrupts our ability to match unfamiliar faces. In this preregistered study, 174 participants completed a face matching task where they were asked to indicate whether two ambient face images belonged to the same person or different people (17,748 total data points). Participants completed 51 match trials consisting of images of the same person that differed substantially on one trait (either trustworthiness, dominance or attractiveness) with minimal differences in the alternate traits. Participants also completed 51 mismatch trials which contained two photos of similar-looking individuals. We hypothesised that participants would make more errors on match trials when images differed in terms of attractiveness ratings than when they differed on trustworthiness or dominance. Contrary to expectations, images that differed in terms of attractiveness were matched most accurately, and there was no relationship between the extent of differences in attractiveness ratings and accuracy. There was some evidence that differences in perceived dominance and, to a lesser extent, trustworthiness were associated with lower face matching performance. However, these relationships were not significant when alternate traits were accounted for. The findings of our study suggest that face matching performance is largely robust against variation in trait judgements.

EXPRESS: Distinct abilities associated with matching same identity faces vs. discriminating different faces: Evidence from individual differences in prosopagnosics and controls

Previous face matching studies provide evidence that matching same identity faces (match trials) and discriminating different face identities (non-match trials) rely on distinct processes. For example, instructional studies geared towards improving face matching in applied settings have often found selective improvements in match or non-match trials only. Additionally, a small study found that developmental prosopagnosics (DPs) have specific deficits in making match but not non-match judgments. In the current study, we sought to replicate this finding in DPs and examine how individual differences across DPs and controls in match vs. non-match performance relate to featural vs. holistic processing abilities. 43 DPs and 27 controls matched face images shown from similar front views or with varied lighting or viewpoint. Participants also performed tasks measuring featural (eyes/mouth) and holistic processing (part-whole task). We found that DPs showed worse overall matching performance than controls and that their relative match vs. non-match deficit depended on image variation condition, indicating that DPs do not consistently show match- or non-match-specific deficits. When examining the association between holistic and featural processing abilities and match vs. non-match trials in the entire group of DPs and controls, we found a very clear dissociation: Match trials significantly correlated with eye processing ability (r=.48) but not holistic processing (r=.11), whereas non-match trials significantly correlated with holistic processing (r=.32) but not eye processing (r=.03). This suggests that matching same identity faces relies more on eye processing while discriminating different faces relies more on holistic processing.

Metacognition during unfamiliar face matching

Kruger and Dunning (1999) described a metacognitive bias in which insight into performance is linked to competence: poorer performers are less aware of their mistakes than better performers. Competence-based insight has been argued to apply generally across task domains, including a recent report investigating social cognition using a variety of face-matching tasks. Problematically, serious statistical and methodological criticisms have been directed against the traditional method of analysis used by researchers in this field. Here, we further illustrate these issues and investigate new sources of insight within unfamiliar face matching. Over two experiments (total N = 1077), where Experiment 2 was a preregistered replication of the key findings from Experiment 1, we found that insight into performance was multi-faceted. Participants demonstrated insight which was not based on competence, in the form of accurate updating of estimated performance. We also found evidence of insight which was based on competence: the difference in confidence on correct versus incorrect trials increased with competence. By providing ways that we can move beyond problematic, traditional approaches, we have begun to reveal a more realistic story regarding the nature of insight into face perception.

Not so fast! Response times in the computerized Benton Face Recognition Test may not reflect face recognition ability

Response times (RTs) are commonly used to assess cognitive abilities, though it is unclear whether face processing RTs predict recognition ability beyond accuracy. In the current study, we examined accuracy and RT on a widely used face matching assessment modified to collect meaningful RT data, the computerized Benton Facial Recognition Test (BFRT-c), and measured whether RTs predicted face recognition ability and developmental prosopagnosia (DP) vs. control group membership. 62 controls and 36 DPs performed the BFRT-c as well as validated measures of face recognition ability: the Cambridge Face Memory Test (CFMT) and a Famous Faces Memory Test (FFMT). In controls, BFRT-c accuracy robustly predicted CFMT (r = .49, p < .001), FFMT (r = .43, p < .001), and a CFMT-FFMT composite (r = .54, p < .001), whereas BFRT-c RT was not significantly associated with these measures (all r's  .21). We also found that BFRT-c accuracy significantly differed between DPs and controls, but RT failed to differentiate the groups.

The Importance of Internal and External Features in Matching Own and Other Race Faces

Research has shown that we are better at discriminating between faces that are our own race, and much less accurate with faces of another race. When the external features of faces were removed, this reduced the accuracy for recognizing other-races faces, more than own-race faces, suggesting that the external features (hair, face shape) are especially important for the recognition of other-race faces. The aim of the current study was to determine whether external features were more useful in matching other-race faces, and whether this was the case for Western and Eastern viewers. The current study employed a face matching task with Caucasian (U.K.) and Asian (Chinese) participants and found that responses were more accurate for own-race faces, and for whole faces when compared with faces where the internal or external features had been removed. Removing the external features of other-race faces increased the own-race bias for Chinese and U.K. participants, demonstrating the importance of viewing whole faces, including the external features when matching other-race faces.

The sibling familiarity effect: Is within-person facial variability shared across siblings?

Humans are experts at familiar face recognition, but poor at unfamiliar face recognition. Familiarity is created when a face is encountered across varied conditions, but the way in which a person's appearance varies is identity-specific, so familiarity with one identity does not benefit recognition of other individuals. However, the faces of biological siblings share structural similarities, so we explored whether the benefits of familiarity are shared across siblings. Results show that familiarity with one half of a sibling pair improves kin detection (experiment 1), and that unfamiliar face matching is more accurate when targets are the siblings of familiar versus unfamiliar individuals (experiment 2). PCA applied to facial images of celebrities and their siblings demonstrates that faces are generally better reconstructed in the principal components of a same-sex sibling than those of an unrelated individual. When we encounter the unfamiliar sibling of someone we already know, our pre-existing representation of their familiar relation may usefully inform processing of the unfamiliar face. This can benefit both kin detection and identity processing, but the benefits are constrained by the degree to which facial variability is shared.

The effect of face masks and sunglasses on identity and expression recognition with super-recognizers and typical observers

Face masks present a new challenge to face identification (here matching) and emotion recognition in Western cultures. Here, we present the results of three experiments that test the effect of masks, and also the effect of sunglasses (an occlusion that individuals tend to have more experienced with) on (i) familiar face matching, (ii) unfamiliar face matching and (iii) emotion categorization. Occlusion reduced accuracy in all three tasks, with most errors in the mask condition; however, there was little difference in performance for faces in masks compared with faces in sunglasses. Super-recognizers, people who are highly skilled at matching unconcealed faces, were impaired by occlusion, but at the group level, performed with higher accuracy than controls on all tasks. Results inform psychology theory with implications for everyday interactions, security and policing in a mask-wearing society.

Body Language Influences on Facial Identification at Passport Control: An Exploration in Virtual Reality

Person identification at airports requires the matching of a passport photograph to its bearer. One aim of this process is to find identity impostors, who use valid identity documents of similar-looking people to avoid detection. In psychology, this process has been studied extensively with static pairs of face photographs that require identity match (same person shown) versus mismatch (two different people) decisions. However, this approach provides a limited proxy for studying how other factors, such as nonverbal behaviour, affect this task. The current study investigated the influence of body language on facial identity matching within a virtual reality airport environment, by manipulating activity levels of person avatars queueing at passport control. In a series of six experiments, detection of identity mismatches was unaffected when observers were not instructed to utilise body language. By contrast, under explicit instruction to look out for unusual body language, these cues enhanced detection of mismatches but also increased false classification of matches. This effect was driven by increased activity levels rather than body language that simply differed from the behaviour of the majority of passengers. The implications and limitations of these findings are discussed.

Enhanced Matching of Children's Faces in "Super-Recognisers" But Not High-Contact Controls

Face matching is notoriously error-prone, and some work suggests additional difficulty when matching the faces of children. It is possible that individuals with natural proficiencies in adult face matching ("super-recognisers" [SRs]) will also excel at the matching of children's faces, although other work implicates facilitations in typical perceivers who have high levels of contact with young children (e.g., nursery teachers). This study compared the performance of both of these groups on adult and child face matching to a group of low-contact controls. High- and low-contact control groups performed at a remarkably similar level in both tasks, whereas facilitations for adult and child face matching were observed in some (but not all) SRs. As a group, the SRs performed better in the adult compared with the child task, demonstrating an extended own-age bias compared with controls. These findings suggest that additional exposure to children's faces does not assist the performance in a face matching task, and the mechanisms underpinning superior recognition of adult faces can also facilitate the child face recognition. Real-world security organisations should therefore seek individuals with general facilitations in face matching for both adult and child face matching tasks.

Towards a 'manifesto' for super-recognizer research

This article provides a response to five excellent commentaries on our article 'Super-recognizers: From the lab to the world and back again'. Specifically, the response summarizes commonalities between these commentaries. Based on this consensus, we propose a flexible framework for the assessment of superior face recognition and outline guiding principles to advance future work in the field.

Face averages and multiple images in a live matching task

We know from previous research that unfamiliar face matching (determining whether two simultaneously presented images show the same person or not) is very error-prone. A small number of studies in laboratory settings have shown that the use of multiple images or a face average, rather than a single image, can improve face matching performance. Here, we tested 1,999 participants using four-image arrays and face averages in two separate live matching tasks. Matching a single image to a live person resulted in numerous errors (79.9% accuracy across both experiments), and neither multiple images (82.4% accuracy) nor face averages (76.9% accuracy) improved performance. These results are important when considering possible alterations which could be made to photo-ID. Although multiple images and face averages have produced measurable improvements in performance in recent laboratory studies, they do not produce benefits in a real-world live face matching context.

Super-recognizers: From the lab to the world and back again

The recent discovery of individuals with superior face processing ability has sparked considerable interest amongst cognitive scientists and practitioners alike. These ‘Super‐recognizers’ (SRs) offer clues to the underlying processes responsible for high levels of face processing ability. It has been claimed that they can help make societies safer and fairer by improving accuracy of facial identity processing in real‐world tasks, for example when identifying suspects from Closed Circuit Television or performing security‐critical identity verification tasks. Here, we argue that the current understanding of superior face processing does not justify widespread interest in SR deployment: There are relatively few studies of SRs and no evidence that high accuracy on laboratory‐based tests translates directly to operational deployment. Using simulated data, we show that modest accuracy benefits can be expected from deploying SRs on the basis of ideally calibrated laboratory tests. Attaining more substantial benefits will require greater levels of communication and collaboration between psychologists and practitioners. We propose that translational and reverse‐translational approaches to knowledge development are critical to advance current understanding and to enable optimal deployment of SRs in society. Finally, we outline knowledge gaps that this approach can help address.

Unfamiliar Face Matching With Driving Licence and Passport Photographs

Matching two different images of an unfamiliar face is difficult, although we rely on this process every day when proving our identity. Although previous work with laboratory photosets has shown that performance is error-prone, few studies have focussed on how accurately people carry out this matching task using photographs taken from official forms of identification. In Experiment 1, participants matched high-resolution, colour face photos with current UK driving licence photos of the same group of people in a sorting task. Averaging 19 mistaken pairings out of 30, our results showed that this task was both difficult and error-prone. In Experiment 2, high-resolution photographs were paired with either driving licence or passport photographs in a typical pairwise matching paradigm. We found no difference in performance levels for the two types of ID image, with both producing unacceptable levels of accuracy (around 75%-79% correct). The current work benefits from increased ecological validity and provides a clear demonstration that these forms of official identification are ineffective and alternatives should be considered.

Face Recognition Using the SR-CNN Model

In order to solve the problem of face recognition in complex environments being vulnerable to illumination change, object rotation, occlusion, and so on, which leads to the imprecision of target position, a face recognition algorithm with multi-feature fusion is proposed. This study presents a new robust face-matching method named SR-CNN, combining the rotation-invariant texture feature (RITF) vector, the scale-invariant feature transform (SIFT) vector, and the convolution neural network (CNN). Furthermore, a graphics processing unit (GPU) is used to parallelize the model for an optimal computational performance. The Labeled Faces in the Wild (LFW) database and self-collection face database were selected for experiments. It turns out that the true positive rate is improved by 10.97–13.24% and the acceleration ratio (the ratio between central processing unit (CPU) operation time and GPU time) is 5–6 times for the LFW face database. For the self-collection, the true positive rate increased by 12.65–15.31%, and the acceleration ratio improved by a factor of 6–7.

Smiles in face matching: Idiosyncratic information revealed through a smile improves unfamiliar face matching performance

Unfamiliar face matching is a surprisingly difficult task, yet we often rely on people's matching decisions in applied settings (e.g., border control). Most attempts to improve accuracy (including training and image manipulation) have had very limited success. In a series of studies, we demonstrate that using smiling rather than neutral pairs of images brings about significant improvements in face matching accuracy. This is true for both match and mismatch trials, implying that the information provided through a smile helps us detect images of the same identity as well as distinguishing between images of different identities. Study 1 compares matching performance when images in the face pair display either an open-mouth smile or a neutral expression. In Study 2, we add an intermediate level, closed-mouth smile, to identify the effect of teeth being exposed, and Study 3 explores face matching accuracy when only information about the lower part of the face is available. Results demonstrate that an open-mouth smile changes the face in an idiosyncratic way which aids face matching decisions. Such findings have practical implications for matching in the applied context where we typically use neutral images to represent ourselves in official documents.

Unfamiliar Face Matching With Frontal and Profile Views

Research has systematically examined how laboratory participants and real-world practitioners decide whether two face photographs show the same person or not using frontal images. In contrast, research has not examined face matching using profile images. In Experiment 1, we ask whether matching unfamiliar faces is easier with frontal compared with profile views. Participants completed the original, frontal version of the Glasgow Face Matching Test, and also an adapted version where all face pairs were presented in profile. There was no difference in performance across the two tasks, suggesting that both views were similarly useful for face matching. Experiments 2 and 3 examined whether matching unfamiliar faces is improved when both frontal and profile views are provided. We compared face matching accuracy when both a frontal and a profile image of each face were presented, with accuracy using each view alone. Surprisingly, we found no benefit when both views were presented together in either experiment. Overall, these results suggest that either frontal or profile views provide substantially overlapping information regarding identity or participants are unable to utilise both sources of information when making decisions. Each of these conclusions has important implications for face matching research and real-world identification development.

Effects of time pressure and time passage on face-matching accuracy

This study investigated the impact of time pressure on matching accuracy with face pairs that combined photographs from student ID cards with high-quality person portraits, and under conditions that provided infrequent identity mismatches. Time pressure was administered via two onscreen displays that observers could use to adjust the amount of time that was allocated to a given trial while completing a block of trials within a required timeframe. Under these conditions, observers matched faces under time pressure that varied from 10 to 2 s (Experiment 1) and 8 to 2 s (Experiment 2). An effect of time pressure was found in each experiment, whereby performance deteriorated under time targets of 4 s. Additionally, a match response bias emerged consistently across blocks, and indicated that separately to time pressure, performance also deteriorated due to time passage. These results therefore indicate that both time passage and pressure exert detrimental effects on face matching.

The Headscarf Effect Revisited: Further Evidence for a Culture-Based Internal Face Processing Advantage

Encoding the internal features of unfamiliar faces poses a perceptual challenge that occasionally results in face recognition errors. Extensive experience with faces framed by a headscarf may, however, enhance perceivers' ability to process internal facial information. To examine this claim empirically, participants in the United Arab Emirates and the United States of America completed a standard part-whole face recognition task. Accuracy on the task was examined using a 2 (perceiver culture: Emirati vs American) x 2 (face race: Arab vs white) x 2 (probe type: part vs whole) x 3 (probe feature: eyes vs nose vs mouth) mixed-measures analysis of variance. As predicted, Emiratis outperformed Americans on the administered task. Although their recognition advantage occurred regardless of probe type, it was most pronounced for Arab faces and for trials that captured the processing of nose or mouth information. The findings demonstrate that culture-based experiences hone perceivers' face processing skills.

Matching Faces Against the Clock

This study examined the effect of time pressure on face-matching accuracy. Across two experiments, observers decided whether pairs of faces depict one person or different people. Time pressure was exerted via two additional displays, which were constantly updated to inform observers on whether they were on track to meet or miss a time target. In this paradigm, faces were matched under increasing or decreasing (Experiment 1) and constant time pressure (Experiment 2), which varied from 10 to 2 seconds. In both experiments, time pressure reduced accuracy, but the point at which this declined varied from 8 to 2 seconds. A separate match response bias was found, which developed over the course of the experiments. These results indicate that both time pressure and the repetitive nature of face matching are detrimental to performance.

The 20 item prosopagnosia index (PI20): relationship with the Glasgow face-matching test

The 20 item prosopagnosia index (PI20) was recently developed to identify individuals with developmental prosopagnosia. While the PI20's principal purpose is to aid researchers and clinicians, it was suggested that it may serve as a useful screening tool to identify people with face recognition difficulties in applied settings where face matching is a critical part of their occupation. Although the PI20 has been validated using behavioural measures of face recognition, it has yet to be validated against a measure of face-matching ability that is more representative of applied settings. In this study, the PI20 was therefore administered with the Glasgow face-matching test (GFMT). A strong correlation was observed between PI20 and GFMT scores, providing further validation for the PI20, indicating that it is likely to be of value in applied settings.

Generalization across view in face memory and face matching

While a change in view is considered to be one of the most damaging manipulations for facial identification, this phenomenon has been measured traditionally with tasks that confound perceptual processes with recognition memory. This study explored facial identification with a pairwise matching task to determine whether view generalization is possible when memory factors are minimised. Experiment 1 showed that the detrimental view effect in recognition memory is attenuated in face matching. Moreover, analysis of individual differences revealed that some observers can identify faces across view with perfect accuracy. This was replicated in Experiment 2, which also showed that view generalization is unaffected when only the internal facial features are shown. These results indicate that the view effect in recognition memory does not arise from data limits, whereby faces contain insufficient visual information to allow identification across views. Instead, these findings point to resource limits, within observers, that hamper such person identification in recognition memory.

Matching faces with emotional expressions

There is some evidence that faces with a happy expression are recognized better than faces with other expressions. However, little is known about whether this happy-face advantage also applies to perceptual face matching, and whether similar differences exist among other expressions. Using a sequential matching paradigm, we systematically compared the effects of seven basic facial expressions on identity recognition. Identity matching was quickest when a pair of faces had an identical happy/sad/neutral expression, poorer when they had a fearful/surprise/angry expression, and poorest when they had a disgust expression. Faces with a happy/sad/fear/surprise expression were matched faster than those with an anger/disgust expression when the second face in a pair had a neutral expression. These results demonstrate that effects of facial expression on identity recognition are not limited to happy-faces when a learned face is immediately tested. The results suggest different influences of expression in perceptual matching and long-term recognition memory.