Variability in photos of the same face

Similar use of shape and texture cues for own- and other-race faces during face learning and recognition

Although the other-race effect (ORE; superior recognition of own- relative to other-race faces) is well established, the mechanisms underlying it are not well understood. We examined whether the ORE is attributable to differential use of shape and texture cues for own- vs. other-race faces. Shape cues are particularly important for detecting that an own-race face is unfamiliar, whereas texture cues are more important for recognizing familiar and newly learned own-race faces. We compared the influence of shape and texture cues on Caucasian participants' recognition of Caucasian and East Asian faces using two complementary approaches. In Experiment 1, participants studied veridical, shape-caricatured, or texture-caricatured faces and then were asked to recognize them in an old/new recognition task. In Experiment 2, all study faces were veridical and we independently removed the diagnosticity of shape (or texture) cues in the test phase by replacing original shape (or texture) with average shape (or texture). Despite an overall own-race advantage, participants' use of shape and texture cues was comparable for own- and other-race faces. These results suggest that the other-race effect is not attributable to qualitative differences in the use of shape and texture cues.

Forgetting faces over a week: investigating self-reported face recognition ability and personality

Background

Although face recognition is now well studied, few researchers have considered the nature of forgetting over longer time periods. Here, I investigated how newly learned faces were recognised over the course of one week. In addition, I considered whether self-reported face recognition ability, as well as Big Five personality dimensions, were able to predict actual performance in a recognition task.

Methods

In this experiment (N = 570), faces were learned through short video interviews, and these identities were later presented in a recognition test (using previously unseen images) after no delay, six hours, twelve hours, one day, or seven days.

Results

The majority of forgetting took place within the first 24 hours, with no significant decrease after that timepoint. Further, self-reported face recognition abilities were moderately predictive of performance, while extraversion showed a small, negative association with performance. In both cases, these associations remained consistent across delay conditions.

Discussion

The current work begins to address important questions regarding face recognition using longitudinal, real-world time intervals, focussing on participant insight into their own abilities, and the process of forgetting more generally.

Unimodal and cross-modal identity judgements using an audio-visual sorting task: Evidence for independent processing of faces and voices

Unimodal and cross-modal information provided by faces and voices contribute to identity percepts. To examine how these sources of information interact, we devised a novel audio-visual sorting task in which participants were required to group video-only and audio-only clips into two identities. In a series of three experiments, we show that unimodal face and voice sorting were more accurate than cross-modal sorting: While face sorting was consistently most accurate followed by voice sorting, cross-modal sorting was at chancel level or below. In Experiment 1, we compared performance in our novel audio-visual sorting task to a traditional identity matching task, showing that unimodal and cross-modal identity perception were overall moderately more accurate than the traditional identity matching task. In Experiment 2, separating unimodal from cross-modal sorting led to small improvements in accuracy for unimodal sorting, but no change in cross-modal sorting performance. In Experiment 3, we explored the effect of minimal audio-visual training: Participants were shown a clip of the two identities in conversation prior to completing the sorting task. This led to small, nonsignificant improvements in accuracy for unimodal and cross-modal sorting. Our results indicate that unfamiliar face and voice perception operate relatively independently with no evidence of mutual benefit, suggesting that extracting reliable cross-modal identity information is challenging.

The sibling familiarity effect: Is within-person facial variability shared across siblings?

Humans are experts at familiar face recognition, but poor at unfamiliar face recognition. Familiarity is created when a face is encountered across varied conditions, but the way in which a person's appearance varies is identity-specific, so familiarity with one identity does not benefit recognition of other individuals. However, the faces of biological siblings share structural similarities, so we explored whether the benefits of familiarity are shared across siblings. Results show that familiarity with one half of a sibling pair improves kin detection (experiment 1), and that unfamiliar face matching is more accurate when targets are the siblings of familiar versus unfamiliar individuals (experiment 2). PCA applied to facial images of celebrities and their siblings demonstrates that faces are generally better reconstructed in the principal components of a same-sex sibling than those of an unrelated individual. When we encounter the unfamiliar sibling of someone we already know, our pre-existing representation of their familiar relation may usefully inform processing of the unfamiliar face. This can benefit both kin detection and identity processing, but the benefits are constrained by the degree to which facial variability is shared.

Getting to Know You: Emerging Neural Representations during Face Familiarization

The successful recognition of familiar persons is critical for social interactions. Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. In three EEG experiments on human participants of both sexes, we elucidated how representations of face familiarity and identity emerge from different qualities of familiarization: brief perceptual exposure (Experiment 1), extensive media familiarization (Experiment 2), and real-life personal familiarization (Experiment 3). Time-resolved representational similarity analysis revealed that familiarization quality has a profound impact on representations of face familiarity: they were strongly visible after personal familiarization, weaker after media familiarization, and absent after perceptual familiarization. Across all experiments, we found no enhancement of face identity representation, suggesting that familiarity and identity representations emerge independently during face familiarization. Our results emphasize the importance of extensive, real-life familiarization for the emergence of robust face familiarity representations, constraining models of face perception and recognition memory.SIGNIFICANCE STATEMENT Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. To elucidate how face representations change as we get familiar with someone, we conducted three EEG experiments where we used brief perceptual exposure, extensive media familiarization, or real-life personal familiarization. Using multivariate representational similarity analysis, we demonstrate that the method of familiarization has a profound impact on face representations, and emphasize the importance of real-life familiarization. Additionally, familiarization shapes representations of face familiarity and identity differently: as we get to know someone, familiarity signals seem to appear before the formation of identity representations.

Psychophysical profiles in super-recognizers

Facial identity matching ability varies widely, ranging from prosopagnosic individuals (who exhibit profound impairments in face cognition/processing) to so-called super-recognizers (SRs), possessing exceptional capacities. Yet, despite the often consequential nature of face matching decisions—such as identity verification in security critical settings—ability assessments tendentially rely on simple performance metrics on a handful of heterogeneously related subprocesses, or in some cases only a single measured subprocess. Unfortunately, methodologies of this ilk leave contributions of stimulus information to observed variations in ability largely un(der)specified. Moreover, they are inadequate for addressing the qualitative or quantitative nature of differences between SRs’ abilities and those of the general population. Here, therefore, we sought to investigate individual differences—among SRs identified using a novel conservative diagnostic framework, and neurotypical controls—by systematically varying retinal availability, bandwidth, and orientation of faces’ spatial frequency content in two face matching experiments. Psychophysical evaluations of these parameters’ contributions to ability reveal that SRs more consistently exploit the same spatial frequency information, rather than suggesting qualitatively different profiles between control observers and SRs. These findings stress the importance of optimizing procedures for SR identification, for example by including measures quantifying the consistency of individuals’ behavior.

Transforming faces to mimic natural kin: A comparison of different paradigms

The ability to detect phenotypic similarity or kinship in third-parties' faces is not perfect, but better than chance. Still, some humans are better than others at this task. Yet researchers in kinship detection have difficulties in building up large and diverse datasets of high-quality pictures of related persons. The current experiments tested a novel method for circumventing this difficulty by using morphing techniques in order to generate a wide array of stimuli derived from a limited number of individual pictures. Six experiments tested various stimuli (standard protocol, mirrored face, other-sex face, other-ethnicity face, other-expression face and antiface). Our benchmarks are the similarity or kinship scores achieved by participants when faced with pictures of real siblings. We show that all stimuli, except the antiface, elicit detection scores similar to those elicited by real pictures of actual siblings. In addition, by exploring different experiment parameters (simultaneous or sequential task, kinship or similarity task) and some individual characteristics, these experiments provide a better understanding of kinship detection in third parties. The validation of our new method will allow widening the range of available stimuli to the research community, and even to develop new ecologically relevant experimental protocols that are hardly or not feasible with veridical images.

Within-Person Variability in First Impressions From Faces

Perceptions of an individual can change dramatically across different images of their face. Questions remain as to whether some traits are more sensitive to image variability than others. To investigate this issue, we constructed a database of 340 naturalistic images consisting of 20 photos of 17 individuals. In this preregistered study, 95 participants rated all 340 images on one of three traits: trustworthiness, dominance, or attractiveness. Across images, participants' trustworthiness ratings tended to vary more than dominance, which in turn varied more than attractiveness; however, the relative differences between traits depended on the identity in question. Importantly, despite the variability in ratings within identities, there were substantial differences between individuals, suggesting that these trait judgements are based to some degree on relatively invariant facial characteristics. We found greater between-identity variability for attractiveness judgements compared to trustworthiness and dominance. Future research should further investigate the extent to which each trait dimension is tied to the identity of the faces.

Chinese Aesthetic Mask: Three Forehead and Five Eyes-Holistic Processing and Facial Attractiveness

Human face processing has been attributed to holistic processing. Here, we ask whether humans are sensitive to configural information when perceiving facial attractiveness. By referring to a traditional Chinese aesthetic theory-Three Forehead and Five Eyes-we generated a series of faces that differed in spacing between facial features. We adopted a two-alternative forced-choice task in Experiment 1 and a rating task in Experiment 2 to assess attractiveness. Both tasks showed a consistent result: The faces which fit the Chinese aesthetic theory were chosen or rated as most attractive. This effect of configural information on facial attractiveness was larger for faces with highly attractive features than for faces with low attractive features. These findings provide experimental evidence for the traditional Chinese aesthetic theory. This issue can be further explored from the perspective of culture in the future.

Two Factors in Face Recognition: Whether You Know the Person's Face and Whether You Share the Person's Race

One of the best-known phenomena in face recognition is the other-race effect, the observation that own-race faces are better remembered than other-race faces. However, previous studies have not put the magnitude of other-race effect in the context of other influences on face recognition. Here, we compared the effects of (a) a race manipulation (own-race/other-race face) and (b) a familiarity manipulation (familiar/unfamiliar face) in a 2 × 2 factorial design. We found that the familiarity effect was several times larger than the race effect in all performance measures. However, participants expected race to have a larger effect on others than it actually did. Face recognition accuracy depends much more on whether you know the person’s face than whether you share the same race.

Digital Detectives: Websleuthing Reduces Eyewitness Identification Accuracy in Police Lineups

Eyewitnesses to crimes sometimes search for a culprit on social media before viewing a police lineup, but it is not known whether this affects subsequent lineup identification accuracy. The present online study was conducted to address this. Two hundred and eighty-five participants viewed a mock crime video, and after a 15–20 min delay either (i) viewed a mock social media site including the culprit, (ii) viewed a mock social media site including a lookalike, or (iii) completed a filler task. A week later, participants made an identification from a photo lineup. It was predicted that searching for a culprit on social media containing the lookalike (rather than the culprit) would reduce lineup identification accuracy. There was a significant association between social media exposure and lineup accuracy for the Target Present lineup (30% more of the participants who saw the lookalike on social media failed to positively identify the culprit than participants in the other conditions), but for the Target Absent lineup (which also included the lookalike) there was no significant association with lineup identification accuracy. The results suggest that if an eyewitness sees a lookalike (where they are expecting to see the culprit) when conducting a self-directed search on social media, they are less likely to subsequently identify the culprit in the formal ID procedure.

Face Morphing, a Modern Threat to Border Security: Recent Advances and Open Challenges

Face morphing poses a serious threat to Automatic Border Control (ABC) and Face Recognition Systems (FRS) in general. The aim of this paper is to present a qualitative assessment of the morphing attack issue, and the challenges it entails, highlighting both the technological and human aspects of the problem. Here, after the face morphing attack scenario is presented, the paper provides an overview of the relevant bibliography and recent advances towards two central directions. First, the morphing of face images is outlined with a particular focus on the three main steps that are involved in the process, namely, landmark detection, face alignment and blending. Second, the detection of morphing attacks is presented under the prism of the so-called on-line and off-line detection scenarios and whether the proposed techniques employ handcrafted features, using classical methods, or automatically generated features, using deep-learning-based methods. The paper, then, presents the evaluation metrics that are employed in the corresponding bibliography and concludes with a discussion on open challenges that need to be address for further advancing automatic detection of morphing attacks. Despite the progress being made, the general consensus of the research community is that significant effort and resources are needed in the near future for the mitigation of the issue, especially, towards the creation of datasets capturing the full extent of the problem at hand and the availability of reference evaluation procedures for comparing novel automatic attack detection algorithms.

Performance of typical and superior face recognizers on a novel interactive face matching procedure

Unfamiliar simultaneous face matching is error prone. Reducing incorrect identification decisions will positively benefit forensic and security contexts. The absence of view-independent information in static images likely contributes to the difficulty of unfamiliar face matching. We tested whether a novel interactive viewing procedure that provides the user with 3D structural information as they rotate a facial image to different orientations would improve face matching accuracy. We tested the performance of 'typical' (Experiment 1) and 'superior' (Experiment 2) face recognizers, comparing their performance using high-quality (Experiment 3) and pixelated (Experiment 4) Facebook profile images. In each trial, participants responded whether two images featured the same person with one of these images being either a static face, a video providing orientation information, or an interactive image. Taken together, the results show that fluid orientation information and interactivity prompt shifts in criterion and support matching performance. Because typical and superior face recognizers both benefited from the structural information provided by the novel viewing procedures, our results point to qualitatively similar reliance on pictorial encoding in these groups. This also suggests that interactive viewing tools can be valuable in assisting face matching in high-performing practitioner groups.

The effect of face masks and sunglasses on identity and expression recognition with super-recognizers and typical observers

Face masks present a new challenge to face identification (here matching) and emotion recognition in Western cultures. Here, we present the results of three experiments that test the effect of masks, and also the effect of sunglasses (an occlusion that individuals tend to have more experienced with) on (i) familiar face matching, (ii) unfamiliar face matching and (iii) emotion categorization. Occlusion reduced accuracy in all three tasks, with most errors in the mask condition; however, there was little difference in performance for faces in masks compared with faces in sunglasses. Super-recognizers, people who are highly skilled at matching unconcealed faces, were impaired by occlusion, but at the group level, performed with higher accuracy than controls on all tasks. Results inform psychology theory with implications for everyday interactions, security and policing in a mask-wearing society.

Multiple-image arrays in face matching tasks with and without memory

Previous research has shown that exposure to within-person variability facilitates face learning. A different body of work has examined potential benefits of providing multiple images in face matching tasks. Viewers are asked to judge whether a target face matches a single face image (as when checking photo-ID) or multiple face images of the same person. The evidence here is less clear, with some studies finding a small multiple-image benefit, and others finding no advantage. In four experiments, we address this discrepancy in the benefits of multiple images from learning and matching studies. We show that multiple-image arrays only facilitate face matching when arrays precede targets. Unlike simultaneous face matching tasks, sequential matching and learning tasks involve memory and require abstraction of a stable representation of the face from the array, for subsequent comparison with a target. Our results show that benefits from multiple-image arrays occur only when this abstraction is required, and not when array and target images are available at once. These studies reconcile apparent differences between face learning and face matching and provide a theoretical framework for the study of within-person variability in face perception.

Seeing through disguise: Getting to know you with a deep convolutional neural network

People use disguise to look unlike themselves (evasion) or to look like someone else (impersonation). Evasion disguise challenges human ability to see an identity across variable images; Impersonation challenges human ability to tell people apart. Personal familiarity with an individual face helps humans to see through disguise. Here we propose a model of familiarity based on high-level visual learning mechanisms that we tested using a deep convolutional neural network (DCNN) trained for face identification. DCNNs generate a face space in which identities and images co-exist in a unified computational framework, that is categorically structured around identity, rather than retinotopy. This allows for simultaneous manipulation of mechanisms that contrast identities and cluster images. In Experiment 1, we measured the DCNN's baseline accuracy (unfamiliar condition) for identification of faces in no disguise and disguise conditions. Disguise affected DCNN performance in much the same way it affects human performance for unfamiliar faces in disguise (cf. Noyes & Jenkins, 2019). In Experiment 2, we simulated familiarity for individual identities by averaging the DCNN-generated representations from multiple images of each identity. Averaging improved DCNN recognition of faces in evasion disguise, but reduced the ability of the DCNN to differentiate identities of similar appearance. In Experiment 3, we implemented a contrast learning technique to simultaneously teach the DCNN appearance variation and identity contrasts between different individuals. This facilitated identification with both evasion and impersonation disguise. Familiar face recognition requires an ability to group images of the same identity together and separate different identities. The deep network provides a high-level visual representation for face recognition that supports both of these mechanisms of face learning simultaneously.

The N170 is Sensitive to Long-term (Personal) Familiarity of a Face Identity

The N170 is a large deflection of the human electroencephalogram (EEG), peaking at about 170 milliseconds over the occipito-temporal cortex after the sudden onset of a face stimulus. The N170 reflects perceptual awareness of a face and its onset corresponds to the emergence of reliable face-selectivity in the human brain. However, whether sensitivity to the long-term familiarity of a face identity emerges already at this early time-point remains debated. Here we provide a brief survey of the 45 published studies comparing the N170 response to unfamiliar and familiar (famous, experimentally familiarized, personally familiar and own) faces. Even though effects of familiarity on the N170 are relatively small and inconsistent across studies, this overview indicates that face familiarity significantly increases the N170 amplitude. This effect is especially present for personally familiar faces, learned in natural conditions. In the human brain, effects linked to familiarity with specific facial identities therefore appear to emerge between 150 and 200 ms in occipito-temporal brain regions, i.e., shortly after the onset of face-selectivity but at the same time as the earliest high-level effects of immediate unfamiliar face identity repetition. This observation challenges standard neurocognitive models with a clear-cut distinction between perceptual and memory stages in human face recognition.

FFA and OFA Encode Distinct Types of Face Identity Information

Faces of different people elicit distinct fMRI patterns in several face-selective regions of the human brain. Here we used representational similarity analysis to investigate what type of identity-distinguishing information is encoded in three face-selective regions: fusiform face area (FFA), occipital face area (OFA), and posterior superior temporal sulcus (pSTS). In a sample of 30 human participants (22 females, 8 males), we used fMRI to measure brain activity patterns elicited by naturalistic videos of famous face identities, and compared their representational distances in each region with models of the differences between identities. We built diverse candidate models, ranging from low-level image-computable properties (pixel-wise, GIST, and Gabor-Jet dissimilarities), through higher-level image-computable descriptions (OpenFace deep neural network, trained to cluster faces by identity), to complex human-rated properties (perceived similarity, social traits, and gender). We found marked differences in the information represented by the FFA and OFA. Dissimilarities between face identities in FFA were accounted for by differences in perceived similarity, Social Traits, Gender, and by the OpenFace network. In contrast, representational distances in OFA were mainly driven by differences in low-level image-based properties (pixel-wise and Gabor-Jet dissimilarities). Our results suggest that, although FFA and OFA can both discriminate between identities, the FFA representation is further removed from the image, encoding higher-level perceptual and social face information.SIGNIFICANCE STATEMENT Recent studies using fMRI have shown that several face-responsive brain regions can distinguish between different face identities. It is however unclear whether these different face-responsive regions distinguish between identities in similar or different ways. We used representational similarity analysis to investigate the computations within three brain regions in response to naturalistically varying videos of face identities. Our results revealed that two regions, the fusiform face area and the occipital face area, encode distinct identity information about faces. Although identity can be decoded from both regions, identity representations in fusiform face area primarily contained information about social traits, gender, and high-level visual features, whereas occipital face area primarily represented lower-level image features.

Contrast Negation Impairs Sorting Unfamiliar Faces by Identity: A Comparison With Original (Contrast-Positive) and Stretched Images

Recognition of unfamiliar faces is difficult in part due to variations in expressions, angles, and image quality. Studies suggest shape and surface properties play varied roles in face learning, and identification of unfamiliar faces uses diagnostic pigmentation/surface reflectance relative to shape information. Here, participants sorted photo-cards of unfamiliar faces by identity, which were shown in their original, stretched, and contrast-negated forms, to examine the utility of diagnostic shape and surface properties in sorting unfamiliar faces by identity. In four experiments, we varied the presentation order of conditions (contrast-negated first or original first with stretched second across experiments) and whether the same or different photo-cards were seen across conditions. Stretching the images did not impair performance in any measures relative to other conditions. Contrast negation generally exacerbated poor sorting by identity compared with the other conditions. However, seeing the contrast-negated photo-cards last mitigated some of the effects of contrast negation. Together, results suggest an important role for surface properties such as pigmentation and reflectance for sorting by identity and add to literatures on informational content and appearance variability in discrimination of facial identity.

Understanding face identification through within-person variability in appearance: Introduction to a virtual special issue

In the effort to determine the cognitive processes underlying the identification of faces, the dissimilarities between images of different people have long been studied. In contrast, the inherent variability between different images of the same face has either been treated as a nuisance variable that should be eliminated from psychological experiments or it has not been considered at all. Over the past decade, research efforts have increased substantially to demonstrate that this within-person variation is meaningful and can give insight into various processes of face identification, such as identity matching, face learning, and familiar face recognition. In this virtual special issue of the Quarterly Journal of Experimental Psychology, we explain the importance of within-person variability for face identification and bring together recent relevant articles published in the journal.

Getting to Know Someone: Familiarity, Person Recognition, and Identification in the Human Brain

In our everyday life, we continuously get to know people, dominantly through their faces. Several neuroscientific experiments showed that familiarization changes the behavioral processing and underlying neural representation of faces of others. Here, we propose a model of the process of how we actually get to know someone. First, the purely visual familiarization of unfamiliar faces occurs. Second, the accumulation of associated, nonsensory information refines person representation, and finally, one reaches a stage where the effortless identification of very well-known persons occurs. We offer here an overview of neuroimaging studies, first evaluating how and in what ways the processing of unfamiliar and familiar faces differs and, second, by analyzing the fMRI adaptation and multivariate pattern analysis results we estimate where identity-specific representation is found in the brain. The available neuroimaging data suggest that different aspects of the information emerge gradually as one gets more and more familiar with a person within the same network. We propose a novel model of familiarity and identity processing, where the differential activation of long-term memory and emotion processing areas is essential for correct identification.

Surgical face masks impair human face matching performance for familiar and unfamiliar faces

In response to the COVID-19 pandemic, many governments around the world now recommend, or require, that their citizens cover the lower half of their face in public. Consequently, many people now wear surgical face masks in public. We investigated whether surgical face masks affected the performance of human observers, and a state-of-the-art face recognition system, on tasks of perceptual face matching. Participants judged whether two simultaneously presented face photographs showed the same person or two different people. We superimposed images of surgical masks over the faces, creating three different mask conditions: control (no masks), mixed (one face wearing a mask), and masked (both faces wearing masks). We found that surgical face masks have a large detrimental effect on human face matching performance, and that the degree of impairment is the same regardless of whether one or both faces in each pair are masked. Surprisingly, this impairment is similar in size for both familiar and unfamiliar faces. When matching masked faces, human observers are biased to reject unfamiliar faces as "mismatches" and to accept familiar faces as "matches". Finally, the face recognition system showed very high classification accuracy for control and masked stimuli, even though it had not been trained to recognise masked faces. However, accuracy fell markedly when one face was masked and the other was not. Our findings demonstrate that surgical face masks impair the ability of humans, and naïve face recognition systems, to perform perceptual face matching tasks. Identification decisions for masked faces should be treated with caution.

UNSW Face Test: A screening tool for super-recognizers

We present a new test-the UNSW Face Test (www.unswfacetest.com)-that has been specifically designed to screen for super-recognizers in large online cohorts and is available free for scientific use. Super-recognizers are people that demonstrate sustained performance in the very top percentiles in tests of face identification ability. Because they represent a small proportion of the population, screening large online cohorts is an important step in their initial recruitment, before confirmatory testing via standardized measures and more detailed cognitive testing. We provide normative data on the UNSW Face Test from 3 cohorts tested via the internet (combined n = 23,902) and 2 cohorts tested in our lab (combined n = 182). The UNSW Face Test: (i) captures both identification memory and perceptual matching, as confirmed by correlations with existing tests of these abilities; (ii) captures face-specific perceptual and memorial abilities, as confirmed by non-significant correlations with non-face object processing tasks; (iii) enables researchers to apply stricter selection criteria than other available tests, which boosts the average accuracy of the individuals selected in subsequent testing. Together, these properties make the test uniquely suited to screening for super-recognizers in large online cohorts.

May I Speak Freely? The Difficulty in Vocal Identity Processing Across Free and Scripted Speech

In the fields of face recognition and voice recognition, a growing literature now suggests that the ability to recognize an individual despite changes from one instance to the next is a considerable challenge. The present paper reports on one experiment in the voice domain designed to determine whether a change in the mere style of speech may result in a measurable difficulty when trying to discriminate between speakers. Participants completed a speaker discrimination task to pairs of speech clips, which represented either free speech or scripted speech segments. The results suggested that speaker discrimination was significantly better when the style of speech did not change compared to when it did change, and was significantly better from scripted than from free speech segments. These results support the emergent body of evidence suggesting that within-identity variability is a challenge, and the forensic implications of such a mild change in speech style are discussed.

A data-driven characterisation of natural facial expressions when giving good and bad news

Facial expressions carry key information about an individual's emotional state. Research into the perception of facial emotions typically employs static images of a small number of artificially posed expressions taken under tightly controlled experimental conditions. However, such approaches risk missing potentially important facial signals and within-person variability in expressions. The extent to which patterns of emotional variance in such images resemble more natural ambient facial expressions remains unclear. Here we advance a novel protocol for eliciting natural expressions from dynamic faces, using a dimension of emotional valence as a test case. Subjects were video recorded while delivering either positive or negative news to camera, but were not instructed to deliberately or artificially pose any specific expressions or actions. A PCA-based active appearance model was used to capture the key dimensions of facial variance across frames. Linear discriminant analysis distinguished facial change determined by the emotional valence of the message, and this also generalised across subjects. By sampling along the discriminant dimension, and back-projecting into the image space, we extracted a behaviourally interpretable dimension of emotional valence. This dimension highlighted changes commonly represented in traditional face stimuli such as variation in the internal features of the face, but also key postural changes that would typically be controlled away such as a dipping versus raising of the head posture from negative to positive valences. These results highlight the importance of natural patterns of facial behaviour in emotional expressions, and demonstrate the efficacy of using data-driven approaches to study the representation of these cues by the perceptual system. The protocol and model described here could be readily extended to other emotional and non-emotional dimensions of facial variance.

Parental reference photos do not always improve the accuracy of forensic age progressions

During long-term missing children cases, forensic artists construct age-progressions to estimate the child's current appearance. It is commonly believed that incorporating information about the child's biological relatives is critical in accurately estimating the child's current appearance. However, some evidence suggests that predicting appearance based on inheritance of features may be error prone. The present studies examine whether age-progressions constructed with the aid of a biological reference photos led to better recognition than those constructed without a biological reference. We also investigated whether there would be any variation depending on the age-range of the age-progressions. Eight professional forensic artists created age-progressions based upon photographs provided by each of our eight targets. Half of their age progressions with the aid of parental reference photos and half without parental reference photos. Furthermore, half were age-progressed across a longer age-range (5-20 years) and half covered a shorter age-range (12-20 years). In Experiment 1 similarity scores were higher over shorter age-ranges. Further, across longer age-ranges age-progressions created with the aid of a parental reference were lower than those without a reference. In Experiment 2 recognition performance was higher across shorter age-ranges. Additionally, across longer age-ranges age-progressions created with the aid of a parental reference were recognized worse than those without a reference. These results suggest that in long-term missing person cases, forensic artists may benefit from not relying on biological references. Finally, consistent with previous research (e.g. Lampinen et al., 2012) age-progressions provided no benefit over using outdated photographs.

Comparing unfamiliar voice and face identity perception using identity sorting tasks

Identity sorting tasks, in which participants sort multiple naturally varying stimuli of usually two identities into perceived identities, have recently gained popularity in voice and face processing research. In both modalities, participants who are unfamiliar with the identities tend to perceive multiple stimuli of the same identity as different people and thus fail to "tell people together." These similarities across modalities suggest that modality-general mechanisms may underpin sorting behaviour. In this study, participants completed a voice sorting and a face sorting task. Taking an individual differences approach, we asked whether participants' performance on voice and face sorting of unfamiliar identities is correlated. Participants additionally completed a voice discrimination (Bangor Voice Matching Test) and a face discrimination task (Glasgow Face Matching Test). Using these tasks, we tested whether performance on sorting related to explicit identity discrimination. Performance on voice sorting and face sorting tasks was correlated, suggesting that common modality-general processes underpin these tasks. However, no significant correlations were found between sorting and discrimination performance, with the exception of significant relationships for performance on "same identity" trials with "telling people together" for voices and faces. Overall, any reported relationships were however relatively weak, suggesting the presence of additional modality-specific and task-specific processes.

Convolutional neural net face recognition works in non-human-like ways

Convolutional neural networks (CNNs) give the state-of-the-art performance in many pattern recognition problems but can be fooled by carefully crafted patterns of noise. We report that CNN face recognition systems also make surprising 'errors'. We tested six commercial face recognition CNNs and found that they outperform typical human participants on standard face-matching tasks. However, they also declare matches that humans would not, where one image from the pair has been transformed to appear a different sex or race. This is not due to poor performance; the best CNNs perform almost perfectly on the human face-matching tasks, but also declare the most matches for faces of a different apparent race or sex. Although differing on the salience of sex and race, humans and computer systems are not working in completely different ways. They tend to find the same pairs of images difficult, suggesting some agreement about the underlying similarity space.

Adults' facial impressions of children's niceness, but not shyness, show modest accuracy

Lay wisdom warns against "judging a book by its cover." However, facial first impressions influence people's behaviour towards others, so it is critical that we understand whether these impressions are at all accurate. Understanding impressions of children's faces is particularly important because these impressions can have social consequences during a crucial time of development. Here, we examined the accuracy of two traits that capture the most variance in impressions of children's faces, niceness and shyness. We collected face images and parental reports of actual niceness/shyness for 86 children (4-11 years old). Different images of the same person can lead to different impressions, and so we employed a novel approach by obtaining impressions from five images of each child. These images were ambient, representing the natural variability in faces. Adult strangers rated the faces for niceness (Study 1) or shyness (Study 2). Niceness impressions were modestly accurate for different images of the same child, regardless of whether these images were presented individually or simultaneously as a group. Shyness impressions were not accurate, for images presented either individually or as a group. Together, these results demonstrate modest accuracy in adults' impressions of niceness, but not shyness, from children's faces. Furthermore, our results reveal that this accuracy can be captured by images which contain natural face variability, and holds across different images of the same child's face. These results invite future research into the cues and causal mechanisms underlying this link between facial impressions of niceness and nice behaviour in children.

Differences between and within individuals, and subprocesses of face cognition: implications for theory, research and personnel selection

Recent investigations of individual differences have demonstrated striking variability in performance both within the same subprocess in face cognition (e.g. face perception), but also between two different subprocesses (i.e. face perception versus face recognition) that are assessed using different tasks (face matching versus face memory). Such differences between and within individuals between and within laboratory tests raise practical challenges. This applies in particular to the development of screening tests for the selection of personnel in real-world settings where faces are routinely processed, such as at passport control. The aim of this study, therefore, was to examine the performance profiles of individuals within and across two different subprocesses of face cognition: face perception and face recognition. To this end, 146 individuals completed four different tests of face matching-one novel tool for assessing proficiency in face perception, as well as three established measures-and two benchmark tests of face memory probing face recognition. In addition to correlational analyses, we further scrutinized individual performance profiles of the highest and lowest performing observers identified per test, as well as across all tests. Overall, a number of correlations emerged between tests. However, there was limited evidence at the individual level to suggest that high proficiency in one test generalized to other tests measuring the same subprocess, as well as those that measured a different subprocess. Beyond emphasizing the need to honour inter-individual differences through careful multivariate assessment in the laboratory, our findings have real-world implications: combinations of tests that most accurately map the task(s) and processes of interest are required for personnel selection.

How familiarity warps representation in the face space

Recognition of familiar as compared to unfamiliar faces is robust and resistant to marked image distortion or degradation. Here we tested the flexibility of familiar face recognition with a morphing paradigm where the appearance of a personally familiar face was mixed with the appearance of a stranger (Experiment 1) and the appearance of one's own face with the appearance of a familiar face and the appearance of a stranger (Experiment 2). The aim of the two experiments was to assess how categorical boundaries for recognition of identity are affected by familiarity. We found a narrower categorical boundary for the identity of personally familiar faces when they were mixed with unfamiliar identities as compared to the control condition, in which the appearance of two unfamiliar faces was mixed. Our results suggest that familiarity warps the representational geometry of face space, amplifying perceptual distances for small changes in the appearance of familiar faces that are inconsistent with the structural features that define their identities.

How does familiarity with a voice affect trait judgements?

From only a single spoken word, listeners can form a wealth of first impressions of a person's character traits and personality based on their voice. However, due to the substantial within-person variability in voices, these trait judgements are likely to be highly stimulus-dependent for unfamiliar voices: The same person may sound very trustworthy in one recording but less trustworthy in another. How trait judgements differ when listeners are familiar with a voice is unclear: Are listeners who are familiar with the voices as susceptible to the effects of within-person variability? Does the semantic knowledge listeners have about a familiar person influence their judgements? In the current study, we tested the effect of familiarity on listeners' trait judgements from variable voices across 3 experiments. Using a between-subjects design, we contrasted trait judgements by listeners who were familiar with a set of voices - either through laboratory-based training or through watching a TV show - with listeners who were unfamiliar with the voices. We predicted that familiarity with the voices would reduce variability in trait judgements for variable voice recordings from the same identity (cf. Mileva, Kramer & Burton, Perception, 48, 471 and 2019, for faces). However, across the 3 studies and two types of measures to assess variability, we found no compelling evidence to suggest that trait impressions were systematically affected by familiarity.

An objective and reliable electrophysiological marker for implicit trustworthiness perception

Trustworthiness is assumed to be processed implicitly from faces, despite the fact that the overwhelming majority of research has only involved explicit trustworthiness judgements. To answer the question whether or not trustworthiness processing can be implicit, we apply an electroencephalography fast periodic visual stimulation (FPVS) paradigm, where electrophysiological cortical activity is triggered in synchrony with facial trustworthiness cues, without explicit judgements. Face images were presented at 6 Hz, with facial trustworthiness varying at 1 Hz. Significant responses at 1 Hz were observed, indicating that differences in the trustworthiness of the faces were reflected in the neural signature. These responses were significantly reduced for inverted faces, suggesting that the results are associated with higher order face processing. The neural responses were reliable, and correlated with explicit trustworthiness judgements, suggesting that the technique is capable of picking up on stable individual differences in trustworthiness processing. By demonstrating neural activity associated with implicit trustworthiness judgements, our results contribute to resolving a key theoretical debate. Moreover, our data show that FPVS is a valuable tool to examine face processing at the individual level, with potential application in pre-verbal and clinical populations who struggle with verbalization, understanding or memory.