Variability in photos of the same face

Decomposing the composite face effect: Evidence for non-holistic processing based on the ex-Gaussian distribution

Composite faces fuse the top and bottom halves from two different faces to create a powerful illusion of a novel face. It has been argued that composite faces are processed holistically, namely that the constituent face parts are perceived as a template, rather than independent features. This study sought to uncover the locus of the composite face effect by relating its empirical reaction time distributions to theoretical ex-Gaussian parameters. The results showed that the composite face effect for unfamiliar (Experiment 1) and familiar (Experiment 2) faces is generated by pure changes in the exponential component of the ex-Gaussian distribution. This held true for both partial and complete design measures. The exponential component has been attributed to working memory and attentional processes. The results suggest the involvement of attentional and working memory processes in the composite face effect and in the perception of faces in general. They cast doubts on the holistic nature of face processing. The results also provide important constraints on future computational theories of the effect.

Normative data for two challenging tests of face matching under ecological conditions

Background

Unfamiliar face processing is an ability that varies considerably between individuals. Numerous studies have aimed to identify its underlying determinants using controlled experimental procedures. While such tests can isolate variables that influence face processing, they usually involve somewhat unrealistic situations and optimized face images as stimulus material. As a consequence, the extent to which the performance observed under laboratory settings is informative for predicting real-life proficiency remains unclear.

Results

We present normative data for two ecologically valid but underused tests of face matching: the Yearbook Test (YBT) and the Facial Identity Card Sorting Test (FICST). The YBT (n = 252) measures identity matching across substantial age-related changes in facial appearance, while the FICST (n = 218) assesses the ability to process unfamiliar facial identity despite superficial image variations. To determine the predictive value of both tests, a subsample of our cohort (n = 181) also completed a commonly used test of face recognition and two tests of face perception (the long form of the Cambridge Face Memory Test (CFMT+), the Expertise in Facial Comparison Test (EFCT) and the Person Identification Challenge Test (PICT)).

Conclusions

Focusing on the top performers identified independently per test, we made two important observations: 1) YBT and FICST performance can predict CFMT+ scores and vice versa; and 2) EFCT and PICT scores neither reliably predict superior performance in ecologically meaningful and challenging tests of face matching, nor in the most commonly used test of face recognition. These findings emphasize the necessity for using challenging and ecologically relevant, and thus highly sensitive, tasks of unfamiliar face processing to identify high-performing individuals in the normal population.

You shall not pass: how facial variability and feedback affect the detection of low-prevalence fake IDs

In many real-world settings, individuals rarely present another person's ID, which increases the likelihood that a screener will fail to detect it. Three experiments examined how within-person variability (i.e., differences between two images of the same person) and feedback may have influenced criterion shifting, thought to be one of the sources of the low-prevalence effect (LPE). Participants made identity judgments of a target face and an ID under either high, medium, or low mismatch prevalence. Feedback appeared after every trial, only error trials, or no trials. Experiment 1 used two controlled images taken on the same day. Experiment 2 used two controlled images taken at least 6 months apart. Experiment 3 used one controlled and one ambient image taken at least 1 year apart. Importantly, receiver operating characteristic curves revealed that feedback and greater within-person variability exacerbated the LPE by affecting both criterion and discriminability. These results carry implications for many real-world settings, such as border crossings and airports, where identity screening plays a major role in securing public safety.

Discrimination and recognition of faces with changed configuration

Subtle metric differences in facial configuration, such as between-person variation in the distances between the eyes, have been used widely in psychology to explain face recognition. However, these studies of configuration have typically utilized unfamiliar faces rather than the familiar faces that the process of recognition ultimately seeks to explain. This study investigates whether face recognition relies on the metric information presumed in configural theory, by manipulating the interocular distance in both unfamiliar and familiar faces. In Experiment 1, observers were asked to detect which face in a pair was presented with its configuration intact. In Experiment 2, this discrimination task was repeated with faces presented individually, and observers were also asked to make familiarity categorizations to the same stimuli. In both experiments, familiarity determined detection of faces in their original configuration, and also enhanced identity categorization in Experiment 2. However, discrimination of configuration was generally low. In turn, recognition accuracy was generally high irrespective of configuration condition. Moreover, observers most sensitive to configuration during discrimination did not appear to rely on this information for recognition of familiar faces. These results demonstrate that configuration theory provides limited explanatory power for the recognition of familiar faces.

Facial identity across the lifespan

We can recognise people that we know across their lifespan. We see family members age, and we can recognise celebrities across long careers. How is this possible, despite the very large facial changes that occur as people get older? Here we analyse the statistical properties of faces as they age, sampling photos of the same people from their 20s to their 70s. Across a number of simulations, we observe that individuals' faces retain some idiosyncratic physical properties across the adult lifespan that can be used to support moderate levels of age-independent recognition. However, we found that models based exclusively on image-similarity only achieved limited success in recognising faces across age. In contrast, more robust recognition was achieved with the introduction of a minimal top-down familiarisation procedure. Such models can incorporate the within-person variability associated with a particular individual to show a surprisingly high level of generalisation, even across the lifespan. The analysis of this variability reveals a powerful statistical tool for understanding recognition, and demonstrates how visual representations may support operations typically thought to require conceptual properties.

Individual Differences in Serial Dependence of Facial Identity are Associated with Face Recognition Abilities

Serial dependence is a perceptual bias where current perception is biased towards prior visual input. This bias occurs when perceiving visual attributes, such as facial identity, and has been argued to play an important functional role in vision, stabilising the perception of objects through integration. In face identity recognition, this bias could assist in building stable representations of facial identity. If so, then individual variation in serial dependence could contribute to face recognition ability. To investigate this possibility, we measured both the strength of serial dependence and the range over which individuals showed this bias (the tuning) in 219 adults, using a new measure of serial dependence of facial identity. We found that better face recognition was associated with stronger serial dependence and narrower tuning, that is, showing serial dependence primarily when sequential faces were highly similar. Serial dependence tuning was further found to be a significant predictor of face recognition abilities independently of both object recognition and face identity aftereffects. These findings suggest that the extent to which serial dependence is used selectively for similar faces is important to face recognition. Our results are consistent with the view that serial dependence plays a functional role in face recognition.

Facial Trustworthiness and Criminal Sentencing: A Comment on Wilson and Rule (2015)

Our first impressions of others, whether accurate or unfounded, have real-world consequences in terms of how we judge and treat those people. Previous research has suggested that criminal sentencing is influenced by the perceived facial trustworthiness of defendants in murder trials. In real cases, those who appeared less trustworthy were more likely to receive death rather than life sentences. Here, we carried out several attempts to replicate this finding, utilizing the original set of stimuli (Study 1), multiple images of each identity (Study 2), and a larger sample of identities (Study 3). In all cases, we found little support for the association between facial trustworthiness and sentencing. Furthermore, there was clear evidence that the specific image chosen to depict each identity had a significant influence on subsequent judgments. Taken together, our findings suggest that perceptions of facial trustworthiness have no real-world influence on sentencing outcomes in serious criminal cases.

Sorting through the impact of familiarity when processing vocal identity: Results from a voice sorting task

The present article reports on one experiment designed to examine the importance of familiarity when processing vocal identity. A voice sorting task was used with participants who were either personally familiar or unfamiliar with three speakers. The results suggested that familiarity supported both an ability to tell different instances of the same voice together, and to tell similar instances of different voices apart. In addition, the results suggested differences between the three speakers in terms of the extent to which they were confusable, underlining the importance of vocal characteristics and stimulus selection within behavioural tasks. The results are discussed with reference to existing debates regarding the nature of stored representations as familiarity develops, and the difficulty when processing voices over faces more generally.

More human than human: a Turing test for photographed faces

Background

Recent experimental work has shown that hyper-realistic face masks can pass for real faces during live viewing. However, live viewing embeds the perceptual task (mask detection) in a powerful social context that may influence respondents’ behaviour. To remove this social context, we assessed viewers’ ability to distinguish photos of hyper-realistic masks from photos of real faces in a computerised two-alternative forced choice (2AFC) procedure.

Results

In experiment 1 (N = 120), we observed an error rate of 33% when viewing time was restricted to 500 ms. In experiment 2 (N = 120), we observed an error rate of 20% when viewing time was unlimited. In both experiments we saw a significant performance cost for other-race comparisons relative to own-race comparisons.

Conclusions

We conclude that viewers could not reliably distinguish hyper-realistic face masks from real faces in photographic presentations. As well as its theoretical interest, failure to detect synthetic faces has important implications for security and crime prevention, which often rely on facial appearance and personal identity being related.

Facial features and unethical behavior - Doped athletes show higher facial width-to-height ratios than non-doping sanctioned athletes

Past research has emphasized the role of facial structures in predicting social behavior. In particular the facial width-to-height ratio (fWHR) was found to be a reliable predictor for antisocial and unethical behavior. The current study was aimed at examining this association in the field of sports: FWHRs of 146 doping sanctioned athletes in athletics (37 male/38 female) and weightlifting (44 male/27 female) were compared to the fWHRs of randomly chosen non-doping sanctioned athletes of the Top Ten at the World Championship 2017 and Olympic Games 2016 in both sports (146 athletes). The results showed that doping sanctioned athletes due to the use of anabolic steroids had larger fWHRs than non-doping sanctioned athletes. However, doping sanctioned athletes due to other doping rule violations than the use of anabolic steroids, did not show this effect. The study provides empirical evidence for the relation between fWHR and unethical behavior in a real-world setting and contributes to the discussion about fWHR’s biological origin, emphasizing the role of anabolic steroids. A mutual interaction between fWHR and doping behavior is discussed, at which a larger fWHR might signify a higher tendency to behave unethically, whereas the consequential intake of anabolic steroids might also shape individuals’ faces.

The Jena Speaker Set (JESS)-A database of voice stimuli from unfamiliar young and old adult speakers

Here we describe the Jena Speaker Set (JESS), a free database for unfamiliar adult voice stimuli, comprising voices from 61 young (18-25 years) and 59 old (60-81 years) female and male speakers uttering various sentences, syllables, read text, semi-spontaneous speech, and vowels. Listeners rated two voice samples (short sentences) per speaker for attractiveness, likeability, two measures of distinctiveness ("deviation"-based [DEV] and "voice in the crowd"-based [VITC]), regional accent, and age. Interrater reliability was high, with Cronbach's α between .82 and .99. Young voices were generally rated as more attractive than old voices, but particularly so when male listeners judged female voices. Moreover, young female voices were rated as more likeable than both young male and old female voices. Young voices were judged to be less distinctive than old voices according to the DEV measure, with no differences in the VITC measure. In age ratings, listeners almost perfectly discriminated young from old voices; additionally, young female voices were perceived as being younger than young male voices. Correlations between the rating dimensions above demonstrated (among other things) that DEV-based distinctiveness was strongly negatively correlated with rated attractiveness and likeability. By contrast, VITC-based distinctiveness was uncorrelated with rated attractiveness and likeability in young voices, although a moderate negative correlation was observed for old voices. Overall, the present results demonstrate systematic effects of vocal age and gender on impressions based on the voice and inform as to the selection of suitable voice stimuli for further research into voice perception, learning, and memory.

Part and whole face representations in immediate and long-term memory

Although there is empricial support for the old adage that "we never forget a face" (Journal of Experimental Psychology: General 104 (1975) 54-75), the cognitive processes responsible for our long-term face memories are not well understood. By manipulating the upright and inverted orientation of faces during encoding and retrieval, we investigated the influence of holistic processing on our ability to recognize faces stored in long-term memory. In Experiment 1, participants were trained to identify 12 novel upright faces (six male, six female) by name (e.g., "Joe," "Sue") to a criterion of 100% accuracy. Following learning, holistic memory for the upright and inverted faces was tested using the parts/wholes face recognition task. Different groups of participants were tested either immediately, one week, or two weeks after learning. A significant holistic effect was found for faces tested in their original upright orientation that was stable over the immediate, one-week, and two-week testing periods. In contrast, recognition of the same faces when shown inverted was poor and showed no evidence of holistic processing. In Experiment 2, faces were learned in their inverted orientations with 100% accuracy and tested in their upright and inverted orientations. At the immediate, one-week, or two-week intervals, recognition of inverted faces was relatively poor and there was no evidence of holistic processing for faces tested either in inverted or upright orientations. Collectively, these results indicate holistic processing provides an efficient means for the encoding and retrieval of faces in long-term memory that are relatively stable with the passage of time.

Learning own- and other-race facial identities: Testing implicit recognition with event-related brain potentials

Exposure to varying images of the same person can encourage the formation of a representation that is sufficiently robust to allow recognition of previously unseen images of this person. While behavioural work suggests that face identity learning is harder for other-race faces, the present experiment investigated the neural correlates underlying own- and other-race face learning. Participants sorted own- and other-race identities into separate identity clusters and were further familiarised with these identities in a matching task. Subsequently, we compared event-related brain potentials (ERPs) in an implicit recognition (butterfly detection) task for learnt and previously unseen identities. We observed better sorting and matching for own- than other-race identities, and behavioural learning effects were restricted to own-race identities. Similarly, the N170 ERP component showed clear learning effects for own-race faces only. The N250, a component more closely associated with face learning was more negative for learnt than novel identities. ERP findings thus suggests a processing advantage for own-race identities at an early perceptual level whereas later correlates of identity learning were unaffected by ethnicity. These results suggest learning advantages for own-race identities, which underscores the importance of perceptual expertise in the own-race bias.

Best-worst scaling improves measurement of first impressions

A common goal in psychological research is the measurement of subjective impressions, such as first impressions of faces. These impressions are commonly measured using Likert ratings. Although these ratings are simple to administer, they are associated with response issues that can limit reliability. Here we examine best-worst scaling (BWS), a forced-choice method, as a potential alternative to Likert ratings for measuring participants' facial first impressions. We find that at the group level, BWS scores correlated almost perfectly with Likert scores, indicating that the two methods measure the same impressions. However, at the individual participant level BWS outperforms Likert ratings, both in terms of ability to predict preferences in a third task, and in terms of test-retest reliability. These benefits highlight the power of BWS, particularly for use in individual differences research.

Face search in CCTV surveillance

Background

We present a series of experiments on visual search in a highly complex environment, security closed-circuit television (CCTV). Using real surveillance footage from a large city transport hub, we ask viewers to search for target individuals. Search targets are presented in a number of ways, using naturally occurring images including their passports and photo ID, social media and custody images/videos. Our aim is to establish general principles for search efficiency within this realistic context.

Results

Across four studies we find that providing multiple photos of the search target consistently improves performance. Three different photos of the target, taken at different times, give substantial performance improvements by comparison to a single target. By contrast, providing targets in moving videos or with biographical context does not lead to improvements in search accuracy.

Conclusions

We discuss the multiple-image advantage in relation to a growing understanding of the importance of within-person variability in face recognition.

Electronic supplementary material

The online version of this article (10.1186/s41235-019-0193-0) contains supplementary material, which is available to authorized users.

Two Sides of Face Learning: Improving Between-Identity Discrimination While Tolerating More Within-Person Variability in Appearance

Two photos of an unfamiliar face are often perceived as belonging to different people—an error that disappears when a face is familiar. Face learning has been characterized as increased tolerance of within-person variability in appearance and is facilitated by exposure to such variability (e.g., differences in expression, lighting, and aesthetics). We hypothesized that increased tolerance of variability in appearance might lead to reduced discrimination and that misidentifications would be reduced if a face was learned in the context of a similar-looking identity. After validating our stimuli (Experiments 1a and 1b), we conducted three experiments investigating face learning. In two of these, participants learned three faces (Experiment 2: 15 images/identity and Experiment 3: 5 images/identity), two of which were similar. In a recognition task, misidentifications did not change as a function of similarity, although participants recognized more images of the target in Experiment 2 (i.e., after learning 15 images). In Experiment 4, participants learned one identity and the number of images studied varied across groups. Recognition of new images increased with the number of images studied, with no changes in false alarms; sensitivity (A′) marginally increased. The results suggest that recognition and discrimination reflect separable processes with minimal influence of between-person similarity on discrimination.

Acoustic voice variation within and between speakers

Little is known about the nature or extent of everyday variability in voice quality. This paper describes a series of principal component analyses to explore within- and between-talker acoustic variation and the extent to which they conform to expectations derived from current models of voice perception. Based on studies of faces and cognitive models of speaker recognition, the authors hypothesized that a few measures would be important across speakers, but that much of within-speaker variability would be idiosyncratic. Analyses used multiple sentence productions from 50 female and 50 male speakers of English, recorded over three days. Twenty-six acoustic variables from a psychoacoustic model of voice quality were measured every 5 ms on vowels and approximants. Across speakers the balance between higher harmonic amplitudes and inharmonic energy in the voice accounted for the most variance (females = 20%, males = 22%). Formant frequencies and their variability accounted for an additional 12% of variance across speakers. Remaining variance appeared largely idiosyncratic, suggesting that the speaker-specific voice space is different for different people. Results further showed that voice spaces for individuals and for the population of talkers have very similar acoustic structures. Implications for prototype models of voice perception and recognition are discussed.

The wisdom of the crowd: A case of post- to ante-mortem face matching by police super-recognisers

This case report describes novel methodology used to identify a 43-year-old post-mortem photo of a drowned male recovered from a London river in the 1970s. Embedded in an array of foils, police super-recognisers (n=25) possessing superior simultaneous face matching ability, and police controls (n=139) provided confidence ratings as to the similarity of the post-mortem photo to an ante-mortem photo of a man who went missing at about the same time. Indicative of a match, compared to controls, super-recognisers provided higher ratings to the target than the foils. Effects were enhanced when drawing on the combined wisdom of super-recogniser crowds, but not control crowds. These findings supported additional case evidence allowing the coroner to rule that the deceased male and missing male were likely one and the same person. A description of how similar super-recogniser wisdom of the crowd procedures could be applied to other visual image identification cases when no other method is feasible is provided.

Perceptual and Semantic Phases of Face Identification Processing: A Multivariate Electroencephalography Study

Rapid identification of a familiar face requires an image-invariant representation of person identity. A varying sample of familiar faces is necessary to disentangle image-level from person-level processing. We investigated the time course of face identity processing using a multivariate electroencephalography analysis. Participants saw ambient exemplars of celebrity faces that differed in pose, lighting, hairstyle, and so forth. A name prime preceded a face on half of the trials to preactivate person-specific information, whereas a neutral prime was used on the remaining half. This manipulation helped dissociate perceptual- and semantic-based identification. Two time intervals within the post-face onset electroencephalography epoch were sensitive to person identity. The early perceptual phase spanned 110-228 msec and was not modulated by the name prime. The late semantic phase spanned 252-1000 msec and was sensitive to person knowledge activated by the name prime. Within this late phase, the identity response occurred earlier in time (300-600 msec) for the name prime with a scalp topography similar to the FN400 ERP. This may reflect a matching of the person primed in memory with the face on the screen. Following a neutral prime, the identity response occurred later in time (500-800 msec) with a scalp topography similar to the P600f ERP. This may reflect activation of semantic knowledge associated with the identity. Our results suggest that processing of identity begins early (110 msec), with some tolerance to image-level variations, and then progresses in stages sensitive to perceptual and then to semantic features.

Face morphing attacks: Investigating detection with humans and computers

Background

In recent years, fraudsters have begun to use readily accessible digital manipulation techniques in order to carry out face morphing attacks. By submitting a morph image (a 50/50 average of two people’s faces) for inclusion in an official document such as a passport, it might be possible that both people sufficiently resemble the morph that they are each able to use the resulting genuine ID document. Limited research with low-quality morphs has shown that human detection rates were poor but that training methods can improve performance. Here, we investigate human and computer performance with high-quality morphs, comparable with those expected to be used by criminals.

Results

Over four experiments, we found that people were highly error-prone when detecting morphs and that training did not produce improvements. In a live matching task, morphs were accepted at levels suggesting they represent a significant concern for security agencies and detection was again error-prone. Finally, we found that a simple computer model outperformed our human participants.

Conclusions

Taken together, these results reinforce the idea that advanced computational techniques could prove more reliable than training people when fighting these types of morphing attacks. Our findings have important implications for security authorities worldwide.

Electronic supplementary material

The online version of this article (10.1186/s41235-019-0181-4) contains supplementary material, which is available to authorized users.

A data-driven approach to stimulus selection reveals an image-based representation of objects in high-level visual areas

The ventral visual pathway is directly involved in the perception and recognition of objects. However, the extent to which the neural representation of objects in this region reflects low-level or high-level properties remains unresolved. A problem in resolving this issue is that only a small proportion of the objects experienced during natural viewing can be shown during a typical experiment. This can lead to an uneven sampling of objects that biases our understanding of how they are represented. To address this issue, we developed a data-driven approach to stimulus selection that involved describing a large number objects in terms of their image properties. In the first experiment, clusters of objects were evenly selected from this multi-dimensional image space. Although the clusters did not have any consistent semantic features, each elicited a distinct pattern of neural response. In the second experiment, we asked whether high-level, category-selective patterns of response could be elicited by objects from other categories, but with similar image properties. Object clusters were selected based on the similarity of their image properties to objects from five different categories (bottle, chair, face, house, and shoe). The pattern of response to each metameric object cluster was similar to the pattern elicited by objects from the corresponding category. For example, the pattern for bottles was similar to the pattern for objects with similar image properties to bottles. In both experiments, the patterns of response were consistent across participants providing evidence for common organising principles. This study provides a more ecological approach to understanding the perceptual representations of objects and reveals the importance of image properties.

'Please sort these voice recordings into 2 identities': Effects of task instructions on performance in voice sorting studies

We investigated the effects of two types of task instructions on performance on a voice sorting task by listeners who were either familiar or unfamiliar with the voices. Listeners were asked to sort 15 naturally varying stimuli from two voice identities into perceived identities. Half of the listeners sorted the recordings freely into as many identities as they perceived; the other half were forced to sort stimuli into two identities only. As reported in previous studies, unfamiliar listeners formed more clusters than familiar listeners. Listeners therefore perceived different naturally varying stimuli from the same identity as coming from different identities, while being highly accurate at telling apart the stimuli from different voices. We further show that a change in task instructions - forcing listeners to sort stimuli into two identities only - helped unfamiliar listeners to overcome this selective failure at 'telling people together'. This improvement, however, came at the cost of an increase in errors in telling people apart. For familiar listeners, similar non-significant trends were apparent. Therefore, even when informed about correct number of identities, listeners may fail to accurately perceive identity further highlighting that voice identity perception in the context of natural within-person variability is a challenging task. We discuss our results in terms of similarities and differences to findings in the face perception literature and their importance in applied settings, such as forensic voice identification.

Learning own- and other-race facial identities from natural variability

Exposure to multiple varying face images of the same person encourages the formation of identity representations which are sufficiently robust to allow subsequent recognition from new, never-before seen images. While recent studies suggest that identity information is initially harder to perceive in images of other- relative to own-race identities, it remains unclear whether these difficulties propagate to face learning, that is, to the formation of robust face representations. We report two experiments in which Caucasian and East Asian participants sorted multiple images of own- and other-race persons according to identity in an implicit learning task and subsequently either matched novel images of learnt and previously unseen faces for identity (Experiment 1) or made old/new decisions for new images of learnt and unfamiliar identities (Experiment 2). Caucasian participants demonstrated own-race advantages during sorting, matching, and old/new recognition, while corresponding effects were absent in East Asian participants with substantial other-race expertise. Surprisingly, East Asian participants showed enhanced learning for other-race identities during matching in Experiment 1, which may reflect their increased motivation to individuate other-race faces. Thus, our results highlight the importance of perceptual expertise for own- and other-race processing, but may also lend support to recent suggestions on how expertise and socio-cognitive factors can interact.

Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements

It is commonly assumed that a person's emotional state can be readily inferred from his or her facial movements, typically called emotional expressions or facial expressions. This assumption influences legal judgments, policy decisions, national security protocols, and educational practices; guides the diagnosis and treatment of psychiatric illness, as well as the development of commercial applications; and pervades everyday social interactions as well as research in other scientific fields such as artificial intelligence, neuroscience, and computer vision. In this article, we survey examples of this widespread assumption, which we refer to as the common view, and we then examine the scientific evidence that tests this view, focusing on the six most popular emotion categories used by consumers of emotion research: anger, disgust, fear, happiness, sadness, and surprise. The available scientific evidence suggests that people do sometimes smile when happy, frown when sad, scowl when angry, and so on, as proposed by the common view, more than what would be expected by chance. Yet how people communicate anger, disgust, fear, happiness, sadness, and surprise varies substantially across cultures, situations, and even across people within a single situation. Furthermore, similar configurations of facial movements variably express instances of more than one emotion category. In fact, a given configuration of facial movements, such as a scowl, often communicates something other than an emotional state. Scientists agree that facial movements convey a range of information and are important for social communication, emotional or otherwise. But our review suggests an urgent need for research that examines how people actually move their faces to express emotions and other social information in the variety of contexts that make up everyday life, as well as careful study of the mechanisms by which people perceive instances of emotion in one another. We make specific research recommendations that will yield a more valid picture of how people move their faces to express emotions and how they infer emotional meaning from facial movements in situations of everyday life. This research is crucial to provide consumers of emotion research with the translational information they require.

Limited evidence of hierarchical encoding in the cheerleader effect

"The cheerleader effect" refers to the increase in attractiveness that an individual face experiences when seen in a group of other faces. It has been proposed that the cheerleader effect occurs because (a) the visual system rapidly summarises a group of faces into an ensemble representation, (b) which is hypothesised to be highly attractive because of its average facial characteristics, and (c) observers remember individual faces to be more alike the ensemble representation than they were, due to hierarchical structure of visual working memory. Across three experiments, we investigated whether the cheerleader effect is consistent with hierarchical encoding, by asking observers to give attractiveness ratings to the same target faces shown in groups and alone. Consistent with hierarchical encoding, the largest attractiveness increases of 1.5-2.0% occurred when target faces were presented in groups of faces that could be mentally summarised to create an ensemble representation with average facial characteristics. Surprisingly, smaller cheerleader effects still occurred in conditions that were incompatible with hierarchical encoding (i.e., groups with non-human images). Together, these results offer only limited evidence for the role of hierarchical encoding in the cheerleader effect, suggesting that alternative mechanisms must be explored in future research.

Modelling face memory reveals task-generalizable representations

Current cognitive theories are cast in terms of information-processing mechanisms that use mental representations^1-4. For example, people use their mental representations to identify familiar faces under various conditions of pose, illumination and ageing, or to draw resemblance between family members. Yet, the actual information contents of these representations are rarely characterized, which hinders knowledge of the mechanisms that use them. Here, we modelled the three-dimensional representational contents of 4 faces that were familiar to 14 participants as work colleagues. The representational contents were created by reverse-correlating identity information generated on each trial with judgements of the face's similarity to the individual participant's memory of this face. In a second study, testing new participants, we demonstrated the validity of the modelled contents using everyday face tasks that generalize identity judgements to new viewpoints, age and sex. Our work highlights that such models of mental representations are critical to understanding generalization behaviour and its underlying information-processing mechanisms.

Objective Patterns of Face Recognition Deficits in 165 Adults with Self-Reported Developmental Prosopagnosia

In the last 15 years, increasing numbers of individuals have self-referred to research laboratories in the belief that they experience severe everyday difficulties with face recognition. The condition “developmental prosopagnosia” (DP) is typically diagnosed when impairment is identified on at least two objective face-processing tests, usually involving assessments of face perception, unfamiliar face memory, and famous face recognition. While existing evidence suggests that some individuals may have a mnemonic form of prosopagnosia, it is also possible that other subtypes exist. The current study assessed 165 adults who believe they experience DP, and 38% of the sample were impaired on at least two of the tests outlined above. While statistical dissociations between face perception and face memory were only observed in four cases, a further 25% of the sample displayed dissociations between impaired famous face recognition and intact short-term unfamiliar face memory and face perception. We discuss whether this pattern of findings reflects (a) limitations within dominant diagnostic tests and protocols, (b) a less severe form of DP, or (c) a currently unrecognized but prevalent form of the condition that affects long-term face memory, familiar face recognition or semantic processing.

Categorization training changes the visual representation of face identity

Previous research suggests that learning to categorize faces along a new dimension changes the perceptual representation of that dimension, but little is known about how the representation of specific face identities changes after such category learning. Here, we trained participants to categorize faces that varied along two morphing dimensions. One dimension was relevant to the categorization task and the other was irrelevant. We used reverse correlation to estimate the internal templates used to identify the two faces at the extremes of the relevant dimension, both before and after training, and at two different levels of the irrelevant dimension. Categorization training changed the internal templates used for face identification, even though identification and categorization tasks impose different demands on the observers. After categorization training, the internal templates became more invariant across changes in the irrelevant dimension. These results suggest that the representation of face identity can be modified by categorization experience.

An objective, sensitive and ecologically valid neural measure of rapid human individual face recognition

Humans may be the only species able to rapidly and automatically recognize a familiar face identity in a crowd of unfamiliar faces, an important social skill. Here, by combining electroencephalography (EEG) and fast periodic visual stimulation (FPVS), we introduce an ecologically valid, objective and sensitive neural measure of this human individual face recognition function. Natural images of various unfamiliar faces are presented at a fast rate of 6 Hz, allowing one fixation per face, with variable natural images of a highly familiar face identity, a celebrity, appearing every seven images (0.86 Hz). Following a few minutes of stimulation, a high signal-to-noise ratio neural response reflecting the generalized discrimination of the familiar face identity from unfamiliar faces is observed over the occipito-temporal cortex at 0.86 Hz and harmonics. When face images are presented upside-down, the individual familiar face recognition response is negligible, being reduced by a factor of 5 over occipito-temporal regions. Differences in the magnitude of the individual face recognition response across different familiar face identities suggest that factors such as exposure, within-person variability and distinctiveness mediate this response. Our findings of a biological marker for fast and automatic recognition of individual familiar faces with ecological stimuli open an avenue for understanding this function, its development and neural basis in neurotypical individual brains along with its pathology. This should also have implications for the use of facial recognition measures in forensic science.

Flexible voices: Identity perception from variable vocal signals

Human voices are extremely variable: The same person can sound very different depending on whether they are speaking, laughing, shouting or whispering. In order to successfully recognise someone from their voice, a listener needs to be able to generalize across these different vocal signals (‘telling people together’). However, in most studies of voice-identity processing to date, the substantial within-person variability has been eliminated through the use of highly controlled stimuli, thus focussing on how we tell people apart. We argue that this obscures our understanding of voice-identity processing by controlling away an essential feature of vocal stimuli that may include diagnostic information. In this paper, we propose that we need to extend the focus of voice-identity research to account for both “telling people together” as well as “telling people apart.” That is, we must account for whether, and to what extent, listeners can overcome within-person variability to obtain a stable percept of person identity from vocal cues. To do this, our theoretical and methodological frameworks need to be adjusted to explicitly include the study of within-person variability.

Social Evaluation of Faces Across Gender and Familiarity

Models of social evaluation aim to capture the information people use to form first impressions of unfamiliar others. However, little is currently known about the relationship between perceived traits across gender. In Study 1, we asked viewers to provide ratings of key social dimensions (dominance, trustworthiness, etc.) for multiple images of 40 unfamiliar identities. We observed clear sex differences in the perception of dominance—with negative evaluations of high dominance in unfamiliar females but not males. In Study 2, we used the social evaluation context to investigate the key predictions about the importance of pictorial information in familiar and unfamiliar face processing. We compared the consistency of ratings attributed to different images of the same identities and demonstrated that ratings of images depicting the same familiar identity are more tightly clustered than those of unfamiliar identities. Such results imply a shift from image rating to person rating with increased familiarity, a finding which generalises results previously observed in studies of identification.

A Data-Driven Test for Cross-Cultural Differences in Face Preferences

Previous research has shown strong cross-cultural agreement in facial attractiveness judgments. However, these studies all used a theory-driven approach in which responses to specific facial characteristics are compared between cultures. This approach is constrained by the predictions that can be derived from existing theories and can therefore bias impressions of the extent of cross-cultural agreement in face preferences. We directly addressed this problem by using a data-driven, rather than theory-driven, approach to compare facial attractiveness judgments made by Chinese-born participants who were resident in China, Chinese-born participants currently resident in the UK, and UK-born and UK-resident White participants. Analyses of the principal components along which faces naturally varied suggested that Chinese and White UK participants used face information in different ways, at least when judging women’s facial attractiveness. In other words, the data-driven approach used in this study revealed some cross-cultural differences in face preferences that were not apparent in studies using theory-driven approaches.

Making a Spectacle of Yourself: The Effect of Glasses and Sunglasses on Face Perception

We investigated the effect of wearing glasses and sunglasses on the perception of social traits from faces and on face matching. Participants rated images of people wearing no glasses, glasses and sunglasses on three social traits (trustworthiness, competence and attractiveness). Wearing sunglasses reduced ratings of trustworthiness. Participants also performed a matching task (telling whether two images show the same person or not) with pairs of images both wearing no glasses, glasses or sunglasses, and all combinations of eyewear. Incongruent eyewear conditions (e.g., one image wearing glasses and the other wearing sunglasses, etc.) reduced performance. Further analysis comparing performance on congruent and incongruent eyewear trials showed that our effects were driven by match trial performance, where differences in eyewear decreased accuracy. For same-eyewear-condition pairs, performance was poorer for pairs of images both wearing sunglasses than no glasses. Our results extend and update previous research on the effect of eyewear on face perception.

Face perception: A brief journey through recent discoveries and current directions

Faces are a rich source of information about the people around us. Identity, state of mind, emotions, intentions, age, gender, ethnic background, attractiveness and a host of other attributes about an individual can be gleaned from a face. When face perception fails, dramatic psycho-social consequences can follow at the individual level, as in the case of prosopagnosic parents who are unable to recognize their children at school pick-up. At the species level, social interaction patterns are shaped by human face perception abilities. The computational feat of recognizing faces and facial attributes, and the challenges overcome by the human brain to achieve this feat, have fascinated generations of vision researchers. In this paper, we present a brief overview of some of the milestones of discovery as well as outline a selected set of current directions and open questions on this topic.

Learning context and the other-race effect: Strategies for improving face recognition

People recognize faces of their own race more accurately than faces of other races-a phenomenon known as the "Other-Race Effect" (ORE). Previous studies show that training with multiple variable images improves face recognition. Building on multi-image training, we take a novel approach to improving own- and other-race face recognition by testing the role of learning context on accuracy. Learning context was either contiguous, with multiple images of each identity seen in sequence, or distributed, with multiple images of an identity randomly interspersed among different identities. In two experiments, East Asian and Caucasian participants learned own- and other-races faces either in a contiguous or distributed order. In Experiment 1, people learned each identity from four highly variable face images. In Experiment 2, identities were learned from one image, repeated four times. In both experiments we found a robust other-race effect. The effect of learning context, however, differed depending on the variability of the learned images. The distributed presentation yielded better recognition when people learned from single repeated images (Exp. 1), but not when they learned from multiple variable images (Exp. 2). Overall, performance was better with multiple-image training than repeated single image training. We conclude that multiple-image training and distributed learning can both improve recognition accuracy, but via distinct processes. The former broadens perceptual tolerance for image variation from a face, when there are diverse images available to learn. The latter effectively strengthens the representation of differences among similar faces, when there is only a single learning image.

Do facial first impressions reflect a shared social reality?

Influential facial impression models have repeatedly shown that trustworthiness, youthful-attractiveness, and dominance dimensions subserve a wide variety of first impressions formed from strangers' faces, suggestive of a shared social reality. However, these models are built from impressions aggregated across observers. Critically, recent work has now shown substantial inter-observer differences in facial impressions, raising the important question of whether these dimensional models based on aggregated group data are meaningful at the individual observer level. We addressed this question with a novel case series approach, using factor analyses of ratings of twelve different traits to build individual models of facial impressions for different observers. Strikingly, three dimensions of trustworthiness, youthful/attractiveness, and competence/dominance appeared across the majority of these individual observer models, demonstrating that the dimensional approach is indeed meaningful at the individual level. Nonetheless, we also found differences in the stability of the competence/dominance dimension across observers. Taken together, results suggest that individual differences in impressions arise in the context of a largely common structure that supports a shared social reality.

Breaking voice identity perception: Expressive voices are more confusable for listeners

The human voice is a highly flexible instrument for self-expression, yet voice identity perception is largely studied using controlled speech recordings. Using two voice-sorting tasks with naturally varying stimuli, we compared the performance of listeners who were familiar and unfamiliar with the TV show Breaking Bad. Listeners organised audio clips of speech with (1) low-expressiveness and (2) high-expressiveness into perceived identities. We predicted that increased expressiveness (e.g., shouting, strained voice) would significantly impair performance. Overall, while unfamiliar listeners were less able to generalise identity across exemplars, the two groups performed equivalently well when telling voices apart when dealing with low-expressiveness stimuli. However, high vocal expressiveness significantly impaired telling apart in both the groups: this led to increased misidentifications, where sounds from one character were assigned to the other. These misidentifications were highly consistent for familiar listeners but less consistent for unfamiliar listeners. Our data suggest that vocal flexibility has powerful effects on identity perception, where changes in the acoustic properties of vocal signals introduced by expressiveness lead to effects apparent in familiar and unfamiliar listeners alike. At the same time, expressiveness appears to have affected other aspects of voice identity processing selectively in one listener group but not the other, thus revealing complex interactions of stimulus properties and listener characteristics (i.e., familiarity) in identity processing.