Effects of lighting on the perception of facial surfaces

High familiar faces have both eye recognition and holistic processing advantages

People recognize familiar faces better than unfamiliar faces. However, it remains unknown whether familiarity affects part-based and/or holistic processing. Wang et al., Frontiers in Psychology, 6, 559 (2015), Vision Research, 157, 89-96 (2019) found both enhanced part-based and holistic processing in eye relative to mouth regions (i.e., in a region-selective manner) for own-race and own-species faces, i.e., faces with more experience. Here, we examined the role of face familiarity in eyes (part-based, region-selective) and holistic processing. Face familiarity was tested at three levels: high-familiar (faces of students from the same department and the same class who attended almost all courses together), low-familiar (faces of students from the same department but different classes who attended some courses together), and unfamiliar (faces of schoolmates from different departments who seldom attended the same courses). Using the old/new task in Experiment 1, we found that participants recognized eyes of high-familiar faces better than low-familiar and unfamiliar ones, while similar performance was observed for mouths, indicating a region-selective, eyes familiarity effect. Using the "Perceptual field" paradigm in Experiment 2, we observed a stronger inversion effect for high-familiar faces, a weaker inversion effect for low-familiar faces, but a non-significant inversion effect for unfamiliar faces, indicating that face familiarity plays a role in holistic processing. Taken together, our results suggest that familiarity, like other experience-based variables (e.g., race and species), can improve both eye processing and holistic processing.

Effect of mask coverage on face identification in Taiwanese men and women

Mask wearing is the easiest and most effective way to avoid COVID-19 infection; however, it affects interpersonal activities, especially face identification. This study examined the effects of three mask coverage levels (full coverage, FC; coverage up to the middle [MB] or bottom of the nose bridge [BB]) on face identification accuracy and time. A total of 115 university students (60 men and 55 women) were recruited to conduct a computer-based simulation test consisting of 30 questions (10 questions [five face images each of men and women] for the three mask coverage levels). One unmasked target face and four face images with a specified mask coverage level were designed for each question, and the participants were requested to select the same face from the four covered face images on the basis of the target face. The ANOVA results indicated that identification accuracy was significantly affected by sex (p < 0.01) and the mask coverage level (p < 0.001), whereas identification time was only influenced by sex (p < 0.05). The multiple comparison results indicated that the identification accuracy rate for faces wearing a mask with FC (90.3%) was significantly lower than for those wearing masks with coverage up to the MB (93.7%) and BB (94.9%) positions; however, no difference in identification accuracy rate was observed between the MB and BB levels. Women exhibited a higher identification accuracy rate than men (94.1% vs. 91.9%) in identifying unfamiliar faces, even though they may spend less time identifying the images. A smaller mask coverage level (i.e., the BB level) does not facilitate face identification. The findings can be served as a reference for people to trade-off between wearing a mask and interpersonal interaction in their daily activities.

The effect of observation angles on facial age perceptions: A case study of Japanese women

Most conventional aging research has limited its approach concerning the head and face shape and skin condition to the frontal face. However, in our daily lives, we observe facial features from various angles, which may reveal or obscure aging features that could only be identified under limited conditions in the past. This study systematically investigates the effect of facial observation angles-specifically, of horizontal and vertical angles-on age impression. A total of 112 Japanese women aged 20-49 years participated as observers who evaluated the age impressions of 280 Japanese women aged 20-69 years. A two-way analysis of the variance of the age impression score was conducted for two factors: observation angle (five angles with yaw and pitch directions) and age group (five ages, from the 20s to the 60s). The results reveal that, as compared with frontal observation, the perceived age tended to decrease with the facial observation angles and that the effect of the angle on perceived age decreased with increasing age, especially for the profile face. Understanding the effect of the facial observation angle on age impression and clarifying the characteristics of the face and skin not perceived in the frontal face will provide useful knowledge to make people look youthful, look more beautiful, and be happier in all aspects of their lives.

COVID-19 masks increase the influence of face recognition algorithm decisions on human decisions in unfamiliar face matching

Face masks, recently adopted to reduce the spread of COVID-19, have had the unintended consequence of increasing the difficulty of face recognition. In security applications, face recognition algorithms are used to identify individuals and present results for human review. This combination of human and algorithm capabilities, known as human-algorithm teaming, is intended to improve total system performance. However, prior work has shown that human judgments of face pair similarity-confidence can be biased by an algorithm's decision even in the case of an error by that algorithm. This can reduce team effectiveness, particularly for difficult face pairs. We conducted two studies to examine whether face masks, now routinely present in security applications, impact the degree to which this cognitive bias is experienced by humans. We first compared the influence of algorithm's decisions on human similarity-confidence ratings in the presence and absence of face masks and found that face masks more than doubled the influence of algorithm decisions on human similarity-confidence ratings. We then investigated if this increase in cognitive bias was dependent on perceived algorithm accuracy by also presenting algorithm accuracy rates in the presence of face masks. We found that making humans aware of the potential for algorithm errors mitigated the increase in cognitive bias due to face masks. Our findings suggest that humans reviewing face recognition algorithm decisions should be made aware of the potential for algorithm errors to improve human-algorithm team performance.

Illumination and gaze effects on face evaluation: The Bi-AGI database

Face evaluation and first impression generation can be affected by multiple face elements such as invariant facial features, gaze direction and environmental context; however, the composite modulation of eye gaze and illumination on faces of different gender and ages has not been previously investigated. We aimed at testing how these different facial and contextual features affect ratings of social attributes. Thus, we created and validated the Bi-AGI Database, a freely available new set of male and female face stimuli varying in age across lifespan from 18 to 87 years, gaze direction and illumination conditions. Judgments on attractiveness, femininity-masculinity, dominance and trustworthiness were collected for each stimulus. Results evidence the interaction of the different variables in modulating social trait attribution, in particular illumination differently affects ratings across age, gaze and gender, with less impact on older adults and greater effect on young faces.

Image variability and face matching

This study investigates whether variability in perceived trait judgements disrupts our ability to match unfamiliar faces. In this preregistered study, 174 participants completed a face matching task where they were asked to indicate whether two ambient face images belonged to the same person or different people (17,748 total data points). Participants completed 51 match trials consisting of images of the same person that differed substantially on one trait (either trustworthiness, dominance or attractiveness) with minimal differences in the alternate traits. Participants also completed 51 mismatch trials which contained two photos of similar-looking individuals. We hypothesised that participants would make more errors on match trials when images differed in terms of attractiveness ratings than when they differed on trustworthiness or dominance. Contrary to expectations, images that differed in terms of attractiveness were matched most accurately, and there was no relationship between the extent of differences in attractiveness ratings and accuracy. There was some evidence that differences in perceived dominance and, to a lesser extent, trustworthiness were associated with lower face matching performance. However, these relationships were not significant when alternate traits were accounted for. The findings of our study suggest that face matching performance is largely robust against variation in trait judgements.

The emergence of view-symmetric neural responses to familiar and unfamiliar faces

Successful recognition of familiar faces is thought to depend on the ability to integrate view-dependent representations of a face into a view-invariant representation. It has been proposed that a key intermediate step in achieving view invariance is the representation of symmetrical views. However, key unresolved questions remain, such as whether these representations are specific for naturally occurring changes in viewpoint and whether view-symmetric representations exist for familiar faces. To address these issues, we compared behavioural and neural responses to natural (canonical) and unnatural (noncanonical) rotations of the face. Similarity judgements revealed that symmetrical viewpoints were perceived to be more similar than non-symmetrical viewpoints for both canonical and non-canonical rotations. Next, we measured patterns of neural response from early to higher level regions of visual cortex. Early visual areas showed a view-dependent representation for natural or canonical rotations of the face, such that the similarity between patterns of response were related to the difference in rotation. View symmetric patterns of neural response to canonically rotated faces emerged in higher visual areas, particularly in face-selective regions. The emergence of a view-symmetric representation from a view-dependent representation for canonical rotations of the face was also evident for familiar faces, suggesting that view-symmetry is an important intermediate step in generating view-invariant representations. Finally, we measured neural responses to unnatural or non-canonical rotations of the face. View-symmetric patterns of response were also found in face-selective regions. However, in contrast to natural or canonical rotations of the face, these view-symmetric responses did not arise from an initial view-dependent representation in early visual areas. This suggests differences in the way that view-symmetrical representations emerge with canonical or non-canonical rotations. The similarity in the neural response to canonical views of familiar and unfamiliar faces in the core face network suggests that the neural correlates of familiarity emerge at later stages of processing.

Face learning via brief real-world social interactions includes changes in face-selective brain areas and hippocampus

Making new acquaintances requires learning to recognise previously unfamiliar faces. In the current study, we investigated this process by staging real-world social interactions between actors and the participants. Participants completed a face-matching behavioural task in which they matched photographs of the actors (whom they had yet to meet), or faces similar to the actors (henceforth called foils). Participants were then scanned using functional magnetic resonance imaging (fMRI) while viewing photographs of actors and foils. Immediately after exiting the scanner, participants met the actors for the first time and interacted with them for 10 min. On subsequent days, participants completed a second behavioural experiment and then a second fMRI scan. Prior to each session, actors again interacted with the participants for 10 min. Behavioural results showed that social interactions improved performance accuracy when matching actor photographs, but not foil photographs. The fMRI analysis revealed a difference in the neural response to actor photographs and foil photographs across all regions of interest (ROIs) only after social interactions had occurred. Our results demonstrate that short social interactions were sufficient to learn and discriminate previously unfamiliar individuals. Moreover, these learning effects were present in brain areas involved in face processing and memory.

Masked face identification is improved by diagnostic feature training

To slow the spread of COVID-19, many people now wear face masks in public. Face masks impair our ability to identify faces, which can cause problems for professional staff who identify offenders or members of the public. Here, we investigate whether performance on a masked face matching task can be improved by training participants to compare diagnostic facial features (the ears and facial marks)—a validated training method that improves matching performance for unmasked faces. We show this brief diagnostic feature training, which takes less than two minutes to complete, improves matching performance for masked faces by approximately 5%. A control training course, which was unrelated to face identification, had no effect on matching performance. Our findings demonstrate that comparing the ears and facial marks is an effective means of improving face matching performance for masked faces. These findings have implications for professions that regularly perform face identification.

Perceptions of familiar and unfamiliar ear- and eyewitnesses

A witness's relationship with a defendant is frequently discussed in criminal trials, yet investigations into perceptions of this relationship have been scarce. Further, an exploration of witnesses other than eyewitnesses has been missing from the literature. The present studies explored how witness type and familiarity with a defendant impact the perceived credibility of a witness. In Study 1, a familiar earwitness was perceived as more credible and honest than a stranger earwitness but the same was not found for eyewitnesses. Results from Study 2 suggest an eyewitness was seen as more credible and believable than an earwitness, and that a familiar witness was seen as more reliable than a stranger, but not than an acquaintance. There was no impact of familiarity or witness type on legal decisions. The present studies indicate that the prior definitions of familiarity might only capture a restricted range of potentially familiar relations.

Surgical face masks impair human face matching performance for familiar and unfamiliar faces

In response to the COVID-19 pandemic, many governments around the world now recommend, or require, that their citizens cover the lower half of their face in public. Consequently, many people now wear surgical face masks in public. We investigated whether surgical face masks affected the performance of human observers, and a state-of-the-art face recognition system, on tasks of perceptual face matching. Participants judged whether two simultaneously presented face photographs showed the same person or two different people. We superimposed images of surgical masks over the faces, creating three different mask conditions: control (no masks), mixed (one face wearing a mask), and masked (both faces wearing masks). We found that surgical face masks have a large detrimental effect on human face matching performance, and that the degree of impairment is the same regardless of whether one or both faces in each pair are masked. Surprisingly, this impairment is similar in size for both familiar and unfamiliar faces. When matching masked faces, human observers are biased to reject unfamiliar faces as "mismatches" and to accept familiar faces as "matches". Finally, the face recognition system showed very high classification accuracy for control and masked stimuli, even though it had not been trained to recognise masked faces. However, accuracy fell markedly when one face was masked and the other was not. Our findings demonstrate that surgical face masks impair the ability of humans, and naïve face recognition systems, to perform perceptual face matching tasks. Identification decisions for masked faces should be treated with caution.

A sparkle in the eye: Illumination cues and lightness constancy in the perception of eye contact

In social interactions, our sense of when we have eye contact with another person relies on the distribution of luminance across their eye region, reflecting the position of the darker iris within the lighter sclera of the human eye. This distribution of luminance can be distorted by the lighting conditions, consistent with the fundamental challenge that the visual system faces in distinguishing the nature of a surface from the pattern of light falling upon it. Here we perform a set of psychophysics experiments in human observers to investigate how illumination impacts on the perception of eye contact. First, we find that simple changes in the direction of illumination can produce systematic biases in our sense of when we have eye contact with another person. Second, we find that the visual system uses information about the lighting conditions to partially discount or 'explain away' the effects of illumination in this context, leading to a significantly more robust sense of when we have eye contact with another person. Third, we find that perceived eye contact is affected by specular reflections from the eye surface in addition to shading patterns, implicating eye glint as a potential cue to gaze direction. Overall, this illustrates how our interpretation of social signals relies on visual mechanisms that both compensate for the effects of illumination on retinal input and potentially exploit novel cues that illumination can produce.

The 4D Space-Time Dimensions of Facial Perception

Facial information is a powerful channel for human-to-human communication. Characteristically, faces can be defined as biological objects that are four-dimensional (4D) patterns, whereby they have concurrently a spatial structure and surface as well as temporal dynamics. The spatial characteristics of facial objects contain a volume and surface in three dimensions (3D), namely breadth, height and importantly, depth. The temporal properties of facial objects are defined by how a 3D facial structure and surface evolves dynamically over time; where time is referred to as the fourth dimension (4D). Our entire perception of another’s face, whether it be social, affective or cognitive perceptions, is therefore built on a combination of 3D and 4D visual cues. Counterintuitively, over the past few decades of experimental research in psychology, facial stimuli have largely been captured, reproduced and presented to participants with two dimensions (2D), while remaining largely static. The following review aims to advance and update facial researchers, on the recent revolution in computer-generated, realistic 4D facial models produced from real-life human subjects. We delve in-depth to summarize recent studies which have utilized facial stimuli that possess 3D structural and surface cues (geometry, surface and depth) and 4D temporal cues (3D structure + dynamic viewpoint and movement). In sum, we have found that higher-order perceptions such as identity, gender, ethnicity, emotion and personality, are critically influenced by 4D characteristics. In future, it is recommended that facial stimuli incorporate the 4D space-time perspective with the proposed time-resolved methods.

Acoustic voice variation within and between speakers

Little is known about the nature or extent of everyday variability in voice quality. This paper describes a series of principal component analyses to explore within- and between-talker acoustic variation and the extent to which they conform to expectations derived from current models of voice perception. Based on studies of faces and cognitive models of speaker recognition, the authors hypothesized that a few measures would be important across speakers, but that much of within-speaker variability would be idiosyncratic. Analyses used multiple sentence productions from 50 female and 50 male speakers of English, recorded over three days. Twenty-six acoustic variables from a psychoacoustic model of voice quality were measured every 5 ms on vowels and approximants. Across speakers the balance between higher harmonic amplitudes and inharmonic energy in the voice accounted for the most variance (females = 20%, males = 22%). Formant frequencies and their variability accounted for an additional 12% of variance across speakers. Remaining variance appeared largely idiosyncratic, suggesting that the speaker-specific voice space is different for different people. Results further showed that voice spaces for individuals and for the population of talkers have very similar acoustic structures. Implications for prototype models of voice perception and recognition are discussed.

I recognise your name but I can't remember your face: An advantage for names in recognition memory

Forgetting someone's name is a common failure of memory, and often occurs despite being able to recognise that person's face. This gives rise to the widespread view that memory for names is generally worse than memory for faces. However, this everyday error confounds stimulus class (faces vs. names) with memory task: recognition versus recall. Here we compare memory for faces and names when both are tested in the same recognition memory framework. Contrary to the common view, we find a clear advantage for names over faces. Across three experiments, we show that recognition of previously unfamiliar names exceeds recognition of previously unfamiliar faces. This advantage persists, even when the same face pictures are repeated at learning and test-a picture-memory task known to produce high levels of performance. Differential performance between names and faces disappears in recognition memory for familiar people. The results are discussed with reference to representational complexity and everyday memory errors.

Person Identification from Drones by Humans: Insights from Cognitive Psychology

The deployment of unmanned aerial vehicles (i.e., drones) in military and police operations implies that drones can provide footage that is of sufficient quality to enable the recognition of strategic targets, criminal suspects, and missing persons. On the contrary, evidence from Cognitive Psychology suggests that such identity judgements by humans are already difficult under ideal conditions, and are even more challenging with drone surveillance footage. In this review, we outline the psychological literature on person identification for readers who are interested in the real-world application of drones. We specifically focus on factors that are likely to affect identification performance from drone-recorded footage, such as image quality, and additional person-related information from the body and gait. Based on this work, we suggest that person identification from drones is likely to be very challenging indeed, and that performance in laboratory settings is still very likely to underestimate the difficulty of this task in real-world settings.

How many voices did you hear? Natural variability disrupts identity perception from unfamiliar voices

Our voices sound different depending on the context (laughing vs. talking to a child vs. giving a speech), making within‐person variability an inherent feature of human voices. When perceiving speaker identities, listeners therefore need to not only ‘tell people apart’ (perceiving exemplars from two different speakers as separate identities) but also ‘tell people together’ (perceiving different exemplars from the same speaker as a single identity). In the current study, we investigated how such natural within‐person variability affects voice identity perception. Using voices from a popular TV show, listeners, who were either familiar or unfamiliar with this show, sorted naturally varying voice clips from two speakers into clusters to represent perceived identities. Across three independent participant samples, unfamiliar listeners perceived more identities than familiar listeners and frequently mistook exemplars from the same speaker to be different identities. These findings point towards a selective failure in ‘telling people together’. Our study highlights within‐person variability as a key feature of voices that has striking effects on (unfamiliar) voice identity perception. Our findings not only open up a new line of enquiry in the field of voice perception but also call for a re‐evaluation of theoretical models to account for natural variability during identity perception.

Photo ID verification remains challenging despite years of practice

Background

Matching unfamiliar faces to photographic identification (ID) documents occurs across many domains, including financial transactions (e.g., mortgage documents), controlling the purchase of age-restricted goods (e.g., alcohol sales), and airport security. Laboratory research has repeatedly documented the fallibility of this process in novice observers, but little research has assessed individual differences based on occupational expertise (cf. White et al., PLoS One 9:e103510, 2014; White et al., Proceedings of the Royal Society B 282(1814):20151292, 2015). In the present study, over 800 professional notaries (who routinely verify identity prior to witnessing signatures on legal documents), 70 bank tellers, and 35 undergraduate students completed an online unfamiliar face-matching test. In this test, observers made match/nonmatch decisions to 30 face ID pairs (half of which were matches), with no time constraints and no trial-by-trial feedback.

Results

Results showed that all groups performed similarly, although age was negatively correlated with accuracy. Critically, weekly and yearly experience with unfamiliar face matching did not impact performance.

Conclusions

These results suggest that accumulated occupational experience has no bearing on unfamiliar face ID abilities and that cognitive declines associated with aging also manifest in unfamiliar face matching.

Aging Strikes the Self-Face Advantage in Featural Processing

Background/Study Context: The face is the most distinctive physical feature of a person. Previous work has shown that one's own face (self-face) is advantageous in perception. Here the authors investigate how aging influences the configural and featural processing of self-face.

METHODS

Older and young adults searched for their own faces and faces of strangers (Experiment 1) or acquaintances (Experiment 2) among distractor faces. The configural and featural processing of faces was assessed with face inversion in Experiment 1 and with changes in point of view in Experiment 2.

RESULTS

Experiment 1 revealed a robust self-face advantage for upright faces in both young and older adults. A similar advantage was observed for inverted faces in young but not in older adults. Experiment 2 revealed a self-face advantage in older adults regardless of the point of view; in young adults, however, the self-face advantage only emerged for frontal view faces.

CONCLUSION

The present study shows that older adults have a self-face advantage in configural but not in featural processing. The authors suggest that the impairment in featural processing in older adults is likely the result of age-related changes in perceptual experience.

How robust is familiar face recognition? A repeat detection study of more than 1000 faces

Recent theories suggest that familiar faces have a robust representation in memory because they have been encountered over a wide variety of contexts and image changes (e.g. lighting, viewpoint and expression). By contrast, unfamiliar faces are encountered only once, and so they do not benefit from such richness of experience and are represented based on image-specific details. In this registered report, we used a repeat detection task to test whether familiar faces are recognized better than unfamiliar faces across image changes. Participants viewed a stream of more than 1000 celebrity face images for 0.5 s each, any of which might be repeated at a later point and has to be detected. Some participants saw the same image at repeats, while others saw a different image of the same face. A post-experimental familiarity check allowed us to determine which celebrities were and were not familiar to each participant. We had three predictions: (i) detection would be better for familiar than unfamiliar faces, (ii) detection would be better across same rather than different images, and (iii) detection of familiar faces would be comparable across same and different images, but detection of unfamiliar faces would be poorer across different images. We obtained support for the first two predictions but not the last. Instead, we found that repeat detection of faces, regardless of familiarity, was poorer across different images. Our study suggests that the robustness of familiar face recognition may have limits, and that under some conditions, familiar face recognition can be just as influenced by image changes as unfamiliar face recognition.

Variation and Likeness in Ambient Artistic Portraiture

An artist-led exploration of portrait accuracy and likeness involved 12 Artists producing 12 portraits referencing a life-size 3D print of the same Sitter. The works were assessed during a public exhibition, and the resulting likeness assessments were compared to portrait accuracy as measured using geometric morphometrics (statistical shape analysis). Our results are that, independently of the assessors' prior familiarity with the Sitter's face, the likeness judgements tended to be higher for less morphologically accurate portraits. The two highest rated were the portrait that most exaggerated the Sitter's distinctive features, and a portrait that was a more accurate (but not the most accurate) depiction. In keeping with research showing photograph likeness assessments involve recognition, we found familiar assessors rated the two highest ranked portraits even higher than those with some or no familiarity. In contrast, those lacking prior familiarity with the Sitter's face showed greater favour for the portrait with the highest morphological accuracy, and therefore most likely engaged in face-matching with the exhibited 3D print. Furthermore, our research indicates that abstraction in portraiture may not enhance likeness, and we found that when our 12 highly diverse portraits were statistically averaged, this resulted in a portrait that is more morphologically accurate than any of the individual artworks comprising the average.

About Face: Matching Unfamiliar Faces Across Rotations of View and Lighting

Matching the identities of unfamiliar faces is heavily influenced by variations in their images. Changes to viewpoint and lighting direction during face perception are commonplace across yaw and pitch axes and can result in dramatic image differences. We report two experiments that, for the first time, factorially investigate the combined effects of lighting and view angle on matching performance for unfamiliar faces. The use of three-dimensional head models allowed control of both lighting and viewpoint. We found viewpoint effects in the yaw axis with little to no effect of lighting. However, for rotations about the pitch axis, there were both viewpoint and lighting effects and these interacted where lighting effects were found only for front views and views from below. The pattern of effects was similar regardless of whether view variation occurred as a result of head (Experiment 1) or camera (Experiment 2) suggesting that face matching is not purely image based. Along with face inversion effects in Experiment 1, the results of this study suggest that face perception is based on shape and surface information and draws on implicit knowledge of upright faces and ecological (top) lighting conditions.

Recognition of computerized facial approximations by familiar assessors

Studies testing the effectiveness of facial approximations typically involve groups of participants who are unfamiliar with the approximated individual(s). This limitation requires the use of photograph arrays including a picture of the subject for comparison to the facial approximation. While this practice is often necessary due to the difficulty in obtaining a group of assessors who are familiar with the approximated subject, it may not accurately simulate the thought process of the target audience (friends and family members) in comparing a mental image of the approximated subject to the facial approximation. As part of a larger process to evaluate the effectiveness and best implementation of the ReFace facial approximation software program, the rare opportunity arose to conduct a recognition study using assessors who were personally acquainted with the subjects of the approximations. ReFace facial approximations were generated based on preexisting medical scans, and co-workers of the scan donors were tested on whether they could accurately pick out the approximation of their colleague from arrays of facial approximations. Results from the study demonstrated an overall poor recognition performance (i.e., where a single choice within a pool is not enforced) for individuals who were familiar with the approximated subjects. Out of 220 recognition tests only 10.5% resulted in the assessor selecting the correct approximation (or correctly choosing not to make a selection when the array consisted only of foils), an outcome that was not significantly different from the 9% random chance rate. When allowed to select multiple approximations the assessors felt resembled the target individual, the overall sensitivity for ReFace approximations was 16.0% and the overall specificity was 81.8%. These results differ markedly from the results of a previous study using assessors who were unfamiliar with the approximated subjects. Some possible explanations for this disparity in performance were examined, and it was ultimately concluded that ReFace facial approximations may have limited effectiveness if used in the traditional way. However, some promising alternative uses are explored that may expand the utility of facial approximations for aiding in the identification of unknown human remains.

The utility of multiple synthesized views in the recognition of unfamiliar faces

The ability to recognize an unfamiliar individual on the basis of prior exposure to a photograph is notoriously poor and prone to errors, but recognition accuracy is improved when multiple photographs are available. In applied situations, when only limited real images are available (e.g., from a mugshot or CCTV image), the generation of new images might provide a technological prosthesis for otherwise fallible human recognition. We report two experiments examining the effects of providing computer-generated additional views of a target face. In Experiment 1, provision of computer-generated views supported better target face recognition than exposure to the target image alone and equivalent performance to that for exposure of multiple photograph views. Experiment 2 replicated the advantage of providing generated views, but also indicated an advantage for multiple viewings of the single target photograph. These results strengthen the claim that identifying a target face can be improved by providing multiple synthesized views based on a single target image. In addition, our results suggest that the degree of advantage provided by synthesized views may be affected by the quality of synthesized material.

Face familiarity promotes stable identity recognition: exploring face perception using serial dependence

Studies suggest that familiar faces are processed in a manner distinct from unfamiliar faces and that familiarity with a face confers an advantage in identity recognition. Our visual system seems to capitalize on experience to build stable face representations that are impervious to variation in retinal input that may occur due to changes in lighting, viewpoint, viewing distance, eye movements, etc. Emerging evidence also suggests that our visual system maintains a continuous perception of a face's identity from one moment to the next despite the retinal input variations through serial dependence. This study investigates whether interactions occur between face familiarity and serial dependence. In two experiments, participants used a continuous scale to rate attractiveness of unfamiliar and familiar faces (either experimentally learned or famous) presented in rapid sequences. Both experiments revealed robust inter-trial effects in which attractiveness ratings for a given face depended on the preceding face's attractiveness. This inter-trial attractiveness effect was most pronounced for unfamiliar faces. Indeed, when participants were familiar with a given face, attractiveness ratings showed significantly less serial dependence. These results represent the first evidence that familiar faces can resist the temporal integration seen in sequential dependencies and highlight the importance of familiarity to visual cognition.

Combined Effects of Gaze and Orientation of Faces on Person Judgments in Social Situations

In social situations, faces of others can vary simultaneously in gaze and orientation. How these variations affect different kinds of social judgments, such as attractiveness or trustworthiness, is only partly understood. Therefore, we studied how different gaze directions, head angles, but also levels of facial attractiveness affect perceived attractiveness and trustworthiness. We always presented pairs of faces - either two average attractive faces or a highly attractive together with a less attractive face. We also varied gaze and head angles showing faces in three different orientations, front, three-quarter and profile view. In Experiment 1 (N = 62), participants rated averted gaze in three-quarter views as more attractive than in front and profile views, and evaluated faces with direct gaze in front views as most trustworthy. Moreover, faces that were being looked at by another face were seen as more attractive. Independent of the head orientation or gaze direction, highly attractive faces were rated as more attractive and more trustworthy. In Experiment 2 (N = 54), we found that the three-quarter advantage vanished when the second face was blurred during judgments, which demonstrates the importance of the presence of another person-as in a triadic social situation-as well as the importance of their visible gaze. The findings emphasize that social evaluations such as trustworthiness are unaffected by the esthetic advantage of three-quarter views of two average attractive faces, and that the effect of a faces' attractiveness is more powerful than the more subtle effects of gaze and orientations.

Things are Looking up: Differential Decline in Face Recognition following Pitch and Yaw Rotation

Previous research into the effects of viewpoint change on face recognition has typically dealt with rotations around the head's vertical axis (yaw). Another common, although less studied, source of viewpoint variation in faces is rotation around the head's horizontal pitch axis (pitch). In the current study we used both a sequential matching task and an old/new recognition task to examine the effect of viewpoint change following rotation about both pitch and yaw axes on human face recognition. The results of both tasks showed that recognition performance was better for faces rotated about yaw compared to pitch. Further, recognition performance for faces rotated upwards on the pitch axis was better than for faces rotated downwards. Thus, equivalent angular rotations about pitch and yaw do not produce equivalent viewpoint-dependent declines in recognition performance.

Tracking the truth: the effect of face familiarity on eye fixations during deception

In forensic investigations, suspects sometimes conceal recognition of a familiar person to protect co-conspirators or hide knowledge of a victim. The current experiment sought to determine whether eye fixations could be used to identify memory of known persons when lying about recognition of faces. Participants' eye movements were monitored whilst they lied and told the truth about recognition of faces that varied in familiarity (newly learned, famous celebrities, personally known). Memory detection by eye movements during recognition of personally familiar and famous celebrity faces was negligibly affected by lying, thereby demonstrating that detection of memory during lies is influenced by the prior learning of the face. By contrast, eye movements did not reveal lies robustly for newly learned faces. These findings support the use of eye movements as markers of memory during concealed recognition but also suggest caution when familiarity is only a consequence of one brief exposure.

Exploring the motion advantage: evaluating the contribution of familiarity and differences in facial motion

Seeing a face move can improve familiar face recognition, face matching, and learning. More specifically, familiarity with a face may facilitate the learning of an individual's "dynamic facial signature". In the outlined research we examine the relationship between participant ratings of familiarity, the distinctiveness of motion, the amount of facial motion, and the recognition of familiar moving faces (Experiment 1) as well as the magnitude of the motion advantage (Experiment 2). Significant positive correlations were found between all factors. Findings suggest that faces rated as moving a lot and in a distinctive manner benefited the most from being seen in motion. Additionally findings indicate that facial motion information becomes a more important cue to recognition the more familiar a face is, suggesting that "dynamic facial signatures" continue to be learnt over time and integrated within the face representation. Results are discussed in relation to theoretical explanations of the moving face advantage.

Super-recognisers in Action: Evidence from Face-matching and Face Memory Tasks

Individuals employed in forensic or security settings are often required to compare faces of ID holders to document photographs, or to recognise the faces of suspects in closed-circuit television footage. It has long been established that both tasks produce a high error rate amongst typical perceivers. This study sought to determine the performance of individuals with exceptionally good face memory ('super-recognisers') on applied facial identity matching and memory tasks. In experiment 1, super-recognisers were significantly better than controls when matching target faces to simultaneously presented line-ups. In experiment 2, super-recognisers were also better at recognising faces from video footage. These findings suggest that super-recognisers are more accurate at face matching and face memory tasks than typical perceivers, and they could be valuable expert employees in national security and forensic settings.

Telling faces together: Learning new faces through exposure to multiple instances

We are usually able to recognize novel instances of familiar faces with little difficulty, yet recognition of unfamiliar faces can be dramatically impaired by natural within-person variability in appearance. In a card-sorting task for facial identity, different photos of the same unfamiliar face are often seen as different people. Here we report two card-sorting experiments in which we manipulate whether participants know the number of identities present. Without constraints, participants sort faces into many identities. However, when told the number of identities present, they are highly accurate. This minimal contextual information appears to support viewers in "telling faces together". In Experiment 2 we show that exposure to within-person variability in the sorting task improves performance in a subsequent face-matching task. This appears to offer a fast route to learning generalizable representations of new faces.

Spatiotemporal changes in neural response patterns to faces varying in visual familiarity

Increasing experience with a previously unfamiliar face improves human ability to recognize it in challenging and novel viewing conditions. Differential neural responses to familiar versus unfamiliar faces in multiple regions of the ventral-temporal and parietal cortex have been reported in previous work, but with limited attention to how behavioral and neural measures change with increasing familiarity. We examined changes in the spatial and temporal characteristics of neural response patterns elicited by faces that vary in their degree of visual familiarity. First, we developed a behavioral paradigm to familiarize participants to low-, medium-, and high-levels of familiarity with faces. Recognition of novel, naturalistic images of the learned individuals improved with increasing familiarity with faces. Next, a new set of participants learned faces using the behavioral paradigm, outside the fMRI scanner, and subsequently viewed blocks of whole-body images of the learned and novel people, inside the scanner. We found that the face-selective FFA and OFA, and a combination of the ventral-temporal areas (e.g., fusiform gyrus) and parietal areas (e.g., precuneus) contained patterns useful for classifying highly familiar versus unfamiliar faces. Classification along the temporal-sequence of the face blocks revealed an early separation of neural patterns elicited in response to highly familiar versus unfamiliar faces in the FFA and OFA, but not in other regions of interest. This indicates the potential for a rapid assessment of the "known versus unknown" status of faces in core face-selective regions of the brain. The present study provides a first look at the perceptual and neural correlates underlying experience gains with faces as they become familiar.

The importance of internal facial features in learning new faces

For familiar faces, the internal features (eyes, nose, and mouth) are known to be differentially salient for recognition compared to external features such as hairstyle. Two experiments are reported that investigate how this internal feature advantage accrues as a face becomes familiar. In Experiment 1, we tested the contribution of internal and external features to the ability to generalize from a single studied photograph to different views of the same face. A recognition advantage for the internal features over the external features was found after a change of viewpoint, whereas there was no internal feature advantage when the same image was used at study and test. In Experiment 2, we removed the most salient external feature (hairstyle) from studied photographs and looked at how this affected generalization to a novel viewpoint. Removing the hair from images of the face assisted generalization to novel viewpoints, and this was especially the case when photographs showing more than one viewpoint were studied. The results suggest that the internal features play an important role in the generalization between different images of an individual's face by enabling the viewer to detect the common identity-diagnostic elements across non-identical instances of the face.

Why has research in face recognition progressed so slowly? The importance of variability

Despite many years of research, there has been surprisingly little progress in our understanding of how faces are identified. Here I argue that there are two contributory factors: (a) Our methods have obscured a critical aspect of the problem, within-person variability; and (b) research has tended to conflate familiar and unfamiliar face processing. Examples of procedures for studying variability are given, and a case is made for studying real faces, of the type people recognize every day. I argue that face recognition (specifically identification) may only be understood by adopting new techniques that acknowledge statistical patterns in the visual environment. As a consequence, some of our current methods will need to be abandoned.

fMRI analysis of contrast polarity in face-selective cortex in humans and monkeys

Recognition is strongly impaired when the normal contrast polarity of faces is reversed. For instance, otherwise-familiar faces become very difficult to recognize when viewed as photographic negatives. Here, we used fMRI to demonstrate related properties in visual cortex: 1) fMRI responses in the human Fusiform Face Area (FFA) decreased strongly (26%) to contrast-reversed faces across a wide range of contrast levels (5.3-100% RMS contrast), in all subjects tested. In a whole brain analysis, this contrast polarity bias was largely confined to the Fusiform Face Area (FFA; p<0.0001), with possible involvement of a left occipital face-selective region. 2) It is known that reversing facial contrast affects three image properties in parallel (absorbance, shading, and specular reflection). Here, comparison of FFA responses to those in V1 suggests that the contrast polarity bias is produced in FFA only when all three component properties were reversed simultaneously, which suggests a prominent non-linearity in FFA processing. 3) Across a wide range (180°) of illumination source angles, 3D face shapes without texture produced response constancy in FFA, without a contrast polarity bias. 4) Consistent with psychophysics, analogous fMRI biases for normal contrast polarity were not produced by non-face objects, with image statistics similar to the face stimuli. 5) Using fMRI, we also demonstrated a contrast polarity bias in awake behaving macaque monkeys, in the cortical region considered homologous to human FFA. Thus common cortical mechanisms may underlie facial contrast processing across ~25 million years of primate evolution.

Facial Coding is Disrupted at Equiluminance

Human face recognition is disrupted by the reversal of luminance contrast polarity (ie photo negatives—see Galper 1970 Psychonomic Science19 207–208; Johnston et al 1992 Perception21 365–375), while recognition of other objects is less impacted (Nederhouser et al 2007 Vision Research47 2134–2142; Subramaniam and Biederman 1997 Investigative Ophthalmology & Visual Science38 998). This suggests that correct patterns of luminance contrast are important for facial coding. Here we investigate this further by minimising luminance contrast. We contrast peoples' ability to categorise cars and faces when images vary in luminance and when images are altered to predominantly contain differences in colour (equiluminance). Eliminating luminance contrast had a greater adverse impact on facial classifications relative to car categorisations. This was true even though precautions were taken to equate visibility, and despite equal levels of performance when images contained luminance contrast. These results were not due to images containing markedly different spectra, as the effect persisted for facial images altered to match car images in this regard, and performance in both tasks dropped off proportionally with increasing levels of image blur. Finally, consistent with previous observations, we show that facial coding is not only adversely impacted at equiluminance but becomes even worse when the polarity of luminance contrast is reversed. Our data show that the correct pattern of luminance contrast is very important for facial coding. We suggest that this is related to the role of luminance contrast in signalling 3-D shape from shading.