Information and viewpoint dependence in face recognition

Multiple images captured from a single encounter do not promote face learning

Viewing multiple images of a newly encountered face improves recognition of that identity in new instances. Studies examining face learning have presented high-variability (HV) images that incorporate changes that occur from moment-to-moment (e.g., head orientation and expression) and over time (e.g., lighting, hairstyle, and health). We examined whether low-variability (LV) images (i.e., images that incorporate only moment-to-moment changes) also promote generalisation of learning such that novel instances are recognised. Participants viewed a single image, six LV images, or six HV images of a target identity before being asked to recognise novel images of that identity in a face matching task (training stimuli remained visible) or a memory task (training stimuli were removed). In Experiment 1 (n = 71), participants indicated which image(s) in 8-image arrays belonged to the target identity. In Experiment 2 (n = 73), participants indicated whether sequentially presented images belonged to the target identity. Relative to the single-image condition, sensitivity to identity improved and response biases were less conservative in the HV condition; we found no evidence of generalisation of learning in the LV condition regardless of testing protocol. Our findings suggest that day-to-day variability in appearance plays an essential role in acquiring expertise with a novel face.

Effect of Action Units, Viewpoint and Immersion on Emotion Recognition Using Dynamic Virtual Faces

Facial affect recognition is a critical skill in human interactions that is often impaired in psychiatric disorders. To address this challenge, tests have been developed to measure and train this skill. Recently, virtual human (VH) and virtual reality (VR) technologies have emerged as novel tools for this purpose. This study investigates the unique contributions of different factors in the communication and perception of emotions conveyed by VHs. Specifically, it examines the effects of the use of action units (AUs) in virtual faces, the positioning of the VH (frontal or mid-profile), and the level of immersion in the VR environment (desktop screen versus immersive VR). Thirty-six healthy subjects participated in each condition. Dynamic virtual faces (DVFs), VHs with facial animations, were used to represent the six basic emotions and the neutral expression. The results highlight the important role of the accurate implementation of AUs in virtual faces for emotion recognition. Furthermore, it is observed that frontal views outperform mid-profile views in both test conditions, while immersive VR shows a slight improvement in emotion recognition. This study provides novel insights into the influence of these factors on emotion perception and advances the understanding and application of these technologies for effective facial emotion recognition training.

Dynamic Face Perception: The Role of Expertise in Dual Processing of Features and Configuration

Face perception is the basis of many types of social information exchange, but there is controversy over its underlying mechanisms. Researchers have theorized two processing pathways underlying facial perception: configural processing and featural processing. Featural processing focuses on the individual features of a face, whereas configural processing focuses on the spatial relations of features. To resolve the debate on the relative contribution of the two pathways in face perception, researchers have proposed a dual processing model that the two pathways contribute to two different perceptions, detecting face-like patterns and identifying individual faces. The dual processing model is based on face perception experiments that primarily use static faces. As we mostly interact with dynamic faces in real life, the generalization of the model to dynamic faces will advance our understanding of how faces are perceived in real life. This paper proposes a refined dual processing model of dynamic face perception, in which expertise in dynamic face perception supports identifying individual faces, and it is a learned behaviour that develops with age. Specifically, facial motions account for the advantages of dynamic faces, compared to static faces. This paper highlights two intrinsic characteristics of facial motions that enable the advantages of dynamic faces in face perception. Firstly, facial motion provides facial information from various viewpoints, and thus supports the generalization of face perception to the unlearned view of faces. Secondly, distinctive motion patterns serve as a cue to the identity of the face.

Perceptual similarity modulates effects of learning from variability on face recognition

Face recognition is a challenging classification task that humans perform effortlessly for familiar faces. Recent studies have emphasized the importance of exposure to high variability appearances of the same identity to perform this task. However, these studies did not explicitly measure the perceptual similarity between the learned images and the images presented at test, which may account for the advantage of learning from high variability. Particularly, randomly selected test images are more likely to be perceptually similar to learned high variability images, and dissimilar to learned low variability images. Here we dissociated effects of learning from variability and study-test perceptual similarity, by collecting human similarity ratings for the study and test images. Using these measures, we independently manipulated the variability between the learning images and their perceptual similarity to the test images. Different groups of participants learned face identities from a low or high variability set of images. The learning phase was followed by a face matching test (Experiment 1) or a face recognition task (Experiment 2) that presented novel images of the learned identities that were perceptually dissimilar or similar to the learned images. Results of both experiments show that perceptual similarity between study and test, rather than image variability at learning per se, predicts face recognition. We conclude that learning from high variability improves face recognition for perceptually similar but not for perceptually dissimilar images. These findings may not be specific to faces and should be similarly evaluated for other domains.

Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models

Discerning the subtle differences between individuals’ faces is crucial for social functioning. It requires us not only to solve general challenges of object recognition (e.g., invariant recognition over changes in view or lighting) but also to be attuned to the specific ways in which face structure varies. Three-dimensional morphable models based on principal component analyses of real faces provide descriptions of statistical differences between faces, as well as tools to generate novel faces. We rendered large sets of realistic face pairs from such a model and collected similarity and same/different identity judgments. The statistical model predicted human perception as well as state-of-the-art image-computable neural networks. Results underscore the statistical tuning of face encoding.

Human vision is attuned to the subtle differences between individual faces. Yet we lack a quantitative way of predicting how similar two face images look and whether they appear to show the same person. Principal component–based three-dimensional (3D) morphable models are widely used to generate stimuli in face perception research. These models capture the distribution of real human faces in terms of dimensions of physical shape and texture. How well does a “face space” based on these dimensions capture the similarity relationships humans perceive among faces? To answer this, we designed a behavioral task to collect dissimilarity and same/different identity judgments for 232 pairs of realistic faces. Stimuli sampled geometric relationships in a face space derived from principal components of 3D shape and texture (Basel face model [BFM]). We then compared a wide range of models in their ability to predict the data, including the BFM from which faces were generated, an active appearance model derived from face photographs, and image-computable models of visual perception. Euclidean distance in the BFM explained both dissimilarity and identity judgments surprisingly well. In a comparison against 16 diverse models, BFM distance was competitive with representational distances in state-of-the-art deep neural networks (DNNs), including novel DNNs trained on BFM synthetic identities or BFM latents. Models capturing the distribution of face shape and texture across individuals are not only useful tools for stimulus generation. They also capture important information about how faces are perceived, suggesting that human face representations are tuned to the statistical distribution of faces.

Staring death in the face: chimpanzees' attention towards conspecific skulls and the implications of a face module guiding their behaviour

Chimpanzees exhibit a variety of behaviours surrounding their dead, although much less is known about how they respond towards conspecific skeletons. We tested chimpanzees' visual attention to images of conspecific and non-conspecific stimuli (cat/chimp/dog/rat), shown simultaneously in four corners of a screen in distinct orientations (frontal/diagonal/lateral) of either one of three types (faces/skulls/skull-shaped stones). Additionally, we compared their visual attention towards chimpanzee-only stimuli (faces/skulls/skull-shaped stones). Lastly, we tested their attention towards specific regions of chimpanzee skulls. We theorized that chimpanzee skulls retaining face-like features would be perceived similarly to chimpanzee faces and thus be subjected to similar biases. Overall, supporting our hypotheses, the chimpanzees preferred conspecific-related stimuli. The results showed that chimpanzees attended: (i) significantly longer towards conspecific skulls than other species skulls (particularly in forward-facing and to a lesser extent diagonal orientations); (ii) significantly longer towards conspecific faces than other species faces at forward-facing and diagonal orientations; (iii) longer towards chimpanzee faces compared with chimpanzee skulls and skull-shaped stones, and (iv) attended significantly longer to the teeth, similar to findings for elephants. We suggest that chimpanzee skulls retain relevant, face-like features that arguably activate a domain-specific face module in chimpanzees' brains, guiding their attention.

The sizable difficulty in matching unfamiliar faces differing only moderately in orientation in depth is a function of image dissimilarity

Attempting to match unfamiliar, highly similar faces at moderate differences in orientation in depth is surprisingly difficult. No neurocomputational account of these costs that addressed the representation of faces by which a face-similarity metric can be derived has been offered. A metric specifying the similarity of the to-be-distinguished faces is required as the rotation costs will be a function of the difficulty in distinguishing the faces. Consequently, rotation costs have typically been described in terms of angle of disparity, rather than the dissimilarity of the faces produced by the rotation. We assessed the effects of orientation disparity in a match-to-sample paradigm of a simultaneous presentation of a triangular display of three faces. Two lower test faces, a matching face and a foil, were always at the same orientation and differed by 0° to 20° from the sample on top. The similarity of the images was scaled by a model based on simple cell tuning, modeled as Gabor wavelets, that correlates almost perfectly with psychophysical similarity. Two measures of face similarity, with approximately additive effects on reaction times, accounted for matching performance: a) the decrease in similarity between the images of the matching and sample faces produced by increases in their orientation disparity, and b) the similarity between the matching face and the selection of a particular foil. The 20° orientation disparity was sufficient to yield a sizeable 301 msec increase in reaction time. An implication of the results is that the activity in V1 produced by viewing a face is fed forward to areas responsible for the individuation of that face.

Within-person variability can improve the identification of unfamiliar faces across changes in viewpoint

Unfamiliar face identification is concerningly error prone, especially across changes in viewing conditions. Within-person variability has been shown to improve matching performance for unfamiliar faces, but this has only been demonstrated using images of a front view. In this study, we test whether the advantage of within-person variability from front views extends to matching to target images of a face rotated in view. Participants completed either a simultaneous matching task (Experiment 1) or a sequential matching task (Experiment 2) in which they were tested on their ability to match the identity of a face shown in an array of either one or three ambient front-view images, with a target image shown in front, three-quarter, or profile view. While the effect was stronger in Experiment 2, we found a consistent pattern in match trials across both experiments in that there was a multiple image matching benefit for front, three-quarter, and profile-view targets. We found multiple image effects for match trials only, indicating that providing observers with multiple ambient images confers an advantage for recognising different images of the same identity but not for discriminating between images of different identities. Signal detection measures also indicate a multiple image advantage despite a more liberal response bias for multiple image trials. Our results show that within-person variability information for unfamiliar faces can be generalised across views and can provide insights into the initial processes involved in the representation of familiar faces.

The roles of gaze and head orientation in face categorization during rapid serial visual presentation

Little is known about how perceived gaze direction and head orientation may influence human categorization of visual stimuli as faces. To address this question, a sequence of unsegmented natural images, each containing a random face or a non-face object, was presented in rapid succession (stimulus duration: 91.7 ms per image) during which human observers were instructed to respond immediately to every face presentation. Faces differed in gaze and head orientation in 7 combinations - full-front views with perceived gaze (1) directed to the observer, (2) averted to the left, or (3) averted to the right, left ¾ side views with (4) direct gaze or (5) averted gaze, and right ¾ side views with (6) direct gaze or (7) averted gaze - were presented randomly throughout the sequence. We found highly accurate and rapid behavioural responses to all kinds of faces. Crucially, both perceived gaze direction and head orientation had comparable, non-interactive effects on response times, where direct gaze was responded faster than averted gaze by 48 ms and full-front view faster than ¾ side view also by 48 ms on average. Presentations of full-front faces with direct gaze led to an additive speed advantage of 96 ms to ¾ faces with averted gaze. The results reveal that the effects of perceived gaze direction and head orientation on the speed of face categorization probably depend on the degree of social relevance of the face to the viewer.

Perpetrator pose reinstatement during a lineup test increases discrimination accuracy

We examined how encoding view influences the information that is stored in and retrieved from memory during an eyewitness identification task. Participants watched a mock crime and we varied the angle from which they viewed the perpetrator. In Experiment 1, participants (N = 2904) were tested with a static photo lineup; the viewing angle of the lineup members was the same or different from the perpetrator at encoding. In Experiment 2, participants (N = 1430) were tested with a novel interactive lineup in which they could rotate the lineup faces into any angle. In both experiments, discrimination accuracy was greater when the viewing angle at encoding and test matched. Participants reinstated the angle of the interactive faces to match their encoding angle. Our results highlight the importance of encoding specificity for eyewitness identification, and show that people actively seek out information in the testing environment that matches the study environment to aid memory retrieval.

Does automatic human face categorization depend on head orientation?

Whether human categorization of visual stimuli as faces is optimal for full-front views, best revealing diagnostic features but lacking depth cues, remains largely unknown. To address this question, we presented 16 human observers with unsegmented natural images of different living and non-living objects at a fast rate (f = 12 Hz), with natural face images appearing at f/9 = 1.33 Hz. Faces posing all full-front or at ¾ side view angles appeared in separate sequences. Robust frequency-tagged 1.33 Hz (and harmonic) occipito-temporal electroencephalographic (EEG) responses reflecting face-selective neural activity did not differ in overall amplitude between full-front and ¾ side views. Despite this, alternating between full-front and ¾ side views within a sequence led to significant responses at specific harmonics of .67 Hz (f/18), objectively isolating view-dependent face-selective responses over occipito-temporal regions. Critically, a time-domain analysis showed that these view-dependent face-selective responses reflected only an earlier response to full-front than ¾ side views by 8-13 ms. Overall, these findings indicate that the face-selective neural representation is as robust for ¾ side faces as for full-front faces in the human brain, but full-front views provide a slightly earlier processing-time advantage as compared to rotated face views.

Performance of typical and superior face recognizers on a novel interactive face matching procedure

Unfamiliar simultaneous face matching is error prone. Reducing incorrect identification decisions will positively benefit forensic and security contexts. The absence of view-independent information in static images likely contributes to the difficulty of unfamiliar face matching. We tested whether a novel interactive viewing procedure that provides the user with 3D structural information as they rotate a facial image to different orientations would improve face matching accuracy. We tested the performance of 'typical' (Experiment 1) and 'superior' (Experiment 2) face recognizers, comparing their performance using high-quality (Experiment 3) and pixelated (Experiment 4) Facebook profile images. In each trial, participants responded whether two images featured the same person with one of these images being either a static face, a video providing orientation information, or an interactive image. Taken together, the results show that fluid orientation information and interactivity prompt shifts in criterion and support matching performance. Because typical and superior face recognizers both benefited from the structural information provided by the novel viewing procedures, our results point to qualitatively similar reliance on pictorial encoding in these groups. This also suggests that interactive viewing tools can be valuable in assisting face matching in high-performing practitioner groups.

The effect of face masks and sunglasses on identity and expression recognition with super-recognizers and typical observers

Face masks present a new challenge to face identification (here matching) and emotion recognition in Western cultures. Here, we present the results of three experiments that test the effect of masks, and also the effect of sunglasses (an occlusion that individuals tend to have more experienced with) on (i) familiar face matching, (ii) unfamiliar face matching and (iii) emotion categorization. Occlusion reduced accuracy in all three tasks, with most errors in the mask condition; however, there was little difference in performance for faces in masks compared with faces in sunglasses. Super-recognizers, people who are highly skilled at matching unconcealed faces, were impaired by occlusion, but at the group level, performed with higher accuracy than controls on all tasks. Results inform psychology theory with implications for everyday interactions, security and policing in a mask-wearing society.

Understanding face identification through within-person variability in appearance: Introduction to a virtual special issue

In the effort to determine the cognitive processes underlying the identification of faces, the dissimilarities between images of different people have long been studied. In contrast, the inherent variability between different images of the same face has either been treated as a nuisance variable that should be eliminated from psychological experiments or it has not been considered at all. Over the past decade, research efforts have increased substantially to demonstrate that this within-person variation is meaningful and can give insight into various processes of face identification, such as identity matching, face learning, and familiar face recognition. In this virtual special issue of the Quarterly Journal of Experimental Psychology, we explain the importance of within-person variability for face identification and bring together recent relevant articles published in the journal.

Does stereopsis improve face identification? A study using a virtual reality display with integrated eye-tracking and pupillometry

Stereopsis is a powerful depth cue for humans, which may also contribute to object recognition. In particular, we surmise that face identification would benefit from the availability of stereoscopic depth cues, since facial perception may be based on three-dimensional (3D) representations. In this study, a virtual reality (VR) headset with integrated eye-tracking was used to present stereoscopic images of faces. As a monoscopic contrast condition, identical images of faces were displayed to the two eyes. We monitored the participants' gaze behavior and pupil diameters while they performed a sample-to-match face identification task. We found that accuracy was superior in the stereoscopic condition compared to the monoscopic condition for frontal and intermediate views, but not profiles. Moreover, pupillary diameters were smaller when identifying stereoscopically seen faces than when viewing them without stereometric cues, which we interpret as lower processing load for the former than the latter conditions. The analysis of gaze showed that participants tended to focus on regions of the face rich in volumetric information, more so in the stereoscopic condition than the monoscopic condition. Together, these findings suggest that a 3D representation of faces may be the natural format used by the visual system when assessing face identity. Stereoscopic information, by providing depth information, assists the construction of robust facial representations in memory.

The 4D Space-Time Dimensions of Facial Perception

Facial information is a powerful channel for human-to-human communication. Characteristically, faces can be defined as biological objects that are four-dimensional (4D) patterns, whereby they have concurrently a spatial structure and surface as well as temporal dynamics. The spatial characteristics of facial objects contain a volume and surface in three dimensions (3D), namely breadth, height and importantly, depth. The temporal properties of facial objects are defined by how a 3D facial structure and surface evolves dynamically over time; where time is referred to as the fourth dimension (4D). Our entire perception of another’s face, whether it be social, affective or cognitive perceptions, is therefore built on a combination of 3D and 4D visual cues. Counterintuitively, over the past few decades of experimental research in psychology, facial stimuli have largely been captured, reproduced and presented to participants with two dimensions (2D), while remaining largely static. The following review aims to advance and update facial researchers, on the recent revolution in computer-generated, realistic 4D facial models produced from real-life human subjects. We delve in-depth to summarize recent studies which have utilized facial stimuli that possess 3D structural and surface cues (geometry, surface and depth) and 4D temporal cues (3D structure + dynamic viewpoint and movement). In sum, we have found that higher-order perceptions such as identity, gender, ethnicity, emotion and personality, are critically influenced by 4D characteristics. In future, it is recommended that facial stimuli incorporate the 4D space-time perspective with the proposed time-resolved methods.

Multi-Item Discriminability Pattern to Faces in Developmental Prosopagnosia Reveals Distinct Mechanisms of Face Processing

Previous studies have shown that individuals with developmental prosopagnosia (DP) show specific deficits in face processing. However, the mechanism underlying the deficits remains largely unknown. One hypothesis suggests that DP shares the same mechanism as normal population, though their faces processing is disproportionally impaired. An alternative hypothesis emphasizes a qualitatively different mechanism of DP processing faces. To test these hypotheses, we instructed DP and normal individuals to perceive faces and objects. Instead of calculating accuracy averaging across stimulus items, we used the discrimination accuracy for each item to construct a multi-item discriminability pattern. We found DP's discriminability pattern was less similar to that of normal individuals when perceiving faces than perceiving objects, suggesting that DP has qualitatively different mechanism in representing faces. A functional magnetic resonance imaging study was conducted to reveal the neural basis and found that multi-voxel activation patterns for faces in the right fusiform face area and occipital face area of DP were deviated away from the mean activation pattern of normal individuals. Further, the face representation was more heterogeneous in DP, suggesting that deficits of DP may come from multiple sources. In short, our study provides the first direct evidence that DP processes faces qualitatively different from normal population.

Female horses spontaneously identify a photograph of their keeper, last seen six months previously

Horses are capable of identifying individual conspecifics based on olfactory, auditory or visual cues. However, this raises the questions of their ability to recognize human beings and on the basis of what cues. This study investigated whether horses could differentiate between a familiar and unfamiliar human from photographs of faces. Eleven horses were trained on a discrimination task using a computer-controlled screen, on which two photographs were presented simultaneously (32 trials/session): touching one was rewarded (S+) and the other not (S−). In the training phase, the S+ faces were of four unfamiliar people which gradually became familiar over the trials. The S− faces were novel for each trial. After the training phase, the faces of the horses’ keepers were presented opposite novel faces to test whether the horses could identify the former spontaneously. A reward was given whichever face was touched to avoid any possible learning effect. Horses touched the faces of keepers significantly more than chance, whether it was their current keeper or one they had not seen for six months (t = 3.65; p < 0.004 and t = 6.24; p < 0.0001). Overall, these results show that horses have advanced human face-recognition abilities and a long-term memory of those human faces.

Viewpoint Invariance of Eye Size Illusion Caused by Eyeshadow

Previous research found that application of eyeshadow on the upper eyelids induces overestimation of eye size. The present study examined whether or not this eyeshadow illusion is dependent on viewpoint. We created a three-dimensional model of a female face and manipulated the presence/absence of eyeshadow and face orientation around the axis of yaw (Experiment 1) or pitch (Experiment 2) rotation. Using the staircase method, we measured perceived eye size for each face stimulus. Results showed that the eyeshadow illusion occurred regardless of face orientation around axes of both yaw and pitch rotations. Crucially, the illusion’s magnitude did not vary across face orientations; lack of interaction between the illusion’s magnitude and face orientation was confirmed by small values of Bayes factors. These findings ruled out the hypothesis that eyeshadow serves as a depth cue and leads to overestimation of eye size due to size-distance scaling. Alternatively, the present findings suggest that the eyeshadow illusion can be well explained by the assimilation between the eyes and eyeshadow, which also facilitates assimilation between the eyes and eyebrows. Practical implications and the present findings’ generalizability are also discussed.

View specific generalisation effects in face recognition: Front and yaw comparison views are better than pitch

It can be difficult to recognise new instances of an unfamiliar face. Recognition errors in this particular situation appear to be viewpoint dependent with error rates increasing with the angular distance between the face views. Studies using front views for comparison have shown that recognising faces rotated in yaw can be difficult and that recognition of faces rotated in pitch is more challenging still. Here we investigate the extent to which viewpoint dependent face recognition depends on the comparison view. Participants were assigned to one of four different comparison view groups: front, ¾ yaw (right), ¾ pitch-up (above) or ¾ pitch-down (below). On each trial, participants matched their particular comparison view to a range of yaw or pitch rotated test views. Results showed that groups with a front or ¾ yaw comparison view had superior overall performance and more successful generalisation to a broader range of both pitch and yaw test views compared to groups with pitch-up or pitch-down comparison views, both of which had a very restricted generalisation range. Regression analyses revealed the importance of image similarity between views for generalisation, with a lesser role for 3D face depth. These findings are consistent with a view interpolation solution to view generalisation of face recognition, with front and ¾ yaw views being most informative.

Three-Quarter Views of Depth-Rotated Faces Induce Face-Specific Capacity Limits in Visual Search

Abstract. Participants made speeded categorization decisions regarding a famous person (politician or film star) accompanied by a peripheral distracter face (either the same or from the opposite category). The first experiment found that processing a peripheral distracter face is independent of load when the search set contains name strings. The search set in the second experiment consisted of faces. Interference effects between the target and distracter face (both shown in frontal views) were found when no additional non-target faces were present (low load), but not when two non-famous faces (high load) accompanied the target face, even when the latter were shown in three-quarter views. These results indicate that face-specific capacity limitations are independent of changes in view (up to 45°) and gaze direction.

Orientation Encoding and Viewpoint Invariance in Face Recognition: Inferring Neural Properties from Large-Scale Signals

Viewpoint-invariant face recognition is thought to be subserved by a distributed network of occipitotemporal face-selective areas that, except for the human anterior temporal lobe, have been shown to also contain face-orientation information. This review begins by highlighting the importance of bilateral symmetry for viewpoint-invariant recognition and face-orientation perception. Then, monkey electrophysiological evidence is surveyed describing key tuning properties of face-selective neurons-including neurons bimodally tuned to mirror-symmetric face-views-followed by studies combining functional magnetic resonance imaging (fMRI) and multivariate pattern analyses to probe the representation of face-orientation and identity information in humans. Altogether, neuroimaging studies suggest that face-identity is gradually disentangled from face-orientation information along the ventral visual processing stream. The evidence seems to diverge, however, regarding the prevalent form of tuning of neural populations in human face-selective areas. In this context, caveats possibly leading to erroneous inferences regarding mirror-symmetric coding are exposed, including the need to distinguish angular from Euclidean distances when interpreting multivariate pattern analyses. On this basis, this review argues that evidence from the fusiform face area is best explained by a view-sensitive code reflecting head angular disparity, consistent with a role of this area in face-orientation perception. Finally, the importance is stressed of explicit models relating neural properties to large-scale signals.

Face Processing Systems: From Neurons to Real-World Social Perception

Primate face processing depends on a distributed network of interlinked face-selective areas composed of face-selective neurons. In both humans and macaques, the network is divided into a ventral stream and a dorsal stream, and the functional similarities of the areas in humans and macaques indicate they are homologous. Neural correlates for face detection, holistic processing, face space, and other key properties of human face processing have been identified at the single neuron level, and studies providing causal evidence have established firmly that face-selective brain areas are central to face processing. These mechanisms give rise to our highly accurate familiar face recognition but also to our error-prone performance with unfamiliar faces. This limitation of the face system has important implications for consequential situations such as eyewitness identification and policing.

About Face: Matching Unfamiliar Faces Across Rotations of View and Lighting

Matching the identities of unfamiliar faces is heavily influenced by variations in their images. Changes to viewpoint and lighting direction during face perception are commonplace across yaw and pitch axes and can result in dramatic image differences. We report two experiments that, for the first time, factorially investigate the combined effects of lighting and view angle on matching performance for unfamiliar faces. The use of three-dimensional head models allowed control of both lighting and viewpoint. We found viewpoint effects in the yaw axis with little to no effect of lighting. However, for rotations about the pitch axis, there were both viewpoint and lighting effects and these interacted where lighting effects were found only for front views and views from below. The pattern of effects was similar regardless of whether view variation occurred as a result of head (Experiment 1) or camera (Experiment 2) suggesting that face matching is not purely image based. Along with face inversion effects in Experiment 1, the results of this study suggest that face perception is based on shape and surface information and draws on implicit knowledge of upright faces and ecological (top) lighting conditions.

Sheep recognize familiar and unfamiliar human faces from two-dimensional images

One of the most important human social skills is the ability to recognize faces. Humans recognize familiar faces easily, and can learn to identify unfamiliar faces from repeatedly presented images. Sheep are social animals that can recognize other sheep as well as familiar humans. Little is known, however, about their holistic face-processing abilities. In this study, we trained eight sheep (Ovis aries) to recognize the faces of four celebrities from photographic portraits displayed on computer screens. After training, the sheep chose the 'learned-familiar' faces rather than the unfamiliar faces significantly above chance. We then tested whether the sheep could recognize the four celebrity faces if they were presented in different perspectives. This ability has previously been shown only in humans. Sheep successfully recognized the four celebrity faces from tilted images. Interestingly, there was a drop in performance with the tilted images (from 79.22 ± 7.5% to 66.5 ± 4.1%) of a magnitude similar to that seen when humans perform this task. Finally, we asked whether sheep could recognize a very familiar handler from photographs. Sheep identified the handler in 71.8 ± 2.3% of the trials without pretraining. Together these data show that sheep have advanced face-recognition abilities, comparable with those of humans and non-human primates.

On Response Bias in the Face Congruency Effect for Internal and External Features

Some years ago Cheung et al. (2008) proposed the complete design (CD) for measuring the failure of selective attention in composite objects. Since the CD is a fully balanced design, analysis of response bias may reveal potential effects of the experimental manipulation, the stimulus material, and/or attributes of the observers. Here we used the CD to prove whether external features modulate perception of internal features with the context congruency paradigm (Nachson et al., 1995; Meinhardt-Injac et al., 2010) in a larger sample of N = 303 subjects. We found a large congruency effect (Cohen's d = 1.78), which was attenuated by face inversion (d = 1.32). The congruency relation also strongly modulated response bias. In incongruent trials the proportion of "different" responses was much larger than in congruent trials (d = 0.79), which was again attenuated by face inversion (d = 0.43). Because in incongruent trials the wholes formed by integrating external and internal features are always different, while in congruent trials same and different wholes occur with the same frequency, a congruency related bias effect is expected from holistic integration. Our results suggest two behavioral markers of holistic processing in the context congruency paradigm: a performance advantage in congruent compared to incongruent trials, and a tendency toward more "different" responses in incongruent, compared to congruent trials. Since the results for both markers differed only quantitatively in upright and inverted presentation, our findings indicate no change of the face processing mode by picture plane rotation. A potential transfer to the composite face paradigm is discussed.

Contributions of individual face features to face discrimination

Faces are highly complex stimuli that contain a host of information. Such complexity poses the following questions: (a) do observers exhibit preferences for specific information? (b) how does sensitivity to individual face parts compare? These questions were addressed by quantifying sensitivity to different face features. Discrimination thresholds were determined for synthetic faces under the following conditions: (i) 'full face': all face features visible; (ii) 'isolated feature': single feature presented in isolation; (iii) 'embedded feature': all features visible, but only one feature modified. Mean threshold elevations for isolated features, relative to full-faces, were 0.84x, 1.08, 2.12, 3.34, 4.07 and 4.47 for head-shape, hairline, nose, mouth, eyes and eyebrows respectively. Hence, when two full faces can be discriminated at threshold, the difference between the eyes is about four times less than what is required when discriminating between isolated eyes. In all cases, sensitivity was higher when features were presented in isolation than when they were embedded within a face context (threshold elevations of 0.94x, 1.74, 2.67, 2.90, 5.94 and 9.94). This reveals a specific pattern of sensitivity to face information. Observers are between two and four times more sensitive to external than internal features. The pattern for internal features (higher sensitivity for the nose, compared to mouth, eyes and eyebrows) is consistent with lower sensitivity for those parts affected by facial dynamics (e.g. facial expressions). That isolated features are easier to discriminate than embedded features supports a holistic face processing mechanism which impedes extraction of information about individual features from full faces.

Recognition of human faces by dogs (Canis familiaris) requires visibility of head contour

Researchers have suggested that dogs are able to recognise human faces, but conclusive evidence has yet to be found. Experiment 1 of this study investigated whether dogs can recognise humans using visual information from the face/head region, and whether this also occurs in conditions of suboptimal visibility of the face. Dogs were presented with their owner's and a stranger's heads, protruding through openings of an apparatus in opposite parts of the experimental setting. Presentations occurred in conditions of either optimal or suboptimal visibility; the latter featured non-frontal orientation, uneven illumination and invisibility of outer contours of the heads. Instances where dogs approached their owners with a higher frequency than predicted by chance were considered evidence of recognition. This occurred only in the optimal condition. With a similar paradigm, Experiment 2 investigated which of the alterations in visibility that characterised the suboptimal condition accounted for dogs' inability to recognise owners. Dogs approached their owners more frequently than predicted by chance if outer head contours were visible, but not if heads were either frontally oriented or evenly illuminated. Moreover, male dogs were slightly better at recognition than females. These findings represent the first clear demonstration that dogs can recognise human faces and that outer face elements are crucial for such a task, complementing previous research on human face processing in dogs. Parallels with face recognition abilities observed in other animal species, as well as with human infants, point to the relevance of these results from a comparative standpoint.

Face recognition is similarly affected by viewpoint in school-aged children and adults

There is an ongoing debate on the question when face processing abilities mature. One aspect that has been part of this debate is the ability to recognize faces in and across different viewpoints. Here, we tested 128 participants consisting of school-age children (ages, 5–10 years) and adults (ages, 19–37 years) in two experiments to investigate the effects of different viewpoints (including front, three-quarter, profile view) on face recognition during development. Furthermore, we compared recognition performance for faces to that of another object category (cars). In the first experiment (n = 88) we tested if the pattern of performance for faces presented in different viewpoints is similar in school-aged children and adults. Participants completed a two-alternative-forced-choice (2AFC) memory task comprising images of both faces and cars in front, three-quarter and profile view, which were presented in the same viewpoint during learning and testing. In the second experiment (n = 40) we tested if face recognition is similarly affected by viewpoint changes in children and adults. In this experiment the 2AFC memory task included a change of viewpoint between learning and testing. While in both experiments we found higher recognition performance for faces with increasing age, the overall pattern of both viewpoint and viewpoint-change-effects and also the difference between view-change- and no-change-conditions was similar across age groups. In contrast to faces, no viewpoint effects were observed in cars (experiment 1), viewpoint change effects, however, were similar for cars and faces (experiment 2). In sum, our results suggest early maturity of the ability to recognize faces in and across different viewpoints.

The utility of multiple synthesized views in the recognition of unfamiliar faces

The ability to recognize an unfamiliar individual on the basis of prior exposure to a photograph is notoriously poor and prone to errors, but recognition accuracy is improved when multiple photographs are available. In applied situations, when only limited real images are available (e.g., from a mugshot or CCTV image), the generation of new images might provide a technological prosthesis for otherwise fallible human recognition. We report two experiments examining the effects of providing computer-generated additional views of a target face. In Experiment 1, provision of computer-generated views supported better target face recognition than exposure to the target image alone and equivalent performance to that for exposure of multiple photograph views. Experiment 2 replicated the advantage of providing generated views, but also indicated an advantage for multiple viewings of the single target photograph. These results strengthen the claim that identifying a target face can be improved by providing multiple synthesized views based on a single target image. In addition, our results suggest that the degree of advantage provided by synthesized views may be affected by the quality of synthesized material.

Image Dependency in the Recognition of Newly Learnt Faces

Research investigating the effect of lighting and viewpoint changes on unfamiliar and newly learnt faces has revealed that such recognition is highly image dependent and that changes in either of these leads to poor recognition accuracy. Three experiments are reported to extend these findings by examining the effect of apparent age on the recognition of newly learnt faces. Experiment 1 investigated the ability to generalize to novel ages of a face after learning a single image. It was found that recognition was best for the learnt image with performance falling the greater the dissimilarity between the study and test images. Experiments 2 and 3 examined whether learning two images aids subsequent recognition of a novel image. The results indicated that interpolation between two studied images (Experiment 2) provided some additional benefit over learning a single view, but that this did not extend to extrapolation (Experiment 3). The results from all studies suggest that recognition was driven primarily by pictorial codes and that the recognition of faces learnt from a limited number of sources operates on stored images of faces as opposed to more abstract, structural, representations.

Face familiarity promotes stable identity recognition: exploring face perception using serial dependence

Studies suggest that familiar faces are processed in a manner distinct from unfamiliar faces and that familiarity with a face confers an advantage in identity recognition. Our visual system seems to capitalize on experience to build stable face representations that are impervious to variation in retinal input that may occur due to changes in lighting, viewpoint, viewing distance, eye movements, etc. Emerging evidence also suggests that our visual system maintains a continuous perception of a face's identity from one moment to the next despite the retinal input variations through serial dependence. This study investigates whether interactions occur between face familiarity and serial dependence. In two experiments, participants used a continuous scale to rate attractiveness of unfamiliar and familiar faces (either experimentally learned or famous) presented in rapid sequences. Both experiments revealed robust inter-trial effects in which attractiveness ratings for a given face depended on the preceding face's attractiveness. This inter-trial attractiveness effect was most pronounced for unfamiliar faces. Indeed, when participants were familiar with a given face, attractiveness ratings showed significantly less serial dependence. These results represent the first evidence that familiar faces can resist the temporal integration seen in sequential dependencies and highlight the importance of familiarity to visual cognition.

Learning the Moves: The Effect of Familiarity and Facial Motion on Person Recognition across Large Changes in Viewing Format

Familiarity with a face or person can support recognition in tasks that require generalization to novel viewing contexts. Using naturalistic viewing conditions requiring recognition of people from face or whole body gait stimuli, we investigated the effects of familiarity, facial motion, and direction of learning/test transfer on person recognition. Participants were familiarized with previously unknown people from gait videos and were tested on faces (experiment 1a) or were familiarized with faces and were tested with gait videos (experiment 1b). Recognition was more accurate when learning from the face and testing with the gait videos, than when learning from the gait videos and testing with the face. The repetition of a single stimulus, either the face or gait, produced strong recognition gains across transfer conditions. Also, the presentation of moving faces resulted in better performance than that of static faces. In experiment 2, we investigated the role of facial motion further by testing recognition with static profile images. Motion provided no benefit for recognition, indicating that structure-from-motion is an unlikely source of the motion advantage found in the first set of experiments.

Things are Looking up: Differential Decline in Face Recognition following Pitch and Yaw Rotation

Previous research into the effects of viewpoint change on face recognition has typically dealt with rotations around the head's vertical axis (yaw). Another common, although less studied, source of viewpoint variation in faces is rotation around the head's horizontal pitch axis (pitch). In the current study we used both a sequential matching task and an old/new recognition task to examine the effect of viewpoint change following rotation about both pitch and yaw axes on human face recognition. The results of both tasks showed that recognition performance was better for faces rotated about yaw compared to pitch. Further, recognition performance for faces rotated upwards on the pitch axis was better than for faces rotated downwards. Thus, equivalent angular rotations about pitch and yaw do not produce equivalent viewpoint-dependent declines in recognition performance.

What is in a View? The Role of Featural Information in the Recognition of Unfamiliar Faces across Viewpoint Transformation

In recognising a face the visual system shows a remarkable ability in overcoming changes in viewpoint. However, the mechanisms involved in solving this complex computational problem, particularly in terms of information processing, have not been clearly defined. Considerable evidence indicates that face recognition involves both featural and configural processing. In this study we examined the contribution of featural information across viewpoint change. Participants were familiarised with unknown faces and were later tested for recognition in complete or part-face format, across changes in view. A striking effect of viewpoint resulting in a reduction in profile recognition compared with the three-quarter and frontal views was found. However, a complete-face over part-face advantage independent of transformation was demonstrated across all views. A hierarchy of feature salience was also demonstrated. Findings are discussed in terms of the problem of object constancy as it applies to faces.

Socially Important Faces Are Processed Preferentially to Other Familiar and Unfamiliar Faces in a Priming Task across a Range of Viewpoints

Using a priming paradigm, we investigate whether socially important faces are processed preferentially compared to other familiar and unfamiliar faces, and whether any such effects are affected by changes in viewpoint. Participants were primed with frontal images of personally familiar, famous or unfamiliar faces, and responded to target images of congruent or incongruent identity, presented in frontal, three quarter or profile views. We report that participants responded significantly faster to socially important faces (a friend's face) compared to other highly familiar (famous) faces or unfamiliar faces. Crucially, responses to famous and unfamiliar faces did not differ. This suggests that, when presented in the context of a socially important stimulus, socially unimportant familiar faces (famous faces) are treated in a similar manner to unfamiliar faces. This effect was not tied to viewpoint, and priming did not affect socially important face processing differently to other faces.

Efficient visual information for unfamiliar face matching despite viewpoint variations: It's not in the eyes!

Faces are encountered in highly diverse angles in real-world settings. Despite this considerable diversity, most individuals are able to easily recognize familiar faces. The vast majority of studies in the field of face recognition have nonetheless focused almost exclusively on frontal views of faces. Indeed, a number of authors have investigated the diagnostic facial features for the recognition of frontal views of faces previously encoded in this same view. However, the nature of the information useful for identity matching when the encoded face and test face differ in viewing angle remains mostly unexplored. The present study addresses this issue using individual differences and bubbles, a method that pinpoints the facial features effectively used in a visual categorization task. Our results indicate that the use of features located in the center of the face, the lower left portion of the nose area and the center of the mouth, are significantly associated with individual efficiency to generalize a face's identity across different viewpoints. However, as faces become more familiar, the reliance on this area decreases, while the diagnosticity of the eye region increases. This suggests that a certain distinction can be made between the visual mechanisms subtending viewpoint invariance and face recognition in the case of unfamiliar face identification. Our results further support the idea that the eye area may only come into play when the face stimulus is particularly familiar to the observer.

Effects of exposure to facial expression variation in face learning and recognition

Facial expression is a major source of image variation in face images. Linking numerous expressions to the same face can be a huge challenge for face learning and recognition. It remains largely unknown what level of exposure to this image variation is critical for expression-invariant face recognition. We examined this issue in a recognition memory task, where the number of facial expressions of each face being exposed during a training session was manipulated. Faces were either trained with multiple expressions or a single expression, and they were later tested in either the same or different expressions. We found that recognition performance after learning three emotional expressions had no improvement over learning a single emotional expression (Experiments 1 and 2). However, learning three emotional expressions improved recognition compared to learning a single neutral expression (Experiment 3). These findings reveal both the limitation and the benefit of multiple exposures to variations of emotional expression in achieving expression-invariant face recognition. The transfer of expression training to a new type of expression is likely to depend on a relatively extensive level of training and a certain degree of variation across the types of expressions.