Holistic Face Processing is Induced by Shape and Texture

From Perugino to Picasso revisited: Electrophysiological responses to faces in paintings from different art styles

Behavioral research (Ventura, et al., 2023) suggested that pictorial representations of faces varying along a realism-distortion spectrum elicit holistic processing as natural faces. Whether holistic face neural responses are engaged similarly remains, however, underexplored. In the present study, we evaluated the neural correlates of naturalist and artistic face processing, by exploring electrophysiological responses to faces in photographs versus in four major painting styles. The N170 response to faces in photographs was indistinguishable from that elicited by faces in the renaissance art style (depicting the most realistic faces), whilst both categories elicited larger N170 than faces in other art styles (post-impressionism, expressionism, and cubism), with a gradation in brain activity. The present evidence suggest that visual processing may become finer grained the more the realistic nature of the face. Despite behavioral equivalence, the neural mechanisms for holistic processing of natural faces and faces in diverse art styles are not equivalent.

The role of facial skin tone and texture in the perception of age

Age and gender perception from looking at people's faces, without any cultural or conventional cues, is primarily based on two independent components: a) the shape or facial structure, and b) surface reflectance (skin tone and texture, STT). This study examined the relative contribution of facial STT to the perception of age. A total of 204 subjects participated in four experiments presenting artificial 3D realistic faces of different age versions under two key experimental conditions: with and without STT. Two experiments involved a discrimination-age task, and other two involved a direct age-estimation task. The faces for the last experiment were generated from the photographs of real people. The results were quite consistent throughout the experiments. Data suggest that the contribution of the STT information leads to roughly 25-33 % of accuracy in age perception. Interestingly, a differential pattern emerges in relation to facial age: the relative contribution of skin information increases sharply with advancing age, to the point that age judgments of the older faces (60 years old) without STT information fall to the chance level. This pattern suggests that facial skin tone and texture are the main sources of information for estimating the age of people past their maturity as those are the principal visual signs of aging beyond the anatomical changes of facial structure.

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

We present TED-Face, a new method for recovering high-fidelity 3D facial geometry and appearance with enhanced textures from single-view images. While vision-based face reconstruction has received intensive research in the past decades due to its broad applications, it remains a challenging problem because human eyes are particularly sensitive to numerically minute yet perceptually significant details. Previous methods that seek to minimize reconstruction errors within a low-dimensional face space can suffer from this issue and generate close yet low-fidelity approximations. The loss of high-frequency texture details is a key factor in their process, which we propose to address by learning to recover both dense radiance residuals and sparse facial texture features from a single image, in addition to the variables solved by previous work-shape, appearance, illumination, and camera. We integrate the estimation of all these factors in a single unified deep neural network and train it on several popular face reconstruction datasets. We also introduce two new metrics, visual fidelity (VIF) and structural similarity (SSIM), to compensate for the fact that reconstruction error is not a consistent perceptual metric of quality. On the popular FaceWarehouse facial reconstruction benchmark, our proposed system achieves a VIF score of 0.4802 and an SSIM score of 0.9622, improving over the state-of-the-art Deep3D method by 6.69% and 0.86%, respectively. On the widely used LS3D-300W dataset, we obtain a VIF score of 0.3922 and an SSIM score of 0.9079 for indoor images, and the scores for outdoor images are 0.4100 and 0.9160, respectively, which also represent an improvement over those of Deep3D. These results show that our method is able to recover visually more realistic facial appearance details compared with previous methods.

Sex perception from facial structure: Categorization with and without skin texture and color

Sex identification of faces without any cultural or conventional sex cue is primarily based on two independent components: a) shape or facial structure, and b) surface reflectance (skin texture and color). The present work studied the relative contribution of each component by means of two experiments based on 3D face models created with different degrees of masculinity-femininity within a sex continuum. The first experiment utilized totally artificial faces created ex novo by computer. The second employed face models created from photographs of real people. The results of both experiments were consistent. As expected, when both components were present in a face, sex was correctly classified in almost all the cases. More interestingly, the contribution of the "pure" facial structure to the sex perception (with no surface reflectance) was about 80%, whereas 20% of the total information was provided by the surface reflectance. Furthermore, examination of the psychometric curves suggests that the information provided by surface reflectance contributes to a categorical perception of facial sex, since when it is removed the sex is perceived in a more continuous / less categorical way. On the other hand, our stimuli presented a certain "male" bias, repeatedly found in the literature on facial sex perception.

Overestimation of eye size: People see themselves with bigger eyes in a holistic approach

A face contains crucial information for identification; moreover, face recognition is superior to other types of recognition. Notably, one's own face is recognized better than other familiar faces. However, it is unclear whether one's own face, especially one's own internal facial features, is represented more accurately than other faces. Here, we investigated how one's own internal facial features were represented. We conducted a psychological experiment in which the participants were required to adjust eye size to the real size in photos of their own or well-known celebrities' faces. To investigate why individuals' own and celebrity facial representations were different, two types of photos were prepared, with and without external features. It was found that the accuracy of eye size for one's own face was better than that for celebrities' faces in the condition without external features, in which holistic processing was less involved than in the condition with external features. This implies that the eye size of one's own face was represented more accurately than that of other familiar faces when external features were removed. Moreover, the accuracy of the eye size of one's own face in the condition with external features was worse than that in the condition without external features; the adjusted eye size in the condition with external features was larger than that in the condition without external features. In contrast, for celebrities' faces, there was no significant difference between the conditions with and without external features. The adjusted eye sizes in all conditions were overestimated compared to real eye sizes. Previous research indicated that eye size was adjusted to a larger size when evaluating as more attractive, in which the evaluation is related to holistic processing. Based on this perspective, it could be that one's own face was represented as more attractive in the condition with external features in the current study. Taken together, the results indicated that the representation of own eye size, which is an internal facial feature, was affected by the visibility of the external features.

The composite face effect is robust against perceptual misfit

The composite face effect—the failure of selective attention toward a target face half—is frequently used to study mechanisms of feature integration in faces. Here we studied how this effect depends on the perceptual fit between attended and unattended halves. We used composite faces that were rated by trained observers as either a seamless fit (i.e., close to a natural and homogeneous face) or as a deliberately bad quality of fit (i.e., unnatural, strongly segregated face halves). In addition, composites created by combining face halves randomly were tested. The composite face effect was measured as the alignment × congruency interaction (Gauthier and Bukach Cognition, 103, 322–330 2007), but also with alternative data analysis procedures (Rossion and Boremanse Journal of Vision, 8, 1–13 2008). We found strong but identical composite effects in all fit conditions. Fit quality neither increased the composite face effect nor was it attenuated by bad or random fit quality. The implications for a Gestalt account of holistic face processing are discussed.

Sensitivity to face animacy and inversion in childhood: Evidence from EEG data

Adults exhibit relative behavioral difficulties in processing inanimate, artificial faces compared to real human faces, with implications for using artificial faces in research and designing artificial social agents. However, the developmental trajectory of inanimate face perception is unknown. To address this gap, we used electroencephalography to investigate inanimate faces processing in cross-sectional groups of 5-10-year-old children and adults. A face inversion manipulation was used to test whether face animacy processing relies on expert face processing strategies. Groups of 5-7-year-olds (N = 18), 8-10-year-olds (N = 18), and adults (N = 16) watched pictures of real or doll faces presented in an upright or inverted orientation. Analyses of event-related potentials revealed larger N170 amplitudes in response to doll faces, irrespective of age group or face orientation. Thus, the N170 is sensitive to face animacy by 5-7 years of age, but such sensitivity may not reflect high-level, expert face processing. Multivariate pattern analyses of the EEG signal additionally assessed whether animacy information could be reliably extracted during face processing. Face orientation, but not face animacy, could be reliably decoded from occipitotemporal channels in children and adults. Face animacy could be decoded from whole scalp channels in adults, but not children. Together, these results suggest that 5-10-year-old children exhibit some sensitivity to face animacy over occipitotemporal regions that is comparable to adults.

Brain Processes While Struggling With Evidence Accumulation During Facial Emotion Recognition: An ERP Study

The human brain is tuned to recognize emotional facial expressions in faces having a natural upright orientation. The relative contributions of featural, configural, and holistic processing to decision-making are as yet poorly understood. This study used a diffusion decision model (DDM) of decision-making to investigate the contribution of early face-sensitive processes to emotion recognition from physiognomic features (the eyes, nose, and mouth) by determining how experimental conditions tapping those processes affect early face-sensitive neuroelectric reflections (P100, N170, and P250) of processes determining evidence accumulation at the behavioral level. We first examined the effects of both stimulus orientation (upright vs. inverted) and stimulus type (photographs vs. sketches) on behavior and neuroelectric components (amplitude and latency). Then, we explored the sources of variance common to the experimental effects on event-related potentials (ERPs) and the DDM parameters. Several results suggest that the N170 indicates core visual processing for emotion recognition decision-making: (a) the additive effect of stimulus inversion and impoverishment on N170 latency; and (b) multivariate analysis suggesting that N170 neuroelectric activity must be increased to counteract the detrimental effects of face inversion on drift rate and of stimulus impoverishment on the stimulus encoding component of non-decision times. Overall, our results show that emotion recognition is still possible even with degraded stimulation, but at a neurocognitive cost, reflecting the extent to which our brain struggles to accumulate sensory evidence of a given emotion. Accordingly, we theorize that: (a) the P100 neural generator would provide a holistic frame of reference to the face percept through categorical encoding; (b) the N170 neural generator would maintain the structural cohesiveness of the subtle configural variations in facial expressions across our experimental manipulations through coordinate encoding of the facial features; and (c) building on the previous configural processing, the neurons generating the P250 would be responsible for a normalization process adapting to the facial features to match the stimulus to internal representations of emotional expressions.

Temporal Contiguity Training Influences Behavioral and Neural Measures of Viewpoint Tolerance

Humans can often recognize faces across viewpoints despite the large changes in low-level image properties a shift in viewpoint introduces. We present a behavioral and an fMRI adaptation experiment to investigate whether this viewpoint tolerance is reflected in the neural visual system and whether it can be manipulated through training. Participants saw training sequences of face images creating the appearance of a rotating head. Half of the sequences showed faces undergoing veridical changes in appearance across the rotation (non-morph condition). The other half were non-veridical: during rotation, the face simultaneously morphed into another face. This procedure should successfully associate frontal face views with side views of the same or a different identity, and, according to the temporal contiguity hypothesis, thus enhance viewpoint tolerance in the non-morph condition and/or break tolerance in the morph condition. Performance on the same/different task in the behavioral experiment (N = 20) was affected by training. There was a significant interaction between training (associated/not associated) and identity (same/different), mostly reflecting a higher confusion of different identities when they were associated during training. In the fMRI study (N = 20), fMRI adaptation effects were found for same-viewpoint images of untrained faces, but no adaptation for untrained faces was present across viewpoints. Only trained faces which were not morphed during training elicited a slight adaptation across viewpoints in face-selective regions. However, both in the behavioral and in the neural data the effects were small and weak from a statistical point of view. Overall, we conclude that the findings are not inconsistent with the proposal that temporal contiguity can influence viewpoint tolerance, with more evidence for tolerance when faces are not morphed during training.

The complete design in the composite face paradigm: role of response bias, target certainty, and feedback

Some years ago an improved design (the “complete design”) was proposed to assess the composite face effect in terms of a congruency effect, defined as the performance difference for congruent and incongruent target to no-target relationships (Cheung et al., 2008). In a recent paper Rossion (2013) questioned whether the congruency effect was a valid hallmark of perceptual integration, because it may contain confounds with face-unspecific interference effects. Here we argue that the complete design is well-balanced and allows one to separate face-specific from face-unspecific effects. We used the complete design for a same/different composite stimulus matching task with face and non-face objects (watches). Subjects performed the task with and without trial-by-trial feedback, and with low and high certainty about the target half. Results showed large congruency effects for faces, particularly when subjects were informed late in the trial about which face halves had to be matched. Analysis of response bias revealed that subjects preferred the “different” response in incongruent trials, which is expected when upper and lower face halves are integrated perceptually at the encoding stage. The results pattern was observed in the absence of feedback, while providing feedback generally attenuated the congruency effect, and led to an avoidance of response bias. For watches no or marginal congruency effects and a moderate global “same” bias were observed. We conclude that the congruency effect, when complemented by an evaluation of response bias, is a valid hallmark of feature integration that allows one to separate faces from non-face objects.