The composite face illusion: A whole window into our understanding of holistic face perception

The heterogeneity of holistic processing profiles in developmental prosopagnosia: holistic processing is impaired but not absent

Although it is generally assumed that face recognition relies on holistic processing, whether face recognition deficits observed in Developmental Prosopagnosics (DPs) can be explained by impaired holistic processing is currently under debate. The mixed findings from past studies could be the consequence of DP's heterogeneous deficit nature and the use of different measures of holistic processing-the inversion, part-whole, and composite tasks-which showed a poor association among each other. The present study aimed to gain further insight into the role of holistic processing in DPs. Groups of DPs and neurotypicals completed three tests measuring holistic face processing and non-face objects (i.e., Navon task). At a group level, DPs showed (1) diminished, but not absent, inversion and part-whole effects, (2) comparable magnitudes of the composite face effect and (3) global precedence effect in the Navon task. However, single-case analyses showed that these holistic processing deficits in DPs are heterogeneous.

Autistic adults exhibit typical sensitivity to changes in interpersonal distance

The visual processing differences seen in autism often impede individuals' visual perception of the social world. In particular, many autistic people exhibit poor face recognition. Here, we sought to determine whether autistic adults also show impaired perception of dyadic social interactions-a class of stimulus thought to engage face-like visual processing. Our focus was the perception of interpersonal distance. Participants completed distance change detection tasks, in which they had to make perceptual decisions about the distance between two actors. On half of the trials, participants judged whether the actors moved closer together; on the other half, whether they moved further apart. In a nonsocial control task, participants made similar judgments about two grandfather clocks. We also assessed participants' face recognition ability using standardized measures. The autistic and nonautistic observers showed similar levels of perceptual sensitivity to changes in interpersonal distance when viewing social interactions. As expected, however, the autistic observers showed clear signs of impaired face recognition. Despite putative similarities between the visual processing of faces and dyadic social interactions, our results suggest that these two facets of social vision may dissociate.

A preference to look closer to the eyes is associated with a position-invariant face neural code

When looking at faces, humans invariably move their eyes to a consistent preferred first fixation location on the face. While most people have the preferred fixation location just below the eyes, a minority have it between the nose-tip and mouth. Not much is known about whether these long-term differences in the preferred fixation location are associated with distinct neural representations of faces. To study this, we used a gaze-contingent face adaptation aftereffect paradigm to test in two groups of observers, one with their mean preferred fixation location closer to the eyes (upper lookers) and the other closer to the mouth (lower lookers). In this task, participants were required to maintain their gaze at either their own group's mean preferred fixation location or that of the other group during adaptation and testing. The two possible fixation locations were 3.6° apart on the face. We measured the face adaptation aftereffects when the adaptation and testing happened while participants maintained fixation at either the same or different locations on the face. Both groups showed equally strong adaptation effects when the adaptation and testing happened at the same fixation location. Crucially, only the upper lookers showed a partial transfer of the FAE across the two fixation locations, when adaptation occurred at the eyes. Lower lookers showed no spatial transfer of the FAE irrespective of the adaptation position. Given the classic finding that neural tuning is increasingly position invariant as one moves higher in the visual hierarchy, this result suggests that differences in the preferred fixation location are associated with distinct neural representations of faces.

Face Feature Change Detection Ability in Developmental Prosopagnosia and Super-Recognisers

In non-clinical populations, facial features (eyes, nose, mouth) may vary in their contribution to face identity perception. Changes to whole faces are easier to detect than changes to individual features, and eye changes are typically easier to detect than mouth changes, which in turn are easier to detect than nose changes. However, how this differs for people with face recognition difficulties (developmental prosopagnosia; DP) and for individuals with superior face recognition abilities (super-recognisers; SR) is not clear; although findings from previous studies have suggested differences, the nature of this difference is not understood. The aim of this study was to examine whether differences in the ability to detect feature changes in DPs and SRs were (a) quantitative, meaning that the pattern across feature changes remained the same but there was an overall upwards or downwards shift in performance, or (b) qualitative, meaning that the pattern across feature changes was different. Using a change detection task in which individual face features (eyes, nose, mouth) changed between sequentially presented faces, we found that while prosopagnosics showed a quantitative difference in performance with a downwards shift across all conditions, super-recognisers only showed qualitative differences: they were better able to detect when the face was the same and were marginally (but not non-significantly) worse at detecting when the eyes changed. Further, the only condition which distinguished between the three groups was the ability to identify when the same face was presented, with SRs being better than controls, and controls being better than DPs. Our findings suggest that, in feature-matching tasks, differences for DPs are due to them being overall worse at the task, while SRs use a qualitatively different strategy.

Facilitation and interference are asymmetric in holistic face processing

A hallmark of face specificity is holistic processing. It is typically measured by paradigms such as the part-whole and composite tasks. However, these tasks show little evidence for common variance, so a comprehensive account of holistic processing remains elusive. One aspect that varies between tasks is whether they measure facilitation or interference from holistic processing. In this study, we examined facilitation and interference in a single paradigm to determine the way in which they manifest during a face perception task. Using congruent and incongruent trials in the complete composite face task, we found that these two aspects are asymmetrically influenced by the location and cueing probabilities of the target facial half, suggesting that they may operate somewhat independently. We argue that distinguishing facilitation and interference has the potential to disentangle mixed findings from different popular paradigms measuring holistic processing in one unified framework.

From Perugino to Picasso revisited: Electrophysiological responses to faces in paintings from different art styles

Behavioral research (Ventura, et al., 2023) suggested that pictorial representations of faces varying along a realism-distortion spectrum elicit holistic processing as natural faces. Whether holistic face neural responses are engaged similarly remains, however, underexplored. In the present study, we evaluated the neural correlates of naturalist and artistic face processing, by exploring electrophysiological responses to faces in photographs versus in four major painting styles. The N170 response to faces in photographs was indistinguishable from that elicited by faces in the renaissance art style (depicting the most realistic faces), whilst both categories elicited larger N170 than faces in other art styles (post-impressionism, expressionism, and cubism), with a gradation in brain activity. The present evidence suggest that visual processing may become finer grained the more the realistic nature of the face. Despite behavioral equivalence, the neural mechanisms for holistic processing of natural faces and faces in diverse art styles are not equivalent.

The role of inversion and face masks on simultaneous and delayed face matching tasks

Although it is generally accepted that face recognition relies on holistic processing, it has been suggested that the simultaneous face matching task may depend on a more analytical or featural processing approach. However, empirical evidence supporting this claim is limited. In two experiments, we further explored the role of holistic and featural processing on simultaneous face matching by manipulating holistic processing through inversion and presenting faces with or without face masks. The results from Experiment 1 revealed that both inversion and face masks impaired matching performance. However, while the inversion effect was evident in both full-view and masked faces, the mask effect was only found in upright, but not inverted, faces. These results were replicated in Experiment 2 but, the inversion and mask effects were stronger in delayed face matching than in simultaneous face matching. Our findings suggest that simultaneous face matching relies on holistic processing, but to a smaller extent compared to higher memory-demanding identification tasks.

The face inversion effect does not provide a pure measure of holistic face processing

It is widely held that upright faces are processed more holistically than inverted faces and that this difference is reflected in the face inversion effect. It is not clear, however, how the inversion effect can best be measured, whether it is task specific, or even whether it specifically correlates with processing of upright faces. We examined these questions in a large sample (N = 420) who provided data on processing of upright and inverted stimuli in two different tasks with faces and one with objects. We find that the inversion effects are task dependent, and that they do not correlate better among face processing tasks than they do across face and object processing tasks. These findings were obtained regardless of whether inversion effects were measured by means of difference scores or regression. In comparison, only inversion effects based on regression predicted performance with upright faces in tasks other than those the inversion effects were derived from. Critically, however, inversion effects based on regression also predicted performance with inverted faces to a similar degree as they predicted performance with upright faces. Consequently, and contrary to what is commonly assumed, inversion effects do not seem to capture effects specific to holistic processing of upright faces. While the present findings do not bring us closer to an understanding of which changes in cognitive processing are induced by inversion, they do suggest that inversion effects do not reflect a unitary construct; an implicit assumption that seems to characterize much of the research regarding face processing.

Navon letters and composite faces: same or different processing mechanisms?

Navon letters and composite faces are two fascinating demonstrations of hierarchical organization in perception. Many researchers believe that the two types of stimuli and their associated tasks gauge comparable holistic mechanisms. This belief is so common that the two paradigms are now being applied in tandem to measure impaired holistic processing in prosopagnosic patients. But are Navon letters and composite faces processed in a similar fashion? In the present study we take a closer look at their apparent affinity. We gain novel insights into their underlying mechanisms by fitting parameters of the linear ballistic accumulator (LBA) model to empirical correct and incorrect response times (RTs). The results reveal major differences in processing between the two tasks. We conclude that despite the presence of a compelling surface similarity, Navon compound letters and composite faces tap into separate psychological processes.

Putting people in context: ERP responses to bodies in natural scenes

The N190 is a body-sensitive ERP component that responds to images of human bodies in different poses. In natural settings, bodies vary in posture and appear within complex, cluttered environments, frequently with other people. In many studies, however, such variability is absent. How does the N190 response change when observers see images that incorporate these sources of variability? In two experiments (N = 16 each), we varied the natural appearance of upright and inverted bodies to examine how the N190 amplitude, latency, and the Body-Inversion Effect (BIE) were affected by natural variability. In Experiment 1, we varied the number of people present in upright and inverted naturalistic scenes such that only one body, a subitizable number of bodies, or a "crowd" was present. In Experiment 2, we varied the natural body appearance by presenting bodies either as silhouettes or with photographic detail. Further, we varied the natural background appearance by either removing it or presenting individual bodies within a rich environment. Using component-based analyses of the N190, we found that the number of bodies in a scene reduced the N190 amplitude, but didn't affect the BIE (Experiment 1). Naturalistic body and background appearance (Experiment 2) also affected the N190, such that component amplitude was dramatically reduced by naturalistic appearance. To complement this analysis, we examined the contribution of spatiotemporal features (i.e., electrode × time point amplitude) via SVM decoding. This technique allows us to examine which timepoints across the entire waveform contribute the most to successful decoding of body orientation in each condition. This analysis revealed that later timepoints (after 300ms) contribute most to successful orientation decoding. These results demonstrate that natural appearance variability affects body processing at the N190 and that later ERP components may make important contributions to body processing in natural scenes.

Holistic and featural processing's link to face recognition varies by individual and task

While it is generally accepted that holistic processing facilitates face recognition, recent studies suggest that poor recognition might also arise from imprecise perception of local features in the face. This study aimed to examine to what extent holistic and featural processing relates to individual differences in face recognition ability (FRA), during face learning (Experiment 1) and face recognition (Experiment 2). Participants performed two tasks: (1) The "Cambridge Face Memory Test-Chinese" which measured participants' FRAs, and (2) an "old/new recognition memory test" encompassing whole faces (preserving holistic and featural processing) and faces revealed through a dynamic aperture (impairing holistic processing but preserving featural processing). Our results showed that participants recognised faces more accurately in conditions when holistic information was preserved, than when it is impaired. We also show that the better use of holistic processing during face learning and face recognition was associated with better FRAs. However, enhanced featural processing during recognition, but not during learning, was related to better FRAs. Together, our findings demonstrate that good face recognition depends on distinct roles played by holistic and featural processing at different stages of face recognition.

Using representational similarity analysis to reveal category and process specificity in visual object recognition

Cross-condition comparisons on neurodevelopmental conditions are central in neurodiversity research. In the realm of visual perception, the performance of participants with different category-specific disorders such as developmental prosopagnosia (problems with faces) and dyslexia (problems with words) have contributed to understanding of perceptual processes involved in word and face recognition. Alterations in face and word recognition are present in several neurodiverse populations, and improved knowledge about their relationship may increase our understanding of this variability of impairment. The present study investigates organizing principles of visual object processing and their implications for developmental disorders of recognition. Some accounts suggest that distinct mechanisms are responsible for recognizing objects of different categories, while others propose that categories share or even compete for cortical resources. We took an individual differences approach to estimate the relationship between abilities in recognition. Neurotypical participants (N = 97 after outlier exclusion) performed a match-to-sample task with faces, houses, and pseudowords. Either individual features or feature configurations were manipulated. To estimate the separability of visual recognition mechanisms, we used representational similarity analysis (RSA) where correlational matrices for accuracy were compared to predicted data patterns. Recognition abilities separated into face recognition on one hand and house/pseudoword recognition on the other, indicating that face recognition may rely on relatively selective mechanisms in neurotypicals. We also found evidence for a general visual object recognition mechanism, while some combinations of category (faces, houses, words) and processing type (featural, configural) likely rely on additional mechanisms. Developmental conditions may therefore reflect combinations of impaired and intact aspects of specific and general visual object recognition mechanisms, where featural and configural processes for one object category separate from the featural or configural processing of another. More generally, RSA is a promising approach for advancing understanding of neurodiversity, including shared aspects and distinctions between neurodevelopmental conditions of visual recognition.

Segregation and integration of sensory features by flexible temporal characteristics of independent neural representations

Segregation and integration are two fundamental yet competing computations in cognition. For example, in serial speech processing, stable perception necessitates the sequential establishment of perceptual representations to remove irrelevant features for achieving invariance. Whereas multiple features need to combine to create a coherent percept. How to simultaneously achieve seemingly contradicted computations of segregation and integration in a serial process is unclear. To investigate their neural mechanisms, we used loudness and lexical tones as a research model and employed a novel multilevel oddball paradigm with Electroencephalogram (EEG) recordings to explore the dynamics of mismatch negativity (MMN) responses to their deviants. When two types of deviants were presented separately, distinct topographies of MMNs to loudness and tones were observed at different latencies (loudness earlier), supporting the sequential dynamics of independent representations for two features. When they changed simultaneously, the latency of responses to tones became shorter and aligned with that to loudness, while the topographies remained independent, yielding the combined MMN as a linear additive of single MMNs of loudness and tones. These results suggest that neural dynamics can be temporally synchronized to distinct sensory features and balance the computational demands of segregation and integration, grounding for invariance and feature binding in serial processing.

Robust holistic face processing in early childhood during the COVID-19 pandemic

The timing of the developmental emergence of holistic face processing and its sensitivity to experience in early childhood are somewhat controversial topics. To investigate holistic face perception in early childhood, we used an online testing platform and administered a two-alternative forced-choice task to 4-, 5-, and 6-year-old children. The children saw pairs of composite faces and needed to decide whether the faces were the same or different. To determine whether experience with masked faces may have negatively affected holistic processing, we also administered a parental questionnaire to assess the children's exposure to masked faces during the COVID-19 pandemic. We found that all three age groups performed holistic face processing when the faces were upright (Experiment 1) but not when the faces were inverted (Experiment 2), that response accuracy increased with age, and that response accuracy was not related to degree of exposure to masked faces. These results indicate that holistic face processing is relatively robust in early childhood and that short-term exposure to partially visible faces does not negatively affect young children's holistic face perception.

Deep learning applied to EEG source-data reveals both ventral and dorsal visual stream involvement in holistic processing of social stimuli

Perception of social stimuli (faces and bodies) relies on "holistic" (i.e., global) mechanisms, as supported by picture-plane inversion: perceiving inverted faces/bodies is harder than perceiving their upright counterpart. Albeit neuroimaging evidence suggested involvement of face-specific brain areas in holistic processing, their spatiotemporal dynamics and selectivity for social stimuli is still debated. Here, we investigate the spatiotemporal dynamics of holistic processing for faces, bodies and houses (adopted as control non-social category), by applying deep learning to high-density electroencephalographic signals (EEG) at source-level. Convolutional neural networks were trained to classify cortical EEG responses to stimulus orientation (upright/inverted), separately for each stimulus type (faces, bodies, houses), resulting to perform well above chance for faces and bodies, and close to chance for houses. By explaining network decision, the 150-200 ms time interval and few visual ventral-stream regions were identified as mostly relevant for discriminating face and body orientation (lateral occipital cortex, and for face only, precuneus cortex, fusiform and lingual gyri), together with two additional dorsal-stream areas (superior and inferior parietal cortices). Overall, the proposed approach is sensitive in detecting cortical activity underlying perceptual phenomena, and by maximally exploiting discriminant information contained in data, may reveal spatiotemporal features previously undisclosed, stimulating novel investigations.

Potentiated early neural responses to fearful faces are not driven by specific face parts

Prioritized processing of fearful compared to neutral faces is reflected in increased amplitudes of components of the event-related potential (ERP). It is unknown whether specific face parts drive these modulations. Here, we investigated the contributions of face parts on ERPs to task-irrelevant fearful and neutral faces using an ERP-dependent facial decoding technique and a large sample of participants (N = 83). Classical ERP analyses showed typical and robust increases of N170 and EPN amplitudes by fearful relative to neutral faces. Facial decoding further showed that the absolute amplitude of these components, as well as the P1, was driven by the low-frequency contrast of specific face parts. However, the difference between fearful and neutral faces was not driven by any specific face part, as supported by Bayesian statistics. Furthermore, there were no correlations between trait anxiety and main effects or interactions. These results suggest that increased N170 and EPN amplitudes to task-irrelevant fearful compared to neutral faces are not driven by specific facial regions but represent a holistic face processing effect.

Face masks are less effective than sunglasses in masking face identity

The effect of covering faces on face identification is recently garnering interest amid the COVID-19 pandemic. Here, we investigated how face identification performance was affected by two types of face disguise: sunglasses and face masks. Observers studied a series of faces; then judged whether a series of test faces, comprising studied and novel faces, had been studied before or not. Face stimuli were presented either without coverings (full faces), wearing sunglasses covering the upper region (eyes, eyebrows), or wearing surgical masks covering the lower region (nose, mouth, chin). We found that sunglasses led to larger reductions in sensitivity (d') to face identity than face masks did, while both disguises increased the tendency to report faces as studied before, a bias that was absent for full faces. In addition, faces disguised during either study or test only (i.e. study disguised faces, test with full faces; and vice versa) led to further reductions in sensitivity from both studying and testing with disguised faces, suggesting that congruence between study and test is crucial for memory retrieval. These findings implied that the upper region of the face, including the eye-region features, is more diagnostic for holistic face-identity processing than the lower face region.

Looking at the upper facial half enlarges the range of holistic face processing

Previous studies suggested that upper and lower facial halves might be involved in the human holistic face processing differently. In this study, we replicated and extended the finding above. In Experiment 1, we used the standard composite-face task to measure holistic face processing when participants made judgements on the upper and lower facial halves separately. Results showed that the composite-face effect was stronger for the upper facial half compared to the lower half. In Experiment 2, we investigated how facial information was integrated when participants focused on different features, using the perceptual field paradigm. Results showed that: (1) more "peripheral faces" were chosen when participants fixated at the eyes than when they fixated at the mouth; (2) less "peripheral faces" were chosen for inverted faces regardless of the fixated features. Findings from both experiments together indicate that more peripheral facial information were integrated when participants focused on the upper facial half, highlighting the significance of focusing on the upper facial half in face processing.

High-frequency transcranial random noise stimulation enhances unfamiliar face matching of high resolution and pixelated faces

Face identification is useful for social interactions and its impairment can lead to severe social and mental problems. This ability is also remarkably important in applied settings, including eyewitness identification and ID verification. Several studies have demonstrated the potential of Transcranial Random Noise Stimulation (tRNS) to enhance different cognitive skills. However, research has produced inconclusive results about the effectiveness of tRNS to improve face identification. The present study aims to further explore the effect of tRNS on face identification using an unfamiliar face matching task. Observers firstly received either high-frequency bilateral tRNS or sham stimulation for 20 min. The stimulation targeted occipitotemporal areas, which have been previously involved in face processing. In a subsequent stage, observers were asked to perform an unfamiliar face matching task consisting of unaltered and pixelated face pictures. Compared to the sham stimulation group, the high-frequency tRNS group showed better unfamiliar face matching performance with both unaltered and pixelated faces. Our results show that a single high-frequency tRNS session might suffice to improve face identification abilities. These results have important consequences for the treatment of face recognition disorders, and potential applications in those scenarios whereby the identification of faces is primordial.

Contributions of low- and high-level contextual mechanisms to human face perception

Contextual modulations at primary stages of visual processing depend on the strength of local input. Contextual modulations at high-level stages of (face) processing show a similar dependence to local input strength. Namely, the discriminability of a facial feature determines the amount of influence of the face context on that feature. How high-level contextual modulations emerge from primary mechanisms is unclear due to the scarcity of empirical research systematically addressing the functional link between the two. We tested (62) young adults' ability to process local input independent of the context using contrast detection and (upright and inverted) morphed facial feature matching tasks. We first investigated contextual modulation magnitudes across tasks to address their shared variance. A second analysis focused on the profile of performance across contextual conditions. In upright eye matching and contrast detection tasks, contextual modulations only correlated at the level of their profile (averaged Fisher-Z transformed r = 1.18, BF10 > 100), but not magnitude (r = .15, BF10 = .61), suggesting the functional independence but similar working principles of the mechanisms involved. Both the profile (averaged Fisher-Z transformed r = .32, BF10 = 9.7) and magnitude (r = .28, BF10 = 4.58) of the contextual modulations correlated between inverted eye matching and contrast detection tasks. Our results suggest that non-face-specialized high-level contextual mechanisms (inverted faces) work in connection to primary contextual mechanisms, but that the engagement of face-specialized mechanisms for upright faces obscures this connection. Such combined study of low- and high-level contextual modulations sheds new light on the functional relationship between different levels of the visual processing hierarchy, and thus on its functional organization.

Idiosyncratic biases in the perception of medical images

Introduction

Radiologists routinely make life-altering decisions. Optimizing these decisions has been an important goal for many years and has prompted a great deal of research on the basic perceptual mechanisms that underlie radiologists' decisions. Previous studies have found that there are substantial individual differences in radiologists' diagnostic performance (e.g., sensitivity) due to experience, training, or search strategies. In addition to variations in sensitivity, however, another possibility is that radiologists might have perceptual biases-systematic misperceptions of visual stimuli. Although a great deal of research has investigated radiologist sensitivity, very little has explored the presence of perceptual biases or the individual differences in these.

Methods

Here, we test whether radiologists' have perceptual biases using controlled artificial and Generative Adversarial Networks-generated realistic medical images. In Experiment 1, observers adjusted the appearance of simulated tumors to match the previously shown targets. In Experiment 2, observers were shown with a mix of real and GAN-generated CT lesion images and they rated the realness of each image.

Results

We show that every tested individual radiologist was characterized by unique and systematic perceptual biases; these perceptual biases cannot be simply explained by attentional differences, and they can be observed in different imaging modalities and task settings, suggesting that idiosyncratic biases in medical image perception may widely exist.

Discussion

Characterizing and understanding these biases could be important for many practical settings such as training, pairing readers, and career selection for radiologists. These results may have consequential implications for many other fields as well, where individual observers are the linchpins for life-altering perceptual decisions.

Two face masks are better than one: congruency effects in face matching

Although the positive effects of congruency between stimuli are well replicated in face memory paradigms, mixed findings have been found in face matching. Due to the current COVID-19 pandemic, face masks are now very common during daily life outdoor activities. Thus, the present study aims to further explore congruency effects in matching faces partially occluded by surgical masks. Observers performed a face matching task consisting of pairs of faces presented in full view (i.e., full-view condition), pairs of faces in which only one of the faces had a mask (i.e., one-mask condition), and pairs of faces in which both faces had a mask (i.e., two-mask condition). Although face masks disrupted performance in identity match and identity mismatch trials, in match trials, we found better performance in the two-mask condition compared to the one-mask condition. This finding highlights the importance of congruency between stimuli on face matching when telling faces together.

Decoding semantics from intermodulation responses in frequency-tagged stereotactic EEG

BACKGROUND

Humans perform object recognition using holistic processing, which is different from computers. Intermodulation responses in the steady-state visual evoked potential (SSVEP) of scalp electroencephalography (EEG) have recently been used as an objective label for holistic processing.

NEW METHOD

Using stereotactic EEG (sEEG) to record SSVEP directly from inside of the brain, we aimed to decode Chinese characters from non-characters with activation from multiple brain areas including occipital, parietal, temporal, and frontal cortices.

RESULTS

Semantic categories could be decoded from responses at the intermodulation frequency with high accuracy (80%-90%), but not the base frequency. Moreover, semantic categories could be decoded with activation from multiple areas including temporal, parietal, and frontal areas.

COMPARISON WITH EXISTING METHOD(S)

Previous studies investigated holistic processing in faces and words with frequency-tagged scalp EEGs. The current study extended the results to stereotactic EEG signals directly recorded from the brain.

CONCLUSIONS

The human brain applies holistic processing in recognizing objects like Chinese characters. Our findings could be extended to an add-on feature in the existing SSVEP BCI speller.

Sex perception from facial structure: Categorization with and without skin texture and color

Sex identification of faces without any cultural or conventional sex cue is primarily based on two independent components: a) shape or facial structure, and b) surface reflectance (skin texture and color). The present work studied the relative contribution of each component by means of two experiments based on 3D face models created with different degrees of masculinity-femininity within a sex continuum. The first experiment utilized totally artificial faces created ex novo by computer. The second employed face models created from photographs of real people. The results of both experiments were consistent. As expected, when both components were present in a face, sex was correctly classified in almost all the cases. More interestingly, the contribution of the "pure" facial structure to the sex perception (with no surface reflectance) was about 80%, whereas 20% of the total information was provided by the surface reflectance. Furthermore, examination of the psychometric curves suggests that the information provided by surface reflectance contributes to a categorical perception of facial sex, since when it is removed the sex is perceived in a more continuous / less categorical way. On the other hand, our stimuli presented a certain "male" bias, repeatedly found in the literature on facial sex perception.

The mechanisms supporting holistic perception of words and faces are not independent

The question of whether word and face recognition rely on overlapping or dissociable neural and cognitive mechanisms received considerable attention in the literature. In the present work, we presented words (aligned or misaligned) superimposed on faces (aligned or misaligned) and tested the interference from the unattended stimulus category on holistic processing of the attended category. In Experiment 1, we found that holistic face processing is reduced when a face was overlaid with an unattended, aligned word (processed holistically). In Experiment 2, we found a similar reduction of holistic processing for words when a word was superimposed on an unattended, aligned face (processed holistically). This reciprocal interference effect indicates a trade-off in holistic processing of the two stimuli, consistent with the idea that word and face recognition may rely on non-independent, overlapping mechanisms.

Registered report: Social face evaluation: ethnicity-specific differences in the judgement of trustworthiness of faces and facial parts

Social face evaluation is a common and consequential element of everyday life based on the judgement of trustworthiness. However, the particular facial regions that guide such trustworthiness judgements are largely unknown. It is also unclear whether different facial regions are consistently utilized to guide judgments for different ethnic groups, and whether previous exposure to specific ethnicities in one's social environment has an influence on trustworthiness judgements made from faces or facial regions. This registered report addressed these questions through a global online survey study that recruited Asian, Black, Latino, and White raters (N = 4580). Raters were shown full faces and specific parts of the face for an ethnically diverse, sex-balanced set of 32 targets and rated targets' trustworthiness. Multilevel modelling showed that in forming trustworthiness judgements, raters relied most strongly on the eyes (with no substantial information loss vis-à-vis full faces). Corroborating ingroup-outgroup effects, raters rated faces and facial parts of targets with whom they shared their ethnicity, sex, or eye color as significantly more trustworthy. Exposure to ethnic groups in raters' social environment predicted trustworthiness ratings of other ethnic groups in nuanced ways. That is, raters from the ambient ethnic majority provided slightly higher trustworthiness ratings for stimuli of their own ethnicity compared to minority ethnicities. In contrast, raters from an ambient ethnic minority (e.g., immigrants) provided substantially lower trustworthiness ratings for stimuli of the ethnic majority. Taken together, the current study provides a new window into the psychological processes underlying social face evaluation and its cultural generalizability. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 7 January 2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.6084/m9.figshare.18319244 .

A more featural based processing for the self-face: An eye-tracking study

Studies have suggested that the holistic advantage in face perception is not always reported for the own face. With two eye-tracking experiments, we explored the role of holistic and featural processing in the processing and the recognition of self, personally familiar, and unfamiliar faces. Observers were asked to freely explore (Exp.1) and recognize (Exp.2) their own, a friend's, and an unfamiliar face. In Exp.1, self-face was fixated more and longer and there was a preference for the mouth region when seeing the own face and for the nose region when seeing a friend and unfamiliar faces. In Exp.2, the viewing strategies did not differ across all faces, with eye fixations mostly directed to the nose region. These results suggest that task demands might modulate the way that the own face is perceived and highlights the importance of considering the role of the distinct visual experience people have for the own face in the processing and recognition of the self-face.

The importance of internal and external features in recognizing faces that vary in familiarity and race

Familiar and unfamiliar faces are recognized in fundamentally different ways. One way in which recognition differs is in terms of the features that facilitate recognition: previous studies have shown that familiar face recognition depends more on internal facial features (i.e., eyes, nose and mouth), whereas unfamiliar face recognition depends more on external facial features (i.e., hair, ears and contour). However, very few studies have examined the recognition of faces that vary in both familiarity and race, and the reliance on different facial features, whilst also using faces that incorporate natural within-person variability. In the current study, we used an online version of the card sorting task to assess adults’ (n = 258) recognition of faces that varied in familiarity and race when presented with either the whole face, internal features only, or external features only. Adults better recognized familiar faces than unfamiliar faces in both the whole face and the internal features only conditions, but not in the external features only condition. Reasons why adults did not show an own-race advantage in recognition are discussed.

Local features drive identity responses in macaque anterior face patches

Humans and other primates recognize one another in part based on unique structural details of the face, including both local features and their spatial configuration within the head and body. Visual analysis of the face is supported by specialized regions of the primate cerebral cortex, which in macaques are commonly known as face patches. Here we ask whether the responses of neurons in anterior face patches, thought to encode face identity, are more strongly driven by local or holistic facial structure. We created stimuli consisting of recombinant photorealistic images of macaques, where we interchanged the eyes, mouth, head, and body between individuals. Unexpectedly, neurons in the anterior medial (AM) and anterior fundus (AF) face patches were predominantly tuned to local facial features, with minimal neural selectivity for feature combinations. These findings indicate that the high-level structural encoding of face identity rests upon populations of neurons specialized for local features.

Anterior face patches in the macaque have been assumed to represent face identity in a holistic manner. Here the authors show that the neural encoding of face identity in the anterior medial and anterior fundus face patches are instead driven principally by local features.

Holistic face processing in 4- and 7-month-old infants

Previous studies found an onset of holistic face processing in the age range between 0-4 and 7 months of age. To validate these studies, the present study investigated infants 4 and 7 months of age with a different experimental approach. In a habituation-dishabituation experiment, the infants were tested with stereoscopic stimuli in which stripes floated above a face, thereby occluding some parts of the face (amodal completion condition), and stereoscopic stimuli in which the same face parts floated above stripes (modal completion condition). Research with adults indicates that faces are processed holistically, that is as global wholes, in the amodal, but as independent parts in the modal completion condition, resulting in superior face recognition when the occluding bars are in front of than when they are behind the visible face parts. The present study found that infants regardless of whether they are 4 or 7 months old reliably recognized and differentiated the faces in the amodal but not in the modal completion condition. Moreover, the difference between the experimental conditions was statistically significant. These findings show that approximately at the age of 4-7 months of life, infants begin to holistically unify disjoint face parts into a coherent whole.

The relation between holistic processing as measured by three composite tasks and face processing: A latent variable modeling approach

We investigated the relationship between holistic processing and face processing using a latent variables approach. Three versions of the composite paradigm were used to measure holistic processing: Vanderbilt Holistic Face Processing Test, a sequential composite matching task, and a simultaneous composite matching task. Three tasks were used to measure face perception and face memory abilities respectively. We had three pairs of tasks such that within each pair (of memory and perception task), the stimuli involved, the requirement for matching across viewpoints, etc., are the same, such that the only difference is whether perception or memory is taxed. There were no significant correlations between the different versions of the composite task. We discovered no evidence to support a distinction between face perception and face memory, suggesting the existence of a general face processing factor. Finally, there was no evidence that holistic processing (as captured by either of the three composite tasks) is predictive of better face processing per se, casting doubts on the role of holistic processing in differentiating different levels of efficiency in face processing.

Healthy aging impairs face discrimination ability

Face images enable individual identities to be discriminated from one another. We aimed to quantify age-related changes in different aspects of face identity discrimination. Face discrimination sensitivity was measured with a memory-free “odd-one-out” task. Five age groups (N = 15) of healthy adults with normal vision were tested: 20, 50-59, 60-69, 70-79, and 80-89. Sensitivity was measured for full-face images (all features visible), external features (head-shape, hairline), internal features (nose, mouth, eyes, and eyebrows) and closed-contour shapes (control object). Sensitivity to full-faces continuously declined by approximately 13% per decade, after 50 years of age. When age-related differences in visual acuity were controlled, the effect of age on face discrimination sensitivity remained. Sensitivity to face features also deteriorated with age. Although the effect for external features was similar to full-faces, the rate of decline was considerably steeper (approximately 3.7 times) for internal, relative to external, features. In contrast, there was no effect of age on sensitivity to shapes. All age groups demonstrated the same overall pattern of sensitivity to different types of face information. Healthy aging was associated with a continuous decline in sensitivity to both full-faces and face features, although encoding of internal features was disproportionately impaired. This age-related deficit was independent of differences in low-level vision. That sensitivity to shapes was unaffected by age suggests these results cannot be explained by general cognitive decline or lower-level visual deficits. Instead, healthy aging is associated with a specific decline in the mechanisms that underlie face discrimination.

Body Processing in Children and Adolescents with Traumatic Brain Injury: An Exploratory Study

Dysfunctions in body processing have been documented in adults with brain damage, while limited information is available for children. This study aimed to investigate body processing in children and adolescents with traumatic brain injury (TBI) (N = 33), compared to peers with typical development. Two well-known computerized body-representation paradigms, namely Visual Body Recognition and Visuo-spatial Imagery, were administered. Through the first paradigm, the body inversion and composite illusion effects were tested with a matching to sample task as measures of configural and holistic processing of others’ bodies, respectively. The second paradigm investigated with a laterality judgement task the ability to perform first-person and object-based mental spatial transformations of own body and external objects, respectively. Body stimuli did not convey any emotional contents or symbolic meanings. Patients with TBI had difficulties with mental transformations of both body and object stimuli, displaying deficits in motor and visual imagery abilities, not limited to body processing. Therefore, cognitive rehabilitation of body processing in TBI might benefit from the inclusion of both general training on visuo-spatial abilities and specific exercises aimed at boosting visual body perception and motor imagery.

Attentional biases toward real images and drawings of negative faces

The allocation of attention is affected by internal emotional states, such as anxiety and depression. The attention captured by real images of negative faces can be quantified by emotional probe tasks. The present study investigated whether attentional bias toward drawings of negative faces (line drawings and cartoon faces) differs from that of real faces. Non-clinical university students indicated their levels of anxiety and depression via self-report questionnaires, and completed a probe discrimination task under three face image conditions in a between-participants design. Significant correlations were found between bias scores and scores on the self-reported BDI-II under the real face condition. However, two types of face drawings were only weakly correlated with self-report scores. In our probe task to investigate attentional bias to facial stimuli in nonclinical adults, the strength of the relationship between depression and attentional bias to negative face was stronger for real faces than for face drawings.

Face specific inversion effects provide evidence for two subtypes of developmental prosopagnosia

Many studies have attempted to identify the perceptual underpinnings of developmental prosopagnosia (DP). The majority have focused on whether holistic and configural processing mechanisms are impaired in DP. However, previous work suggests that there is substantial heterogeneity in holistic and configural processing within the DP population; further, there is disagreement as to whether any deficits are face-specific or reflect a broader perceptual deficit. This study used a data-driven approach to examine whether there are systematic patterns of variability in DP that reflect different underpinning perceptual deficits. A group of individuals with DP (N = 37) completed a cognitive battery measuring holistic/configural and featural processing in faces and non-face objects. A two-stage cluster analysis on data from the Cambridge Face Perception Test identified two subgroups of DPs. Across several tasks, the first subgroup (N = 21) showed typical patterns of holistic/configural processing (measured via inversion effects); the second (N = 16) was characterised by reduced or abolished inversion effects compared to age-matched control participants (N = 91). The subgroups did not differ on tasks measuring upright face matching, object matching, non-face holistic processing, or composite effects. These findings indicate two separable pathways to face recognition impairment, one characterised by impaired configural processing and the other potentially by impaired featural processing. Comparisons to control participants provide some preliminary evidence that the deficit in featural processing may extend to some non-face stimuli. Our results demonstrate the utility of examining both the variability between and consistency across individuals with DP as a means of illuminating our understanding of face recognition in typical and atypical populations.

A featural account for own-face processing? Looking for support from face inversion, composite face, and part-whole tasks

It is widely accepted that face perception relies on holistic processing. However, this holistic advantage is not always found in the processing of the own face. Our study aimed to explore the role of holistic and featural processing in the identification of the own face, using three standard, but largely independent measures of holistic face processing: the face inversion task, the composite face task, and the part-whole task. Participants were asked to identify their face, a friend’s face, and an unfamiliar face in three different experimental blocks: (a) inverted versus upright; (b) top and bottom halves of the face aligned versus misaligned; and (c) facial features presented in isolation versus whole foil face context. Inverting a face impaired its identification, regardless of the identity. However, alignment effects were only found when identifying a friend or an unfamiliar face. In addition, a stronger feature advantage (i.e., better recognition for isolated features compared to in a whole-face context) was observed for the own face compared to the friend and unfamiliar faces. Altogether, these findings suggest that the own face is processed in a more featural manner but also relies on holistic processing. This work also highlights the importance of taking into consideration that different holistic processing paradigms could tap different forms of holistic processing.

The relationships between reading fluency and different measures of holistic word processing

Recently, paradigms in the face recognition literature have been adopted to reveal holistic processing in word recognition. It is unknown, however, whether different measures of holistic word processing share similar underlying mechanisms, and whether fluent word reading relies on holistic word processing. We measured holistic processing effects in three paradigms (composite, configural sensitivity, part-whole) as well as in reading fluency (3DM task: reading aloud high- and low-frequency words and pseudowords). Bin scores were used to combine accuracy and response time variables in the quest for a more comprehensive, reliable, and valid measure of holistic processing. Weak correlations were found between the different holistic processing measures, with only a significant correlation between the configural sensitivity effect and part-whole effect (r = .32) and a trend of a positive correlation between the word composite effect and configural sensitivity effect (r = .21). Of the three holistic processing measures, only one (part-whole effect) correlated with a lexical access measure of 3DM (r = .23). We also performed a principal component analysis (PCA) of performance in the three lists of 3DM, with the second most probably reflecting lexical access processes. There was a tendency for a positive correlation between part-whole bin measure and Component 2 of PCA. We also found a positive correlation between composite aligned in accuracy and Component 2 of PCA.Our results show that different measures of holistic word processing reflect predominantly different mechanisms, and that differences among normal readers in word reading do not seem to depend highly on holistic processing.

Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part II: Neural basis

Patient PS sustained her dramatic brain injury in 1992, the same year as the first report of a neuroimaging study of human face recognition. The present paper complements the review on the functional nature of PS's prosopagnosia (part I), illustrating how her case study directly, i.e., through neuroimaging investigations of her brain structure and activity, but also indirectly, through neural studies performed on other clinical cases and neurotypical individuals, inspired and constrained neural models of human face recognition. In the dominant right hemisphere for face recognition in humans, PS's main lesion concerns (inputs to) the inferior occipital gyrus (IOG), in a region where face-selective activity is typically found in normal individuals ('Occipital Face Area', OFA). Her case study initially supported the criticality of this region for face identity recognition (FIR) and provided the impetus for transcranial magnetic stimulation (TMS), intracerebral electrical stimulation, and cortical surgery studies that have generally supported this view. Despite PS's right IOG lesion, typical face-selectivity is found anteriorly in the middle portion of the fusiform gyrus, a hominoid structure (termed the right 'Fusiform Face Area', FFA) that is widely considered to be the most important region for human face recognition. This finding led to the original proposal of direct anatomico-functional connections from early visual cortices to the FFA, bypassing the IOG/OFA (lulu), a hypothesis supported by further neuroimaging studies of PS, other neurological cases and neuro-typical individuals with original visual stimulation paradigms, data recordings and analyses. The proposal of a lack of sensitivity to face identity in PS's right FFA due to defective reentrant inputs from the IOG/FFA has also been supported by other cases, functional connectivity and cortical surgery studies. Overall, neural studies of, and based on, the case of prosopagnosia PS strongly question the hierarchical organization of the human neural face recognition system, supporting a more flexible and dynamic view of this key social brain function.

Non-monotonic developmental trend of holistic processing in visual expertise: the case of Chinese character recognition

Holistic processing has been identified as an expertise marker of face and object recognition. By contrast, reduced holistic processing is purportedly an expertise marker in recognising orthographic characters in Chinese. Does holistic processing increase or decrease in expertise development? Is orthographic recognition a domain-specific exception to all other kinds of recognition (e.g. face and objects)? In two studies, we examined the developmental trend of holistic processing in Chinese character recognition in Chinese and non-Chinese children, and its relationship with literacy abilities: Chinese first graders—with emergent Chinese literacy acquired in kindergarten—showed increased holistic processing perhaps as an inchoate expertise marker when compared with kindergartners and non-Chinese first graders; however, the holistic processing effect was reduced in higher-grade Chinese children. These results suggest a non-monotonic inverted U-shape trend of holistic processing in visual expertise development: An increase in holistic processing due to initial reading experience followed by a decrease in holistic processing due to literacy enhancement. This result marks the development of holistic and analytic processing skills, both of which can be essential for mastering visual recognition. This study is the first to investigate the developmental trend of holistic processing in Chinese character recognition using the composite paradigm.

Change detection vs. change localization for own-race and other-race faces

The other-race effect (ORE) is a well-known phenomenon in which people discriminate and recognize faces from their ethnic group more accurately than faces from other ethnic groups. Holistic processing, or the mandatory tendency to process all parts of an object together, has been proposed as an explanation for the ORE. According to the holistic perspective of the ORE, other-race faces might be subject to weaker holistic processing than own-race faces. However, evidence for this hypothesis is inconsistent. Although it is generally assumed that holistic processing helps the individuation of objects, holistic processing may also come at a cost. Specifically, holistic processing may reduce the capacity to localize changes in the constituent parts of an object, but not in detecting changes to an object as a whole. In the present study, we examined change detection and change localization accuracy for Caucasian and African faces, and houses. Performance was better for change detection than change localization for Caucasian faces. While clear costs of holistic processing for Caucasian faces were thus found, the difference between change localization and change detection was not obvious for African faces. However, childhood exposure to other-race people correlated with change detection for African faces, but not with change localization for African faces. Our results thus show that holistic processing of other-race faces may depend on early contact with other-race people.

Symmetric in the striate but asymmetric in the extrastriate cortex when processing three-quarter faces: Neural underpinnings for aesthetic appreciations

Faces and their aesthetic appreciation are a core element of social interaction. Although studies have been made on facial processing when looking at faces with different perspectives, a direct comparison of faces in the left to the right perspective is missing. Portraits in classical Western art indicate a preference of the left compared to the right perspective, but the neural underpinnings of such an asymmetry still have to be clarified. Using functional magnetic resonance imaging, the current study focuses on the processing of three-quarter faces seen with different perspectives. Seventeen participants were asked to passively look at photographs of six male and six female faces with a neutral expression; the photographs were taken from the left, right, and frontal perspectives while keeping their focus on the eyes. The results showed that specific brain areas were involved in processing the three-quarter faces in either symmetric or asymmetric ways. Viewing left and right three-quarter faces resulted in two mirror-like activations in the striate cortex corresponding to the symmetric layout of the left and right perspectives. Viewing the left face resulted additionally in an enhanced activation also in the left extrastriate cortex. The right perspective of male faces elicited a lower activation compared to other perspectives in face-selective areas of the brain. Our findings suggest that the preference of the left three-quarter face emerges already in the early visual pathway presumably prior to facial identification, emotional processing, and aesthetic appreciation. Our observations may have general importance in disentangling different neural components and processing stages in the spatiotemporal characteristics of artistic expressions.

The Effect of Surgical Masks on the Featural and Configural Processing of Emotions

From the start of the COVID-19 pandemic, the use of surgical masks became widespread. However, they occlude an important part of the face and make it difficult to decode and interpret other people's emotions. To clarify the effect of surgical masks on configural and featural processing, participants completed a facial emotion recognition task to discriminate between happy, sad, angry, and neutral faces. Stimuli included fully visible faces, masked faces, and a cropped photo of the eyes or mouth region. Occlusion due to the surgical mask affects emotion recognition for sadness, anger, and neutral faces, although no significative differences were found in happiness recognition. Our findings suggest that happiness is recognized predominantly via featural processing.

Misaligned dynamic faces are processed holistically

It was recently proposed that both experience-driven and object-based perceptual grouping contribute to holistic face processing. We investigated whether motion-as a common fate perceptual grouping cue-could enhance the holistic processing of misaligned faces. We manipulated alignment and motion (dynamic and static) of study and test faces in a modified complete composite task in which the congruency effect was regarded as an indicator of holistic processing. Participants made same-different judgments about the top halves of two sequentially presented composite faces. We observed that when the study faces were dynamic, regardless of whether the test faces were dynamic or static, misaligned-misaligned face pairs were processed holistically. When the study faces were static, misaligned-misaligned face pairs showed no holistic processing, and neither did inverted faces. These results indicate that motion can promote the holistic processing of misaligned faces. Our findings provide important insights into different types of holistic face processing, and we discuss these types as well as their relationships with each other in depth.

Dissociated face- and word-selective intracerebral responses in the human ventral occipito-temporal cortex

The extent to which faces and written words share neural circuitry in the human brain is actively debated. Here, we compare face-selective and word-selective responses in a large group of patients (N = 37) implanted with intracerebral electrodes in the ventral occipito-temporal cortex (VOTC). Both face-selective (i.e., significantly different responses to faces vs. non-face visual objects) and word-selective (i.e., significantly different responses to words vs. pseudofonts) neural activity is isolated with frequency-tagging. Critically, this sensitive approach allows to objectively quantify category-selective neural responses and disentangle them from general visual responses. About 70% of significant electrode contacts show either face-selectivity or word-selectivity only, with the expected right and left hemispheric dominance, respectively. Spatial dissociations are also found within core regions of face and word processing, with a medio-lateral dissociation in the fusiform gyrus (FG) and surrounding sulci, respectively. In the 30% of overlapping face- and word-selective contacts across the VOTC or in the FG and surrounding sulci, between-category-selective amplitudes (faces vs. words) show no-to-weak correlations, despite strong correlations in both the within-category-selective amplitudes (face-face, word-word) and the general visual responses to words and faces. Overall, these observations support the view that category-selective circuitry for faces and written words is largely dissociated in the human adult VOTC.