Inverted faces are (eventually) processed holistically

Two faces of holistic face processing: Facilitation and interference underlying part-whole and composite effects

Holistic processing, a strong tendency to process multiple features together, is regarded as a hallmark of face perception. Holistic effects can be revealed by several tasks, including the part-whole task, standard composite task, and complete composite task. Although holistic effects are readily observed using these tasks, the lack of correlations among these effects and the mixed findings across these tasks when examining the effects among various populations or manipulations pose questions about how these effects should be understood. We distinguished facilitation and interference effects within the holistic effects in the complete composite task and found that the holistic effect in the part-whole task appeared to be correlated with facilitation but not interference in the complete composite task, whereas the holistic effect in the standard composite task was correlated with interference but not facilitation in the complete composite task. These findings suggest that clarifying the roles of facilitation and interference is critical for understanding holistic face processing.

The role of transitional probabilities in word holistic processing

In recent years, increased attention has been devoted to visual word recognition under a perceptual expertise framework. Because the information required to identify words is distributed across the word, a holistic attentional strategy is optimal and develops with experience. It is, however, an open question the extent to which other information embedded in a word may contribute to word holistic processing, namely sublexical word properties. In the present research, we therefore explore the role of sublexical properties-specifically bigram transition probabilities-in this processing strategy. We used a common task in the holistic processing literature (i.e., composite task) and four-letter disyllabic words, where two of the bigrams reinforce the cohesiveness of each syllable and one of the bigrams reinforces the cohesiveness between the syllables. We found preliminary evidence of a role of these sublexical properties in word holistic processing.

Perceptual heterogeneity in developmental prosopagnosia is continuous, not categorical

Developmental prosopagnosia (DP) is associated with considerable perceptual heterogeneity, though the nature of this heterogeneity and whether there are discrete subgroups versus continuous deficits remains unclear. Bennetts et al. (2022) recently found that holistic versus featural processing deficits distinguished discrete DP subgroups, but their sample was relatively small (N = 37), and subgroups were defined using a single task. To characterize perceptual heterogeneity in DPs more comprehensively, we administered a broad face perception battery to a large sample of 109 DPs and 134 controls, including validated measures of face matching (Cambridge Face Perception Test - CFPT, Computerized Benton Facial Recognition Test, Same/Different Face Matching Task), holistic processing (Part-Whole Task), and feature processing (Georges Task and Part-Whole part trials). When examining face matching measures, DPs exhibited a similar distribution of performance as controls, though shifted towards impairment by an average of 1.4 SD. We next applied Bennetts (2022) hierarchical clustering approach and k-means clustering to the CFPT upright, inverted, and inversion index measures, similarly finding one group of DPs with poorer inverted face performance and another with a decreased face inversion effect (holistic processing). However, these subgroup differences failed to generalize to other measures of feature and holistic processing beyond the CFPT. We finally ran hierarchical and k-means cluster analyses on our larger battery of face matching, feature, and holistic processing measures. Results clearly showed subgroups with generally better versus worse performance across all measures, with the distinction between groups being somewhat arbitrary. Together, these findings support a continuous account of DP perceptual heterogeneity, with performance differing primarily across all aspects of face perception.

The role of inversion and face masks on simultaneous and delayed face matching tasks

Although it is generally accepted that face recognition relies on holistic processing, it has been suggested that the simultaneous face matching task may depend on a more analytical or featural processing approach. However, empirical evidence supporting this claim is limited. In two experiments, we further explored the role of holistic and featural processing on simultaneous face matching by manipulating holistic processing through inversion and presenting faces with or without face masks. The results from Experiment 1 revealed that both inversion and face masks impaired matching performance. However, while the inversion effect was evident in both full-view and masked faces, the mask effect was only found in upright, but not inverted, faces. These results were replicated in Experiment 2 but, the inversion and mask effects were stronger in delayed face matching than in simultaneous face matching. Our findings suggest that simultaneous face matching relies on holistic processing, but to a smaller extent compared to higher memory-demanding identification tasks.

Red background color biases gender categorization of human faces

Color carries gender information (e.g., red-female). This study explored whether background color could influence the gender categorization of human faces. Visual stimuli were generated from faces whose sexually dimorphic content was morphed monotonically from female to male perception. The face stimulus was presented upright (Experiment 1) and inverted (Experiment 2) with three background colors (i.e., red, green, and gray). Participants were instructed to categorize the gender of the face stimulus as male or female by pressing one of two labelled keys. Results showed that a red background could bias the gender of an ambiguous upright face toward a female compared with green and gray background colors (Experiment 1). However, this red effect was diminished when the face stimulus was inverted (Experiment 2). These results suggest that red background color interacting with facial configuration features biases gender perception toward a female face, possibly through top-down processing of learned associations between the color red and femininity.

Holistic processing and visual characteristics of regulated and spontaneous expressions

The rapid and efficient recognition of facial expressions is crucial for adaptive behaviors, and holistic processing is one of the critical processing methods to achieve this adaptation. Therefore, this study integrated the effects and attentional characteristics of the authenticity of facial expressions on holistic processing. The results show that both regulated and spontaneous expressions were processed holistically. However, the spontaneous expression details did not indicate typical holistic processing, with the congruency effect observed equally for aligned and misaligned conditions. No significant difference between the two expressions was observed in terms of reaction times and eye movement characteristics (i.e., total fixation duration, fixation counts, and first fixation duration). These findings suggest that holistic processing strategies differ between the two expressions. Nevertheless, the difference was not reflected in attentional engagement.

Looking at the upper facial half enlarges the range of holistic face processing

Previous studies suggested that upper and lower facial halves might be involved in the human holistic face processing differently. In this study, we replicated and extended the finding above. In Experiment 1, we used the standard composite-face task to measure holistic face processing when participants made judgements on the upper and lower facial halves separately. Results showed that the composite-face effect was stronger for the upper facial half compared to the lower half. In Experiment 2, we investigated how facial information was integrated when participants focused on different features, using the perceptual field paradigm. Results showed that: (1) more "peripheral faces" were chosen when participants fixated at the eyes than when they fixated at the mouth; (2) less "peripheral faces" were chosen for inverted faces regardless of the fixated features. Findings from both experiments together indicate that more peripheral facial information were integrated when participants focused on the upper facial half, highlighting the significance of focusing on the upper facial half in face processing.

Spotting lesions in thorax X-rays at a glance: holistic processing in radiology

Radiologists often need only a glance to grasp the essence of complex medical images. Here, we use paradigms and manipulations from perceptual learning and expertise fields to elicit mechanisms and limits of holistic processing in radiological expertise. In the first experiment, radiologists were significantly better at categorizing thorax X-rays when they were presented for 200 ms in an upright orientation than when they were presented upside-down. Medical students, in contrast, were guessing in both situations. When the presentation time was increased to 500 ms, allowing for a couple more glances, the radiologists improved their performance on the upright stimuli, but remained at the same level on the inverted presentation. The second experiment circumvented the holistic processing by immediately cueing a tissue within the X-rays, which may or may not contain a nodule. Radiologists were again better than medical students at recognizing whether the cued tissue was a nodule, but this time neither the inverted presentation nor additional time affected their performance. Our study demonstrates that holistic processing is most likely a continuous recurring process which is just as susceptible to the inversion effect as in other expertise domains. More importantly, our study also indicates that holistic-like processing readily occurs in complex stimuli (e.g., whole thorax X-rays) but is more difficult to find in uniform single parts of such stimuli (e.g., nodules).

Multiple latent variables but functionally dependent output mappings underlying the recognition of own- and other-race faces for Chinese individuals: Evidence from state-trace analysis

To explore the number of latent variables underlying recognition of own- and other-race faces for Chinese observers, we conducted a study-recognition task where orientation, stimuli type, and duration were manipulated in the study phase and applied state trace analysis as a statistic method. Results showed that each state trace plot on each pair of stimuli types matched a single monotonic curve when stimuli type was set to state factor, but separate curves between face and non-face showed up when the state factor was orientation. The results implied that at least two latent variables affected recognition performance in the inversion paradigm. Besides, the unidimensional structure between own- and other-race faces regardless of the state factor suggested that Chinese participants used the same recognition mechanism for unfamiliar own- and other-race faces in the inversion paradigm.

A featural account for own-face processing? Looking for support from face inversion, composite face, and part-whole tasks

It is widely accepted that face perception relies on holistic processing. However, this holistic advantage is not always found in the processing of the own face. Our study aimed to explore the role of holistic and featural processing in the identification of the own face, using three standard, but largely independent measures of holistic face processing: the face inversion task, the composite face task, and the part-whole task. Participants were asked to identify their face, a friend’s face, and an unfamiliar face in three different experimental blocks: (a) inverted versus upright; (b) top and bottom halves of the face aligned versus misaligned; and (c) facial features presented in isolation versus whole foil face context. Inverting a face impaired its identification, regardless of the identity. However, alignment effects were only found when identifying a friend or an unfamiliar face. In addition, a stronger feature advantage (i.e., better recognition for isolated features compared to in a whole-face context) was observed for the own face compared to the friend and unfamiliar faces. Altogether, these findings suggest that the own face is processed in a more featural manner but also relies on holistic processing. This work also highlights the importance of taking into consideration that different holistic processing paradigms could tap different forms of holistic processing.

The relationships between reading fluency and different measures of holistic word processing

Recently, paradigms in the face recognition literature have been adopted to reveal holistic processing in word recognition. It is unknown, however, whether different measures of holistic word processing share similar underlying mechanisms, and whether fluent word reading relies on holistic word processing. We measured holistic processing effects in three paradigms (composite, configural sensitivity, part-whole) as well as in reading fluency (3DM task: reading aloud high- and low-frequency words and pseudowords). Bin scores were used to combine accuracy and response time variables in the quest for a more comprehensive, reliable, and valid measure of holistic processing. Weak correlations were found between the different holistic processing measures, with only a significant correlation between the configural sensitivity effect and part-whole effect (r = .32) and a trend of a positive correlation between the word composite effect and configural sensitivity effect (r = .21). Of the three holistic processing measures, only one (part-whole effect) correlated with a lexical access measure of 3DM (r = .23). We also performed a principal component analysis (PCA) of performance in the three lists of 3DM, with the second most probably reflecting lexical access processes. There was a tendency for a positive correlation between part-whole bin measure and Component 2 of PCA. We also found a positive correlation between composite aligned in accuracy and Component 2 of PCA.Our results show that different measures of holistic word processing reflect predominantly different mechanisms, and that differences among normal readers in word reading do not seem to depend highly on holistic processing.

The inversion effect in word recognition: the effect of language familiarity and handwriting

Humans have expertise with visual words and faces. One marker of this expertise is the inversion effect. This is attributed to experience with those objects being biased towards a canonical orientation, rather than some inherent property of object structure or perceptual anisotropy. To confirm the role of experience, we measured inversion effects in word matching for familiar and unfamiliar languages. Second, we examined whether there may be more demands on reading expertise with handwritten stimuli rather than computer font, given the greater variability and irregularities in the former, with the prediction of larger inversion effects for handwriting. We recruited two cohorts of subjects, one fluent in Farsi and the other in Punjabi, neither of whom were able to read the other's language. Subjects performed a match-to-sample task with words in either computer fonts or handwritings. Subjects were more accurate and faster with their familiar language, even when it was inverted. Inversion effects were present for the familiar but not the unfamiliar language. The inversion effect in accuracy for handwriting was larger than that for computer fonts in the familiar language. We conclude that the word inversion effect is generated solely by orientation-biased experience, and that demands on this expertise are greater with handwriting than computer font.

The time course of holistic processing is similar for face and non-face Gestalt stimuli

There is evidence that holistic processing of faces and other stimuli rich in Gestalt perceptual grouping cues recruit overlapping mechanisms at early processing stages, but not at later stages where faces and objects of expertise likely overlap. This has led to suggestions of dual pathways supporting holistic processing; an early stimulus-based pathway (supporting processing of stimuli rich in perceptual grouping cues) and an experience-based pathway (supporting processing of object of expertise), with both pathways supporting face processing. Holistic processing markers are present when upright faces are presented for as little as 50-ms. If the overlap between holistic processing of faces and stimuli rich in grouping cues occurs early in processing, markers of holistic processing for these Gestalt stimuli should be present as early as those for faces. In Experiment 1, we investigate the time-course of the emergence of holistic processing markers for face and non-face Gestalt stimuli. The emergence of these markers for faces and the Gestalt stimuli was strikingly similar; both emerged with masked presentations as little as 50-ms. In Experiment 2, where the stimulus presentation was not masked, thus the presentation duration, but not the post-presentation perceptual processing, was constrained, patterns of holistic processing for these stimuli still did not diverge. These findings are consistent with an early, and possibly extended, temporal locus for the overlap in the holistic processing of faces and non-face stimuli rich in grouping cues.

Can faces affect object-based attention? Evidence from online experiments

This study tested how human faces affect object-based attention (OBA) through two online experiments in a modified double-rectangle paradigm. The results of Experiment 1 revealed that faces did not elicit the OBA effect as non-face objects, which was caused by a longer response time (RT) when attention is focused on faces relative to non-face objects. In addition, by observing faster RTs when attention was engaged horizontally rather than vertically, we found a significant horizontal attention bias, which might override the OBA effect if vertical rectangles were the only items presented; these results were replicated in Experiment 2 (using only vertical rectangles) after directly measuring horizontal bias and excluding its influence on the OBA effect. This study suggested that faces cannot elicit the same-object advantage in the double-rectangle paradigm and provided a method to measure the OBA effect free from horizontal bias.

Change detection vs. change localization for own-race and other-race faces

The other-race effect (ORE) is a well-known phenomenon in which people discriminate and recognize faces from their ethnic group more accurately than faces from other ethnic groups. Holistic processing, or the mandatory tendency to process all parts of an object together, has been proposed as an explanation for the ORE. According to the holistic perspective of the ORE, other-race faces might be subject to weaker holistic processing than own-race faces. However, evidence for this hypothesis is inconsistent. Although it is generally assumed that holistic processing helps the individuation of objects, holistic processing may also come at a cost. Specifically, holistic processing may reduce the capacity to localize changes in the constituent parts of an object, but not in detecting changes to an object as a whole. In the present study, we examined change detection and change localization accuracy for Caucasian and African faces, and houses. Performance was better for change detection than change localization for Caucasian faces. While clear costs of holistic processing for Caucasian faces were thus found, the difference between change localization and change detection was not obvious for African faces. However, childhood exposure to other-race people correlated with change detection for African faces, but not with change localization for African faces. Our results thus show that holistic processing of other-race faces may depend on early contact with other-race people.

Misaligned dynamic faces are processed holistically

It was recently proposed that both experience-driven and object-based perceptual grouping contribute to holistic face processing. We investigated whether motion-as a common fate perceptual grouping cue-could enhance the holistic processing of misaligned faces. We manipulated alignment and motion (dynamic and static) of study and test faces in a modified complete composite task in which the congruency effect was regarded as an indicator of holistic processing. Participants made same-different judgments about the top halves of two sequentially presented composite faces. We observed that when the study faces were dynamic, regardless of whether the test faces were dynamic or static, misaligned-misaligned face pairs were processed holistically. When the study faces were static, misaligned-misaligned face pairs showed no holistic processing, and neither did inverted faces. These results indicate that motion can promote the holistic processing of misaligned faces. Our findings provide important insights into different types of holistic face processing, and we discuss these types as well as their relationships with each other in depth.

Faces and words are both associated and dissociated as evidenced by visual problems in dyslexia

Faces and words are traditionally assumed to be independently processed. Dyslexia is also traditionally thought to be a non-visual deficit. Counter to both ideas, face perception deficits in dyslexia have been reported. Others report no such deficits. We sought to resolve this discrepancy. 60 adults participated in the study (24 dyslexic, 36 typical readers). Feature-based processing and configural or global form processing of faces was measured with a face matching task. Opposite laterality effects in these tasks, dependent on left-right orientation of faces, supported that they tapped into separable visual mechanisms. Dyslexic readers tended to be poorer than typical readers at feature-based face matching while no differences were found for global form face matching. We conclude that word and face perception are associated when the latter requires the processing of visual features of a face, while processing the global form of faces apparently shares minimal-if any-resources with visual word processing. The current results indicate that visual word and face processing are both associated and dissociated-but this depends on what visual mechanisms are task-relevant. We suggest that reading deficits could stem from multiple factors, and that one such factor is a problem with feature-based processing of visual objects.

Influence of Perceptual Discriminability and Spatial Distance on Holistic Processing of Faces

Background: Holistic processing is defined as the perceptual integration of facial features, and plays an important role in face recognition. While researchers recognize the crucial role played by holistic processing in face perception, a complete delineation of the underlying mechanisms is impending. Very few studies have examined the effects of perceptual discrimination and spatial perception on holistic processing. Hence, the present study aimed to examine the influence of perceptual discrimination and spatial perception on face recognition.

Methods: We conducted two experiments by manipulating the perceptual discriminability of the target (the top-half faces) and non-target face (the bottom-half faces) parts in the composite-face task and examined how perceptual discriminability may affect holistic processing of faces.

Results: The results of Experiment 1 illustrated that holistic processing was modulated by the perceptual discriminability of the face. Furthermore, differential patterns of perceptual discriminability with the target and non-target parts suggested that different mechanisms may be responsible for the influence of target and non-target parts on face perception. The results of Experiment 2 illustrated that holistic processing was modulated by spatial distance between two faces, implicating that feature-by-feature strategy might decrease the magnitude of holistic processing.

Conclusion: The results of the present study suggest that holistic processing may lead to augmented perception effect exaggerating the differences between the two faces and may also be affected by the feature-by-feature strategy.

Whole-object effects in visual word processing: Parallels with and differences from face recognition

Visual words and faces differ in their structural properties, but both are objects of high expertise. Holistic processing is said to characterize expert face recognition, but the extent to which whole-word processes contribute to word recognition is unclear, particularly as word recognition is thought to proceed by a component-based process. We review the evidence for experimental effects in word recognition that parallel those used to support holistic face processing, namely inversion effects, the part-whole task, and composite effects, as well as the status of whole-word processing in pure alexia and developmental dyslexia, contrasts between familiar and unfamiliar languages, and the differences between handwriting and typeset font. The observations support some parallels in whole-object influences between face and visual word recognition, but do not necessarily imply similar expert mechanisms. It remains to be determined whether and how the relative balance between part-based and whole-object processing differs for visual words and faces.

The Efficiency of Demography in Face Perception

When we look at a face, we cannot help but "read" it: Beyond simply processing its identity, we also form robust impressions of both transient psychological states (e.g., surprise) and stable character traits (e.g., trustworthiness). But perhaps the most fundamental traits we extract from faces are their social demographics, for example, race, age, and gender. How much exposure is required to extract such properties? Curiously, despite extensive work on the temporal efficiency of extracting both higher-level social properties (such as competence and dominance) and more basic characteristics (such as identity and familiarity), this question remains largely unexplored for demography. We correlated observers' percepts of the race/age/gender of unfamiliar faces viewed at several brief durations (and then masked) with their judgments after unlimited exposure. Performance reached asymptote by 100 ms, was above chance by only 33.33 ms, and had a similar temporal profile to detecting faces in the first place. This was true even when the property to be reported wasn't revealed until after the face had disappeared, and when the faces were matched for several lower-level visual properties. Collectively, these results demonstrate that the extraction of demographic features from faces is highly efficient, and can truly be done at a glance.

Similar use of shape and texture cues for own- and other-race faces during face learning and recognition

Although the other-race effect (ORE; superior recognition of own- relative to other-race faces) is well established, the mechanisms underlying it are not well understood. We examined whether the ORE is attributable to differential use of shape and texture cues for own- vs. other-race faces. Shape cues are particularly important for detecting that an own-race face is unfamiliar, whereas texture cues are more important for recognizing familiar and newly learned own-race faces. We compared the influence of shape and texture cues on Caucasian participants' recognition of Caucasian and East Asian faces using two complementary approaches. In Experiment 1, participants studied veridical, shape-caricatured, or texture-caricatured faces and then were asked to recognize them in an old/new recognition task. In Experiment 2, all study faces were veridical and we independently removed the diagnosticity of shape (or texture) cues in the test phase by replacing original shape (or texture) with average shape (or texture). Despite an overall own-race advantage, participants' use of shape and texture cues was comparable for own- and other-race faces. These results suggest that the other-race effect is not attributable to qualitative differences in the use of shape and texture cues.

Different measures of holistic face processing tap into distinct but partially overlapping mechanisms

Holistic processing, which includes the integration of facial features and analysis of their relations to one another, is a hallmark of what makes faces 'special'. Various experimental paradigms purport to measure holistic processing but these have often produced inconsistent results. This has led researchers to question the nature and structure of the mechanism(s) underlying holistic processing. Using an individual differences approach, researchers have examined relations between various measures of holistic processing in an attempt to resolve these questions. In keeping with this, we examined relationships between four commonly used measures of holistic face processing in a large group of participants (N = 223): (1) The Face Inversion Effect, (2) the Part Whole Effect (PWE), (3) the Composite Face Effect, and (4) the Configural Featural Detection Task (CFDT). Several novel methodological and analytical elements were introduced, including the use of factor analysis and the inclusion of control conditions to confirm the face specificity of all of the effects measured. The four indexes of holistic processing derived from each measure loaded onto two factors, one encompassing the PWE and the CFDT, and one encompassing the CE. The 16 conditions tested across the four tasks loaded onto four factors, each factor corresponding to a different measure. These results, together with those of other studies, suggest that holistic processing is a multifaceted construct and that different measures tap into distinct but partially overlapping elements of it.

Children process faces holistically with the same efficiency as adults

Previous studies had reported qualitatively identical holistic face processing (using the composite task) already in 6-year-olds. In the present study, we evaluated these processes quantitatively by evaluating whether children are less efficient in dealing with and encoding faces. Thus, in the present study we explored the time course of holistic processing in children and adults by manipulating stimulus presentation time. The study composite task was presented for 800 ms. The test composite face was presented either for 183 ms or 800 ms. Our youngest participants, 6-year-olds, process faces holistically with the same efficiency as typical adults.

Subcortical regions of the human visual system do not process faces holistically

Face perception is considered to be evolutionarily adaptive and conserved across species. While subcortical visual brain areas are implicated in face perception based on existing evidence from phylogenetic and ontogenetic studies, whether these subcortical structures contribute to more complex visual computations such as the holistic processing (HP) of faces in humans is unknown. To address this issue, we used a well-established marker of HP, the composite face effect (CFE), with a group of adult human observers, and presented two sequential faces in a trial monocularly or interocularly using a Wheatstone stereoscope. HP refers to the finding that two identical top (or bottom) halves of a face are judged to be different when their task-irrelevant bottom (or top) halves belong to different faces. Because humans process faces holistically, they are unable to ignore the information from the irrelevant half of the composite face, and this is true to an even greater extent when the two halves of the faces are aligned compared with when they are misaligned ('Alignment effect'). The results revealed the HP effect and also uncovered the Alignment effect, a key marker of the CFE. The findings also indicated a monocular advantage, replicating the known subcortical contribution to face perception. There was, however, no statistically significant difference in the CFE when the images were presented in the monocular versus interocular conditions. These findings indicate that HP is not necessarily mediated by the subcortical visual pathway, and suggest that further investigation of cortical, rather than subcortical, structures might advance our understanding of HP and its role in face processing.

The composite face effect is robust against perceptual misfit

The composite face effect—the failure of selective attention toward a target face half—is frequently used to study mechanisms of feature integration in faces. Here we studied how this effect depends on the perceptual fit between attended and unattended halves. We used composite faces that were rated by trained observers as either a seamless fit (i.e., close to a natural and homogeneous face) or as a deliberately bad quality of fit (i.e., unnatural, strongly segregated face halves). In addition, composites created by combining face halves randomly were tested. The composite face effect was measured as the alignment × congruency interaction (Gauthier and Bukach Cognition, 103, 322–330 2007), but also with alternative data analysis procedures (Rossion and Boremanse Journal of Vision, 8, 1–13 2008). We found strong but identical composite effects in all fit conditions. Fit quality neither increased the composite face effect nor was it attenuated by bad or random fit quality. The implications for a Gestalt account of holistic face processing are discussed.

Face and word composite effects are similarly affected by priming of local and global processing

Holistic processing has been shown with both faces and words, but it is unclear how similar their underlying mechanisms are. In this study attention to global and local features was manipulated and the consequences for holistic word and face processing were examined. On each trial participants were presented two Navon figures and told to focus on either the global or the local level. Then they performed a composite task in which they indicated whether the target halves of two sequentially presented faces or words were the same or different, ignoring the irrelevant halves. Similar stronger global priming effects were found for faces and words, indicating that holistic processing for the two types of stimuli were susceptible to attention manipulations to similar degrees, which was confirmed with Bayesian analyses. The findings add to the investigation of the similarity and differences between holistic processing and help reveal those aspects of holistic processing that are domain general and those specific to individual categories.

The constancy of the holistic processing of unfamiliar faces: Evidence from the study-test consistency effect and the within-person motion and viewpoint invariance

It has been well documented that static face processing is holistic. Faces contain variant (e.g., motion, viewpoint) and invariant (race, sex) features. However, little research has focused on whether holistic face representations are tolerant of within-person variations. The present study thus investigated whether holistic face representations of faces are tolerant of within-person motion and viewpoint variations by manipulating study-test consistency using a complete composite paradigm. Participants were shown two faces sequentially and were asked to judge whether the faces' top halves were identical or different. The first face was a static face or a dynamic face rotated in depth at 30°, 60°, and 90°. The second face was either a different front-view static face (Experiment 1a, study-test inconsistent) or identical to the first face (Experiment 1b, study-test consistent). In Experiment 2, study-test consistency was manipulated within subjects, and inverted faces were included. Our results show that study-test consistency significantly enhanced the holistic processing of upright and inverted faces; this study-test consistency effect and holistic processing were not modulated by motion and viewpoint changes via depth rotation. Interestingly, we found holistic processing for moving study-test consistent inverted faces, but not for static inverted faces. What these results tell us about the nature of holistic face representation is discussed in depth with respect to earlier and current theories on face processing.

Cortical Pathways or Mechanism in the Face Inversion Effect in Patients with First-Episode Schizophrenia

OBJECTIVE

Impaired face perception is considered as a hallmark of social disability in schizophrenia. It is widely believed that inverted faces and upright faces are processed by distinct mechanisms. Previous studies have identified that individuals with schizophrenia display poorer face processing than controls. However, the mechanisms underlying the face inversion effect (FIE) in patients with first-episode schizophrenia (FSZ) remain unclear.

METHODS

We designed an fMRI task to investigate the FIE mechanism in patients with schizophrenia. Thirty-four patients with FSZ and thirty-five healthy controls (CON) underwent task-related fMRI scanning, clinical assessment, anhedonia experience examination, and social function and cognitive function evaluation.

RESULTS

The patients with FSZ exhibited distinct functional activity regarding upright and inverted face processing within the cortical face and non-face network. These results suggest that the differences in quantitative processing might mediate the FIE in schizophrenia. Compared with controls, affected patients showed impairments in processing both upright and inverted faces; and for these patients with FSZ, upright face processing was associated with more severe and broader impairment than inverted face processing. Reduced response in the left middle occipital gyrus for upright face processing was related to poorer performance of social function outcomes evaluated using the Personal and Social Performance Scale.

CONCLUSION

Our data suggested that patients with FSZ exhibited similar performance in processing inverted faces and upright faces, but were less efficient than controls; and for these patients, inverted faces are processed less efficiently than upright faces. We also provided a clue that the mechanism under abnormal FIE might be related to an aberrant activation of non-face-selective areas instead of abnormal activation of face-specific areas in patients with schizophrenia. Finally, our study indicated that the neural pathway for upright recognition might be relevant in determining the functional outcomes of this devastating disorder.

Holistic ensemble perception

In a glance, observers can evaluate gist characteristics from crowds of faces, such as the average emotional tenor or the average family resemblance. Prior research suggests that high-level ensemble percepts rely on holistic and viewpoint-invariant information. However, it is also possible that feature-based analysis was sufficient to yield successful ensemble percepts in many situations. To confirm that ensemble percepts can be extracted holistically, we asked observers to report the average emotional valence of Mooney face crowds. Mooney faces are two-tone, shadow-defined images that cannot be recognized in a part-based manner. To recognize features in a Mooney face, one must first recognize the image as a face by processing it holistically. Across experiments, we demonstrated that observers successfully extracted the average emotional valence from crowds that were spatially distributed or viewed in a rapid temporal sequence. In a subsequent set of experiments, we maximized holistic processing by including only those Mooney faces that were difficult to recognize when inverted. Under these conditions, participants remained highly sensitive to the average emotional valence of Mooney face crowds. Taken together, these experiments provide evidence that ensemble perception can operate selectively on holistic representations of human faces, even when feature-based information is not readily available.

Investigating holistic face processing within and outside of face-responsive brain regions

It has been shown that human faces are processed holistically (i.e. as indecomposable wholes, rather than by their component parts) and this holistic face processing is linked to brain activity in face-responsive brain regions. Although several brain regions outside of the face-responsive network are also sensitive to relational processing and perceptual grouping, whether these non-face-responsive regions contribute to holistic processing remains unclear. Here, we investigated holistic face processing in the composite face paradigm both within and outside of face-responsive brain regions. We recorded participants' brain activity using fMRI while they performed a composite face task. Behavioural results indicate that participants tend to judge the same top face halves as different when they are aligned with different bottom face halves but not when they are misaligned, demonstrating a composite face effect. Neuroimaging results revealed significant differences in responses to aligned and misaligned faces in the lateral occipital complex (LOC), and trends in the anterior part of the fusiform face area (FFA2) and transverse occipital sulcus (TOS), suggesting that these regions are sensitive to holistic versus part-based face processing. Furthermore, the retrosplenial cortex (RSC) and the parahippocampal place area (PPA) showed a pattern of neural activity consistent with a holistic representation of face identity, which also correlated with the strength of the behavioural composite face effect. These results suggest that neural activity in brain regions both within and outside of the face-responsive network contributes to the composite-face effect.

Flexible face processing: Holistic processing of facial identity is modulated by task-irrelevant facial expression

Holistic processing is a hallmark of face perception and has been observed separately for facial identity and expression. While the identity of a face remains constant regardless of any changes in facial expressions, to what extent is holistic processing of facial identity affected by task-irrelevant facial expressions? If holistic processing is flexible and integrates both identity and expression information, the magnitude of holistic processing of facial identity may be systematically modulated by different facial expressions, due to either visual or emotional differences among the expressions. In Experiment 1, participants matched the identities of target halves of two sequentially presented face composites, with both composites showing either positive (happy) or negative (angry) expressions. The presentation duration for the test composite was either short (200 ms) or long (until a response). With a short presentation duration, the magnitude of holistic processing of identity for happy and angry composites was comparable. In contrast, with a long presentation duration, holistic processing of identity was reduced for angry compared with happy face composites. Experiment 2 replicated the results and showed reduced holistic processing of identity for face composites with either angry or neutral expressions, compared with happy expressions, given a long presentation duration. Because the modulation of facial expressions on holistic processing of facial identity was observed with long, but not short, presentation durations, these results suggest that such influence is unlikely solely due to visual differences, but may instead arise from cognitive evaluation of the emotions conveyed by facial expressions.

Further evidence for a late locus of holistic word processing: Exploring vertex effect in the word composite task

Previous studies have shown a rather late and lexical level for holistic word processing. In the present study, we evaluated whether there are early effects in holistic processing of words, taking into consideration the role of lower-level visual processes that are critical in the hierarchy of visual word recognition: the extraction of viewpoint-invariant line junctions/vertices. We used contour-deleted words in two conditions: preservation of the vertices versus preservation of midsegments and an all-contour condition. We found evidence of a composite effect that was equivalent for all materials. Thus, we found no evidence of an early contribution of holistic processing to word recognition, and confirmed that holistic word processing is related to late lexical orthographic representations.

Age Differences in Face Processing: The Role of Perceptual Degradation and Holistic Processing

OBJECTIVES

We simultaneously investigated the role of three hypotheses regarding age-related differences in face processing: perceptual degradation, impaired holistic processing, and an interaction between the two.

METHODS

Young adults (YA) aged 20-33-year olds, middle-age adults (MA) aged 50-64-year olds, and older adults (OA) aged 65-82-year olds were tested on the context congruency paradigm, which allows measurement of face-specific holistic processing across the life span (Meinhardt-Injac, Persike & Meinhardt, 2014. Acta Psychologica, 151, 155-163). Perceptual degradation was examined by measuring performance with faces that were not filtered (FSF), with faces filtered to preserve low spatial frequencies (LSF), and with faces filtered to preserve high spatial frequencies (HSF).

RESULTS

We found that reducing perceptual signal strength had a greater impact on MA and OA for HSF faces, but not LSF faces. Context congruency effects were significant and of comparable magnitude across ages for FSF, LSF, and HSF faces. By using watches as control objects, we show that these holistic effects reflect face-specific mechanisms in all age groups.

DISCUSSION

Our results support the perceptual degradation hypothesis for faces containing only HSF and suggest that holistic processing is preserved in aging even under conditions of reduced signal strength.

Unmasking individual differences in adult reading procedures by disrupting holistic orthographic perception

Word identification is undeniably important for skilled reading and ultimately reading comprehension. Interestingly, both lexical and sublexical procedures can support word identification. Recent cross-linguistic comparisons have demonstrated that there are biases in orthographic coding (e.g., holistic vs. analytic) linked with differences in writing systems, such that holistic orthographic coding is correlated with lexical-level reading procedures and vice versa. The current study uses a measure of holistic visual processing used in the face processing literature, orientation sensitivity, to test individual differences in word identification within a native English population. Results revealed that greater orientation sensitivity (i.e., greater holistic processing) was associated with a reading profile that relies less on sublexical phonological measures and more on lexical-level characteristics within the skilled English readers. Parallels to Chinese procedures of reading and a proposed alternative route to skilled reading are discussed.

Understanding the role of configural processing in face emotion recognition in Parkinson's disease

This investigation examined whether impairment in configural processing could explain deficits in face emotion recognition in people with Parkinson's disease (PD). Stimuli from the Radboud Faces Database were used to compare recognition of four negative emotion expressions by older adults with PD (n = 16) and matched controls (n = 17). Participants were tasked with categorizing emotional expressions from upright and inverted whole faces and facial composites; it is difficult to derive configural information from these two types of stimuli so featural processing should play a larger than usual role in accurate recognition of emotional expressions. We found that the PD group were impaired relative to controls in recognizing anger, disgust and fearful expressions in upright faces. Then, consistent with a configural processing deficit, participants with PD showed no composite effect when attempting to identify facial expressions of anger, disgust and fear. A face inversion effect, however, was observed in the performance of all participants in both the whole faces and facial composites tasks. These findings can be explained in terms of a configural processing deficit if it is assumed that the disruption caused by facial composites was specific to configural processing, whereas inversion reduced performance by making it difficult to derive both featural and configural information from faces.

Holistic face representation is highly orientation-specific

It has long been argued that face processing requires disproportionate reliance on holistic processing (HP), relative to that required for nonface object recognition. Nevertheless, whether the holistic nature of face perception is achieved via a unique internal representation or by the employment of an automated attention mechanism is still debated. Previous studies had used the face inversion effect (FIE), a unique face-processing marker, or the face composite task, a gold standard paradigm measuring holistic processing, to examine the validity of these two different hypotheses, with some studies combining the two paradigms. However, the results of such studies remain inconclusive, particularly pertaining to the issue of the two proposed HP mechanisms-an internal representation as opposed to an automated attention mechanism. Here, using the complete composite paradigm design, we aimed to examine whether face rotation yields a nonlinear or a linear drop in HP, thus supporting an account that face processing is based either on an orientation-dependent internal representation or on automated attention. Our results reveal that even a relatively small perturbation in face orientation (30 deg away from upright) already causes a sharp decline in HP. These findings support the face internal representation hypothesis and the notion that the holistic processing of faces is highly orientation-specific.

Individual differences in object recognition

There is substantial evidence for individual differences in personality and cognitive abilities, but we lack clear intuitions about individual differences in visual abilities. Previous work on this topic has typically compared performance with only 2 categories, each measured with only 1 task. This approach is insufficient for demonstration of domain-general effects. Most previous work has used familiar object categories, for which experience may vary between participants and categories, thereby reducing correlations that would stem from a common factor. In Study 1, we adopted a latent variable approach to test for the first time whether there is a domain-general object recognition ability, o. We assessed whether shared variance between latent factors representing performance for each of 5 novel object categories could be accounted for by a single higher-order factor. On average, 89% of the variance of lower-order factors denoting performance on novel object categories could be accounted for by a higher-order factor, providing strong evidence for o. Moreover, o also accounted for a moderate proportion of variance in tests of familiar object recognition. In Study 2, we assessed whether the strong association across categories in object recognition is due to third-variable influences. We find that o has weak to moderate associations with a host of cognitive, perceptual, and personality constructs and that a clear majority of the variance in and covariance between performance on different categories is independent of fluid intelligence. This work provides the first demonstration of a reliable, specific, and domain-general object recognition ability, and suggest a rich framework for future work in this area. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Investigating the Time Course of Part-Based and Holistic Processing in Face Perception

Human has an exceptional ability for face recognition to keep up social network. However, it is unclear to understand the mechanisms of face recognition until now. Specifically, there is less research to examine the time course of part-based and holistic processing when these two routes trigger and finish. In the present experiments, the exposure time was manipulated to examine the time course of face processing and found evidence suggesting that holistic processing occurs shortly after part-based processing at about 200 ms, and can last for a relatively long duration up to 2,000 ms. These results may support to a dual-route model comprising holistic processing and part-based processing in face perception. Moreover, our findings were inconsistent with the previous study which suggests that no holistic processing was observed at the relatively long duration, and suspected that perceptual discriminability may have been responsible for the discrepancy.

Orientation-invariance of individual differences in three face processing tasks

Numerous studies have reported impairments in perception and recognition, and, particularly, in part-integration of faces following picture-plane inversion. Whether these findings support the notion that inversion changes face processing qualitatively remains a topic of debate. To examine whether associations and dissociations of the human face processing ability depend on stimulus orientation, we measured face recognition with the Cambridge Face Memory Test (CFMT), along with experimental tests of face perception and selective attention to faces and non-face objects in a sample of 314 participants. Results showed strong inversion effects for all face-related tasks, and modest ones for non-face objects. Individual differences analysis revealed that the CFMT shared common variance with face perception and face-selective attention, however, independent of orientation. Regardless of whether predictor and criterion had same or different orientation, face recognition was best predicted by the same test battery. Principal component decomposition revealed a common factor for face recognition and face perception, a second common factor for face recognition and face-selective attention, and two unique factors. The patterns of factor loadings were nearly identical for upright and inverted presentation. These results indicate orientation-invariance of common variance in three domains of face processing. Since inversion impaired performance, but did not affect domain-related associations and dissociations, the findings suggest process-specific but orientation-general mechanisms. Specific limitations by constraints of individual differences analysis and test selection are discussed.

The development of holistic face processing: An evaluation with the complete design of the composite task

The composite paradigm is widely used to quantify holistic processing (HP) of faces: participants perform a sequential same-different task on one half (e.g., top) of a test-face relative to the corresponding half of a study-face. There is, however, debate regarding the appropriate design in this task. In the partial design, the irrelevant halves (e.g., bottom) of test- and study-faces are always different; an alignment effect indexes HP. In the complete design, besides alignment, congruency between the irrelevant and critical halves of the test-face is manipulated regarding the same/different response status of the study-face. The HP indexed in the complete design does not confound congruency and alignment and has good construct and convergent validities. De Heering, Houthuys, & Rossion (2007) argued that HP is mature as early as 4-year-olds but employed the partial design. Here we revisit this claim, testing four groups of 4- to 9/10 year-old children and two groups of adults. We found evidence of HP only from 6-year-olds on when considering the complete design, whereas significant alignment effects were found in the index adopted in the partial design already in 4-year-olds but which we demonstrate that reflects other factors besides HP, including response bias associated with congruency.

Early identity recognition of familiar faces is not dependent on holistic processing

It is widely accepted that holistic processing is critical for early face recognition, but recent work has suggested a larger role for feature-based processing. The earliest step in familiar face recognition is thought to be matching a perceptual representation of a familiar face to a stored representation of that face, which is thought to be indexed by the N250r event-related potential (ERP). In the current face-priming studies, we investigated whether this perceptual representation can be effectively activated by feature-based processing. In the first experiment, prime images were familiar whole faces, isolated eyes or isolated mouths. Whole faces and isolated eyes, but not isolated mouths, effectively modulated the N250r. In the second experiment, prime images were familiar whole faces presented either upright or inverted. Inverted face primes were no less effective than upright face primes in modulating the N250r. Together, the results of these studies indicate that activation of the earliest face recognition processes is not dependent on holistic processing of a typically configured face. Rather, feature-based processing can effectively activate the perceptual memory of a familiar face. However, not all features are effective primes as we found eyes, but not mouths, were effective in activating early face recognition.

Perceived Race Affects Configural Processing but Not Holistic Processing in the Composite-Face Task

One explanation for the own-race bias in face recognition is the loss of holistic processing for other-race faces. The composite-face task (involving matching the top halves of faces when the bottom halves are either changed or the same) tests holistic processing but it has been inconsistent in revealing other-race effects. Two composite-face experiments are reported using pairs of faces that have common internal features but can be perceived as either being racially Black or White depending on their external features. In Experiment 1 (matching the top halves of faces) holistic processing was found for both face races for White participants (shown by both a mis-alignment advantage when bottom halves were different and also by a congruence-by-alignment interaction in discrimination). Bayesian analysis supported there being no effect of race. However, the size of the simple congruence effect was larger for own- than for other-race faces. Experiment 2 found that this race-by-congruence interaction was not present when matching the bottom halves of faces. The results are interpreted in of terms of the perceived race affecting the processing of second-order relational information rather than holistic processing.

Hemispheric asymmetry in holistic processing of words

Holistic processing has been regarded as a hallmark of face perception, indicating the automatic and obligatory tendency of the visual system to process all face parts as a perceptual unit rather than in isolation. Studies involving lateralized stimulus presentation suggest that the right hemisphere dominates holistic face processing. Holistic processing can also be shown with other categories such as words and thus it is not specific to faces or face-like expertize. Here, we used divided visual field presentation to investigate the possibly different contributions of the two hemispheres for holistic word processing. Observers performed same/different judgment on the cued parts of two sequentially presented words in the complete composite paradigm. Our data indicate a right hemisphere specialization for holistic word processing. Thus, these markers of expert object recognition are domain general.

Inversion Impairs Expert Budgerigar Identity Recognition: A Face-Like Effect for a Nonface Object of Expertise

The face-inversion effect is the finding that picture-plane inversion disproportionately impairs face recognition compared to object recognition and is now attributed to greater orientation-sensitivity of holistic processing for faces but not common objects. Yet, expert dog judges have showed similar recognition deficits for inverted dogs and inverted faces, suggesting that holistic processing is not specific to faces but to the expert recognition of perceptually similar objects. Although processing changes in expert object recognition have since been extensively documented, no other studies have observed the distinct recognition deficits for inverted objects-of-expertise that people as face experts show for faces. However, few studies have examined experts who recognize individual objects similar to how people recognize individual faces. Here we tested experts who recognize individual budgerigar birds. The effect of inversion on viewpoint-invariant budgerigar and face recognition was compared for experts and novices. Consistent with the face-inversion effect, novices showed recognition deficits for inverted faces but not for inverted budgerigars. By contrast, experts showed equal recognition deficits for inverted faces and budgerigars. The results are consistent with the hypothesis that processes underlying the face-inversion effect are specific to the expert individuation of perceptually similar objects.

How holistic processing of faces relates to cognitive control and intelligence

The Vanderbilt Holistic Processing Test for faces (VHPT-F) is the first standard test designed to measure individual differences in holistic processing. The test measures failures of selective attention to face parts through congruency effects, an operational definition of holistic processing. However, this conception of holistic processing has been challenged by the suggestion that it may tap into the same selective attention or cognitive control mechanisms that yield congruency effects in Stroop and Flanker paradigms. Here, we report data from 130 subjects on the VHPT-F, several versions of Stroop and Flanker tasks, as well as fluid IQ. Results suggested a small degree of shared variance in Stroop and Flanker congruency effects, which did not relate to congruency effects on the VHPT-F. Variability on the VHPT-F was also not correlated with Fluid IQ. In sum, we find no evidence that holistic face processing as measured by congruency in the VHPT-F is accounted for by domain-general control mechanisms.