Revealing the mechanisms of human face perception using dynamic apertures

The heterogeneity of holistic processing profiles in developmental prosopagnosia: holistic processing is impaired but not absent

Although it is generally assumed that face recognition relies on holistic processing, whether face recognition deficits observed in Developmental Prosopagnosics (DPs) can be explained by impaired holistic processing is currently under debate. The mixed findings from past studies could be the consequence of DP's heterogeneous deficit nature and the use of different measures of holistic processing-the inversion, part-whole, and composite tasks-which showed a poor association among each other. The present study aimed to gain further insight into the role of holistic processing in DPs. Groups of DPs and neurotypicals completed three tests measuring holistic face processing and non-face objects (i.e., Navon task). At a group level, DPs showed (1) diminished, but not absent, inversion and part-whole effects, (2) comparable magnitudes of the composite face effect and (3) global precedence effect in the Navon task. However, single-case analyses showed that these holistic processing deficits in DPs are heterogeneous.

The face inversion effect does not provide a pure measure of holistic face processing

It is widely held that upright faces are processed more holistically than inverted faces and that this difference is reflected in the face inversion effect. It is not clear, however, how the inversion effect can best be measured, whether it is task specific, or even whether it specifically correlates with processing of upright faces. We examined these questions in a large sample (N = 420) who provided data on processing of upright and inverted stimuli in two different tasks with faces and one with objects. We find that the inversion effects are task dependent, and that they do not correlate better among face processing tasks than they do across face and object processing tasks. These findings were obtained regardless of whether inversion effects were measured by means of difference scores or regression. In comparison, only inversion effects based on regression predicted performance with upright faces in tasks other than those the inversion effects were derived from. Critically, however, inversion effects based on regression also predicted performance with inverted faces to a similar degree as they predicted performance with upright faces. Consequently, and contrary to what is commonly assumed, inversion effects do not seem to capture effects specific to holistic processing of upright faces. While the present findings do not bring us closer to an understanding of which changes in cognitive processing are induced by inversion, they do suggest that inversion effects do not reflect a unitary construct; an implicit assumption that seems to characterize much of the research regarding face processing.

Holistic and featural processing's link to face recognition varies by individual and task

While it is generally accepted that holistic processing facilitates face recognition, recent studies suggest that poor recognition might also arise from imprecise perception of local features in the face. This study aimed to examine to what extent holistic and featural processing relates to individual differences in face recognition ability (FRA), during face learning (Experiment 1) and face recognition (Experiment 2). Participants performed two tasks: (1) The "Cambridge Face Memory Test-Chinese" which measured participants' FRAs, and (2) an "old/new recognition memory test" encompassing whole faces (preserving holistic and featural processing) and faces revealed through a dynamic aperture (impairing holistic processing but preserving featural processing). Our results showed that participants recognised faces more accurately in conditions when holistic information was preserved, than when it is impaired. We also show that the better use of holistic processing during face learning and face recognition was associated with better FRAs. However, enhanced featural processing during recognition, but not during learning, was related to better FRAs. Together, our findings demonstrate that good face recognition depends on distinct roles played by holistic and featural processing at different stages of face recognition.

Face perception: computational insights from phylogeny

Studies of face perception in primates elucidate the psychological and neural mechanisms that support this critical and complex ability. Recent progress in characterizing face perception across species, for example in insects and reptiles, has highlighted the ubiquity over phylogeny of this key ability for social interactions and survival. Here, we review the competence in face perception across species and the types of computation that support this behavior. We conclude that the computational complexity of face perception evinced by a species is not related to phylogenetic status and is, instead, largely a product of environmental context and social and adaptive pressures. Integrating findings across evolutionary data permits the derivation of computational principles that shed further light on primate face perception.

Linear Integration of Sensory Evidence over Space and Time Underlies Face Categorization

Visual object recognition relies on elaborate sensory processes that transform retinal inputs to object representations, but it also requires decision-making processes that read out object representations and function over prolonged time scales. The computational properties of these decision-making processes remain underexplored for object recognition. Here, we study these computations by developing a stochastic multifeature face categorization task. Using quantitative models and tight control of spatiotemporal visual information, we demonstrate that human subjects (five males, eight females) categorize faces through an integration process that first linearly adds the evidence conferred by task-relevant features over space to create aggregated momentary evidence and then linearly integrates it over time with minimum information loss. Discrimination of stimuli along different category boundaries (e.g., identity or expression of a face) is implemented by adjusting feature weights of spatial integration. This linear but flexible integration process over space and time bridges past studies on simple perceptual decisions to complex object recognition behavior.SIGNIFICANCE STATEMENT Although simple perceptual decision-making such as discrimination of random dot motion has been successfully explained as accumulation of sensory evidence, we lack rigorous experimental paradigms to study the mechanisms underlying complex perceptual decision-making such as discrimination of naturalistic faces. We develop a stochastic multifeature face categorization task as a systematic approach to quantify the properties and potential limitations of the decision-making processes during object recognition. We show that human face categorization could be modeled as a linear integration of sensory evidence over space and time. Our framework to study object recognition as a spatiotemporal integration process is broadly applicable to other object categories and bridges past studies of object recognition and perceptual decision-making.

The constancy of the holistic processing of unfamiliar faces: Evidence from the study-test consistency effect and the within-person motion and viewpoint invariance

It has been well documented that static face processing is holistic. Faces contain variant (e.g., motion, viewpoint) and invariant (race, sex) features. However, little research has focused on whether holistic face representations are tolerant of within-person variations. The present study thus investigated whether holistic face representations of faces are tolerant of within-person motion and viewpoint variations by manipulating study-test consistency using a complete composite paradigm. Participants were shown two faces sequentially and were asked to judge whether the faces' top halves were identical or different. The first face was a static face or a dynamic face rotated in depth at 30°, 60°, and 90°. The second face was either a different front-view static face (Experiment 1a, study-test inconsistent) or identical to the first face (Experiment 1b, study-test consistent). In Experiment 2, study-test consistency was manipulated within subjects, and inverted faces were included. Our results show that study-test consistency significantly enhanced the holistic processing of upright and inverted faces; this study-test consistency effect and holistic processing were not modulated by motion and viewpoint changes via depth rotation. Interestingly, we found holistic processing for moving study-test consistent inverted faces, but not for static inverted faces. What these results tell us about the nature of holistic face representation is discussed in depth with respect to earlier and current theories on face processing.

Patterns of perceptual performance in developmental prosopagnosia: An in-depth case series

Developmental prosopagnosia (DP) is a syndrome characterized by lifelong impairment in face recognition in the absence of brain damage. A key question regarding DP concerns which process(es) might be affected to selectively/disproportionally impair face recognition. We present evidence from a group of DPs, combining an overview of previous results with additional analyses important for understanding their pattern of preserved and impaired perceptual abilities. We argue that for most of these individuals, the common denominator is a deficit in (rapid) processing of global shape information. We conclude that the deficit in this group of DPs is not face-selective, but that it may appear so because faces are more visually similar-and recognized at a more fine-grained level-than objects. Indeed, when the demand on perceptual differentiation and visual similarity are held constant for faces and objects, we find no evidence for a disproportionate deficit for faces in this group of DPs.

Look up to the body: An eye-tracking investigation of 7-months-old infants' visual exploration of emotional body expressions

The human body is an important source of information to infer a person's emotional state. Research with adult observers indicate that the posture of the torso, arms and hands provide important perceptual cues for recognising anger, fear and happy expressions. Much less is known about whether infants process body regions differently for different body expressions. To address this issue, we used eye tracking to investigate whether infants' visual exploration patterns differed when viewing body expressions. Forty-eight 7-months-old infants were randomly presented with static images of adult female bodies expressing anger, fear and happiness, as well as an emotionally-neutral posture. Facial cues to emotional state were removed by masking the faces. We measured the proportion of looking time, proportion and number of fixations, and duration of fixations on the head, upper body and lower body regions for the different expressions. We showed that infants explored the upper body more than the lower body. Importantly, infants at this age fixated differently on different body regions depending on the expression of the body posture. In particular, infants spent a larger proportion of their looking times and had longer fixation durations on the upper body for fear relative to the other expressions. These results extend and replicate the information about infant processing of emotional expressions displayed by human bodies, and they support the hypothesis that infants' visual exploration of human bodies is driven by the upper body.

Holistic processing of facial identity in developmental prosopagnosia

The nature of the perceptual deficit seen in developmental prosopagnosia remains poorly understood. One possibility is that these individuals experience face recognition difficulties because they fail to process faces holistically; they may be less able to analyze distal regions in parallel and therefore struggle to integrate information from different regions into a unified perceptual whole. Consequently, developmental prosopagnosics may be forced to base perceptual decisions on a slow, effortful piecemeal analysis of local facial features. In the present study, we sought to test this view by comparing the face recognition of developmental prosopagnosics and typical observers under two viewing conditions: when target faces were briefly presented in their entirety, and when they were inspected region-by-region through a dynamic aperture. If developmental prosopagnosics are forced to base perceptual decisions on information accumulated from a serial piecemeal analysis, one would expect little if any decrement in performance when target faces are viewed through apertures. Contrary to this prediction, however, developmental prosopagnosics showed strong aperture effects comparable with typical observers; their perceptual decisions were more accurate in the whole-face condition than when targets were viewed through the aperture. As expected, the developmental prosopagnosics were less accurate than typical controls when judging briefly presented faces shown in their entirety. Strikingly, however, they were also less able to accumulate perceptual evidence from a serial region-by-region analysis, than typical observers. Our results suggest that the perceptual problems seen in this population arise from imprecise descriptions of local regions, not aberrant holistic processing.

Assessing the presence of face biases by means of anorthoscopic perception

Anorthoscopy is a presentation mode in which an image is shown sliding behind a slit-shaped aperture, so that it is visible only part by part and never in its entirety. With the aims to assess (1) whether the processing of complex stimuli (faces) correctly occurs in anorthoscopy, and (2) whether the Own-Gender Bias (OGB: the better recognition of stimuli belonging to the same gender of the observer: faster and more accurate) and the Left-Face Bias (LFB: the preference to analyze the left half of the face) occur in such a part by part presentation, we presented female and male faces as whole stimuli (Experiment 1) and in anorthoscopy (Experiments 2 and 3), as well as female/male chimeric faces (Experiment 4), during a gender categorization task. Experiment 1 confirmed that participants correctly categorized the gender of faces, but the OGB was not found. In Experiments 2 and 3 we manipulated the direction (Experiment 2: upward/downward; Experiment 3: leftward/rightward), the speed (slow and fast) of the sliding faces, and the width of the aperture (small and large). Both tasks revealed that facial gender was correctly categorized in anorthoscopy. The OGB was found, but only for males/females in Experiments 2/3, respectively. In Experiment 4 the LFB emerged only in the tachistoscopic session, suggesting that this perceptual bias does not extend to anorthoscopy.

The discrimination of facial sex in developmental prosopagnosia

Developmental prosopagnosia (DP) is a neurodevelopmental condition characterised by difficulties recognising and discriminating faces. It is currently unclear whether the perceptual impairments seen in DP are restricted to identity information, or also affect the perception of other facial characteristics. To address this question, we compared the performance of 17 DPs and matched controls on two sensitive sex categorisation tasks. First, in a morph categorisation task, participants made binary decisions about faces drawn from a morph continuum that blended incrementally an average male face and an average female face. We found that judgement precision was significantly lower in the DPs than in the typical controls. Second, we used a sex discrimination task, where female or male facial identities were blended with an androgynous average face. We manipulated the relative weighting of each facial identity and the androgynous average to create four levels of signal strength. We found that DPs were significantly less sensitive than controls at each level of difficulty. Together, these results suggest that the visual processing difficulties in DP extend beyond the extraction of facial identity and affects the extraction of other facial characteristics. Deficits of facial sex categorisation accord with an apperceptive characterisation of DP.

Face perception in autism spectrum disorder: Modulation of holistic processing by facial emotion

Individuals with Autism Spectrum Disorder (ASD; autistic individuals) may exhibit atypical face perception because they fail to process faces holistically. In the context of this hypothesis, it is critical to determine whether autistic individuals exhibit diminished susceptibility to the composite face illusion, widely regarded as a key marker of holistic face processing. To date, however, previous studies have yielded inconsistent findings. In light of recent evidence suggesting that facial emotion cues increase the strength of the composite face illusion in typical individuals, the present study sought to determine whether the presence of facial emotion also modulates the strength of the composite face illusion in autistic individuals, many of whom experience difficulties recognizing facial expressions. We therefore measured composite face effects in a sample of autistic individuals (N = 20) and matched typical controls (N = 29) using an incidental emotion procedure in which distractor regions varied systematically in their emotion strength. As expected, the presence of facial emotion in the distractor regions of composite face arrangements increased the strength of the illusory distortion induced. The extent of the modulation by facial emotion was similar in the two groups. The composite effects seen in the ASD group were qualitatively and quantitatively similar to those seen in the typical group, suggestive of intact holistic processing in this population.