Whole not hole: Expert face recognition requires holistic perception

Facial and Emotion Recognition Deficits in Myasthenia Gravis

Myasthenia gravis (MG) is a neuromuscular disease of autoimmune etiology and chronic evolution. In addition to the muscle weakness and fatigue that characterize MG, in some studies patients show an inferior performance in cognitive tasks and difficulties in recognizing basic emotions from facial expressions. However, it remains unclear if these difficulties are due to anxious-depressive symptoms that these patients present or related to cognitive abilities, such as facial recognition. This study had a descriptive cross-sectional design with a sample of 92 participants, 52 patients with MG and 40 healthy controls. The data collection protocol included measures to assess recognition of facial expressions (BRFT), facial emotional expression (FEEL), and levels of anxiety and depression (HADS). The MG group had worse performance than the control group in recognizing "fear" (p = 0.001; r = 0.344), "happiness" (p = 0.000; r = 0.580), "disgust" (p = 0.000; r = 0.399), "surprise" (p = 0.000; r = 0.602), and "anger" (p = 0.007; r = 0.284). Likewise, the MG group also underperformed in facial recognition (p = 0.001; r = 0.338). These difficulties were not related to their levels of anxiety and depression. Alterations were observed both in the recognition of facial emotions and in facial recognition, without being mediated by emotional variables. These difficulties can influence the interpersonal interaction of patients with MG.

The role of inversion and face masks on simultaneous and delayed face matching tasks

Although it is generally accepted that face recognition relies on holistic processing, it has been suggested that the simultaneous face matching task may depend on a more analytical or featural processing approach. However, empirical evidence supporting this claim is limited. In two experiments, we further explored the role of holistic and featural processing on simultaneous face matching by manipulating holistic processing through inversion and presenting faces with or without face masks. The results from Experiment 1 revealed that both inversion and face masks impaired matching performance. However, while the inversion effect was evident in both full-view and masked faces, the mask effect was only found in upright, but not inverted, faces. These results were replicated in Experiment 2 but, the inversion and mask effects were stronger in delayed face matching than in simultaneous face matching. Our findings suggest that simultaneous face matching relies on holistic processing, but to a smaller extent compared to higher memory-demanding identification tasks.

A perceptual field test in object experts using gaze-contingent eye tracking

A hallmark of expert object recognition is rapid and accurate subordinate-category recognition of visually homogenous objects. However, the perceptual strategies by which expert recognition is achieved is less known. The current study investigated whether visual expertise changes observers' perceptual field (e.g., their ability to use information away from fixation for recognition) for objects in their domain of expertise, using a gaze-contingent eye-tracking paradigm. In the current study, bird experts and novices were presented with two bird images sequentially, and their task was to determine whether the two images were of the same species (e.g., two different song sparrows) or different species (e.g., song sparrow and chipping sparrow). The first study bird image was presented in full view. The second test bird image was presented fully visible (full-view), restricted to a circular window centered on gaze position (central-view), or restricted to image regions beyond a circular mask centered on gaze position (peripheral-view). While experts and novices did not differ in their eye-movement behavior, experts' performance on the discrimination task for the fastest responses was less impaired than novices in the peripheral-view condition. Thus, the experts used peripheral information to a greater extent than novices, indicating that the experts have a wider perceptual field to support their speeded subordinate recognition.

Disrupted visual input unveils the computational details of artificial neural networks for face perception

Background

Convolutional Neural Network (DCNN), with its great performance, has attracted attention of researchers from many disciplines. The studies of the DCNN and that of biological neural systems have inspired each other reciprocally. The brain-inspired neural networks not only achieve great performance but also serve as a computational model of biological neural systems.

Methods

Here in this study, we trained and tested several typical DCNNs (AlexNet, VGG11, VGG13, VGG16, DenseNet, MobileNet, and EfficientNet) with a face ethnicity categorization task for experiment 1, and an emotion categorization task for experiment 2. We measured the performance of DCNNs by testing them with original and lossy visual inputs (various kinds of image occlusion) and compared their performance with human participants. Moreover, the class activation map (CAM) method allowed us to visualize the foci of the "attention" of these DCNNs.

Results

The results suggested that the VGG13 performed the best: Its performance closely resembled human participants in terms of psychophysics measurements, it utilized similar areas of visual inputs as humans, and it had the most consistent performance with inputs having various kinds of impairments.

Discussion

In general, we examined the processing mechanism of DCNNs using a new paradigm and found that VGG13 might be the most human-like DCNN in this task. This study also highlighted a possible paradigm to study and develop DCNNs using human perception as a benchmark.

A featural account for own-face processing? Looking for support from face inversion, composite face, and part-whole tasks

It is widely accepted that face perception relies on holistic processing. However, this holistic advantage is not always found in the processing of the own face. Our study aimed to explore the role of holistic and featural processing in the identification of the own face, using three standard, but largely independent measures of holistic face processing: the face inversion task, the composite face task, and the part-whole task. Participants were asked to identify their face, a friend’s face, and an unfamiliar face in three different experimental blocks: (a) inverted versus upright; (b) top and bottom halves of the face aligned versus misaligned; and (c) facial features presented in isolation versus whole foil face context. Inverting a face impaired its identification, regardless of the identity. However, alignment effects were only found when identifying a friend or an unfamiliar face. In addition, a stronger feature advantage (i.e., better recognition for isolated features compared to in a whole-face context) was observed for the own face compared to the friend and unfamiliar faces. Altogether, these findings suggest that the own face is processed in a more featural manner but also relies on holistic processing. This work also highlights the importance of taking into consideration that different holistic processing paradigms could tap different forms of holistic processing.

Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part II: Neural basis

Patient PS sustained her dramatic brain injury in 1992, the same year as the first report of a neuroimaging study of human face recognition. The present paper complements the review on the functional nature of PS's prosopagnosia (part I), illustrating how her case study directly, i.e., through neuroimaging investigations of her brain structure and activity, but also indirectly, through neural studies performed on other clinical cases and neurotypical individuals, inspired and constrained neural models of human face recognition. In the dominant right hemisphere for face recognition in humans, PS's main lesion concerns (inputs to) the inferior occipital gyrus (IOG), in a region where face-selective activity is typically found in normal individuals ('Occipital Face Area', OFA). Her case study initially supported the criticality of this region for face identity recognition (FIR) and provided the impetus for transcranial magnetic stimulation (TMS), intracerebral electrical stimulation, and cortical surgery studies that have generally supported this view. Despite PS's right IOG lesion, typical face-selectivity is found anteriorly in the middle portion of the fusiform gyrus, a hominoid structure (termed the right 'Fusiform Face Area', FFA) that is widely considered to be the most important region for human face recognition. This finding led to the original proposal of direct anatomico-functional connections from early visual cortices to the FFA, bypassing the IOG/OFA (lulu), a hypothesis supported by further neuroimaging studies of PS, other neurological cases and neuro-typical individuals with original visual stimulation paradigms, data recordings and analyses. The proposal of a lack of sensitivity to face identity in PS's right FFA due to defective reentrant inputs from the IOG/FFA has also been supported by other cases, functional connectivity and cortical surgery studies. Overall, neural studies of, and based on, the case of prosopagnosia PS strongly question the hierarchical organization of the human neural face recognition system, supporting a more flexible and dynamic view of this key social brain function.

Is human face recognition lateralized to the right hemisphere due to neural competition with left-lateralized visual word recognition? A critical review

The right hemispheric lateralization of face recognition, which is well documented and appears to be specific to the human species, remains a scientific mystery. According to a long-standing view, the evolution of language, which is typically substantiated in the left hemisphere, competes with the cortical space in that hemisphere available for visuospatial processes, including face recognition. Over the last decade, a specific hypothesis derived from this view according to which neural competition in the left ventral occipito-temporal cortex with selective representations of letter strings causes right hemispheric lateralization of face recognition, has generated considerable interest and research in the scientific community. Here, a systematic review of studies performed in various populations (infants, children, literate and illiterate adults, left-handed adults) and methodologies (behavior, lesion studies, (intra)electroencephalography, neuroimaging) offers little if any support for this reading lateralized neural competition hypothesis. Specifically, right-lateralized face-selective neural activity already emerges at a few months of age, well before reading acquisition. Moreover, consistent evidence of face recognition performance and its right hemispheric lateralization being modulated by literacy level during development or at adulthood is lacking. Given the absence of solid alternative hypotheses and the key role of neural competition in the sensory-motor cortices for selectivity of representations, learning, and plasticity, a revised language-related neural competition hypothesis for the right hemispheric lateralization of face recognition should be further explored in future research, albeit with substantial conceptual clarification and advances in methodological rigor.

Holistic face recognition is an emergent phenomenon of spatial processing in face-selective regions

Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial processing in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical holistic processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. Here we show that in face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage are smaller and shifted downward in response to face inversion. From these measurements, we successfully predict the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial processing in face-selective regions may enable holistic perception. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.

Peripheral facial features guiding eye movements and reducing fixational variability

Face processing is a fast and efficient process due to its evolutionary and social importance. A majority of people direct their first eye movement to a featureless point just below the eyes that maximizes accuracy in recognizing a person's identity and gender. Yet, the exact properties or features of the face that guide the first eye movements and reduce fixational variability are unknown. Here, we manipulated the presence of the facial features and the spatial configuration of features to investigate their effect on the location and variability of first and second fixations to peripherally presented faces. Our results showed that observers can utilize the face outline, individual facial features, and feature spatial configuration to guide the first eye movements to their preferred point of fixation. The eyes have a preferential role in guiding the first eye movements and reducing fixation variability. Eliminating the eyes or altering their position had the greatest influence on the location and variability of fixations and resulted in the largest detriment to face identification performance. The other internal features (nose and mouth) also contribute to reducing fixation variability. A subsequent experiment measuring detection of single features showed that the eyes have the highest detectability (relative to other features) in the visual periphery providing a strong sensory signal to guide the oculomotor system. Together, the results suggest a flexible multiple-cue approach that might be a robust solution to cope with how the varying eccentricities in the real world influence the ability to resolve individual feature properties and the preferential role of the eyes.

Attentional shift within and between faces: Evidence from children with and without a diagnosis of autism spectrum disorder

Evidence of attentional atypicalities for faces in Autism Spectrum Disorders (ASD) are far from being confirmed. Using eye-tracking technology we compared space-based and object-based attention in children with, and without, a diagnosis of ASD. By capitalizing on Egly's paradigm, we presented two objects (2 faces and their phase-scrambled equivalent) and cued a location in one of the two objects. Then, a target appeared at the same location as the cue (Valid condition), or at a different location within the same object (Same Object condition), or at a different location in another object (Different Object condition). The attentional benefit/cost in terms of time for target detection in each of the three conditions was computed. The findings revealed that target detection was always faster in the valid condition than in the invalid condition, regardless of the type of stimulus and the group of children. Thus, no difference emerged between the two groups in terms of space-based attention. Conversely the two groups differed in object-based attention. Children without a diagnosis of ASD showed attentional shift cost with phase-scrambled stimuli, but not with faces. Instead, children with a diagnosis of ASD deployed similar attentional strategies to focus on faces and their phase-scrambled version.

Mapping the Featural and Holistic Face Processing of Bad and Good Face Recognizers

Individual abilities in face recognition (good versus bad recognizers) were explored by means of event-related potentials (ERPs). The adaptation response profile of the N170 component to whole faces, eyes and mouths was used in order to highlight the crucial role of individual abilities in identity repetition processes for unfamiliar faces. The main point of this study is to underline the importance of characterizing the performance (bad or good) of the participants and to show that behaviorally selected groups might reveal neural differences. Good recognizers showed selective right hemisphere N170 repetition effects for whole faces and not for features. On the contrary, bad recognizers showed a general repetition effect not specifically related to faces and more pronounced processing for features. These findings suggest a different contribution of holistic and featural analysis in bad and good performers. In conclusion, we propose that the N170 might be used as a tool to tease apart face encoding processes as a function of individual differences.

Intracerebral electrical stimulation to understand the neural basis of human face identity recognition

Recognizing people's identity by their faces is a key function in the human species, supported by regions of the ventral occipito-temporal cortex (VOTC). In the last decade, there have been several reports of perceptual face distortion during direct electrical stimulation (DES) with subdural electrodes positioned over a well-known face-selective VOTC region of the right lateral middle fusiform gyrus (LatMidFG; i.e., the "Fusiform Face Area", FFA). However, transient impairments of face identity recognition (FIR) have been extremely rare and only behaviorally quantified during DES with intracerebral (i.e., depth) electrodes in stereo-electroencephalography (SEEG). The three detailed cases reported so far, summarized here, were specifically impaired at FIR during DES inside different anatomical VOTC regions of the right hemisphere: the inferior occipital gyrus (IOG) and the LatMidFG, as well as a region that lies at the heart of a large magnetic susceptibility artifact in functional magnetic resonance imaging (fMRI): the anterior fusiform gyrus (AntFG). In the first two regions, the eloquent electrode contacts were systematically associated with the highest face-selective and (unfamiliar) face individuation responses as measured with intracerebral electrophysiology. Stimulation in the right AntFG did not lead to perceptual changes but also caused an inability to remember having been presented face pictures, as if the episode was never recorded in memory. These observations support the view of an extensive network of face-selective VOTC regions subtending human FIR, with at least three critical nodes in the right hemisphere associated with differential intrinsic and extrinsic patterns of reentrant connectivity.

Kinesthetic engagement in Gestalt evaluation outscores analytical 'atomic feature' evaluation in perceiving aging in crystallization images of agricultural products

There is an increasing interest in a systemic approach to food quality. From this perspective, the copper chloride crystallization method is an interesting asset as it enables an estimation of a sample's 'resilience' in response to controlled degradation. In previous studies, we showed that an ISO-standardized visual evaluation panel could correctly rank crystallization images of diverse agricultural products according to their degree of induced degradation. In this paper we examined the role of contextual sensitivity herein, with the aim to further improve the visual evaluation. To this end, we compared subjects' performance in ranking tests, while primed according to two perceptional strategies (levels: analytical vs. kinesthetic engagement), according to a within-subject design. The ranking test consisted out of wheat and rocket lettuce crystallization images, exhibiting four levels of induced degradation. The perceptual strategy imbuing kinesthetic engagement improved the performance of the ranking test in both samples tested. To the best of our knowledge, this is the first report on the training and application of such a perceptual strategy in visual evaluation.

Body-Posture Recognition by Undergraduate Students Majoring in Physical Education and Other Disciplines

Humans are more proficient at processing visual display of body posture when the body is in upright orientation, compared to when inverted (inversion effect). Here we investigated whether extensive exposure or expertise on body posture recognition would affect the efficiency with which body-posture is processed. Using whole-body and piecemeal-body postures as stimuli, we performed two experiments to investigate whether body-posture recognition differed between two groups of participants: undergraduates majoring in physical education (PE) and those in other subjects (non-PE), respectively. These two groups differed significantly in the frequency and intensity of exercise per day and/or accumulated exercise time. In our experiments, following initial presentation of an image of a body posture, participants were shown the same or a different stimulus and were asked to report whether or not they had been previously shown the same image. The orientations of the body postures were also varied between trials. Our results showed that, in Experiment 1, for whole-body posture recognition, both the PE and non-PE groups showed a robust body-inversion effect in terms of both error rate and reaction time (RT), but the magnitude of the body-inversion effect in the RT measure was greater in the PE than the non-PE group. In Experiment 2, for piecemeal-body postures, both groups showed the inversion effect in terms of both error rate and RT measures and the PE group made fewer overall errors than the non-PE group. These cumulative results suggest that a superiority effect exists for PE participants compared with non-PE participants. Our results are generally consistent with the expertise hypothesis.

Developmental Prosopagnosia and Elastic Versus Static Face Recognition in an Incidental Learning Task

Previous research on the beneficial effect of motion has postulated that learning a face in motion provides additional cues to recognition. Surprisingly, however, few studies have examined the beneficial effect of motion in an incidental learning task and developmental prosopagnosia (DP) even though such studies could provide more valuable information about everyday face recognition compared to the perception of static faces. In the current study, 18 young adults (Experiment 1) and five DPs and 10 age-matched controls (Experiment 2) participated in an incidental learning task during which both static and elastically moving unfamiliar faces were sequentially presented and were to be recognized in a delayed visual search task during which the faces could either keep their original presentation or switch (from static to elastically moving or vice versa). In Experiment 1, performance in the elastic-elastic condition reached a significant improvement relative to the elastic-static and static-elastic condition, however, no significant difference could be detected relative to the static-static condition. Except for higher scores in the elastic-elastic compared to the static-elastic condition in the age-matched group, no other significant differences were detected between conditions for both the DPs and the age-matched controls. The current study could not provide compelling evidence for a general beneficial effect of motion. Age-matched controls performed generally worse than DPs, which may potentially be explained by their higher rates of false alarms. Factors that could have influenced the results are discussed.

Test-Retest-Reliability of Video-Oculography During Free Visual Exploration in Right-Hemispheric Stroke Patients With Neglect

The Mean gaze position during free visual exploration (FVE) is a sensitive tool to detect neglect in patients after a right-hemispheric stroke. Here we investigated the test-retest-reliability of mean gaze position during FVE in 23 patients with left-sided neglect after a first-ever sub-acute right-hemispheric stroke. We analyzed the reliability between different test sets administered within 11 days (test sets A and B, each including different images and their mirrored versions), and between repeated measures using the same test set administered three times within 2 days (test set C, including the same images and their mirrored versions). The intra-class correlation coefficient (ICC) showed good reliability between the two different test sets (test sets A and B; ICC = 0.819), and excellent reliability for the repeated measures with the same test set C (ICC = 0.964). FVE can therefore be recommended for the longitudinal assessments of patients’ neglect severity during neurorehabilitation as well as in treatment trials.

Holistic processing of facial identity in developmental prosopagnosia

The nature of the perceptual deficit seen in developmental prosopagnosia remains poorly understood. One possibility is that these individuals experience face recognition difficulties because they fail to process faces holistically; they may be less able to analyze distal regions in parallel and therefore struggle to integrate information from different regions into a unified perceptual whole. Consequently, developmental prosopagnosics may be forced to base perceptual decisions on a slow, effortful piecemeal analysis of local facial features. In the present study, we sought to test this view by comparing the face recognition of developmental prosopagnosics and typical observers under two viewing conditions: when target faces were briefly presented in their entirety, and when they were inspected region-by-region through a dynamic aperture. If developmental prosopagnosics are forced to base perceptual decisions on information accumulated from a serial piecemeal analysis, one would expect little if any decrement in performance when target faces are viewed through apertures. Contrary to this prediction, however, developmental prosopagnosics showed strong aperture effects comparable with typical observers; their perceptual decisions were more accurate in the whole-face condition than when targets were viewed through the aperture. As expected, the developmental prosopagnosics were less accurate than typical controls when judging briefly presented faces shown in their entirety. Strikingly, however, they were also less able to accumulate perceptual evidence from a serial region-by-region analysis, than typical observers. Our results suggest that the perceptual problems seen in this population arise from imprecise descriptions of local regions, not aberrant holistic processing.

Assessing the presence of face biases by means of anorthoscopic perception

Anorthoscopy is a presentation mode in which an image is shown sliding behind a slit-shaped aperture, so that it is visible only part by part and never in its entirety. With the aims to assess (1) whether the processing of complex stimuli (faces) correctly occurs in anorthoscopy, and (2) whether the Own-Gender Bias (OGB: the better recognition of stimuli belonging to the same gender of the observer: faster and more accurate) and the Left-Face Bias (LFB: the preference to analyze the left half of the face) occur in such a part by part presentation, we presented female and male faces as whole stimuli (Experiment 1) and in anorthoscopy (Experiments 2 and 3), as well as female/male chimeric faces (Experiment 4), during a gender categorization task. Experiment 1 confirmed that participants correctly categorized the gender of faces, but the OGB was not found. In Experiments 2 and 3 we manipulated the direction (Experiment 2: upward/downward; Experiment 3: leftward/rightward), the speed (slow and fast) of the sliding faces, and the width of the aperture (small and large). Both tasks revealed that facial gender was correctly categorized in anorthoscopy. The OGB was found, but only for males/females in Experiments 2/3, respectively. In Experiment 4 the LFB emerged only in the tachistoscopic session, suggesting that this perceptual bias does not extend to anorthoscopy.

The Use of Eye Tracking to Discern the Threshold at Which Metopic Orbitofrontal Deformity Attracts Attention

INTRODUCTION AND OBJECTIVES

Surgical treatment for trigonocephaly aims to eliminate a stigmatizing deformity, yet the severity that captures unwanted attention is unknown. Surgeons intervene at different points of severity, eliciting controversy. This study used eye tracking to investigate when deformity is perceived.

MATERIAL AND METHODS

Three-dimensional photogrammetric images of a normal child and a child with trigonocephaly were mathematically deformed, in 10% increments, to create a spectrum of 11 images. These images were shown to participants using an eye tracker. Participants' gaze patterns were analyzed, and participants were asked if each image looked "normal" or "abnormal."

RESULTS

Sixty-six graduate students were recruited. Average dwell time toward pathologic areas of interest (AOIs) increased proportionally, from 0.77 ± 0.33 seconds at 0% deformity to 1.08 ± 0.75 seconds at 100% deformity (P < .0001). A majority of participants did not agree an image looked "abnormal" until 90% deformity from any angle.

CONCLUSION

Eye tracking can be used as a proxy for attention threshold toward orbitofrontal deformity. The amount of attention toward orbitofrontal AOIs increased proportionally with severity. Participants did not generally agree there was "abnormality" until deformity was severe. This study supports the assertion that surgical intervention may be best reserved for more severe deformity.

Assessing the Big Five personality traits using real-life static facial images

There is ample evidence that morphological and social cues in a human face provide signals of human personality and behaviour. Previous studies have discovered associations between the features of artificial composite facial images and attributions of personality traits by human experts. We present new findings demonstrating the statistically significant prediction of a wider set of personality features (all the Big Five personality traits) for both men and women using real-life static facial images. Volunteer participants (N = 12,447) provided their face photographs (31,367 images) and completed a self-report measure of the Big Five traits. We trained a cascade of artificial neural networks (ANNs) on a large labelled dataset to predict self-reported Big Five scores. The highest correlations between observed and predicted personality scores were found for conscientiousness (0.360 for men and 0.335 for women) and the mean effect size was 0.243, exceeding the results obtained in prior studies using 'selfies'. The findings strongly support the possibility of predicting multidimensional personality profiles from static facial images using ANNs trained on large labelled datasets. Future research could investigate the relative contribution of morphological features of the face and other characteristics of facial images to predicting personality.

Flexible Orientation Tuning of Visual Representations of Human Body Postures: Evidence From Long-Term Priming

The proficiency of human observers to identify body postures is examined in three experiments. We use a posture decision task in which participants are primed with either anatomically possible or impossible postures (in the latter case the upper and lower body face in opposite directions). In a long-term priming paradigm (i.e., in an initial priming block of trials and a subsequent test phase several minutes later), we manipulate the relation between priming and test postures with respect to the identity of the person in the body postures (Experiment 1), the prototypicality of the depth orientations (Experiment 2), and the variability of the priming orientations (Experiment 3). Reaction time to the test postures is the main dependent variable. In Experiment 1 it is found that priming of postures does not depend on the exact visual appearance of the actor (either same priming and test female or male figure or different figures), supporting the hypothesis that posture priming primarily is determined by the spatial relations between the body parts and much less by characteristics of the person involved. Long-term priming in our paradigm apparently is based on the reactivation of high-level posture representations that make abstraction of the identity of the human figure. In Experiment 2 we observe that privileged or prototypical orientations (e.g., 3/4 views) do not affect long-term priming of body postures. In Experiment 3, we find that increasing or decreasing the variability between the priming and test figures influences reaction time performance. Collectively, these results provide a better understanding of the flexibility (e.g., invariant to identity) and limits (e.g., depending on depth orientation) of the processes supporting human posture recognition.

Atypical holistic processing of facial identity and expression in a case of acquired prosopagnosia

Typical face perception is mediated by holistic processing (i.e., the simultaneous integration of face parts into a whole representation). People with Acquired Prosopagnosia (AP), who have lost the ability to recognise faces after a brain lesion, should thus show atypical holistic coding. Our aim is to use the composite-face effect (CFE) as a measure of holistic processing in ST, a 48-year-old woman with AP but normal recognition of facial expressions of emotions, and matched healthy control participants. Two experiments examining the CFE for identity (Experiment 1) and for expression of emotions (Experiment 2) were conducted. Contrary to controls, in both experiments, ST showed an atypical (i.e., reversed) CFE, thus suggesting altered holistic mechanisms affecting both components of perceptual judgement. Results also suggest that normal facial expression recognition is achievable even with holistic processing difficulties, possibly through compensatory, part-based, mechanisms.

Artificial Faces Predict Gaze Allocation in Complex Dynamic Scenes

Both low-level physical saliency and social information, as presented by human heads or bodies, are known to drive gaze behavior in free-viewing tasks. Researchers have previously made use of a great variety of face stimuli, ranging from photographs of real humans to schematic faces, frequently without systematically differentiating between the two. In the current study, we used a Generalized Linear Mixed Model (GLMM) approach to investigate to what extent schematic artificial faces can predict gaze when they are presented alone or in competition with real human faces. Relative differences in predictive power became apparent, while GLMMs suggest substantial effects for real and artificial faces in all conditions. Artificial faces were accordingly less predictive than real human faces but still contributed significantly to gaze allocation. These results help to further our understanding of how social information guides gaze in complex naturalistic scenes.

The functional neuroanatomy of face perception: from brain measurements to deep neural networks

A central goal in neuroscience is to understand how processing within the ventral visual stream enables rapid and robust perception and recognition. Recent neuroscientific discoveries have significantly advanced understanding of the function, structure and computations along the ventral visual stream that serve as the infrastructure supporting this behaviour. In parallel, significant advances in computational models, such as hierarchical deep neural networks (DNNs), have brought machine performance to a level that is commensurate with human performance. Here, we propose a new framework using the ventral face network as a model system to illustrate how increasing the neural accuracy of present DNNs may allow researchers to test the computational benefits of the functional architecture of the human brain. Thus, the review (i) considers specific neural implementational features of the ventral face network, (ii) describes similarities and differences between the functional architecture of the brain and DNNs, and (iii) provides a hypothesis for the computational value of implementational features within the brain that may improve DNN performance. Importantly, this new framework promotes the incorporation of neuroscientific findings into DNNs in order to test the computational benefits of fundamental organizational features of the visual system.

Inversion Impairs Expert Budgerigar Identity Recognition: A Face-Like Effect for a Nonface Object of Expertise

The face-inversion effect is the finding that picture-plane inversion disproportionately impairs face recognition compared to object recognition and is now attributed to greater orientation-sensitivity of holistic processing for faces but not common objects. Yet, expert dog judges have showed similar recognition deficits for inverted dogs and inverted faces, suggesting that holistic processing is not specific to faces but to the expert recognition of perceptually similar objects. Although processing changes in expert object recognition have since been extensively documented, no other studies have observed the distinct recognition deficits for inverted objects-of-expertise that people as face experts show for faces. However, few studies have examined experts who recognize individual objects similar to how people recognize individual faces. Here we tested experts who recognize individual budgerigar birds. The effect of inversion on viewpoint-invariant budgerigar and face recognition was compared for experts and novices. Consistent with the face-inversion effect, novices showed recognition deficits for inverted faces but not for inverted budgerigars. By contrast, experts showed equal recognition deficits for inverted faces and budgerigars. The results are consistent with the hypothesis that processes underlying the face-inversion effect are specific to the expert individuation of perceptually similar objects.

Face Processing Systems: From Neurons to Real-World Social Perception

Primate face processing depends on a distributed network of interlinked face-selective areas composed of face-selective neurons. In both humans and macaques, the network is divided into a ventral stream and a dorsal stream, and the functional similarities of the areas in humans and macaques indicate they are homologous. Neural correlates for face detection, holistic processing, face space, and other key properties of human face processing have been identified at the single neuron level, and studies providing causal evidence have established firmly that face-selective brain areas are central to face processing. These mechanisms give rise to our highly accurate familiar face recognition but also to our error-prone performance with unfamiliar faces. This limitation of the face system has important implications for consequential situations such as eyewitness identification and policing.

The Functional Neuroanatomy of Human Face Perception

Face perception is critical for normal social functioning and is mediated by a network of regions in the ventral visual stream. In this review, we describe recent neuroimaging findings regarding the macro- and microscopic anatomical features of the ventral face network, the characteristics of white matter connections, and basic computations performed by population receptive fields within face-selective regions composing this network. We emphasize the importance of the neural tissue properties and white matter connections of each region, as these anatomical properties may be tightly linked to the functional characteristics of the ventral face network. We end by considering how empirical investigations of the neural architecture of the face network may inform the development of computational models and shed light on how computations in the face network enable efficient face perception.

Information-Theoretic Evidence for Predictive Coding in the Face-Processing System

Predictive coding suggests that the brain infers the causes of its sensations by combining sensory evidence with internal predictions based on available prior knowledge. However, the neurophysiological correlates of (pre)activated prior knowledge serving these predictions are still unknown. Based on the idea that such preactivated prior knowledge must be maintained until needed, we measured the amount of maintained information in neural signals via the active information storage (AIS) measure. AIS was calculated on whole-brain beamformer-reconstructed source time courses from MEG recordings of 52 human subjects during the baseline of a Mooney face/house detection task. Preactivation of prior knowledge for faces showed as α-band-related and β-band-related AIS increases in content-specific areas; these AIS increases were behaviorally relevant in the brain's fusiform face area. Further, AIS allowed decoding of the cued category on a trial-by-trial basis. Our results support accounts indicating that activated prior knowledge and the corresponding predictions are signaled in low-frequency activity (<30 Hz).SIGNIFICANCE STATEMENT Our perception is not only determined by the information our eyes/retina and other sensory organs receive from the outside world, but strongly depends also on information already present in our brains, such as prior knowledge about specific situations or objects. A currently popular theory in neuroscience, predictive coding theory, suggests that this prior knowledge is used by the brain to form internal predictions about upcoming sensory information. However, neurophysiological evidence for this hypothesis is rare, mostly because this kind of evidence requires strong a priori assumptions about the specific predictions the brain makes and the brain areas involved. Using a novel, assumption-free approach, we find that face-related prior knowledge and the derived predictions are represented in low-frequency brain activity.

Adaptive Anchoring Model: How Static and Dynamic Presentations of Time Series Influence Judgments and Predictions

When attempting to predict future events, people commonly rely on historical data. One psychological characteristic of judgmental forecasting of time series, established by research, is that when people make forecasts from series, they tend to underestimate future values for upward trends and overestimate them for downward ones, so‐called trend‐damping (modeled by anchoring on, and insufficient adjustment from, the average of recent time series values). Events in a time series can be experienced sequentially (dynamic mode), or they can also be retrospectively viewed simultaneously (static mode), not experienced individually in real time. In one experiment, we studied the influence of presentation mode (dynamic and static) on two sorts of judgment: (a) predictions of the next event (forecast) and (b) estimation of the average value of all the events in the presented series (average estimation). Participants' responses in dynamic mode were anchored on more recent events than in static mode for all types of judgment but with different consequences; hence, dynamic presentation improved prediction accuracy, but not estimation. These results are not anticipated by existing theoretical accounts; we develop and present an agent‐based model—the adaptive anchoring model (ADAM)—to account for the difference between processing sequences of dynamically and statically presented stimuli (visually presented data). ADAM captures how variation in presentation mode produces variation in responses (and the accuracy of these responses) in both forecasting and judgment tasks. ADAM's model predictions for the forecasting and judgment tasks fit better with the response data than a linear‐regression time series model. Moreover, ADAM outperformed autoregressive‐integrated‐moving‐average (ARIMA) and exponential‐smoothing models, while neither of these models accounts for people's responses on the average estimation task.

Explaining Sad People's Memory Advantage for Faces

Sad people recognize faces more accurately than happy people (Hills et al., 2011). We devised four hypotheses for this finding that are tested between in the current study. The four hypotheses are: (1) sad people engage in more expert processing associated with face processing; (2) sad people are motivated to be more accurate than happy people in an attempt to repair their mood; (3) sad people have a defocused attentional strategy that allows more information about a face to be encoded; and (4) sad people scan more of the face than happy people leading to more facial features to be encoded. In Experiment 1, we found that dysphoria (sad mood often associated with depression) was not correlated with the face-inversion effect (a measure of expert processing) nor with response times but was correlated with defocused attention and recognition accuracy. Experiment 2 established that dysphoric participants detected changes made to more facial features than happy participants. In Experiment 3, using eye-tracking we found that sad-induced participants sampled more of the face whilst avoiding the eyes. Experiment 4 showed that sad-induced people demonstrated a smaller own-ethnicity bias. These results indicate that sad people show different attentional allocation to faces than happy and neutral people.

Perceptual Learning of Faces: A Rehabilitative Study of Acquired Prosopagnosia

Despite many studies of acquired prosopagnosia, there have been only a few attempts at its rehabilitation, all in single cases, with a variety of mnemonic or perceptual approaches, and of variable efficacy. In a cohort with acquired prosopagnosia, we evaluated a perceptual learning program that incorporated variations in view and expression, which was aimed at training perceptual stages of face processing with an emphasis on ecological validity. Ten patients undertook an 11-week face training program and an 11-week control task. Training required shape discrimination between morphed facial images, whose similarity was manipulated by a staircase procedure to keep training near a perceptual threshold. Training progressed from blocks of neutral faces in frontal view through increasing variations in view and expression. Whereas the control task did not change perception, training improved perceptual sensitivity for the trained faces and generalized to new untrained expressions and views of those faces. There was also a significant transfer to new faces. Benefits were maintained over a 3-month period. Training efficacy was greater for those with more perceptual deficits at baseline. We conclude that perceptual learning can lead to persistent improvements in face discrimination in acquired prosopagnosia. This reflects both acquisition of new skills that can be applied to new faces as well as a degree of overlearning of the stimulus set at the level of 3-D expression-invariant representations.

Seeing the eyes in acquired prosopagnosia

Case reports have suggested that perception of the eye region may be impaired more than that of other facial regions in acquired prosopagnosia. However, it is unclear how frequently this occurs, whether such impairments are specific to a certain anatomic subtype of prosopagnosia, and whether these impairments are related to changes in the scanning of faces. We studied a large cohort of 11 subjects with this rare disorder, who had a variety of occipitotemporal or anterior temporal lesions, both unilateral and bilateral. Lesions were characterized by functional and structural imaging. Subjects performed a perceptual discrimination test in which they had to discriminate changes in feature position, shape, or external contour. Test conditions were manipulated to stress focused or divided attention across the whole face. In a second experiment we recorded eye movements while subjects performed a face memory task. We found that greater impairment for eye processing was more typical of subjects with occipitotemporal lesions than those with anterior temporal lesions. This eye selectivity was evident for both eye position and shape, with no evidence of an upper/lower difference for external contour. A greater impairment for eye processing was more apparent under attentionally more demanding conditions. Despite these perceptual deficits, most subjects showed a normal tendency to scan the eyes more than the mouth. We conclude that occipitotemporal lesions are associated with a partially selective processing loss for eye information and that this deficit may be linked to loss of the right fusiform face area, which has been shown to have activity patterns that emphasize the eye region.

Real-life experience with personally familiar faces enhances discrimination based on global information

Despite the agreement that experience with faces leads to more efficient processing, the underlying mechanisms remain largely unknown. Building on empirical evidence from unfamiliar face processing in healthy populations and neuropsychological patients, the present experiment tested the hypothesis that personal familiarity is associated with superior discrimination when identity information is derived based on global, as opposed to local facial information. Diagnosticity and availability of local and global information was manipulated through varied physical similarity and spatial resolution of morph faces created from personally familiar or unfamiliar faces. We found that discrimination of subtle changes between highly similar morph faces was unaffected by familiarity. Contrariwise, relatively more pronounced physical (i.e., identity) differences were more efficiently discriminated for personally familiar faces, indicating more efficient processing of global, as opposed to local facial information through real-life experience.

Facial expression training optimises viewing strategy in children and adults

This study investigated whether training-related improvements in facial expression categorization are facilitated by spontaneous changes in gaze behaviour in adults and nine-year old children. Four sessions of a self-paced, free-viewing training task required participants to categorize happy, sad and fear expressions with varying intensities. No instructions about eye movements were given. Eye-movements were recorded in the first and fourth training session. New faces were introduced in session four to establish transfer-effects of learning. Adults focused most on the eyes in all sessions and increased expression categorization accuracy after training coincided with a strengthening of this eye-bias in gaze allocation. In children, training-related behavioural improvements coincided with an overall shift in gaze-focus towards the eyes (resulting in more adult-like gaze-distributions) and towards the mouth for happy faces in the second fixation. Gaze-distributions were not influenced by the expression intensity or by the introduction of new faces. It was proposed that training enhanced the use of a uniform, predominantly eyes-biased, gaze strategy in children in order to optimise extraction of relevant cues for discrimination between subtle facial expressions.

Recognizing the bank robber and spotting the difference: emotional state and global vs. local attentional set

In two experiments (161 participants in total), we investigated how current mood influences processing styles (global vs. local). Participants watched a video of a bank robbery before receiving a positive, negative or neutral induction, and they performed two tasks: a face-recognition task about the bank robber as global processing measure, and a spot-the-difference task using neutral pictures (Experiment-1) or emotional scenes (Experiment-2) as local processing measure. Results showed that positive mood induction favoured a global processing style, enhancing participants' ability to correctly identify a face even when they watched the video before the mood-induction. This shows that, besides influencing encoding processes, mood state can be also related to retrieval processes. On the contrary, negative mood induction enhanced a local processing style, making easier and faster the detection of differences between nearly identical pictures, independently of their valence. This dissociation supports the hypothesis that current mood modulates processing through activation of different cognitive styles.

Holistic processing of human body postures: evidence from the composite effect

The perception of socially relevant stimuli (e.g., faces and bodies) has received considerable attention in the vision science community. It is now widely accepted that human faces are processed holistically and not only analytically. One observation that has been taken as evidence for holistic face processing is the face composite effect: two identical top halves of a face tend to be perceived as being different when combined with different bottom halves. This supports the hypothesis that face processing proceeds holistically. Indeed, the interference effect disappears when the two face parts are misaligned (blocking holistic perception). In the present study, we investigated whether there is also a composite effect for the perception of body postures: are two identical body halves perceived as being in different poses when the irrelevant body halves differ from each other? Both a horizontal (i.e., top-bottom body halves; Experiment 1) and a vertical composite effect (i.e., left-right body halves; Experiment 2) were examined by means of a delayed matching-to-sample task. Results of both experiments indicate the existence of a body posture composite effect. This provides evidence for the hypothesis that body postures, as faces, are processed holistically.

A family at risk: congenital prosopagnosia, poor face recognition and visuoperceptual deficits within one family

Congenital prosopagnosia (CP) describes a severe face processing impairment despite intact early vision and in the absence of overt brain damage. CP is assumed to be present from birth and often transmitted within families. Previous studies reported conflicting findings regarding associated deficits in nonface visuoperceptual tasks. However, diagnostic criteria for CP significantly differed between studies, impeding conclusions on the heterogeneity of the impairment. Following current suggestions for clinical diagnoses of CP, we administered standardized tests for face processing, a self-report questionnaire and general visual processing tests to an extended family (N=28), in which many members reported difficulties with face recognition. This allowed us to assess the degree of heterogeneity of the deficit within a large sample of suspected CPs of similar genetic and environmental background. (a) We found evidence for a severe face processing deficit but intact nonface visuoperceptual skills in three family members - a father and his two sons - who fulfilled conservative criteria for a CP diagnosis on standardized tests and a self-report questionnaire, thus corroborating findings of familial transmissions of CP. (b) Face processing performance of the remaining family members was also significantly below the mean of the general population, suggesting that face processing impairments are transmitted as a continuous trait rather than in a dichotomous all-or-nothing fashion. (c) Self-rating scores of face recognition showed acceptable correlations with standardized tests, suggesting this method as a viable screening procedure for CP diagnoses. (d) Finally, some family members revealed severe impairments in general visual processing and nonface visual memory tasks either in conjunction with face perception deficits or as an isolated impairment. This finding may indicate an elevated risk for more general visuoperceptual deficits in families with prosopagnosic members.